From nobody Thu Oct 1 01:32:10 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EEF871B2BD7 for ; Thu, 1 Oct 2015 01:32:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.144 X-Spam-Level: X-Spam-Status: No, score=0.144 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, FRT_BELOW2=2.154, J_CHICKENPOX_36=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YU7dtsaVyvAL for ; Thu, 1 Oct 2015 01:32:04 -0700 (PDT) Received: from mailfilter.orange.md (mailfilter.orange.md [94.243.64.204]) by ietfa.amsl.com (Postfix) with ESMTP id 0FD381B2BD3 for ; Thu, 1 Oct 2015 01:32:02 -0700 (PDT) Received: from localhost.localdomain (antispam.orange.md [127.0.0.1]) by localhost (Postfix) with ESMTP id 9E41C27E038; Thu, 1 Oct 2015 11:32:20 +0300 (EEST) Received: from antispam.orange.md by antispam.orange.md with queue id 249653-1; Thu, 01 Oct 2015 08:32:20 GMT Received: from XCHSRV03.main.orange.md (unknown [192.168.200.63]) by mailfilter.orange.md (Postfix) with ESMTP id 807323B2678; Thu, 1 Oct 2015 11:32:20 +0300 (EEST) Received: from XCHSRV01.main.orange.md ([fe80::685f:aef6:93b0:dea7]) by XCHSRV03.main.orange.md ([fe80::6dbc:28dd:213b:8931%14]) with mapi id 14.02.0328.009; Thu, 1 Oct 2015 11:32:00 +0300 From: Veaceslav ROMAN To: Ingemar Johansson S , Eric Dumazet , Neal Cardwell Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw Date: Thu, 1 Oct 2015 08:31:59 +0000 Message-ID: <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> In-Reply-To: <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> Accept-Language: en-US, ro-RO Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.107.165] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Cc: Eric Dumazet , "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2015 08:32:09 -0000 WWVzLCBIWVNUQVJUX0RFTEFZX01JTiAgKDRVPDwzKSAuIA0KQnV0IGN1cnJfcnR0IGFuZCBkZWxh eV9taW4gY29tZSBmcm9tIGJpY3RjcF9hY2tlZCB3aGljaCBpcyBhbHNvIDw8MyA6IA0KCUluIHRj cF9jdWJpYy5jLCBmdW5jdGlvbiAgbGluZSA0MzMgKGtlcm5lbCA0LjIpIA0KCQlkZWxheSA9IChy dHRfdXMgPDwgMykgLyBVU0VDX1BFUl9NU0VDOw0KU28sIDRVPDwzIGlzIGVxdWl2YWxlbnQgdG8g NCBtcyAuDQoNClJlZ2FyZGluZyBwYWNpbmcgSSBhbSBub3Qgc3VyZSB3aWxsIGhlbHAgdGhlIHBl cmZvcm1hbmNlOiB0aGVyZSBhcmUgZ2FwcyBpbiBBQ0sgdHJhaW5zIG9mIHRoZSBvcmRlciBvZiBt aWxsaXNlY29uZHMgYW5kIHRoaXMgaXMgYSBsb3N0IHRpbWUgYW5kIHRoZXJlIGlzIG5vIHdheSB0 byBjb21wZW5zYXRlIGl0LiBJZiBpbnRyb2R1Y2UgQUNLIHBhY2luZyBvciBkYXRhIHBhY2tldHMg cGFjaW5nIHRoaXMgd291bGQgYWRkIHRvIHRoZSBvdmVyYWxsIGxvc3QgdGltZS4NClBhY2luZyBj YW4gaGVscCBvbmx5IHRvIGFsbGV2aWF0ZSBhIHBvdGVudGlhbCBwcm9ibGVtIG9mIHNvbWUgZG93 bnN0cmVhbSByb3V0ZXJzIHdpdGggaW5zdWZmaWNpZW50IGJ1ZmZlcnMgd2hpY2ggbWF5IGludHJv ZHVjZSB1bmV4cGVjdGVkIHBhY2tldHMgZHJvcHMgd2hlbiBsYXJnZSB0cmFpbiBzZW50IGF0IGEg aGlnaCBzcGVlZCBhcyBhIHJlcGx5IHRvIGhpZ2hseSBjb21wcmVzc2VkIEFDSyB0cmFpbiwgYnV0 LCBvbiBhbm90aGVyIGhhbmRzLg0KQnV0IGhvdyBtdWNoIHBhY2luZyBhbmQgaG93IHRvIGFjY29t bW9kYXRlIHRvIGxhcmdlbHkgY2hhbmdpbmcgcmFkaW8gY29uZGl0aW9ucyA/IEFuZCB3aXRoIHBl cm1hbmVudGx5IGNoYW5naW5nIHRlY2hub2xvZ3kgYW5kIHNwZWVkID8gQW55d2F5IHRoZSBwYWNp bmcgd2lsbCBhZGQgdG8gdGhlIHByb2JsZW0gb2YgZ2FwcyBhbmQgYmV0dGVyIHNpemUgcHJvcGVy bHkgcm91dGVycyBidWZmZXJzLiANCg0KSWRlYWxseSBBQ0sgdHJhaW4gaXMgc3BsaXQgaW4gMiBw YXJ0cywgYnV0IGluIHByYWN0aWNlIEkgZG8gc2VlIG1vcmUuIEJ1dCBhcyB0aGUgc3RyZWFtIHBy b2dyZXNzIHRoZSBsZXNzIGdhcHMuIFRoZSBiaWdnZXIgdGhlIGRvd25saW5rIHByZXNzdXJlIHRo ZSBtb3JlIEFDSyBhbmQgbGVzcyBjaGFuY2VzIHRvIGhhdmUgYSBwYXVzZSBpbiBBQ0sgc2VuZGlu ZywgdGhlbiB0aGVyZSB3aWxsIGJlIGEgY29udGludW91cyBmbG93IG9mIEJ1ZmZlciBTdGF0dXMg UmVwb3J0cyBhbmQgbGVzcyBvZiB0aGUgU2NoZWR1bGluZyBSZXF1ZXN0IChTUiwgIkhleSwgSSBo YXZlIHNvbWUgZGF0YSB0byBzZW5kIikuIFRoZSBvbmx5IGlzc3VlIEkgYW0gbm90IHN1cmUgaXMg b24gdGhlIHBvdGVudGlhbCBnYXBzIGluIG90aGVyIG5ldHdvcmtzLiBJIGRpZCBub3QgcHV0IGJl bGxvdyBhbGwgZGV0YWlscywgYnV0IHlvdSBzaG91bGQga25vdyB0aGF0IGRldmljZSBjYW4gbm90 IHNlbmQgU1IgaW4gYXJiaXRyYXJ5IG1vbWVudHMgb2YgdGltZSBidXQgaW4gcHJlZGVmaW5lZCBm b3IgaGltIGludGVydmFscy4gSW4gbXkgbmV0d29yayB0aGlzIHBlcmlvZCBpcyA1IG1zLCB0aGVy ZWZvcmUgdGhlIFJUVCBvZiB0aGUgZmlyc3QgQUNLIGluIHRoZSB0cmFpbiB3aWxsIGhhdmUgdmFy aWF0aW9ucyBvZiAyLjUgbXMuIEJ1dCBzdGFuZGFyZHMgYWRtaXQgdGhpcyBwZXJpb2QgdG8gYmUg YWxzbyAxMCBtcyAsMjAgbXMsIDQwIG1zLCA4MCBtcyBhbmQgSSBjYW4gbm90IHNheSB3aGF0IGlz IHRoZSBwcmFjdGljZSBvZiBvdGhlcnMuIEkgbWF5IG9ubHkgZ3Vlc3MgdGhhdCBtb3N0IHdpbGwg dHJ5IHRvIHVzZSB0aGUgYmVzdCBhbmQgZmFzdGVzdCwgaS5lLiA1IG1zLg0KQW55d2F5LCB0byBn ZXQgYSByZWFsaXN0aWMgbW9kZWxpbmcgb2YgTFRFIGludGVyYWN0aW9uIHdpdGggVENQIHRoZSBk ZXRhaWxlZCBNQUMgc2NoZWR1bGVyIG1vZGVsIGlzIG5lZWRlZC4gVW5mb3J0dW5hdGVseSwgbW9z dCBvZiB0aGUgbGl0ZXJhdHVyZSBmb2N1cyBvbiBzY2hlZHVsaW5nIG9mIHRoZSBQaHlzaWNhbCBS ZXNvdXJjZSBCbG9ja3MsIEFkYXB0aXZlIE1vZHVsYXRpb24gYW5kIENvZGluZywgTUlNTywgaS5l LiwgZm9jdXNpbmcgb24gdGhlIGRpc3RyaWJ1dGlvbiBvZiByZXNvdXJjZXMgYmV0d2VlbiB1c2Vy cywgd2hpY2ggaXMgZ29vZCBhbmQgbmVlZGVkLCBidXQgZG8gbm90IGFkZHJlc3MgdmVyeSBtdWNo IHRoZSB0aW1pbmcgaW50ZXJhY3Rpb24gd2l0aCBUQ1AuICBBdCBsZWFzdCwgSSBuZXZlciBtZXQg aW4gTFRFIHNjaGVkdWxpbmcgbGl0ZXJhdHVyZSBhIHRlcm0gIlRDUCBzZWxmLWNsb2NraW5nIi4g DQoNClRoZSA4IG1zICBIWVNUQVJUX0RFTEFZX01JTiBjb3VsZCBvbmx5IGJlIGEgc2ltcGxlIHNv bHV0aW9uIGZvciB0aGUgcHJvYmxlbSBvZiBjb3VudGluZyBvZiB0aGUgcm91bmQnIGN1cnJfcnR0 IGZyb20gdGhlIHNlY29uZCBBQ0sgb2YgdGhlIHRyYWluIChhcyB0aGUgc2Vjb25kIEFDSyBSVFQg d2lsbCBiZSwgdHlwaWNhbGx5LCBkdWUgdG8gc2NoZWR1bGluZywgNS03IG1zIGxvbmdlciB0aGFu IHRoZSBmaXJzdCBvbmUpLCBidXQgdGhpcyBvbmx5IGlmIFNSIHBlcmlvZGljaXR5IGlzIDUgbXMu ICANClRoZSBvdGhlciAyIHByb2JsZW1zIHdpbGwgcmVtYWluIHVuc29sdmVkLiAgIA0KDQpWZWFj ZXNsYXYgUm9tYW4NCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IEluZ2VtYXIg Sm9oYW5zc29uIFMgW21haWx0bzppbmdlbWFyLnMuam9oYW5zc29uQGVyaWNzc29uLmNvbV0gDQpT ZW50OiBUaHVyc2RheSwgMDEgT2N0b2JlciAyMDE1IDA4OjU5DQpUbzogVmVhY2VzbGF2IFJPTUFO OyBFcmljIER1bWF6ZXQ7IE5lYWwgQ2FyZHdlbGwNCkNjOiB0Y3BtQGlldGYub3JnOyBQaWVycyBP J0hhbmxvbjsgU2FuZ3RhZSBIYTsgRXJpYyBEdW1hemV0OyBlbmQyZW5kLWludGVyZXN0QHBvc3Rl bC5vcmc7IEluZ2VtYXIgSm9oYW5zc29uIFMNClN1YmplY3Q6IFJFOiBbdGNwbV0gW2UyZV0gVENQ IEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KDQpIaQ0KDQpUaGFua3MgZm9yIHRoZSBkZXRhaWxl ZCByZXBvcnQuIFdoYXQgc2VlbXMgdW5jbGVhciB0byBtZSBpcyB0aGUgdmFsdWUgb2YgSFlTVEFS VF9ERUxBWV9NSU4uIFlvdSBtZW50aW9uIDhtcyBidXQgd2hlbiBJIGxvb2sgaW50byB0aGUgNC4y IGtlcm5lbCBjb2RlIEkgZ2V0IHRoZSBmZWVsaW5nIHRoYXQgaXQgaXMgMzJtcyAoICNkZWZpbmUg SFlTVEFSVF9ERUxBWV9NSU4gICAgICAgKDRVPDwzKSAgICkuIElzIHRoaXMgYSBtaXN0YWtlIGZy b20gbXkgc2lkZSAgPw0KSSBoYXZlIHJ1biBMVEUgc2ltdWxhdGlvbnMgYW5kIHRyaWVkIHRvIHdy aW5nIHRoaXMgSHlTdGFydCBpc3N1ZSBpbnNpZGUgb3V0IGFuZCB1cHNpZGUgZG93biBidXQgc29m YXIgaXQgaGFzIGJlZW4gdmVyeSBkaWZmaWN1bHQgdG8gbWFrZSBIeVN0YXJ0IGV4aXQgcHJlbWF0 dXJlbHkgYnV0IHRoZW4gb2YgY291cnNlIEkgaGF2ZSBydW4gd2l0aCBIWVNUQVJUX0RFTEFZX01J TiA9IDMybXMuIA0KDQpBcyByZWdhcmRzIHRvIHRoZSBBQ0stY29tcHJlc3Npb24gcHJvYmxlbSwg eWVzLCBBQ0sgY29tcHJlc3Npb24gZWFzaWx5IG9jY3VyIGluIExURSwgZXZlbiBpbiBzaW11bGF0 aW9ucy4gV2hhdCBJIGhhdmUgc2VlbiBpcyB0aGF0IHBhY2tldCBwYWNpbmcgaXMgYSB2ZXJ5IGVm ZmljaWVudCByZW1lZHksIGZvciBpbnN0YW5jZSB0aGUgcGFja2V0IGltcGxlbWVudGVkIGluIFFV SUMgc2VlbXMgdG8gc29sdmUgQUNLIGNvbXByZXNzaW9uIGlzc3VlcyB2ZXJ5IHdlbGwuIA0KDQpJ dCBpcyBpbnRlcmVzdGluZyB3aGF0IHlvdSBzYXkgdGhhdCBBQ0sgdHJhaW5zIGFyZSBzbyBtdWNo IHNwbGl0IHVwLCBJIGNvdWxkIHVuZGVyc3RhbmQgdGhhdCB0aGV5IGFyZSBzcGxpdCB1cCBpbiB0 d28gcGFydHMgKGluaXRpYWwgZ3JhbnQgKyBhZGRpdGlvbmFsIGFmdGVyIHJlY2VpdmVkIEJTUiku ICANCg0KL0luZ2VtYXINCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBW ZWFjZXNsYXYgUk9NQU4gW21haWx0bzpWZWFjZXNsYXYuUm9tYW5Ab3JhbmdlLm1kXQ0KPiBTZW50 OiBkZW4gMSBva3RvYmVyIDIwMTUgMDI6MjUNCj4gVG86IEVyaWMgRHVtYXpldDsgTmVhbCBDYXJk d2VsbA0KPiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0YWUgSGE7IElu Z2VtYXIgSm9oYW5zc29uIFM7IA0KPiBFcmljIER1bWF6ZXQ7IGVuZDJlbmQtaW50ZXJlc3RAcG9z dGVsLm9yZw0KPiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRl cGxveW1lbnQNCj4gDQo+IEZpbmFsbHkgY2FuIHNoYXJlIHNvbWUgcmVzdWx0cyBvZiBIeXN0YXJ0 IGludGVyYWN0aW9uIHdpdGggTFRFLg0KPiAgRmV3IHdvcmRzIGFib3V0IHRoZSB0ZXN0aW5nIGNv bmZpZ3VyYXRpb246DQo+ICAJRGV2aWNlIFNhbXN1bmcgR2FsYXh5IE5vdGU0TFRFICwgdXAgdG8g MTUwIE1icHMgY2FwYWJsZQ0KPiAgICAgICAgICAgICAgUmFkaW8gTmV0d29yayBMVEUgMjYwMC8x ODAwDQo+ICAgICAgICAgICAgICBEaXN0YW5jZSBiZXR3ZWVuIHRoZSBDb3JlIE5ldHdvcmsgKEVQ QykgYW5kIEJhc2UgU3RhdGlvbiAtIDEwIGttDQo+ICAgICAgICAgICAgICBIVFRQIFNlcnZlciBj b25uZWN0ZWQgZGlyZWN0bHkgdG8gdGhlIENvcmUgTmV0d29yayAxIEdicHMNCj4gICAgICAgICAg ICAgIENvcmUgbmV0d29yayBzd2l0Y2hpbmcoRVBDKSAxMCBHYnBzDQo+ICAgICAgICAgICAgICBC YWNraGF1bCB0cmFuc3BvcnQgIDEgR2JwcywgZnVsbCBJUCwgTVBMUw0KPiAgICAgICAgICAgICAg TGFzdCBtaWxlIHRyYW5zcG9ydCBmdWxsIElQLCBNUExTLCAgMzAwIE1icHMNCj4gDQo+ICAgICAg ICAgICAgIFNlcnZlcjogTGludXggazQwc3J2IDQuMS4wLTA0MDEwMC1nZW5lcmljICMyMDE1MDcw MzA5NDAgDQo+IFNNUCBGcmkgSnVsIDMNCj4gMDk6NDE6NDcgVVRDIDIwMTUgeDg2XzY0IHg4Nl82 NCB4ODZfNjQgR05VL0xpbnV4DQo+ICAgICAgICAgICAgICAgICAgICAgICAgIEFwYWNoZS8yLjQu MTAgKFVidW50dSksIGJ1aWx0OiAgIEp1bCAyNCAyMDE1IDE3OjI1OjE4DQo+IA0KPiAgICAgICAg ICAgIEh5c3RhcnRfZGV0ZWN0OiAyIChIWVNUQVJUX0RFTEFZIG9ubHkpLCBhcyBzdWdnZXN0ZWQu DQo+ICAgICAgICAgICAgbmV0LmlwdjQudGNwX25vX21ldHJpY3Nfc2F2ZSA9IDEgdG8gYXZvaWQg dGNwIG1ldHJpY3MgDQo+IGNhY2hpbmcgaW50ZXJmZXJlbmNlDQo+IA0KPiAJVGVzdHMgdHlwZXM6 DQo+IAkJMS4gRG93bmxvYWQgMyBNQiBmaWxlLCBpbnRlcmNoYW5naW5nIEh5c3RhcnQgT04gYW5k IE9GRiwgaW4gDQo+IGNvbmRpdGlvbnMgb2YgTG93IEJhbmR3aWR0aCAoMjUtMzUgTWJwcywgcG9v ciByYWRpbykgYW5kIEhpZ2ggDQo+IEJhbmR3aWR0aCAoMTAwLTEyMCBNYnBzLCBnb29kIHJhZGlv KS4NCj4gCQkyLiBEb3dubG9hZCAxMCBNQiBmaWxlLCBpbnRlcmNoYW5naW5nIEh5c3RhcnQgT04g YW5kIE9GRiwgaW4gDQo+IGNvbmRpdGlvbnMgb2YgTG93IEJhbmR3aWR0aCAoMjUtMzUgTWJwcywg cG9vciByYWRpbykgYW5kIEhpZ2ggDQo+IEJhbmR3aWR0aCAoMTAwLTEyMCBNYnBzLCBnb29kIHJh ZGlvKS4NCj4gDQo+ICAJMTAwIERvd25sb2FkIHRlc3RzIHBlciB0eXBlLg0KPiAgICAgICAgICAg ICAgRG93bmxvYWQgZHVyYXRpb24gYW5kIGJhbmR3aWR0aCBtZWFzdXJlZCBhdCB0aGUgY2xpZW50 IA0KPiBzaWRlIGFzIHNob3duIGJ5IGN1cmwgZnJvbSB0aGUgZmlyc3QgcmVjZWl2ZWQgZGF0YSBw YWNrZXQgdGlsbCB0aGUgZW5kIA0KPiBvZiBzZXNzaW9uICh0aGlzIGV4Y2x1ZGUgRE5TLCAzIHdh eSBoYW5kc2hha2UsIGh0dHAgR0VUIGFuZCBpdHMgQUNLKS4NCj4gICAgICAgICAgICAgIEh5c3Rh cnQgZXhpdCBjd25kIGV4dHJhY3RlZCBmcm9tIG5zdGF0ICh0aGFuayB5b3UgRXJpYyB0byANCj4g cG9pbnRpbmcgb3V0IHRvIGl0KS4NCj4gDQo+IE92ZXJhbGwgcmVzdWx0czoNCj4gCURvd25sb2Fk IDNNQiwgTG93IEJhbmR3aWR0aHM6DQo+IAkJSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6 IDEuNDEgcw0KPiAgCQlOb0h5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAxLjM2IHMNCj4g DQo+IAlEb3dubG9hZCAzTUIsIEhpZ2ggQmFuZHdpZHRoczoNCj4gCQlIeXN0YXJ0IGF2ZXJhZ2Ug ZG93bmxvYWQgdGltZTogMC42NSBzDQo+ICAJCU5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRp bWU6IDAuMzcgIHMNCj4gDQo+IAlEb3dubG9hZCAxME1CLCBMb3cgQmFuZHdpZHRoczoNCj4gCQlI eXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMi40MCBzDQo+ICAJCU5vSHlzdGFydCBhdmVy YWdlIGRvd25sb2FkIHRpbWU6IDIuMTIgcw0KPiANCj4gCURvd25sb2FkIDEwTUIsIEhpZ2ggQmFu ZHdpZHRoczoNCj4gCQlIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS40MiBzDQo+ICAJ CU5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDAuODkgIHMNCj4gDQo+IFRoZXNlIHJl c3VsdHMgc2hvdyB0aGF0IEh5c3RhcnQgaGFzIG5vIHNpZ25pZmljYW50IGltcGFjdCBpbiANCj4g Y29uZGl0aW9ucyBvZiBsb3cgYXZhaWxhYmxlIGJhbmR3aWR0aCwgYnV0IGF0IGhpZ2ggYXZhaWxh YmxlIHRoZSANCj4gZGVjcmVhc2Ugb2YgcGVyZm9ybWFuY2UgY2FuIHJlYWNoIDYwLTcwICUuDQo+ IFdpdGggdGhlIGRldmVsb3BtZW50IG9mIHRoZSBMVEUtQSB3aGljaCB1c2VzIGxhcmdlciBzcGVj dHJ1bSB0byByZWFjaCANCj4gMzAwIE1icHMgYW5kIG1vcmUgdGhlIGltcGFjdCBvZiBIeXN0YXJ0 IHdpbGwgYmVjb21lIGV2ZW4gbW9yZSB2aXNpYmxlLg0KPiANCj4gSSBkbyBzaGFyZSB3aXRoIEVy aWMgYW5kIE5lYWwgc29tZSBmaWxlcyBkZXNjcmliZWQgYmVsb3cgYW5kIGFsc28gY2FuIA0KPiBw cm92aWRlIGxpbmtzIHRvIGFueW9uZSBpbnRlcmVzdGVkLg0KPiANCj4gU3VtbWFyeSBvZiB0aGVz ZSB0ZXN0cyByZXN1bHRzIGFyZSBnYXRoZXJlZCBpbiB0aGUgZm9sbG93aW5nIEV4Y2VsIA0KPiBm aWxlcyAobmFtZXMsIEkgYmVsaWV2ZSwgYXJlIHNlbGYgZXhwbGFuYXRvcnkpOiBUZXN0XzNNX2xv d19CVy54bHN4LCANCj4gVGVzdF8zTV9oaWdoX0JXLnhsc3gsIFRlc3RfMTBNX2xvd19CVy54bHN4 LCBUZXN0XzEwTV9oaWdoX0JXLnhsc3guDQo+IFNpbWlsYXJseSB0Y3BkdW1wIHRyYWNlcyBhcmUg Y2FsbGVkIFRlc3RfM01fbG93X0JXLnBjYXAsIA0KPiBUZXN0XzNNX2hpZ2hfQlcuIHBjYXAsIFRl c3RfMTBNX2xvd19CVy4gcGNhcCwgVGVzdF8xME1faGlnaF9CVy4NCj4gcGNhcCAoYWJsZSB0byBj b2xsZWN0IG9ubHkgc2VydmVyIHNpZGUgdHJhY2VzLCBidXQgdGhleSBhcmUgc2VsZiBzdWZmaWNp ZW50KS4NCj4gRm9yIHRoZSAxME0gZG93bmxvYWRzIHRoZXJlIGFyZSBmZXcgbWlzc2luZyB0cmFj ZXMgYXMgdGhlcmUgdHJhY2UgDQo+IGZpbGVzIHdlcmUgcmVjeWNsZWQuIEluIHRyYWNlcyBhcmUg YWxzbyBzc2ggc2Vzc2lvbnMsIHBsZWFzZSwgaWdub3JlIA0KPiB0aGVtLCB0aGV5IHdlcmUgdXNl ZCB0byBwdXQgaHlzdGFydCBPTiBhbmQgT0ZGIG9uIHNlcnZlciBhbmQgZXh0cmFjdCANCj4gbnN0 YXQgcmVzdWx0cyBiZWZvcmUgYW5kIGFmdGVyIGVhY2ggZG93bmxvYWQuDQo+IA0KPiBFeGNlbCBm aWxlcyBoYXZlIDIgdGFiczogY3VybF9zdW0gYW5kIFN1bW1hcnkuDQo+IEluIHRoZSBjdXJsX3N1 bSB0YWIgdGhlIHN0YXQgaXMgY3VtdWxhdGVkIGluIGNocm9ub2xvZ2ljYWwgb3JkZXIgb2YgdGVz dHMuDQo+IEluIHRoZSBTdW1tYXJ5X3RhYiAgdGhlIEh5c3RhcnQgYW5kIE5vSHlzdGFydCB0ZXN0 cyBhcmUgc2hvd24gc2lkZSBieSANCj4gc2lkZSAoYXMgc2FpZCwgRG93bmxvYWQgMSB3YXMgd2l0 aCBIeXN0YXJ0LCBEb3dubG9hZCAyIHdpdGhvdXQsIA0KPiBEb3dubG9hZCAzIHdpdGggSHlzdGFy dCwgRG93bmxvYWQgNCB3aXRob3V0LCBldGMuKS4gSXQgYWxzbyBpbmNsdWRlIA0KPiBncmFwaHMg b2YgVHJhbnNmZXIgdGltZSBhbmQgYSBncmFwaCB3aGljaCBzaG93cyB0aGUgZGlzdHJpYnV0aW9u IGluICUgDQo+IG9mIGRpZmZlcmVudCBIeXN0YXJ0IGV4aXQgd2luZG93IERlbGF5Q1dORC4NCj4g DQo+IEZpZWxkczoNCj4gY29tbWVudHMJLSB0ZXN0IGNhc2UNCj4gVHJhbnNmX1RpbWUJLSBjYWxj dWxhdGVkIGZyb20gY3VybCBvdXRwdXQgdGhlIGR1cmF0aW9uIGJldHdlZW4gZmlyc3QgZGF0YQ0K PiBwYWNrZXQgYW5kIGVuZCBvZiB0cmFuc2Zlcg0KPiBUcmFuc2Zfc3BlZWQJLSBjYWxjdWxhdGVk IGZyb20gY3VybCBvdXRwdXQgdGhlIHJhdGlvIG9mIGZpbGUgc2l6ZSBhbmQgZHVyYXRpb24NCj4g YmV0d2VlbiBmaXJzdCBkYXRhIHBhY2tldCBhbmQgZW5kIG9mIHRyYW5zZmVyDQo+IEh5c3RhcnQJ RF9EZWxheSAtICBoeXN0YXJ0IGV4aXQgb2NjdXJyZW5jZSAoZGlmZmVyZW5jZSBiZXR3ZWVuDQo+ IFRjcEV4dFRDUEh5c3RhcnREZWxheURldGVjdCBiZWZvcmUgYW5kIGFmdGVyIGRvd25sb2FkKQ0K PiBEX0RlbGF5Q3duZAkgIC0gSHlzdGFydCBleGl0IGN3bmQgZm9yIHRoaXMgZG93bmxvYWQNCj4g KGRpZmZlcmVuY2Ugb2YgVGNwRXh0VENQSHlzdGFydERlbGF5Q3duZCBiZWZvcmUgYW5kIGFmdGVy IHRoZSANCj4gZG93bmxvYWQpIERhdGVfT25fU2VydmVyIC0gdGltZSBqdXN0IGJlZm9yZSB0aGUg ZG93bmxvYWQgc3RhcnQsIHRvIA0KPiBoZWxwIG5hdmlnYXRlIHRoZSBwY2FwDQo+IA0KPiBBZGRp dGlvbmFsbHksIGluIHRoZSBmaWxlIFRlc3RfMTBNX2xvd19CVy54bHN4IGluIHRoZSB0YWIgY3Vy bF9zdW0gYXJlIA0KPiBpbmNsdWRlZCBmb3IgZWFjaCBkb3dubG9hZCB0Y3Auc3RyZWFtIGV4dHJh Y3QgZnJvbSBwY2FwIG9mIHRoZSBmaXJzdCANCj4gMTAwIHRjcC5hbmFseXNpcy5hY2tfcnR0IHdo aWNoIG1heSBoZWxwIGluIHVuZGVyc3RhbmRpbmcgb2YgdGhlIA0KPiBoeXN0YXJ0IGJlaGF2aW9y IChmaWVsZHMgYWNrX3J0dF8xIHRvIGFja19ydHRfMTAwKS4NCj4gSW4gdGhpcyBmaWxlIHRoZXJl IGFyZSBhbHNvIGZpZWxkcyBVUklfZnJhbWUsIFVSSV90aW1lLCBVUklfdGltZV9yZWxhdGl2ZSwN Cj4gCXRjcF9zdHJlYW0uDQo+IFRoZW4gdGhlcmUgYXJlIGNhbGN1bGF0ZWQgZmllbGRzOiBtaW5S VFQgYXMgYSBtaW5pbXVtIG9mIGFsbCAxMDAgQUNLLCByMV81LA0KPiByMl84LCByM184LAlyNF84 IHdoaWNoIHNpbXVsYXRlIEh5c3RhcnQgY2FsY3VsYXRpb25zIG9mIHRoZSByb3VuZCBjdXJyX3J0 dA0KPiBhbmQgcjEgbWluLCByMiBtaW4sIHIzIG1pbiwgcjQgbWluIHJlcHJlc2VudCBtaW5SVFQg b2YgdGhlIHdob2xlIHJvdW5kLg0KPiBQbGVhc2UsIGNvcnJlY3QgbWUgaWYgSSBhbSB3cm9uZywg YnV0IGluIG15IHVuZGVyc3RhbmRpbmcgZm9yIGVhY2ggQUNLIA0KPiB0aGUgaHlzdGFydF91cGRh dGUgaXMgY2FsbGVkIGJlZm9yZSB0aGUgaHlzdGFydF9yZXNldCwgYW5kIGFzIGEgcmVzdWx0IA0K PiB0aGUgY29tcHV0YXRpb24gb2YgY3VyX3J0dCBvZiB0aGUgOCBwYWNrZXRzIG9mIHRoZSByb3Vu ZCBzdGFydHMgd2l0aCANCj4gdGhlIDItbmQgcGFja2V0IG9mIHRoZSByb3VuZCwgbm90IHRoZSBm aXJzdCBvbmUuDQo+ICBJJ3ZlIHVzZSByMV81IGFzIGEgbWluKGFja19ydHRfNy4uLmFja19ydHRf MTEpLCBkdWUgdG8gDQo+IGh5c3RhcnRfbG93X3dpbmRvdz0xNiwgcjJfOCBhcyBhIG1pbihhY2tf cnR0XzEyIC4uLiBhY2tfcnR0XzE5KSwgcjNfOCANCj4gYXMgYQ0KPiBtaW4oYWNrX3J0dF8zMiAu Li4gYWNrX3J0dF8zOSkgYW5kICksIHI0XzggYXMgYSBtaW4oYWNrX3J0dF83MiAuLi4gYWNrX3J0 dF83OSkuDQo+IFRvIHVuZGVyc3RhbmQgdGhlIGltcGFjdCBvZiB0aGlzIHRoZXJlIGFyZSBhbHNv IHIxIEhPTCwgcjIgSE9MLCByMyANCj4gSE9MLCByNCBIT0wgKEhlYWQgT2YgTGluZSkgd2hpY2gg Y29tcHV0ZSB0aGUgc2FtZSB3aXRoIGZpcnN0IHBhY2tldCBpbiANCj4gdGhlIHJvdW5kIGluY2x1 ZGVkIGluIHRoZSByZWxldmFudCByb3VuZDogIHIxXzUgYXMgYSANCj4gbWluKGFja19ydHRfNi4u LmFja19ydHRfMTApLCByMl84IGFzIGEgbWluKGFja19ydHRfMTEgLi4uIGFja19ydHRfMTgpLCAN Cj4gcjNfOCBhcyBhIG1pbihhY2tfcnR0XzMxIC4uLiBhY2tfcnR0XzM4KSBhbmQgKSwgcjRfOCBh cyBhIG1pbihhY2tfcnR0XzcxIC4uLiBhY2tfcnR0Xzc4KS4NCj4gDQo+IElmIG15IHVuZGVyc3Rh bmRpbmcgb2YgSHlzdGFydCBpcyBjb3JyZWN0IGFuZCwgYXNzdW1pbmcgaW4gZWFjaCB0cmFpbiAN Cj4gdGhlIGZpcnN0IHBhY2tldCBoYXMgdGhlIGxvd2VzdCBSVFQgd2hpY2gsIG1vc3QgcHJvYmFi bHkgaXMgdmFsaWQgaW4gDQo+IGFsbCB0eXBlIG9mIG5ldHdvcmtzLCB0aGVuIHRoZSBleGl0IGZy b20gaHlzdGFydCBpbiB0aGlzIA0KPiBpbXBsZW1lbnRhdGlvbiBtYXkgaGFwcGVuIGF0IGN3bmQg MjksIDQ5LCA4OSwgMTY5LCBldGMuIChnaXZlbiB0aGUgSVcgb2YgMTApLg0KPiBUaGVzZSB0ZXN0 cyBzaG93IHRoYXQgdGhpcyBpcyB0aGUgY2FzZSwgaG93ZXZlciB0aGVyZSBhbHNvIA0KPiBpbnRl cm1lZGlhdGUgdmFsdWVzLiBIb3dldmVyIHRoZSBjd25kIG9mIDI5IHNoYWxsIGJlIHRoZSBsb3dl c3QgDQo+IHBvc3NpYmxlIHZhbHVlIGFuZCB0aGlzIGlzIHRoZSBjYXNlIGluIHRoZXNlIHRlc3Rz LiBVbmZvcnR1bmF0ZWx5LCBpbiANCj4gTFRFIHRoZSBmaXJzdCBwYWNrZXQgaW4gdGhlIHRyYWlu IGhhcyBjaGFuY2VzIHRvIGFsd2F5cyBoYXZlIH42IG1zIA0KPiBsb3dlciB0aGFuIHRoZSBmb2xs b3dpbmcgb25lcyBhbmQsIGJlY2F1c2UgaXQgaXMgbm90IGNvdW50ZWQgaW4gdGhlIA0KPiAyLW5k IHRyYWluLCB0aGVyZSBpcyBxdWl0ZSBoaWdoIHBlcmNlbnRhZ2Ugb2YgZXhpdCBhdCAyOSAoMTQt MzYlIGluIA0KPiB0aGVzZSB0ZXN0cyksIGV2ZW4gdGhvdWdoIGxhdGVyIG9uIGN1YmljIGluY3Jl YXNlcyB0aGUgY3duZCB3aXRob3V0IA0KPiBsb3NzZXMgdXAgdG8gbWFueSBodW5kcmVkcy4gRm9y IG1lIHRoaXMgaXMgdG9vIGVhcmx5IGV4aXQgZnJvbSBzbG93IHN0YXJ0Lg0KPiBBcyBpdCBjYW4g YmUgc2VlbiBmcm9tIGNvbXBhcmlzb24gb2YgdGhlIHNpbXVsYXRlZCBjYWxjdWxhdGlvbiwgDQo+ IGluY2x1c2lvbiBvZiB0aGUgZmlyc3QgcGFja2V0IGluIHRoZSByb3VuZCBjdXJyX3J0dCBjYWxj dWxhdGlvbiB3b3VsZCANCj4gZWxpbWluYXRlIHRoZSBjd25kIDI5IGFuZCBzaGlmdCB0byBjd25k IDQ5IG9yIDg5IG9yIGhpZ2hlci4NCj4gDQo+IEJ1dCB0aGVyZSBhcmUgYWxzbyBpbnRlcm1lZGlh dGUgdmFsdWVzLCBlLmcuIDY5LiBMb29raW5nIHRocm91Z2ggDQo+IHRyYWNlcyBvbmUgbWF5IHNl ZSB0aGF0IHRoaXMgaXMgZHVlIHRvICJjb21wcmVzc2VkIEFDSyIgd2hpY2ggcmVjZWl2ZXMgDQo+ IGEgdGlnaHRseSBwYWNrZWQgdHJhaW4gb2YgbWFueSBBQ0sgc28gdGhhdCBpbiB0cmFjZXMgdGhl cmUgYXJlIG5vdCANCj4gZGF0YSBwYWNrZXRzIHNlbnQgaW4gdHVybiBieSBzZXJ2ZXIuIExvb2tz IHRvIG1lLCBiZWNhdXNlIA0KPiBoeXNyYXJ0X3Jlc2V0IHNldHMgdGhlIGVuZF9zZXEgdG8gc25k X25leHQgYW5kIHRoZSBzZXJ2ZXIgZGlkIG5vdCANCj4gbWFuYWdlZCB0byBzZW5kIHlldCBwYWNr ZXRzIGluIHJlcGx5IHRvIHRoaXMgaGlnaCBzcGVlZCBBQ0sgdHJhaW4gdGhlIA0KPiBzbmRfbmV4 dCBwb2ludHMgdG8gbXVjaCBsb3dlciB2YWx1ZSB0aGFuIHdvdWxkIG5vcm1hbGx5IGhhcHBlbiBh bmQgDQo+IHRoaXMgYWN0dWFsbHkgZGVzdHJveXMgdGhlIGNvcnJlY3QgaWRlbnRpZmljYXRpb24g b2YgdGhlIGJvcmRlcnMgb2YgDQo+IHRoZSByb3VuZCwgdGhlIHJvdW5kIGZhaWxzIHNvbWV3aGVy ZSBpbiB0aGUgbWlkZGxlIG9mIHRoZSB0cmFpbiBub3QgYXQgDQo+IGl0cyBib3JkZXIsIHRoaXMg bWFrZXMgdmVyeSBsaWtlbHkgaW5jcmVhc2VkIGRlbGF5IGR1ZSB0byBxdWV1ZWluZyBhbmQgDQo+ IGVhcmxpZXIgZXhpdCBmcm9tIHNsb3dfc3RhcnQgYW5kIGludGVybWVkaWF0ZSB2YWx1ZXMgb2Yg ZXhpdCBjd25kIHdoZXJlIHRoZXkgYXJlIG5vdCBleHBlY3RlZC4NCj4gDQo+IEFuZCBsYXN0IGJ1 dCBub3QgbGVhc3Q6IHRoZXJlIGFyZSBnYXBzICg0LTcgbXMpIGluIEFDSyB0cmFpbnMuIE9uIHRo ZSANCj4gcGVyZmVjdGx5IHNlbnQgSVcgb2YgMTAgZGF0YSBwYWNrZXRzIHRyYWluIHRoZSB0eXBp Y2FsIGZvciBMVEUgd2lsbCBiZSANCj4gdG8gcmVjZWl2ZSAxIG9yIDIgQUNLLCB0aGVuIGEgZ2Fw IG9mIDQtNiBtcyB0aGVuIGFub3RoZXIgY291cGxlIG9mIA0KPiBBQ0ssIHRoZW4gYW5vdGhlciBn YXAsIHRoZW4gNSBBQ0sgY29tcHJlc3NlZCB0b2dldGhlciBpbiBhIGZyYWN0aW9uIG9mIA0KPiBt aWNyb3NlY29uZC4gT2YgY2F1c2UsIHRoZSBuZXh0IHRyYWluIGZyb20gdGhlIHNlcnZlciB3aWxs IGNvbnRhaW4gdGhlIA0KPiBzYW1lIGdhcHMuIFZlcnkgcXVpY2tseSwgZS5nLiwgdGhlIEhlYWQg b2YgTGluZSAoSE9MKSBvZiB0aGUgMy1kIHRyYWluIA0KPiB3aWxsIGZvbGxvdyBpbW1lZGlhdGVs eSB0aGUgdGFpbCBvZiB0aGUgMi1uZCB0cmFpbiB3aGljaCB3aWxsIGxlYWQgdG8gDQo+IHF1ZXVl aW5nIGluIHRoZSBlbmQgcm91dGVyIChhY3R1YWxseSB0aGUgcmFkaW8gYmFzZSBzdGF0aW9uLCBt dWNoIA0KPiBlYXJsaWVyIHRoYW4gQkRQIHJlYWNoZWQpLCBidXQgdGhpcyB3aWxsIGxlYWQgdG8g aW5jcmVhc2Ugb2YgdGhlIHRyYWluIA0KPiBSVFQgYW5kIHRvbyB5ZWFybHkgZXhpdCBmcm9tIHNs b3cgc3RhcnQuIFdoeSB0b28gZWFybHkgZXhpdCA/IEJlY2F1c2UsIA0KPiBzaG91bGQgc2VydmVy IGNvbnRpbnVlIGluY3JlYXNpbmcgdGhlIHNwZWVkIHRoaXMgd291bGQgcmVkdWNlIHF1aWNrZXIg DQo+IHRoZSBnYXBzIGJvdGggaW4gZG93bmxpbmsgYW5kIHVwbGluaywgdGhlIHByZWljZSBmb3Ig dGhpcyBiZWluZyB0aGUgDQo+IGluY3JlYXNlIG9mIHRoZSBSVFQgKEkgZ3Vlc3MgZG91YmxlKSBh Z2FpbnN0IHdoYXQgb25lIHdvdWxkIG5vcm1hbGx5IHNlZSBpbiB0aGUgZW5kLXRvLWVuZCBFdGhl cm5ldCBuZXR3b3Jrcy4NCj4gDQo+IA0KPiBJIGRvIGxpa2UgaWRlYXMgb2YgSHlzdGFydCwgYnV0 IEkgIGRvbid0IGtub3cgaG93IGNvdWxkIGl0IGJlIHBvc3NpYmx5IA0KPiByZWNvbmNpbGVkIHdp dGggbXkgTFRFIHByb2JsZW1zLiAgV2l0aCB0aGUgaW5jcmVhc2Ugb2YgTFRFIHNwZWVkcyB0aGlz IA0KPiBlYXJseSBleGl0IGZyb20gc2xvdyBzdGFydCB3aWxsIGJlY29tZSBldmVuIG1vcmUgb2Yg dGhlIHByb2JsZW0uDQo+IEZvciBMVEUsIG5vdCBzdXJlIGhvdyBnb29kIGFuZCBnZW5lcmFsbHkg YWNjZXB0YWJsZSwgc29sdXRpb24gd291bGQgDQo+IGJlLCBwb3NzaWJseSwgdG8gaW5jcmVhc2Ug dGhlIEhZU1RBUlRfREVMQVlfTUlOIHRvIDggbXMsIHdoaWNoLCBhdCANCj4gbGVhc3QgaW4gbXkg Y2FzZSwgd291bGQgZGltaW5pc2ggdGhlIGVhcmxpZXIgZXhpdCBzaWduaWZpY2FudGx5LCBpZiBJ IA0KPiBiZWxpZXZlIGluIHRoZXNlIHRlc3RzIHJlc3VsdHMuDQo+IEFub3RoZXIgc29sdXRpb24s IHdoaWNoIEkgZG9uJ3QgbGlrZSB2ZXJ5IG11Y2gsIGJ1dCBhcyBJIGtub3cgc29tZSANCj4gb3Bl cmF0b3JzIHVzZSwgd291bGQgYmUgdG8gaW5zZXJ0IGEga2luZCBvZiBUQ1AgYWNjZWxlcmF0b3Ig YmV0d2VlbiANCj4gdGhlIEludGVybmV0IGFuZCB0aGUgbW9iaWxlIG5ldHdvcmsgd2hpY2ggd2ls bCBpbnRlcmNlcHQgYWxsIFRDUCBhbmQgDQo+IHdvdWxkIHNlbmQgaXQgdG8gbW9iaWxlIHdpdGhv dXQgSHlzdGFydCBidXQgdGhpcyB3b3VsZCBraWxsIGFueSANCj4gZW5kLXRvLWVuZCBwcmluY2lw bGUgYW5kIHRoaXMgd2lsbCBub3QgYmUgdGhlIEludGVybmV0IGFueW1vcmUuDQo+IA0KPiANCj4g SWYgeW91IGhhdmUgdGltZSBhbmQgbmV2ZXIgbG9vayBjbG9zZWx5IHRvIExURSB0ZWNobm9sb2d5 IEkgZG8gYSBxdWljayANCj4gc3VtbWFyeSBiZWxsb3cuDQo+IA0KPiANCj4gTFRFLCBUQ1Agc2Vs Zi1jbG9ja2luZyBhbmQgQUNLIGNvbXByZXNzaW9uLg0KPiANCj4gU3BlbnQgc29tZSB0aW1lIHRv IHVuZGVyc3RhbmQgdGhlIGJhc2ljcyBvZiB3aGF0IGNvdWxkIGJlIHRoZSByZWFzb24gDQo+IGZv ciBhbGwgdGhvc2UgdGltaW5nIGVmZmVjdHMgZHVlIHRvIExURS4NCj4gCUZpcnN0LCB1bmxpa2Ug d2lyZSBiYXNlZCB0cmFuc3BvcnQgdGVjaG5vbG9naWVzLCBMVEUgKGJ1dCBhbHNvIFVNVFMgDQo+ IGFuZCBmdXR1cmUgNUcpIHNlbmRzIGRhdGEsIGJvdGggdXBsaW5rIGFuZCBkb3dubGluayBpbiBw cmVkZWZpbmVkIA0KPiB0aW1lc2xvdHMgY2FsbGVkIFRyYW5zbWlzc2lvbiBUaW1lIEludGVydmFs IChUVEkpLiBJbiBVTVRTIHRoaXMgaXMgDQo+IDJtcywgaW4gTFRFIGlzDQo+IDEgbXMgYW5kIGl0 IHNheXMgdGhhdCBpbiA1RyBpdCB3aWxsIGJlIDAuNSBtcy4NCj4gVGhlc2UgVFRJIGhhdmUgZXhh Y3QgYm9yZGVycyBhbmQgdGlnaHRseSBmb2xsb3cgZWFjaCBvdGhlciBhbmQgcmVxdWlyZSANCj4g ZXhhY3Qgc3luY2hyb25pemF0aW9uIGJldHdlZW4gdGhlIG1vYmlsZSBkZXZpY2UgYW5kIFJhZGlv IEJhc2UgU3RhdGlvbiANCj4gKGNhbGxlZCBlTm9kZUIgaW4gTFRFKS4gVG8gY29wZSB3aXRoIHZh cnlpbmcgcmFkaW8gY29uZGl0aW9ucyB0aGUgDQo+IHNlbmRpbmcgZGF0YSBiYW5kd2lkdGggaXMg bW9kaWZpZWQgaW4gc3VjaCBhIHdheSB0aGF0IHRvIGtlZXAgYSANCj4gY29uc3RhbnQgcHJvYmFi aWxpdHkgb2YgZGF0YSBsb3NzIG9mIDEwJS4gQmFkIHJhZGlvIC0gbGVzcyBiYW5kd2lkdGgsIGdv b2QgcmFkaW8gLSBtb3JlIGJhbmR3aWR0aC4NCj4gVGhlIGJhbmR3aWR0aCBpcyBtb2RpZmllZCBi eSBwYWNraW5nIGxlc3Mgb3IgbW9yZSBiaXRzIGluIGEgVFRJIG9mIDEgbXMgaW4gTFRFLg0KPiBC dXQgVFRJIHJlbWFpbiB1bmNoYW5nZWQgb2YgZXhhY3RseSAxIG1zLiBUaGF0IG1lYW5zIHRoYXQs IGF0IHRoZSANCj4gVENQL0lQIGxheWVyIHRoZSBwcmVjaXNpb24gb2YgdGhlIHRpbWluZyBpbmZv cm1hdGlvbiBmcm9tIGRldmljZSB0byANCj4gdGhlIG5ldHdvcmsgaXMgMSBtcy4gSW4gdGhlIGNh c2Ugb2YgYSBiYW5kd2lkdGggb2YgMSBNYnBzIHRoZXJlIHdpbGwgDQo+IGJlIDgzIElQIHBhY2tl dHMgcGVyIHNlY29uZCBvciBvbmUgcGFja2V0IGV2ZXJ5IDEyIG1zLiBJbiB0aGlzIGNhc2UsIA0K PiBwb3NzaWJseSB0aGUgcmV0dXJuIG9mIG9uZSBBQ0sgZXZlcnkgMTIgbXMgKGV2ZXJ5IDEyIFRU SSkgd291bGQgYmUgDQo+IG1vcmUgb3IgbGVzcyBzdWZmaWNpZW50IHRvIGtlZXAgYXQgYSBnb29k IGxldmVsIHRoZSBUQ1Agc2VsZi1jbG9ja2luZyANCj4gbWVjaGFuaXNtLiBCdXQsIHdoZW4gdGFs a2luZyBhYm91dCAxMDAgTWJwcyAob3IgMzAwIE1icHMpIHRoYXQgbWVhbnMgDQo+IG9uZSBwYWNr ZXQgKGFuZCBvbmUgQUNLKSBldmVyeSAwLjEyIG1zIGFuZCB0aGVyZSBpcyBubyB3YXkgdG8gcHJv dmlkZSANCj4gaXQgd2l0aCBhIFRUSSBvZiAxIG1zLiBUaGlzIGlzIHRoZSByZWFzb24gZm9yIEFD SyBjb21wcmVzc2lvbiBpbiBMVEUgDQo+IGFuZCB0aGUgZnV0dXJlIDVHIHdoaWNoIHByb21pc2Vz IG1vcmUgdGhhbiAxR2JwcyB3aXRoIGl0cyAwLjUgbXMgVFRJIHdpbGwgYmUgaW4gYSBiaWcgY29u ZmxpY3Qgd2l0aCBUQ1Agc2VsZi1jbG9ja2luZy4NCj4gCUJ1dCB0aGUgcHJvYmxlbSBkbyBub3Qg c3RvcHMgaGVyZS4gVGhlcmUgaXMgYSBzaWduaWZpY2FudGx5IGRpZmZlcmVudCANCj4gbWVjaGFu aXNtcyBvZiB0cmFuc21pc3Npb24gaW4gdGhlIGRvd25saW5rIChmcm9tIGVOb2RlQiB0byBtb2Jp bGUgDQo+IGRldmljZSkgYW5kIGluIHVwbGluayAoZnJvbSBtb2JpbGUgZGV2aWNlIHRvIHRoZSBl Tm9kZUIpLiBUaGUgDQo+IGRpZmZlcmVuY2UgY29tZSBmcm9tIHRoZSBmYWN0IHRoYXQgdGhlIHNj aGVkdWxlciBmb3IgYm90aCBkb3dubGluayBhbmQgDQo+IGluIHVwbGluayBpcyBsb2NhdGVkIGlu IHRoZSBlTm9kZUIuIEFzIG9mIG15IHVuZGVyc3RhbmRpbmcgdGhlIHB1cnBvc2UgDQo+IG9mIHRo aXMgaXMgdG8gc2ltcGxpZnkgdG8gdGhlIG1heGltdW0gdGhlIGRldmljZSBhbmQgc2F2ZSBpdHMg YmF0dGVyeS4gDQo+IEFzIGEgcmVzdWx0IG9mIHRoaXMgZGVzaWduLCB0aGUgZU5vZGVCIGNhbiBz ZW5kIGluIGVhY2ggVFRJIHRvd2FyZCB0aGUgDQo+IG1vYmlsZSBkZXZpY2Ugd2hlbmV2ZXIgZGF0 YSBleGlzdCBpbiBpdHMgYnVmZmVyIChlTm9kZUIgaXMgYSBraW5kIG9mIA0KPiBJUCB3aXJlbGVz cyByb3V0ZXIpLiBCdXQgZm9yIG1vYmlsZSBkZXZpY2UsIGluIG9yZGVyIHRvIHNlbmQgdGhlIGRh dGEgDQo+IGluIHVwbGluaywgYWZ0ZXIgYSBwYXVzZSwgaXQgaGFzIGZpcnN0IHRvIGFzayB0aGUg bmV0d29yayB0byBzY2hlZHVsZSANCj4gaXQuIEFuZCBtb2JpbGUgaGFzIGZvciB0aGlzIHJlcXVl c3QgKGFmdGVyIGEgcGF1c2Ugb2Ygc2VuZGluZykganVzdCANCj4gb25lIGJpdCwgYWN0dWFsbHkg aXMgbm90IGV2ZW4gYSBkYXRhIGJpdCwgaXMgYSBzcGVjaWFsIHJhZGlvIHNpZ25hbCANCj4gd2hp Y2ggdGVsbHMgIkhleSwgSSBoYXZlIHNvbWUgZGF0YSBpbiB0aGUgYnVmZmVyIHRvIHNlbmQiLiBM ZXRzIGxvb2sgDQo+IGF0IGluaXRpYWwgdHJhaW4gb2YgSVcgb2YgMTAgcGFja2V0cy4gSW4gZ29v ZCByYWRpbyBjb25kaXRpb25zIHRoZSANCj4gZU5vZGVCIHdpbGwgZml0IGFsbCAxMCBwYWNrZXRz IGluIDEgVFRJIG9mIDEgbXMgYW5kIHNlbmQgdGhlbSBhbGwgdG8gDQo+IHRoZSBkZXZpY2UuIERl dmljZSB3aWxsIHVucGFjayB0aGVtIGFuZCBnZW5lcmF0ZSAxMCBBQ0sgYW5kIHRoZW4gdGVsbCAN Cj4gdG8gdGhlIGVOb2RlQiAiSGV5LCBJIGhhdmUgc29tZSBkYXRhIGluIHRoZSBidWZmZXIgdG8g c2VuZCIuIFRoZSANCj4gZU5vZGVCIGhhcyBub3Qga25vd2xlZGdlIGhvdyBtdWNoIGRhdGEgdGhl IG1vYmlsZSBoYXMgdG8gc2VuZCwgDQo+IHRoZXJlZm9yIHdpbGwgYWxsb2NhdGUgc29tZSBtaW5p bXVtDQo+IGNhcGFjaXR5OiAiV2VsbCwgSSBnaXZlIHlvdSBhIGdyYW50IHRvIHNlbmQgdXAgdG8g MjAwIEJ5dGVzIGFuZCB5b3UgDQo+IG11c3Qgc2VuZCB0aGVtIGV4YWN0bHkgYWZ0ZXIgNCBtcyB5 b3UgZ2V0IHRoaXMgbm90aWZpY2F0aW9uIi4gVGhlc2UgNCANCj4gbXMgYXJlIGFsbG93YW5jZSBm b3IgbW9iaWxlIGRldmljZSB0byBwcmVwYXJlIGRhdGEgZm9yIHNlbmRpbmcgd2hpY2ggDQo+IGlz IGEgdmVyeSBjb21wbGljYXRlZCBwcm9jZXNzLCB0aGVzZSA0IG1zIGlzIGEgbXVzdCBkZWxheSBi ZXR3ZWVuIHRoZSBncmFudCBhbmQgdGhlIHRyYW5zbWlzc2lvbi4NCj4gSWYgbW9iaWxlIGhhcyBp biBpdHMgYnVmZmVyIDEwIEFDSyB0byBzZW5kIGl0IHdpbGwgc2VuZCBvbmx5IDIgQUNLIGFuZCAN Cj4gd2lsbCBjb21wbGVtZW50IHRoZW0gd2l0aCwgc28gY2FsbGVkLCBidWZmZXIgc3RhdHVzIHJl cG9ydDogIkkgZG8gaGF2ZSANCj4gc3RpbGwgNDgwIGJ5dGVzIHRvIHNlbmQiLiBOb3cgbmV0d29y ayBoYXMgbW9yZSBpbmZvcm1hdGlvbiBhbmQgDQo+IGFsbG9jYXRlIGFzDQo+IHJlcXVlc3RlZDog IldlbGwsIEkgZ2l2ZSB5b3UgYSBncmFudCB0byBzZW5kIHVwIHRvIDIwMCBCeXRlcyBhbmQgeW91 IA0KPiBtdXN0IHNlbmQgdGhlbSBleGFjdGx5IGFmdGVyIDQgbXMgeW91IGdldCB0aGlzIG5vdGlm aWNhdGlvbiIuIEJ1dCwgaW4gDQo+IHRoZSBtZWFuIHRpbWUgc29tZSBvdGhlciBpbmZvcm1hdGlv biBvZiBoaWdoZXIgcHJpb3JpdHkgbWF5IGFwcGVhciBpbiB0aGUgZGV2aWNlIChlLmcuDQo+IHNp Z25hbGluZykgYW5kIHRoZSBkZXZpY2Ugd2lsbCBzZW5kIG9ubHkgNiBvdXQgb2YgOCByZW1haW5p bmcgQUNLIA0KPiB0b2dldGhlciB3aXRoIHNpZ25hbGluZyBhbmQgYW5vdGhlciBCdWZmZXIgU3Rh dHVzIFJlcG9ydDogIkhleSwgSSBkbyANCj4gaGF2ZSBhbm90aGVyIDEyMCBieXRlcyB0byBzZW5k Ii4gQW5kIGFnYWluICJXZWxsLCBJIGdpdmUgeW91IGEgZ3JhbnQgDQo+IHRvIHNlbmQgdXAgdG8g MTIwIEJ5dGVzIGFuZCB5b3UgbXVzdCBzZW5kIHRoZW0gZXhhY3RseSBhZnRlciA0IG1zIHlvdSAN Cj4gZ2V0IHRoaXMgbm90aWZpY2F0aW9uIi4gQXMgYSByZXN1bHQgdGhlIHNlcnZlciB3aWxsIHJl Y2VpdmUgMiBBQ0ssIA0KPiB0aGVuIGFmdGVyIGEgZ2FwIG9mIDYgbXMgYW5vdGhlciA2IEFDSywg dGhlbiwgYWZ0ZXIgYW5vdGhlciA2IG1zLCB0aGUgDQo+IHJlbWFpbmluZyAyIEFDSy4gV2l0aCB0 aGUgc2NoZWR1bGVyIGluIHRoZSBlTm9kZUIgdGhlc2UgZ2FwcyBhcmUgDQo+IGluZXZpdGFibGUu IEFuZCBvbmUgbW9yZSB0aGluazogbW9iaWxlIHBhY2sgdG9nZXRoZXIgdGhvc2UgNiBBQ0sgaW4g MSANCj4gVFRJIG9mIDEgbXMgYW5kIHNlbmQgdGhlbSwgdGhlbiB0aGUgZU5vZGVCIHVucGFjayB0 aGVtIGFuZCB0aGVyZSBpcyBubyANCj4gd2F5IHRvIHJlY292ZXIgdGhlIHNwYWNpbmcgb2YgMC4x MiBtcyBiZXR3ZWVuIHRoZW0uIGVOb2RlQiBzaW1wbHkgc2VuZCANCj4gdGhlbSB0byB0aGUgc2Vy dmVyIGJhY2sgdG8gYmFjayBhdCB0aGUgc3BlZWQgb2YgaXRzIGRhdGEgY2FyZCB3aGljaCBpcyAN Cj4gdHlwaWNhbGx5IDEgR2JwcyBhbmQgd2UgZmFjZSB0aGUgQUNLIGNvbXByZXNzaW9uLiBBZnRl ciB0aGUgbGFzdCAyIEFDSyANCj4gd2VyZSBzZW50IHRoZXJlIGlzIG5vdGhpbmcgZWxzZSB0byBz ZW5kLCBzbyB0aGVyZSBpcyBubyBCdWZmZXIgU3RhdHVzIA0KPiBSZXBvcnQsIHRoZXJlZm9yZSwg YWZ0ZXIgcGFja2V0cyBvZiB0aGUgbmV4dCB0cmFpbiBhcnJpdmUgYW5kIEFDSyBhcmUgDQo+IGdl bmVyYXRlZCBldmVyeXRoaW5nIHN0YXJ0cyBhZ2FpbiB3aXRoIG9uZSBiaXQgb2YgaW5mb3JtYXRp b24gIkhleSwgSSBoYXZlIHNvbWUgZGF0YSBpbiB0aGUgYnVmZmVyIHRvIHNlbmQiLg0KPiBUaGF0 J3Mgd2h5IHRoZXJlIGFyZSBnYXBzIGFuZCBBQ0sgY29tcHJlc3Npb24sIGxvc3MsIG9yIG1vcmUg ZXhhY3RseSwgDQo+IGxhY2sgb2YgYW55IHRpbWluZyBpbmZvcm1hdGlvbiB3aGljaCB3b3VsZCBi ZSB1c2VmdWwgZm9yIHRoZSBUQ1Agc2VsZi1jbG9ja2luZy4NCj4gDQo+IFRoYW5rIHlvdSBhbmQg c29ycnkgZm9yIHRha2luZyB5b3VyIHRpbWUuDQo+IA0KPiANCj4gVmVhY2VzbGF2IFJvbWFuDQo+ IA0KPiANCj4gDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFZlYWNlc2xh diBST01BTg0KPiBTZW50OiBTYXR1cmRheSwgMDUgU2VwdGVtYmVyIDIwMTUgMDE6MTANCj4gVG86 ICdFcmljIER1bWF6ZXQnOyBOZWFsIENhcmR3ZWxsDQo+IENjOiB0Y3BtQGlldGYub3JnOyBQaWVy cyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgSW5nZW1hciBKb2hhbnNzb24gUzsgDQo+IEVyaWMgRHVt YXpldDsgZW5kMmVuZC1pbnRlcmVzdEBwb3N0ZWwub3JnDQo+IFN1YmplY3Q6IFJFOiBbdGNwbV0g W2UyZV0gVENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiANCj4gSWYgd2UgbG9vayBhdCB0 aGUgZmlyc3QgdHJhaW46IGFsbCBwYWNrZXRzIHJlY2VpdmVkIGluIGxlc3MgdGhhbiAxIG1zLiAN Cj4gUHJvYmFibHkgdGhpcyBpcyBvbmx5IGFuIGFwcGVhcmFuY2UgYXMgTFRFIHRyYW5zbWl0cyBp biwgc28gY2FsbGVkIA0KPiB0cmFuc21pc3Npb24gdGltZSBpbnRlcnZhbCAoVFRJKSBvZiAxIG1z LCBhbmQgd2hhdCB3ZSBzZWUgaGVyZSBpcyB0aGF0IA0KPiBhbGwgMTAgcGFja2V0cyBvZiBpbml0 aWFsIHdpbmRvdyBmaXR0ZWQgaW4gMSBtcywgYW5kLCB3aGVuIGRlY29kZWQsIA0KPiB3ZXJlIHBy ZXNlbnRlZCB0byB0aGUgVENQL0lQIGxheWVyIChhbmQgcGNhcCkgYWxsIGF0IG9uY2UuIEJUVywg OCANCj4gcGFja2V0cyBvZiAxMzAyMiBieXRlcyBpbiAxIG1zIG1lYW5zIGluc3RhbnRhbmVvdXMg c3BlZWQgb2YgMTA0IE1icHMsIA0KPiBnb29kIHJhZGlvIGNvbmRpdGlvbnMuIFRDUCBnZW5lcmF0 ZXMgMTAgQUNLIHRyYWluIG9mIHRoZSBkdXJhdGlvbiBvZiANCj4gMS4yIG1zLiBXaWxsIGl0IGJl IGEgRmFzdCBFdGhlcm5ldCBwb3NzaWJseSB3ZSB3b3VsZCBjb25zaWRlciB0aGlzIG5vcm1hbCwg aXNuJ3QgaXQgPw0KPiBJIGxvb2tlZCBhdCBob3cgc2VydmVyIHJlcGx5IHRvIHRoZXNlIDEwIEFD S3MuIFRoZXJlIGFyZSAzIGdhcHMgb2YgNCwgDQo+IDIgYW5kIDUgbXMgaW4gdGhlIHJlcGx5IHRy YWluIGFuZCB0aGUgdG90YWwgdHJhaW4gb2YgMTggKD8sIGl0IHNob3VsZCANCj4gYmUgMjApIHBh Y2tldHMgcmVhY2hlcyBhIGR1cmF0aW9uIG9mIDE0IG1zLiBBRkFJSyBMVEUgbWF5IGludHJvZHVj ZSANCj4gZ2FwcyBpbiBBQ0sgdHJhaW4gZHVlIHRvIHVwbGluayBzY2hlZHVsaW5nIG1lY2hhbmlz bS4NCj4gTWF5IGJlIHRoZXNlIGdhcHMgdHJpZ2dlciBIeXN0YXJ0IGVhcmx5IGV4aXQgPw0KPiAN Cj4gVmVhY2VzbGF2IFJvbWFuDQo+IFRlY2huaWNhbCBhbmQgSVQgZGlyZWN0b3INCj4gT3Jhbmdl IE1vbGRvdmEgUy5BLg0KPiBGaXg6ICszNzMyMjU3NTQwMA0KPiBNb2I6ICszNzM2OTE5ODQwMA0K PiBGYXg6ICszNzMyMjU3NTMwNg0KPiANCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4g RnJvbTogRXJpYyBEdW1hemV0IFttYWlsdG86ZXJpYy5kdW1hemV0QGdtYWlsLmNvbV0NCj4gU2Vu dDogRnJpZGF5LCAwNCBTZXB0ZW1iZXIgMjAxNSAxOTo0OA0KPiBUbzogTmVhbCBDYXJkd2VsbA0K PiBDYzogVmVhY2VzbGF2IFJPTUFOOyB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2Fu Z3RhZSBIYTsgDQo+IEluZ2VtYXIgSm9oYW5zc29uIFM7IEVyaWMgRHVtYXpldDsgZW5kMmVuZC1p bnRlcmVzdEBwb3N0ZWwub3JnDQo+IFN1YmplY3Q6IFJlOiBbdGNwbV0gW2UyZV0gVENQIEh5U3Rh cnQgcGF0Y2ggZGVwbG95bWVudA0KPiANCj4gT24gRnJpLCAyMDE1LTA5LTA0IGF0IDExOjMyIC0w NDAwLCBOZWFsIENhcmR3ZWxsIHdyb3RlOg0KPiA+IEhpIFZlYWNlc2xhdiwNCj4gPg0KPiA+IEkg YWdyZWUgdGhhdCBpbiB5b3VyIExURSB0cmFjZXMgaXQgbG9va3MgbGlrZSBDVUJJQyBIeXN0YXJ0 IGlzIA0KPiA+IGV4aXRpbmcgc2xvdyBzdGFydCB0b28gZWFybHkuDQo+ID4NCj4gPiBTSW5jZSB5 b3Ugc2VlbSB0byBoYXZlIGEgbmljZSBMVEUgdGVzdGJlZCwgd291bGQgeW91IGJlIGFibGUgdG8g ZG8gDQo+ID4gc29tZSBleHBlcmltZW50cyB0byBmaW5kIGEgc2V0IG9mIHBhcmFtZXRlcnMgZm9y IEh5c3RhcnQgdGhhdCB3b3JrIA0KPiA+IGJldHRlciBmb3IgeW91ciBMVEUgZW52aXJvbm1lbnQ/ IEZvciBleGFtcGxlLCB5b3UgbWlnaHQgdHJ5IHRoZSB0d28gDQo+ID4gdmFyaWF0aW9ucyBJIHN1 Z2dlc3RlZCBlYXJsaWVyIGluIHRoZSB0aHJlYWQ6DQo+ID4NCj4gPiAgICAgICAgICAgICAgICAg ICAgICAgICBpZiAoY2EtPmN1cnJfcnR0ID4gY2EtPmRlbGF5X21pbiArDQo+ID4gICAgICAgICAg ICAgICAgICAgICAgICAgICAgIEhZU1RBUlRfREVMQVlfVEhSRVNIKGNhLT5kZWxheV9taW4gPj4g DQo+ID4gMikpIHsgb3INCj4gPiAgICAgICAgICAgICAgICAgICAgICAgICBpZiAoY2EtPmN1cnJf cnR0ID4gY2EtPmRlbGF5X21pbiArDQo+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAgIEhZ U1RBUlRfREVMQVlfVEhSRVNIKGNhLT5kZWxheV9taW4gPj4gDQo+ID4gMSkpIHsNCj4gPg0KPiA+ IERvIGFueSBvZiB0aG9zZSBnaXZlIGJldHRlciByZXN1bHRzIGZvciB5b3VyIHRlc3RzPw0KPiA+ DQo+ID4gbmVhbA0KPiANCj4gQWxzbywgaHlzdGFydCBpcyBmb29sZWQgYnkgdG9vIG1hbnkgQUNL IHJlY2VpdmVkIGluIHNob3J0IHBlcmlvZC4NCj4gDQo+IFRoaXMgcHJvYmxlbSB3b3VsZCBiZSBz b2x2ZWQgaWYgR1JPIHdhcyB1c2VkIGF0IHJlY2VpdmVyLCBhcyBsZXNzIEFDSyANCj4gd291bGQg YmUgc2VudC4NCj4gDQo+IFByZXN1bWFibHkgcmVjZWl2ZXIgaXMgbm90IGEgbGludXggVENQIHN0 YWNrID8NCj4gDQo+IE1heWJlIHdlIHNob3VsZCBhZGQgYSBsb2dpYyBpbiBoeXN0YXJ0X3VwZGF0 ZSgpIHRvIHRha2Ugb25lIEFDSyBwZXIgbXMuDQo+IA0KPiAwNTozMjoyNC4xMzQxNzIgSVAgOTQu MjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MgDQo+IFtTXSwgc2Vx IDQyOTQ0MjMyMTUsIHdpbiA2NTUzNSwgb3B0aW9ucyBbbXNzIDE0NjAsc2Fja09LLFRTIHZhbCAN Cj4gMTExMjM1MSBlY3IgMCxub3Asd3NjYWxlIDhdLCBsZW5ndGggMA0KPiAwNTozMjoyNC4xNjE5 ODYgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MgDQo+ IFtTLl0sIHNlcSAxMTk4ODc3NzIxLCBhY2sgNDI5NDQyMzIxNiwgd2luIDI4OTYwLCBvcHRpb25z IFttc3MgDQo+IDE0MTYsc2Fja09LLFRTIHZhbA0KPiAyODA4ODA4NjM0IGVjciAxMTEyMzUxLG5v cCx3c2NhbGUgN10sIGxlbmd0aCAwDQo+IDA1OjMyOjI0LjE2MjEwOCBJUCA5NC4yNDMuMTA0LjE1 Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncyANCj4gWy5dLCBhY2sgMSwgd2luIDM0 Mywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1NCBlY3IgMjgwODgwODYzNF0sIA0KPiBs ZW5ndGggMA0KPiAwNTozMjoyNC4xNjMwNzIgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUu OTUuMTc4LjIwNC44MDogRmxhZ3MgDQo+IFtQLl0sIHNlcSAxOjY1OCwgYWNrIDEsIHdpbiAzNDMs IG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTQgZWNyIA0KPiAyODA4ODA4NjM0XSwgbGVu Z3RoIDY1Nw0KPiAwNTozMjoyNC4yMDM5ODQgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMu MTA0LjE1Ni4zMjkyNDogRmxhZ3MgDQo+IFsuXSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwgMjgwODgwODY3NSBlY3IgDQo+IDExMTIzNTRdLCBsZW5ndGggMA0KPiAw NTozMjoyNC4yMTAyNzUgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjky NDogRmxhZ3MgDQo+IFtQLl0sIHNlcSAxOjM4NywgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwgMjgwODgwODY3NyANCj4gZWNyIDExMTIzNTRdLCBsZW5ndGggMzg2DQo+ IDA1OjMyOjI0LjIxMDMxNSBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMy OTI0OiBGbGFncyANCj4gWy5dLCBzZXEgMzg3OjE3OTEsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlv bnMgW25vcCxub3AsVFMgdmFsIA0KPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0 MDQNCj4gMDU6MzI6MjQuMjEwMzMyIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4x NTYuMzI5MjQ6IEZsYWdzIA0KPiBbLl0sIHNlcSAxNzkxOjMxOTUsIGFjayA2NTgsIHdpbiAyMzcs IG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIA0KPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVu Z3RoIDE0MDQNCj4gMDU6MzI6MjQuMjEwMzQ3IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQz LjEwNC4xNTYuMzI5MjQ6IEZsYWdzIA0KPiBbLl0sIHNlcSAzMTk1OjQ1OTksIGFjayA2NTgsIHdp biAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIA0KPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0 XSwgbGVuZ3RoIDE0MDQNCj4gMDU6MzI6MjQuMjEwMzUwIElQIDk0LjI0My4xMDQuMTU2LjMyOTI0 ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzIA0KPiBbLl0sIGFjayAzODcsIHdpbiAzNDcsIG9w dGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyIA0KPiAyODA4ODA4Njc3XSwgbGVuZ3Ro IDANCj4gMDU6MzI6MjQuMjEwMzYxIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4x NTYuMzI5MjQ6IEZsYWdzIA0KPiBbLl0sIHNlcSA0NTk5OjYwMDMsIGFjayA2NTgsIHdpbiAyMzcs IG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIA0KPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVu Z3RoIDE0MDQNCj4gMDU6MzI6MjQuMjEwMzc0IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQz LjEwNC4xNTYuMzI5MjQ6IEZsYWdzIA0KPiBbLl0sIHNlcSA2MDAzOjc0MDcsIGFjayA2NTgsIHdp biAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIA0KPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0 XSwgbGVuZ3RoIDE0MDQNCj4gMDU6MzI6MjQuMjEwMzg5IElQIDE5NS45NS4xNzguMjA0LjgwID4g OTQuMjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdzIA0KPiBbLl0sIHNlcSA3NDA3Ojg4MTEsIGFjayA2 NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIA0KPiAyODA4ODA4Njc3IGVjciAx MTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gMDU6MzI6MjQuMjEwNDAyIElQIDE5NS45NS4xNzguMjA0 LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdzIA0KPiBbLl0sIHNlcSA4ODExOjEwMjE1 LCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCANCj4gMjgwODgwODY3 NyBlY3IgMTExMjM1NF0sIGxlbmd0aCAxNDA0DQo+IDA1OjMyOjI0LjIxMDQxNiBJUCAxOTUuOTUu MTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncyANCj4gWy5dLCBzZXEgMTAy MTU6MTE2MTksIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIA0KPiAy ODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gMDU6MzI6MjQuMjEwNDE4IElQ IDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzIA0KPiBbLl0s IGFjayAxNzkxLCB3aW4gMzU4LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjciAN Cj4gMjgwODgwODY3N10sIGxlbmd0aCAwDQo+IDA1OjMyOjI0LjIxMDQzMSBJUCAxOTUuOTUuMTc4 LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncyANCj4gWy5dLCBzZXEgMTE2MTk6 MTMwMjMsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIA0KPiAyODA4 ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gMDU6MzI6MjQuMjEwNDU1IElQIDk0 LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzIA0KPiBbLl0sIGFj ayAzMTk1LCB3aW4gMzY5LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjciANCj4g MjgwODgwODY3N10sIGxlbmd0aCAwDQo+IDA1OjMyOjI0LjIxMDY4NSBJUCA5NC4yNDMuMTA0LjE1 Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncyANCj4gWy5dLCBhY2sgNDU5OSwgd2lu IDM4MCwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3IgDQo+IDI4MDg4MDg2Nzdd LCBsZW5ndGggMA0KPiAwNTozMjoyNC4yMTA3MzEgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAx OTUuOTUuMTc4LjIwNC44MDogRmxhZ3MgDQo+IFsuXSwgYWNrIDYwMDMsIHdpbiAzOTEsIG9wdGlv bnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyIA0KPiAyODA4ODA4Njc3XSwgbGVuZ3RoIDAN Cj4gMDU6MzI6MjQuMjEwOTM3IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4y MDQuODA6IEZsYWdzIA0KPiBbLl0sIGFjayA3NDA3LCB3aW4gNDAyLCBvcHRpb25zIFtub3Asbm9w LFRTIHZhbCAxMTEyMzU5IGVjciANCj4gMjgwODgwODY3N10sIGxlbmd0aCAwDQo+IDA1OjMyOjI0 LjIxMTA3OCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFn cyANCj4gWy5dLCBhY2sgODgxMSwgd2luIDQxMywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTEx MjM1OSBlY3IgDQo+IDI4MDg4MDg2NzddLCBsZW5ndGggMA0KPiAwNTozMjoyNC4yMTEyMjggSVAg OTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MgDQo+IFsuXSwg YWNrIDEwMjE1LCB3aW4gNDI0LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjciAN Cj4gMjgwODgwODY3N10sIGxlbmd0aCAwDQo+IDA1OjMyOjI0LjIxMTM3NyBJUCA5NC4yNDMuMTA0 LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncyANCj4gWy5dLCBhY2sgMTE2MTks IHdpbiA0MzUsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyIA0KPiAyODA4ODA4 Njc3XSwgbGVuZ3RoIDANCj4gMDU6MzI6MjQuMjExNTQ0IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0 ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzIA0KPiBbLl0sIGFjayAxMzAyMywgd2luIDQ0Niwg b3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3IgDQo+IDI4MDg4MDg2NzddLCBsZW5n dGggMA0KPiAwNTozMjoyNC4yMzc5OTkgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0 LjE1Ni4zMjkyNDogRmxhZ3MgDQo+IFsuXSwgc2VxIDEzMDIzOjE0NDI3LCBhY2sgNjU4LCB3aW4g MjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCANCj4gMjgwODgwODcwOSBlY3IgMTExMjM1OV0s IGxlbmd0aCAxNDA0DQo+IDA1OjMyOjI0LjIzODA1MyBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0 LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncyANCj4gWy5dLCBzZXEgMTQ0Mjc6MTU4MzEsIGFjayA2 NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIA0KPiAyODA4ODA4NzA5IGVjciAx MTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4gMDU6MzI6MjQuMjM4MDkwIElQIDk0LjI0My4xMDQuMTU2 LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzIA0KPiBbLl0sIGFjayAxNDQyNywgd2lu IDQ1Nywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM2MiBlY3IgDQo+IDI4MDg4MDg3MDld LCBsZW5ndGggMA0KPiAwNTozMjoyNC4yMzgxNjQgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAx OTUuOTUuMTc4LjIwNC44MDogRmxhZ3MgDQo+IFsuXSwgYWNrIDE1ODMxLCB3aW4gNDY4LCBvcHRp b25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzYyIGVjciANCj4gMjgwODgwODcwOV0sIGxlbmd0aCAw DQo+IDA1OjMyOjI0LjI0NDAyNCBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2 LjMyOTI0OiBGbGFncyANCj4gWy5dLCBzZXEgMTU4MzE6MTcyMzUsIGFjayA2NTgsIHdpbiAyMzcs IG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIA0KPiAyODA4ODA4NzEzIGVjciAxMTEyMzU5XSwgbGVu Z3RoIDE0MDQNCj4gMDU6MzI6MjQuMjQ0MDYzIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQz LjEwNC4xNTYuMzI5MjQ6IEZsYWdzIA0KPiBbLl0sIHNlcSAxNzIzNToxODYzOSwgYWNrIDY1OCwg d2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgDQo+IDI4MDg4MDg3MTMgZWNyIDExMTIz NTldLCBsZW5ndGggMTQwNA0KPiANCg0K From nobody Thu Oct 1 02:59:49 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 366991A1AAA for ; Thu, 1 Oct 2015 02:59:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.447 X-Spam-Level: X-Spam-Status: No, score=-1.447 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FRT_BELOW2=2.154, J_CHICKENPOX_36=0.6, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O6DyK7AMy5xO for ; Thu, 1 Oct 2015 02:59:43 -0700 (PDT) Received: from sesbmg23.ericsson.net (sesbmg23.ericsson.net [193.180.251.37]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8314E1A079D for ; Thu, 1 Oct 2015 02:59:42 -0700 (PDT) X-AuditID: c1b4fb25-f79a26d00000149a-79-560d040c90a7 Received: from ESESSHC007.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg23.ericsson.net (Symantec Mail Security) with SMTP id E9.AC.05274.C040D065; Thu, 1 Oct 2015 11:59:40 +0200 (CEST) Received: from ESESSMB205.ericsson.se ([169.254.5.99]) by ESESSHC007.ericsson.se ([153.88.183.39]) with mapi id 14.03.0248.002; Thu, 1 Oct 2015 11:59:40 +0200 From: Ingemar Johansson S To: Veaceslav ROMAN , Eric Dumazet , Neal Cardwell Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpA= Date: Thu, 1 Oct 2015 09:59:38 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> In-Reply-To: <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.19] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprDIsWRmVeSWpSXmKPExsUyM+JvjS4PC2+Ywbd7hhZPjz1it9h9biqL xb73Z9ksOu7sBbJmTGS0OHLlGKPFtpPzmSze/9/I7MDhsXPWXXaPBZtKPZYs+cnksWTZU0aP juMbmANYo7hsUlJzMstSi/TtErgy1m0/yFbQtoOpYsrtJUwNjC0bmLoYOTkkBEwktm7ZxwJh i0lcuLeerYuRi0NI4CijxKpJuxkhnEWMEg1/v7GCVLEJ2EisPPSdEcQWEaiTWHryMxNIEbNA L5PE1WdbwMYKC5hJ3Gp6xwxRZC4x6XQbM0iRiMA8RonD59+zgyRYBFQkdhz5AVbEK+ArcWFO H5gtJPCYQ2LzwwAQm1MgUOL/qytsIDajgKzE/e/3wG5lFhCXuPVkPtQPAhJL9pxnhrBFJV4+ /scKYStKtD9tALqUA6heU2L9Ln2IVkWJKd0P2SHWCkqcnPmEZQKj2CwkU2chdMxC0jELSccC RpZVjKLFqcVJuelGxnqpRZnJxcX5eXp5qSWbGIHReXDLb9UdjJffOB5iFOBgVOLhXTCZJ0yI NbGsuDL3EKM0B4uSOG8z04NQIYH0xJLU7NTUgtSi+KLSnNTiQ4xMHJxSDYxqzKo2su/0Arcy Jwi4ng/aVPa+8mXfIe4WWZNljwQDfX17N8VwOMUbKIcqW2e0h+xV43KpeT5r8q7E+L2nV1zI L5NuviV4+t9e44Py8Wtn8z96GnY8uuJYp3xrz4SXy1JMny5oTGs7OvtSzJKeKmOhx1PCVry6 f72R7ZuPoE18a8G603w9f5VYijMSDbWYi4oTAefibyWvAgAA Archived-At: Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , Ingemar Johansson S , Eric Dumazet , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2015 09:59:48 -0000 SGkNCg0KWWVzLCB0aGUgdmFyaWFibGUgZGVsYXlfbWluIGlzIGluIFEzIGZvciByZWFzb25zIG9m IHByZWNpc2lvbiBJIGd1ZXNzDQpCdXQgdGhlIGNhbGwgdG8gSFlTVEFSVF9ERUxBWV9USFJFU0gg aXMgd2l0aCBkZWxheV9taW4gc2hpZnRlZCBkb3duIHRvIFEwDQoNCkhZU1RBUlRfREVMQVlfVEhS RVNIKGNhLT5kZWxheV9taW4gPj4gMykpICAoc2VlIGh0dHA6Ly9seHIuZnJlZS1lbGVjdHJvbnMu Y29tL3NvdXJjZS9uZXQvaXB2NC90Y3BfY3ViaWMuYyNMNDAzICkNCkkgbWF5IGJlIG9mZiBoZXJl Li4gDQoNClJlZ2FyZGluZyBwYWNpbmcuIEkgZG9uJ3QgaGF2ZSByZWFsIGRhdGEgdG8gc2hvdyB0 aGUgZWZmZWN0cyBvZiBwYWNpbmcsIGluIHNpbXVsYXRpb25zIGl0IGlzIGhvd2V2ZXIgcG9zc2li bGUgdG8gc2VlIHRoZSBnYWlucyBpbiB0ZXJtcyBvZiBsZXNzIGNvYWxlc2NpbmcsIHllcyBwYWNp bmcgZG9lcyBub3Qgc29sdmUgdGhlIEFDSyBjb21wcmVzc2lvbiBidXQgaXQgc2VlbSB0byBzb2x2 ZSB0aGUgZm9sbG93IHVwIGVmZmVjdCB0aGF0IEFDSyBjb21wcmVzc2lvbiBnaXZlcy4NCg0KL0lu Z2VtYXINCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBWZWFjZXNsYXYg Uk9NQU4gW21haWx0bzpWZWFjZXNsYXYuUm9tYW5Ab3JhbmdlLm1kXQ0KPiBTZW50OiBkZW4gMSBv a3RvYmVyIDIwMTUgMTA6MzINCj4gVG86IEluZ2VtYXIgSm9oYW5zc29uIFM7IEVyaWMgRHVtYXpl dDsgTmVhbCBDYXJkd2VsbA0KPiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNh bmd0YWUgSGE7IEVyaWMgRHVtYXpldDsgZW5kMmVuZC0NCj4gaW50ZXJlc3RAcG9zdGVsLm9yZw0K PiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQN Cj4gDQo+IFllcywgSFlTVEFSVF9ERUxBWV9NSU4gICg0VTw8MykgLg0KPiBCdXQgY3Vycl9ydHQg YW5kIGRlbGF5X21pbiBjb21lIGZyb20gYmljdGNwX2Fja2VkIHdoaWNoIGlzIGFsc28gPDwzIDoN Cj4gCUluIHRjcF9jdWJpYy5jLCBmdW5jdGlvbiAgbGluZSA0MzMgKGtlcm5lbCA0LjIpDQo+IAkJ ZGVsYXkgPSAocnR0X3VzIDw8IDMpIC8gVVNFQ19QRVJfTVNFQzsgU28sDQo+IDRVPDwzIGlzIGVx dWl2YWxlbnQgdG8gNCBtcyAuDQo+IA0KPiBSZWdhcmRpbmcgcGFjaW5nIEkgYW0gbm90IHN1cmUg d2lsbCBoZWxwIHRoZSBwZXJmb3JtYW5jZTogdGhlcmUgYXJlIGdhcHMgaW4NCj4gQUNLIHRyYWlu cyBvZiB0aGUgb3JkZXIgb2YgbWlsbGlzZWNvbmRzIGFuZCB0aGlzIGlzIGEgbG9zdCB0aW1lIGFu ZCB0aGVyZSBpcyBubw0KPiB3YXkgdG8gY29tcGVuc2F0ZSBpdC4gSWYgaW50cm9kdWNlIEFDSyBw YWNpbmcgb3IgZGF0YSBwYWNrZXRzIHBhY2luZyB0aGlzDQo+IHdvdWxkIGFkZCB0byB0aGUgb3Zl cmFsbCBsb3N0IHRpbWUuDQo+IFBhY2luZyBjYW4gaGVscCBvbmx5IHRvIGFsbGV2aWF0ZSBhIHBv dGVudGlhbCBwcm9ibGVtIG9mIHNvbWUgZG93bnN0cmVhbQ0KPiByb3V0ZXJzIHdpdGggaW5zdWZm aWNpZW50IGJ1ZmZlcnMgd2hpY2ggbWF5IGludHJvZHVjZSB1bmV4cGVjdGVkIHBhY2tldHMNCj4g ZHJvcHMgd2hlbiBsYXJnZSB0cmFpbiBzZW50IGF0IGEgaGlnaCBzcGVlZCBhcyBhIHJlcGx5IHRv IGhpZ2hseSBjb21wcmVzc2VkDQo+IEFDSyB0cmFpbiwgYnV0LCBvbiBhbm90aGVyIGhhbmRzLg0K PiBCdXQgaG93IG11Y2ggcGFjaW5nIGFuZCBob3cgdG8gYWNjb21tb2RhdGUgdG8gbGFyZ2VseSBj aGFuZ2luZyByYWRpbw0KPiBjb25kaXRpb25zID8gQW5kIHdpdGggcGVybWFuZW50bHkgY2hhbmdp bmcgdGVjaG5vbG9neSBhbmQgc3BlZWQgPyBBbnl3YXkNCj4gdGhlIHBhY2luZyB3aWxsIGFkZCB0 byB0aGUgcHJvYmxlbSBvZiBnYXBzIGFuZCBiZXR0ZXIgc2l6ZSBwcm9wZXJseSByb3V0ZXJzDQo+ IGJ1ZmZlcnMuDQo+IA0KPiBJZGVhbGx5IEFDSyB0cmFpbiBpcyBzcGxpdCBpbiAyIHBhcnRzLCBi dXQgaW4gcHJhY3RpY2UgSSBkbyBzZWUgbW9yZS4gQnV0IGFzIHRoZQ0KPiBzdHJlYW0gcHJvZ3Jl c3MgdGhlIGxlc3MgZ2Fwcy4gVGhlIGJpZ2dlciB0aGUgZG93bmxpbmsgcHJlc3N1cmUgdGhlIG1v cmUNCj4gQUNLIGFuZCBsZXNzIGNoYW5jZXMgdG8gaGF2ZSBhIHBhdXNlIGluIEFDSyBzZW5kaW5n LCB0aGVuIHRoZXJlIHdpbGwgYmUgYQ0KPiBjb250aW51b3VzIGZsb3cgb2YgQnVmZmVyIFN0YXR1 cyBSZXBvcnRzIGFuZCBsZXNzIG9mIHRoZSBTY2hlZHVsaW5nIFJlcXVlc3QNCj4gKFNSLCAiSGV5 LCBJIGhhdmUgc29tZSBkYXRhIHRvIHNlbmQiKS4gVGhlIG9ubHkgaXNzdWUgSSBhbSBub3Qgc3Vy ZSBpcyBvbiB0aGUNCj4gcG90ZW50aWFsIGdhcHMgaW4gb3RoZXIgbmV0d29ya3MuIEkgZGlkIG5v dCBwdXQgYmVsbG93IGFsbCBkZXRhaWxzLCBidXQgeW91DQo+IHNob3VsZCBrbm93IHRoYXQgZGV2 aWNlIGNhbiBub3Qgc2VuZCBTUiBpbiBhcmJpdHJhcnkgbW9tZW50cyBvZiB0aW1lIGJ1dCBpbg0K PiBwcmVkZWZpbmVkIGZvciBoaW0gaW50ZXJ2YWxzLiBJbiBteSBuZXR3b3JrIHRoaXMgcGVyaW9k IGlzIDUgbXMsIHRoZXJlZm9yZSB0aGUNCj4gUlRUIG9mIHRoZSBmaXJzdCBBQ0sgaW4gdGhlIHRy YWluIHdpbGwgaGF2ZSB2YXJpYXRpb25zIG9mIDIuNSBtcy4gQnV0IHN0YW5kYXJkcw0KPiBhZG1p dCB0aGlzIHBlcmlvZCB0byBiZSBhbHNvIDEwIG1zICwyMCBtcywgNDAgbXMsIDgwIG1zIGFuZCBJ IGNhbiBub3Qgc2F5IHdoYXQNCj4gaXMgdGhlIHByYWN0aWNlIG9mIG90aGVycy4gSSBtYXkgb25s eSBndWVzcyB0aGF0IG1vc3Qgd2lsbCB0cnkgdG8gdXNlIHRoZSBiZXN0DQo+IGFuZCBmYXN0ZXN0 LCBpLmUuIDUgbXMuDQo+IEFueXdheSwgdG8gZ2V0IGEgcmVhbGlzdGljIG1vZGVsaW5nIG9mIExU RSBpbnRlcmFjdGlvbiB3aXRoIFRDUCB0aGUgZGV0YWlsZWQNCj4gTUFDIHNjaGVkdWxlciBtb2Rl bCBpcyBuZWVkZWQuIFVuZm9ydHVuYXRlbHksIG1vc3Qgb2YgdGhlIGxpdGVyYXR1cmUgZm9jdXMN Cj4gb24gc2NoZWR1bGluZyBvZiB0aGUgUGh5c2ljYWwgUmVzb3VyY2UgQmxvY2tzLCBBZGFwdGl2 ZSBNb2R1bGF0aW9uIGFuZA0KPiBDb2RpbmcsIE1JTU8sIGkuZS4sIGZvY3VzaW5nIG9uIHRoZSBk aXN0cmlidXRpb24gb2YgcmVzb3VyY2VzIGJldHdlZW4gdXNlcnMsDQo+IHdoaWNoIGlzIGdvb2Qg YW5kIG5lZWRlZCwgYnV0IGRvIG5vdCBhZGRyZXNzIHZlcnkgbXVjaCB0aGUgdGltaW5nDQo+IGlu dGVyYWN0aW9uIHdpdGggVENQLiAgQXQgbGVhc3QsIEkgbmV2ZXIgbWV0IGluIExURSBzY2hlZHVs aW5nIGxpdGVyYXR1cmUgYSB0ZXJtDQo+ICJUQ1Agc2VsZi1jbG9ja2luZyIuDQo+IA0KPiBUaGUg OCBtcyAgSFlTVEFSVF9ERUxBWV9NSU4gY291bGQgb25seSBiZSBhIHNpbXBsZSBzb2x1dGlvbiBm b3IgdGhlDQo+IHByb2JsZW0gb2YgY291bnRpbmcgb2YgdGhlIHJvdW5kJyBjdXJyX3J0dCBmcm9t IHRoZSBzZWNvbmQgQUNLIG9mIHRoZSB0cmFpbg0KPiAoYXMgdGhlIHNlY29uZCBBQ0sgUlRUIHdp bGwgYmUsIHR5cGljYWxseSwgZHVlIHRvIHNjaGVkdWxpbmcsIDUtNyBtcyBsb25nZXINCj4gdGhh biB0aGUgZmlyc3Qgb25lKSwgYnV0IHRoaXMgb25seSBpZiBTUiBwZXJpb2RpY2l0eSBpcyA1IG1z Lg0KPiBUaGUgb3RoZXIgMiBwcm9ibGVtcyB3aWxsIHJlbWFpbiB1bnNvbHZlZC4NCj4gDQo+IFZl YWNlc2xhdiBSb21hbg0KPiANCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTog SW5nZW1hciBKb2hhbnNzb24gUyBbbWFpbHRvOmluZ2VtYXIucy5qb2hhbnNzb25AZXJpY3Nzb24u Y29tXQ0KPiBTZW50OiBUaHVyc2RheSwgMDEgT2N0b2JlciAyMDE1IDA4OjU5DQo+IFRvOiBWZWFj ZXNsYXYgUk9NQU47IEVyaWMgRHVtYXpldDsgTmVhbCBDYXJkd2VsbA0KPiBDYzogdGNwbUBpZXRm Lm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0YWUgSGE7IEVyaWMgRHVtYXpldDsgZW5kMmVuZC0N Cj4gaW50ZXJlc3RAcG9zdGVsLm9yZzsgSW5nZW1hciBKb2hhbnNzb24gUw0KPiBTdWJqZWN0OiBS RTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4gDQo+IEhpDQo+ IA0KPiBUaGFua3MgZm9yIHRoZSBkZXRhaWxlZCByZXBvcnQuIFdoYXQgc2VlbXMgdW5jbGVhciB0 byBtZSBpcyB0aGUgdmFsdWUgb2YNCj4gSFlTVEFSVF9ERUxBWV9NSU4uIFlvdSBtZW50aW9uIDht cyBidXQgd2hlbiBJIGxvb2sgaW50byB0aGUgNC4yIGtlcm5lbA0KPiBjb2RlIEkgZ2V0IHRoZSBm ZWVsaW5nIHRoYXQgaXQgaXMgMzJtcyAoICNkZWZpbmUgSFlTVEFSVF9ERUxBWV9NSU4NCj4gKDRV PDwzKSAgICkuIElzIHRoaXMgYSBtaXN0YWtlIGZyb20gbXkgc2lkZSAgPw0KPiBJIGhhdmUgcnVu IExURSBzaW11bGF0aW9ucyBhbmQgdHJpZWQgdG8gd3JpbmcgdGhpcyBIeVN0YXJ0IGlzc3VlIGlu c2lkZSBvdXQgYW5kDQo+IHVwc2lkZSBkb3duIGJ1dCBzb2ZhciBpdCBoYXMgYmVlbiB2ZXJ5IGRp ZmZpY3VsdCB0byBtYWtlIEh5U3RhcnQgZXhpdA0KPiBwcmVtYXR1cmVseSBidXQgdGhlbiBvZiBj b3Vyc2UgSSBoYXZlIHJ1biB3aXRoIEhZU1RBUlRfREVMQVlfTUlOID0NCj4gMzJtcy4NCj4gDQo+ IEFzIHJlZ2FyZHMgdG8gdGhlIEFDSy1jb21wcmVzc2lvbiBwcm9ibGVtLCB5ZXMsIEFDSyBjb21w cmVzc2lvbiBlYXNpbHkNCj4gb2NjdXIgaW4gTFRFLCBldmVuIGluIHNpbXVsYXRpb25zLiBXaGF0 IEkgaGF2ZSBzZWVuIGlzIHRoYXQgcGFja2V0IHBhY2luZyBpcyBhDQo+IHZlcnkgZWZmaWNpZW50 IHJlbWVkeSwgZm9yIGluc3RhbmNlIHRoZSBwYWNrZXQgaW1wbGVtZW50ZWQgaW4gUVVJQyBzZWVt cw0KPiB0byBzb2x2ZSBBQ0sgY29tcHJlc3Npb24gaXNzdWVzIHZlcnkgd2VsbC4NCj4gDQo+IEl0 IGlzIGludGVyZXN0aW5nIHdoYXQgeW91IHNheSB0aGF0IEFDSyB0cmFpbnMgYXJlIHNvIG11Y2gg c3BsaXQgdXAsIEkgY291bGQNCj4gdW5kZXJzdGFuZCB0aGF0IHRoZXkgYXJlIHNwbGl0IHVwIGlu IHR3byBwYXJ0cyAoaW5pdGlhbCBncmFudCArIGFkZGl0aW9uYWwgYWZ0ZXINCj4gcmVjZWl2ZWQg QlNSKS4NCj4gDQo+IC9JbmdlbWFyDQo+IA0KPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0t DQo+ID4gRnJvbTogVmVhY2VzbGF2IFJPTUFOIFttYWlsdG86VmVhY2VzbGF2LlJvbWFuQG9yYW5n ZS5tZF0NCj4gPiBTZW50OiBkZW4gMSBva3RvYmVyIDIwMTUgMDI6MjUNCj4gPiBUbzogRXJpYyBE dW1hemV0OyBOZWFsIENhcmR3ZWxsDQo+ID4gQ2M6IHRjcG1AaWV0Zi5vcmc7IFBpZXJzIE8nSGFu bG9uOyBTYW5ndGFlIEhhOyBJbmdlbWFyIEpvaGFuc3NvbiBTOw0KPiA+IEVyaWMgRHVtYXpldDsg ZW5kMmVuZC1pbnRlcmVzdEBwb3N0ZWwub3JnDQo+ID4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJl XSBUQ1AgSHlTdGFydCBwYXRjaCBkZXBsb3ltZW50DQo+ID4NCj4gPiBGaW5hbGx5IGNhbiBzaGFy ZSBzb21lIHJlc3VsdHMgb2YgSHlzdGFydCBpbnRlcmFjdGlvbiB3aXRoIExURS4NCj4gPiAgRmV3 IHdvcmRzIGFib3V0IHRoZSB0ZXN0aW5nIGNvbmZpZ3VyYXRpb246DQo+ID4gIAlEZXZpY2UgU2Ft c3VuZyBHYWxheHkgTm90ZTRMVEUgLCB1cCB0byAxNTAgTWJwcyBjYXBhYmxlDQo+ID4gICAgICAg ICAgICAgIFJhZGlvIE5ldHdvcmsgTFRFIDI2MDAvMTgwMA0KPiA+ICAgICAgICAgICAgICBEaXN0 YW5jZSBiZXR3ZWVuIHRoZSBDb3JlIE5ldHdvcmsgKEVQQykgYW5kIEJhc2UgU3RhdGlvbiAtIDEw IGttDQo+ID4gICAgICAgICAgICAgIEhUVFAgU2VydmVyIGNvbm5lY3RlZCBkaXJlY3RseSB0byB0 aGUgQ29yZSBOZXR3b3JrIDEgR2Jwcw0KPiA+ICAgICAgICAgICAgICBDb3JlIG5ldHdvcmsgc3dp dGNoaW5nKEVQQykgMTAgR2Jwcw0KPiA+ICAgICAgICAgICAgICBCYWNraGF1bCB0cmFuc3BvcnQg IDEgR2JwcywgZnVsbCBJUCwgTVBMUw0KPiA+ICAgICAgICAgICAgICBMYXN0IG1pbGUgdHJhbnNw b3J0IGZ1bGwgSVAsIE1QTFMsICAzMDAgTWJwcw0KPiA+DQo+ID4gICAgICAgICAgICAgU2VydmVy OiBMaW51eCBrNDBzcnYgNC4xLjAtMDQwMTAwLWdlbmVyaWMgIzIwMTUwNzAzMDk0MA0KPiA+IFNN UCBGcmkgSnVsIDMNCj4gPiAwOTo0MTo0NyBVVEMgMjAxNSB4ODZfNjQgeDg2XzY0IHg4Nl82NCBH TlUvTGludXgNCj4gPiAgICAgICAgICAgICAgICAgICAgICAgICBBcGFjaGUvMi40LjEwIChVYnVu dHUpLCBidWlsdDogICBKdWwgMjQgMjAxNSAxNzoyNToxOA0KPiA+DQo+ID4gICAgICAgICAgICBI eXN0YXJ0X2RldGVjdDogMiAoSFlTVEFSVF9ERUxBWSBvbmx5KSwgYXMgc3VnZ2VzdGVkLg0KPiA+ ICAgICAgICAgICAgbmV0LmlwdjQudGNwX25vX21ldHJpY3Nfc2F2ZSA9IDEgdG8gYXZvaWQgdGNw IG1ldHJpY3MNCj4gPiBjYWNoaW5nIGludGVyZmVyZW5jZQ0KPiA+DQo+ID4gCVRlc3RzIHR5cGVz Og0KPiA+IAkJMS4gRG93bmxvYWQgMyBNQiBmaWxlLCBpbnRlcmNoYW5naW5nIEh5c3RhcnQgT04N Cj4gYW5kIE9GRiwgaW4NCj4gPiBjb25kaXRpb25zIG9mIExvdyBCYW5kd2lkdGggKDI1LTM1IE1i cHMsIHBvb3IgcmFkaW8pIGFuZCBIaWdoDQo+ID4gQmFuZHdpZHRoICgxMDAtMTIwIE1icHMsIGdv b2QgcmFkaW8pLg0KPiA+IAkJMi4gRG93bmxvYWQgMTAgTUIgZmlsZSwgaW50ZXJjaGFuZ2luZyBI eXN0YXJ0DQo+IE9OIGFuZCBPRkYsIGluDQo+ID4gY29uZGl0aW9ucyBvZiBMb3cgQmFuZHdpZHRo ICgyNS0zNSBNYnBzLCBwb29yIHJhZGlvKSBhbmQgSGlnaA0KPiA+IEJhbmR3aWR0aCAoMTAwLTEy MCBNYnBzLCBnb29kIHJhZGlvKS4NCj4gPg0KPiA+ICAJMTAwIERvd25sb2FkIHRlc3RzIHBlciB0 eXBlLg0KPiA+ICAgICAgICAgICAgICBEb3dubG9hZCBkdXJhdGlvbiBhbmQgYmFuZHdpZHRoIG1l YXN1cmVkIGF0IHRoZSBjbGllbnQNCj4gPiBzaWRlIGFzIHNob3duIGJ5IGN1cmwgZnJvbSB0aGUg Zmlyc3QgcmVjZWl2ZWQgZGF0YSBwYWNrZXQgdGlsbCB0aGUgZW5kDQo+ID4gb2Ygc2Vzc2lvbiAo dGhpcyBleGNsdWRlIEROUywgMyB3YXkgaGFuZHNoYWtlLCBodHRwIEdFVCBhbmQgaXRzIEFDSyku DQo+ID4gICAgICAgICAgICAgIEh5c3RhcnQgZXhpdCBjd25kIGV4dHJhY3RlZCBmcm9tIG5zdGF0 ICh0aGFuayB5b3UgRXJpYyB0bw0KPiA+IHBvaW50aW5nIG91dCB0byBpdCkuDQo+ID4NCj4gPiBP dmVyYWxsIHJlc3VsdHM6DQo+ID4gCURvd25sb2FkIDNNQiwgTG93IEJhbmR3aWR0aHM6DQo+ID4g CQlIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS40MSBzDQo+ID4gIAkJTm9IeXN0YXJ0 IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS4zNiBzDQo+ID4NCj4gPiAJRG93bmxvYWQgM01CLCBI aWdoIEJhbmR3aWR0aHM6DQo+ID4gCQlIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC42 NSBzDQo+ID4gIAkJTm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC4zNyAgcw0KPiA+ DQo+ID4gCURvd25sb2FkIDEwTUIsIExvdyBCYW5kd2lkdGhzOg0KPiA+IAkJSHlzdGFydCBhdmVy YWdlIGRvd25sb2FkIHRpbWU6IDIuNDAgcw0KPiA+ICAJCU5vSHlzdGFydCBhdmVyYWdlIGRvd25s b2FkIHRpbWU6IDIuMTIgcw0KPiA+DQo+ID4gCURvd25sb2FkIDEwTUIsIEhpZ2ggQmFuZHdpZHRo czoNCj4gPiAJCUh5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAxLjQyIHMNCj4gPiAgCQlO b0h5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAwLjg5ICBzDQo+ID4NCj4gPiBUaGVzZSBy ZXN1bHRzIHNob3cgdGhhdCBIeXN0YXJ0IGhhcyBubyBzaWduaWZpY2FudCBpbXBhY3QgaW4NCj4g PiBjb25kaXRpb25zIG9mIGxvdyBhdmFpbGFibGUgYmFuZHdpZHRoLCBidXQgYXQgaGlnaCBhdmFp bGFibGUgdGhlDQo+ID4gZGVjcmVhc2Ugb2YgcGVyZm9ybWFuY2UgY2FuIHJlYWNoIDYwLTcwICUu DQo+ID4gV2l0aCB0aGUgZGV2ZWxvcG1lbnQgb2YgdGhlIExURS1BIHdoaWNoIHVzZXMgbGFyZ2Vy IHNwZWN0cnVtIHRvIHJlYWNoDQo+ID4gMzAwIE1icHMgYW5kIG1vcmUgdGhlIGltcGFjdCBvZiBI eXN0YXJ0IHdpbGwgYmVjb21lIGV2ZW4gbW9yZSB2aXNpYmxlLg0KPiA+DQo+ID4gSSBkbyBzaGFy ZSB3aXRoIEVyaWMgYW5kIE5lYWwgc29tZSBmaWxlcyBkZXNjcmliZWQgYmVsb3cgYW5kIGFsc28g Y2FuDQo+ID4gcHJvdmlkZSBsaW5rcyB0byBhbnlvbmUgaW50ZXJlc3RlZC4NCj4gPg0KPiA+IFN1 bW1hcnkgb2YgdGhlc2UgdGVzdHMgcmVzdWx0cyBhcmUgZ2F0aGVyZWQgaW4gdGhlIGZvbGxvd2lu ZyBFeGNlbA0KPiA+IGZpbGVzIChuYW1lcywgSSBiZWxpZXZlLCBhcmUgc2VsZiBleHBsYW5hdG9y eSk6IFRlc3RfM01fbG93X0JXLnhsc3gsDQo+ID4gVGVzdF8zTV9oaWdoX0JXLnhsc3gsIFRlc3Rf MTBNX2xvd19CVy54bHN4LA0KPiBUZXN0XzEwTV9oaWdoX0JXLnhsc3guDQo+ID4gU2ltaWxhcmx5 IHRjcGR1bXAgdHJhY2VzIGFyZSBjYWxsZWQgVGVzdF8zTV9sb3dfQlcucGNhcCwNCj4gPiBUZXN0 XzNNX2hpZ2hfQlcuIHBjYXAsIFRlc3RfMTBNX2xvd19CVy4gcGNhcCwgVGVzdF8xME1faGlnaF9C Vy4NCj4gPiBwY2FwIChhYmxlIHRvIGNvbGxlY3Qgb25seSBzZXJ2ZXIgc2lkZSB0cmFjZXMsIGJ1 dCB0aGV5IGFyZSBzZWxmIHN1ZmZpY2llbnQpLg0KPiA+IEZvciB0aGUgMTBNIGRvd25sb2FkcyB0 aGVyZSBhcmUgZmV3IG1pc3NpbmcgdHJhY2VzIGFzIHRoZXJlIHRyYWNlDQo+ID4gZmlsZXMgd2Vy ZSByZWN5Y2xlZC4gSW4gdHJhY2VzIGFyZSBhbHNvIHNzaCBzZXNzaW9ucywgcGxlYXNlLCBpZ25v cmUNCj4gPiB0aGVtLCB0aGV5IHdlcmUgdXNlZCB0byBwdXQgaHlzdGFydCBPTiBhbmQgT0ZGIG9u IHNlcnZlciBhbmQgZXh0cmFjdA0KPiA+IG5zdGF0IHJlc3VsdHMgYmVmb3JlIGFuZCBhZnRlciBl YWNoIGRvd25sb2FkLg0KPiA+DQo+ID4gRXhjZWwgZmlsZXMgaGF2ZSAyIHRhYnM6IGN1cmxfc3Vt IGFuZCBTdW1tYXJ5Lg0KPiA+IEluIHRoZSBjdXJsX3N1bSB0YWIgdGhlIHN0YXQgaXMgY3VtdWxh dGVkIGluIGNocm9ub2xvZ2ljYWwgb3JkZXIgb2YgdGVzdHMuDQo+ID4gSW4gdGhlIFN1bW1hcnlf dGFiICB0aGUgSHlzdGFydCBhbmQgTm9IeXN0YXJ0IHRlc3RzIGFyZSBzaG93biBzaWRlIGJ5DQo+ ID4gc2lkZSAoYXMgc2FpZCwgRG93bmxvYWQgMSB3YXMgd2l0aCBIeXN0YXJ0LCBEb3dubG9hZCAy IHdpdGhvdXQsDQo+ID4gRG93bmxvYWQgMyB3aXRoIEh5c3RhcnQsIERvd25sb2FkIDQgd2l0aG91 dCwgZXRjLikuIEl0IGFsc28gaW5jbHVkZQ0KPiA+IGdyYXBocyBvZiBUcmFuc2ZlciB0aW1lIGFu ZCBhIGdyYXBoIHdoaWNoIHNob3dzIHRoZSBkaXN0cmlidXRpb24gaW4gJQ0KPiA+IG9mIGRpZmZl cmVudCBIeXN0YXJ0IGV4aXQgd2luZG93IERlbGF5Q1dORC4NCj4gPg0KPiA+IEZpZWxkczoNCj4g PiBjb21tZW50cwktIHRlc3QgY2FzZQ0KPiA+IFRyYW5zZl9UaW1lCS0gY2FsY3VsYXRlZCBmcm9t IGN1cmwgb3V0cHV0IHRoZSBkdXJhdGlvbg0KPiBiZXR3ZWVuIGZpcnN0IGRhdGENCj4gPiBwYWNr ZXQgYW5kIGVuZCBvZiB0cmFuc2Zlcg0KPiA+IFRyYW5zZl9zcGVlZAktIGNhbGN1bGF0ZWQgZnJv bSBjdXJsIG91dHB1dCB0aGUgcmF0aW8gb2YgZmlsZSBzaXplDQo+IGFuZCBkdXJhdGlvbg0KPiA+ IGJldHdlZW4gZmlyc3QgZGF0YSBwYWNrZXQgYW5kIGVuZCBvZiB0cmFuc2Zlcg0KPiA+IEh5c3Rh cnQJRF9EZWxheSAtICBoeXN0YXJ0IGV4aXQgb2NjdXJyZW5jZSAoZGlmZmVyZW5jZSBiZXR3ZWVu DQo+ID4gVGNwRXh0VENQSHlzdGFydERlbGF5RGV0ZWN0IGJlZm9yZSBhbmQgYWZ0ZXIgZG93bmxv YWQpDQo+ID4gRF9EZWxheUN3bmQJICAtIEh5c3RhcnQgZXhpdCBjd25kIGZvciB0aGlzIGRvd25s b2FkDQo+ID4gKGRpZmZlcmVuY2Ugb2YgVGNwRXh0VENQSHlzdGFydERlbGF5Q3duZCBiZWZvcmUg YW5kIGFmdGVyIHRoZQ0KPiA+IGRvd25sb2FkKSBEYXRlX09uX1NlcnZlciAtIHRpbWUganVzdCBi ZWZvcmUgdGhlIGRvd25sb2FkIHN0YXJ0LCB0bw0KPiA+IGhlbHAgbmF2aWdhdGUgdGhlIHBjYXAN Cj4gPg0KPiA+IEFkZGl0aW9uYWxseSwgaW4gdGhlIGZpbGUgVGVzdF8xME1fbG93X0JXLnhsc3gg aW4gdGhlIHRhYiBjdXJsX3N1bSBhcmUNCj4gPiBpbmNsdWRlZCBmb3IgZWFjaCBkb3dubG9hZCB0 Y3Auc3RyZWFtIGV4dHJhY3QgZnJvbSBwY2FwIG9mIHRoZSBmaXJzdA0KPiA+IDEwMCB0Y3AuYW5h bHlzaXMuYWNrX3J0dCB3aGljaCBtYXkgaGVscCBpbiB1bmRlcnN0YW5kaW5nIG9mIHRoZQ0KPiA+ IGh5c3RhcnQgYmVoYXZpb3IgKGZpZWxkcyBhY2tfcnR0XzEgdG8gYWNrX3J0dF8xMDApLg0KPiA+ IEluIHRoaXMgZmlsZSB0aGVyZSBhcmUgYWxzbyBmaWVsZHMgVVJJX2ZyYW1lLCBVUklfdGltZSwg VVJJX3RpbWVfcmVsYXRpdmUsDQo+ID4gCXRjcF9zdHJlYW0uDQo+ID4gVGhlbiB0aGVyZSBhcmUg Y2FsY3VsYXRlZCBmaWVsZHM6IG1pblJUVCBhcyBhIG1pbmltdW0gb2YgYWxsIDEwMCBBQ0ssIHIx XzUsDQo+ID4gcjJfOCwgcjNfOCwJcjRfOCB3aGljaCBzaW11bGF0ZSBIeXN0YXJ0IGNhbGN1bGF0 aW9ucyBvZiB0aGUgcm91bmQgY3Vycl9ydHQNCj4gPiBhbmQgcjEgbWluLCByMiBtaW4sIHIzIG1p biwgcjQgbWluIHJlcHJlc2VudCBtaW5SVFQgb2YgdGhlIHdob2xlIHJvdW5kLg0KPiA+IFBsZWFz ZSwgY29ycmVjdCBtZSBpZiBJIGFtIHdyb25nLCBidXQgaW4gbXkgdW5kZXJzdGFuZGluZyBmb3Ig ZWFjaCBBQ0sNCj4gPiB0aGUgaHlzdGFydF91cGRhdGUgaXMgY2FsbGVkIGJlZm9yZSB0aGUgaHlz dGFydF9yZXNldCwgYW5kIGFzIGEgcmVzdWx0DQo+ID4gdGhlIGNvbXB1dGF0aW9uIG9mIGN1cl9y dHQgb2YgdGhlIDggcGFja2V0cyBvZiB0aGUgcm91bmQgc3RhcnRzIHdpdGgNCj4gPiB0aGUgMi1u ZCBwYWNrZXQgb2YgdGhlIHJvdW5kLCBub3QgdGhlIGZpcnN0IG9uZS4NCj4gPiAgSSd2ZSB1c2Ug cjFfNSBhcyBhIG1pbihhY2tfcnR0XzcuLi5hY2tfcnR0XzExKSwgZHVlIHRvDQo+ID4gaHlzdGFy dF9sb3dfd2luZG93PTE2LCByMl84IGFzIGEgbWluKGFja19ydHRfMTIgLi4uIGFja19ydHRfMTkp LCByM184DQo+ID4gYXMgYQ0KPiA+IG1pbihhY2tfcnR0XzMyIC4uLiBhY2tfcnR0XzM5KSBhbmQg KSwgcjRfOCBhcyBhIG1pbihhY2tfcnR0XzcyIC4uLg0KPiBhY2tfcnR0Xzc5KS4NCj4gPiBUbyB1 bmRlcnN0YW5kIHRoZSBpbXBhY3Qgb2YgdGhpcyB0aGVyZSBhcmUgYWxzbyByMSBIT0wsIHIyIEhP TCwgcjMNCj4gPiBIT0wsIHI0IEhPTCAoSGVhZCBPZiBMaW5lKSB3aGljaCBjb21wdXRlIHRoZSBz YW1lIHdpdGggZmlyc3QgcGFja2V0IGluDQo+ID4gdGhlIHJvdW5kIGluY2x1ZGVkIGluIHRoZSBy ZWxldmFudCByb3VuZDogIHIxXzUgYXMgYQ0KPiA+IG1pbihhY2tfcnR0XzYuLi5hY2tfcnR0XzEw KSwgcjJfOCBhcyBhIG1pbihhY2tfcnR0XzExIC4uLiBhY2tfcnR0XzE4KSwNCj4gPiByM184IGFz IGEgbWluKGFja19ydHRfMzEgLi4uIGFja19ydHRfMzgpIGFuZCApLCByNF84IGFzIGEgbWluKGFj a19ydHRfNzEgLi4uDQo+IGFja19ydHRfNzgpLg0KPiA+DQo+ID4gSWYgbXkgdW5kZXJzdGFuZGlu ZyBvZiBIeXN0YXJ0IGlzIGNvcnJlY3QgYW5kLCBhc3N1bWluZyBpbiBlYWNoIHRyYWluDQo+ID4g dGhlIGZpcnN0IHBhY2tldCBoYXMgdGhlIGxvd2VzdCBSVFQgd2hpY2gsIG1vc3QgcHJvYmFibHkg aXMgdmFsaWQgaW4NCj4gPiBhbGwgdHlwZSBvZiBuZXR3b3JrcywgdGhlbiB0aGUgZXhpdCBmcm9t IGh5c3RhcnQgaW4gdGhpcw0KPiA+IGltcGxlbWVudGF0aW9uIG1heSBoYXBwZW4gYXQgY3duZCAy OSwgNDksIDg5LCAxNjksIGV0Yy4gKGdpdmVuIHRoZSBJVyBvZg0KPiAxMCkuDQo+ID4gVGhlc2Ug dGVzdHMgc2hvdyB0aGF0IHRoaXMgaXMgdGhlIGNhc2UsIGhvd2V2ZXIgdGhlcmUgYWxzbw0KPiA+ IGludGVybWVkaWF0ZSB2YWx1ZXMuIEhvd2V2ZXIgdGhlIGN3bmQgb2YgMjkgc2hhbGwgYmUgdGhl IGxvd2VzdA0KPiA+IHBvc3NpYmxlIHZhbHVlIGFuZCB0aGlzIGlzIHRoZSBjYXNlIGluIHRoZXNl IHRlc3RzLiBVbmZvcnR1bmF0ZWx5LCBpbg0KPiA+IExURSB0aGUgZmlyc3QgcGFja2V0IGluIHRo ZSB0cmFpbiBoYXMgY2hhbmNlcyB0byBhbHdheXMgaGF2ZSB+NiBtcw0KPiA+IGxvd2VyIHRoYW4g dGhlIGZvbGxvd2luZyBvbmVzIGFuZCwgYmVjYXVzZSBpdCBpcyBub3QgY291bnRlZCBpbiB0aGUN Cj4gPiAyLW5kIHRyYWluLCB0aGVyZSBpcyBxdWl0ZSBoaWdoIHBlcmNlbnRhZ2Ugb2YgZXhpdCBh dCAyOSAoMTQtMzYlIGluDQo+ID4gdGhlc2UgdGVzdHMpLCBldmVuIHRob3VnaCBsYXRlciBvbiBj dWJpYyBpbmNyZWFzZXMgdGhlIGN3bmQgd2l0aG91dA0KPiA+IGxvc3NlcyB1cCB0byBtYW55IGh1 bmRyZWRzLiBGb3IgbWUgdGhpcyBpcyB0b28gZWFybHkgZXhpdCBmcm9tIHNsb3cgc3RhcnQuDQo+ ID4gQXMgaXQgY2FuIGJlIHNlZW4gZnJvbSBjb21wYXJpc29uIG9mIHRoZSBzaW11bGF0ZWQgY2Fs Y3VsYXRpb24sDQo+ID4gaW5jbHVzaW9uIG9mIHRoZSBmaXJzdCBwYWNrZXQgaW4gdGhlIHJvdW5k IGN1cnJfcnR0IGNhbGN1bGF0aW9uIHdvdWxkDQo+ID4gZWxpbWluYXRlIHRoZSBjd25kIDI5IGFu ZCBzaGlmdCB0byBjd25kIDQ5IG9yIDg5IG9yIGhpZ2hlci4NCj4gPg0KPiA+IEJ1dCB0aGVyZSBh cmUgYWxzbyBpbnRlcm1lZGlhdGUgdmFsdWVzLCBlLmcuIDY5LiBMb29raW5nIHRocm91Z2gNCj4g PiB0cmFjZXMgb25lIG1heSBzZWUgdGhhdCB0aGlzIGlzIGR1ZSB0byAiY29tcHJlc3NlZCBBQ0si IHdoaWNoIHJlY2VpdmVzDQo+ID4gYSB0aWdodGx5IHBhY2tlZCB0cmFpbiBvZiBtYW55IEFDSyBz byB0aGF0IGluIHRyYWNlcyB0aGVyZSBhcmUgbm90DQo+ID4gZGF0YSBwYWNrZXRzIHNlbnQgaW4g dHVybiBieSBzZXJ2ZXIuIExvb2tzIHRvIG1lLCBiZWNhdXNlDQo+ID4gaHlzcmFydF9yZXNldCBz ZXRzIHRoZSBlbmRfc2VxIHRvIHNuZF9uZXh0IGFuZCB0aGUgc2VydmVyIGRpZCBub3QNCj4gPiBt YW5hZ2VkIHRvIHNlbmQgeWV0IHBhY2tldHMgaW4gcmVwbHkgdG8gdGhpcyBoaWdoIHNwZWVkIEFD SyB0cmFpbiB0aGUNCj4gPiBzbmRfbmV4dCBwb2ludHMgdG8gbXVjaCBsb3dlciB2YWx1ZSB0aGFu IHdvdWxkIG5vcm1hbGx5IGhhcHBlbiBhbmQNCj4gPiB0aGlzIGFjdHVhbGx5IGRlc3Ryb3lzIHRo ZSBjb3JyZWN0IGlkZW50aWZpY2F0aW9uIG9mIHRoZSBib3JkZXJzIG9mDQo+ID4gdGhlIHJvdW5k LCB0aGUgcm91bmQgZmFpbHMgc29tZXdoZXJlIGluIHRoZSBtaWRkbGUgb2YgdGhlIHRyYWluIG5v dCBhdA0KPiA+IGl0cyBib3JkZXIsIHRoaXMgbWFrZXMgdmVyeSBsaWtlbHkgaW5jcmVhc2VkIGRl bGF5IGR1ZSB0byBxdWV1ZWluZyBhbmQNCj4gPiBlYXJsaWVyIGV4aXQgZnJvbSBzbG93X3N0YXJ0 IGFuZCBpbnRlcm1lZGlhdGUgdmFsdWVzIG9mIGV4aXQgY3duZCB3aGVyZQ0KPiB0aGV5IGFyZSBu b3QgZXhwZWN0ZWQuDQo+ID4NCj4gPiBBbmQgbGFzdCBidXQgbm90IGxlYXN0OiB0aGVyZSBhcmUg Z2FwcyAoNC03IG1zKSBpbiBBQ0sgdHJhaW5zLiBPbiB0aGUNCj4gPiBwZXJmZWN0bHkgc2VudCBJ VyBvZiAxMCBkYXRhIHBhY2tldHMgdHJhaW4gdGhlIHR5cGljYWwgZm9yIExURSB3aWxsIGJlDQo+ ID4gdG8gcmVjZWl2ZSAxIG9yIDIgQUNLLCB0aGVuIGEgZ2FwIG9mIDQtNiBtcyB0aGVuIGFub3Ro ZXIgY291cGxlIG9mDQo+ID4gQUNLLCB0aGVuIGFub3RoZXIgZ2FwLCB0aGVuIDUgQUNLIGNvbXBy ZXNzZWQgdG9nZXRoZXIgaW4gYSBmcmFjdGlvbiBvZg0KPiA+IG1pY3Jvc2Vjb25kLiBPZiBjYXVz ZSwgdGhlIG5leHQgdHJhaW4gZnJvbSB0aGUgc2VydmVyIHdpbGwgY29udGFpbiB0aGUNCj4gPiBz YW1lIGdhcHMuIFZlcnkgcXVpY2tseSwgZS5nLiwgdGhlIEhlYWQgb2YgTGluZSAoSE9MKSBvZiB0 aGUgMy1kIHRyYWluDQo+ID4gd2lsbCBmb2xsb3cgaW1tZWRpYXRlbHkgdGhlIHRhaWwgb2YgdGhl IDItbmQgdHJhaW4gd2hpY2ggd2lsbCBsZWFkIHRvDQo+ID4gcXVldWVpbmcgaW4gdGhlIGVuZCBy b3V0ZXIgKGFjdHVhbGx5IHRoZSByYWRpbyBiYXNlIHN0YXRpb24sIG11Y2gNCj4gPiBlYXJsaWVy IHRoYW4gQkRQIHJlYWNoZWQpLCBidXQgdGhpcyB3aWxsIGxlYWQgdG8gaW5jcmVhc2Ugb2YgdGhl IHRyYWluDQo+ID4gUlRUIGFuZCB0b28geWVhcmx5IGV4aXQgZnJvbSBzbG93IHN0YXJ0LiBXaHkg dG9vIGVhcmx5IGV4aXQgPyBCZWNhdXNlLA0KPiA+IHNob3VsZCBzZXJ2ZXIgY29udGludWUgaW5j cmVhc2luZyB0aGUgc3BlZWQgdGhpcyB3b3VsZCByZWR1Y2UgcXVpY2tlcg0KPiA+IHRoZSBnYXBz IGJvdGggaW4gZG93bmxpbmsgYW5kIHVwbGluaywgdGhlIHByZWljZSBmb3IgdGhpcyBiZWluZyB0 aGUNCj4gPiBpbmNyZWFzZSBvZiB0aGUgUlRUIChJIGd1ZXNzIGRvdWJsZSkgYWdhaW5zdCB3aGF0 IG9uZSB3b3VsZCBub3JtYWxseSBzZWUNCj4gaW4gdGhlIGVuZC10by1lbmQgRXRoZXJuZXQgbmV0 d29ya3MuDQo+ID4NCj4gPg0KPiA+IEkgZG8gbGlrZSBpZGVhcyBvZiBIeXN0YXJ0LCBidXQgSSAg ZG9uJ3Qga25vdyBob3cgY291bGQgaXQgYmUgcG9zc2libHkNCj4gPiByZWNvbmNpbGVkIHdpdGgg bXkgTFRFIHByb2JsZW1zLiAgV2l0aCB0aGUgaW5jcmVhc2Ugb2YgTFRFIHNwZWVkcyB0aGlzDQo+ ID4gZWFybHkgZXhpdCBmcm9tIHNsb3cgc3RhcnQgd2lsbCBiZWNvbWUgZXZlbiBtb3JlIG9mIHRo ZSBwcm9ibGVtLg0KPiA+IEZvciBMVEUsIG5vdCBzdXJlIGhvdyBnb29kIGFuZCBnZW5lcmFsbHkg YWNjZXB0YWJsZSwgc29sdXRpb24gd291bGQNCj4gPiBiZSwgcG9zc2libHksIHRvIGluY3JlYXNl IHRoZSBIWVNUQVJUX0RFTEFZX01JTiB0byA4IG1zLCB3aGljaCwgYXQNCj4gPiBsZWFzdCBpbiBt eSBjYXNlLCB3b3VsZCBkaW1pbmlzaCB0aGUgZWFybGllciBleGl0IHNpZ25pZmljYW50bHksIGlm IEkNCj4gPiBiZWxpZXZlIGluIHRoZXNlIHRlc3RzIHJlc3VsdHMuDQo+ID4gQW5vdGhlciBzb2x1 dGlvbiwgd2hpY2ggSSBkb24ndCBsaWtlIHZlcnkgbXVjaCwgYnV0IGFzIEkga25vdyBzb21lDQo+ ID4gb3BlcmF0b3JzIHVzZSwgd291bGQgYmUgdG8gaW5zZXJ0IGEga2luZCBvZiBUQ1AgYWNjZWxl cmF0b3IgYmV0d2Vlbg0KPiA+IHRoZSBJbnRlcm5ldCBhbmQgdGhlIG1vYmlsZSBuZXR3b3JrIHdo aWNoIHdpbGwgaW50ZXJjZXB0IGFsbCBUQ1AgYW5kDQo+ID4gd291bGQgc2VuZCBpdCB0byBtb2Jp bGUgd2l0aG91dCBIeXN0YXJ0IGJ1dCB0aGlzIHdvdWxkIGtpbGwgYW55DQo+ID4gZW5kLXRvLWVu ZCBwcmluY2lwbGUgYW5kIHRoaXMgd2lsbCBub3QgYmUgdGhlIEludGVybmV0IGFueW1vcmUuDQo+ ID4NCj4gPg0KPiA+IElmIHlvdSBoYXZlIHRpbWUgYW5kIG5ldmVyIGxvb2sgY2xvc2VseSB0byBM VEUgdGVjaG5vbG9neSBJIGRvIGEgcXVpY2sNCj4gPiBzdW1tYXJ5IGJlbGxvdy4NCj4gPg0KPiA+ DQo+ID4gTFRFLCBUQ1Agc2VsZi1jbG9ja2luZyBhbmQgQUNLIGNvbXByZXNzaW9uLg0KPiA+DQo+ ID4gU3BlbnQgc29tZSB0aW1lIHRvIHVuZGVyc3RhbmQgdGhlIGJhc2ljcyBvZiB3aGF0IGNvdWxk IGJlIHRoZSByZWFzb24NCj4gPiBmb3IgYWxsIHRob3NlIHRpbWluZyBlZmZlY3RzIGR1ZSB0byBM VEUuDQo+ID4gCUZpcnN0LCB1bmxpa2Ugd2lyZSBiYXNlZCB0cmFuc3BvcnQgdGVjaG5vbG9naWVz LCBMVEUgKGJ1dCBhbHNvDQo+IFVNVFMNCj4gPiBhbmQgZnV0dXJlIDVHKSBzZW5kcyBkYXRhLCBi b3RoIHVwbGluayBhbmQgZG93bmxpbmsgaW4gcHJlZGVmaW5lZA0KPiA+IHRpbWVzbG90cyBjYWxs ZWQgVHJhbnNtaXNzaW9uIFRpbWUgSW50ZXJ2YWwgKFRUSSkuIEluIFVNVFMgdGhpcyBpcw0KPiA+ IDJtcywgaW4gTFRFIGlzDQo+ID4gMSBtcyBhbmQgaXQgc2F5cyB0aGF0IGluIDVHIGl0IHdpbGwg YmUgMC41IG1zLg0KPiA+IFRoZXNlIFRUSSBoYXZlIGV4YWN0IGJvcmRlcnMgYW5kIHRpZ2h0bHkg Zm9sbG93IGVhY2ggb3RoZXIgYW5kIHJlcXVpcmUNCj4gPiBleGFjdCBzeW5jaHJvbml6YXRpb24g YmV0d2VlbiB0aGUgbW9iaWxlIGRldmljZSBhbmQgUmFkaW8gQmFzZSBTdGF0aW9uDQo+ID4gKGNh bGxlZCBlTm9kZUIgaW4gTFRFKS4gVG8gY29wZSB3aXRoIHZhcnlpbmcgcmFkaW8gY29uZGl0aW9u cyB0aGUNCj4gPiBzZW5kaW5nIGRhdGEgYmFuZHdpZHRoIGlzIG1vZGlmaWVkIGluIHN1Y2ggYSB3 YXkgdGhhdCB0byBrZWVwIGENCj4gPiBjb25zdGFudCBwcm9iYWJpbGl0eSBvZiBkYXRhIGxvc3Mg b2YgMTAlLiBCYWQgcmFkaW8gLSBsZXNzIGJhbmR3aWR0aCwgZ29vZA0KPiByYWRpbyAtIG1vcmUg YmFuZHdpZHRoLg0KPiA+IFRoZSBiYW5kd2lkdGggaXMgbW9kaWZpZWQgYnkgcGFja2luZyBsZXNz IG9yIG1vcmUgYml0cyBpbiBhIFRUSSBvZiAxIG1zIGluDQo+IExURS4NCj4gPiBCdXQgVFRJIHJl bWFpbiB1bmNoYW5nZWQgb2YgZXhhY3RseSAxIG1zLiBUaGF0IG1lYW5zIHRoYXQsIGF0IHRoZQ0K PiA+IFRDUC9JUCBsYXllciB0aGUgcHJlY2lzaW9uIG9mIHRoZSB0aW1pbmcgaW5mb3JtYXRpb24g ZnJvbSBkZXZpY2UgdG8NCj4gPiB0aGUgbmV0d29yayBpcyAxIG1zLiBJbiB0aGUgY2FzZSBvZiBh IGJhbmR3aWR0aCBvZiAxIE1icHMgdGhlcmUgd2lsbA0KPiA+IGJlIDgzIElQIHBhY2tldHMgcGVy IHNlY29uZCBvciBvbmUgcGFja2V0IGV2ZXJ5IDEyIG1zLiBJbiB0aGlzIGNhc2UsDQo+ID4gcG9z c2libHkgdGhlIHJldHVybiBvZiBvbmUgQUNLIGV2ZXJ5IDEyIG1zIChldmVyeSAxMiBUVEkpIHdv dWxkIGJlDQo+ID4gbW9yZSBvciBsZXNzIHN1ZmZpY2llbnQgdG8ga2VlcCBhdCBhIGdvb2QgbGV2 ZWwgdGhlIFRDUCBzZWxmLWNsb2NraW5nDQo+ID4gbWVjaGFuaXNtLiBCdXQsIHdoZW4gdGFsa2lu ZyBhYm91dCAxMDAgTWJwcyAob3IgMzAwIE1icHMpIHRoYXQgbWVhbnMNCj4gPiBvbmUgcGFja2V0 IChhbmQgb25lIEFDSykgZXZlcnkgMC4xMiBtcyBhbmQgdGhlcmUgaXMgbm8gd2F5IHRvIHByb3Zp ZGUNCj4gPiBpdCB3aXRoIGEgVFRJIG9mIDEgbXMuIFRoaXMgaXMgdGhlIHJlYXNvbiBmb3IgQUNL IGNvbXByZXNzaW9uIGluIExURQ0KPiA+IGFuZCB0aGUgZnV0dXJlIDVHIHdoaWNoIHByb21pc2Vz IG1vcmUgdGhhbiAxR2JwcyB3aXRoIGl0cyAwLjUgbXMgVFRJIHdpbGwNCj4gYmUgaW4gYSBiaWcg Y29uZmxpY3Qgd2l0aCBUQ1Agc2VsZi1jbG9ja2luZy4NCj4gPiAJQnV0IHRoZSBwcm9ibGVtIGRv IG5vdCBzdG9wcyBoZXJlLiBUaGVyZSBpcyBhIHNpZ25pZmljYW50bHkNCj4gZGlmZmVyZW50DQo+ ID4gbWVjaGFuaXNtcyBvZiB0cmFuc21pc3Npb24gaW4gdGhlIGRvd25saW5rIChmcm9tIGVOb2Rl QiB0byBtb2JpbGUNCj4gPiBkZXZpY2UpIGFuZCBpbiB1cGxpbmsgKGZyb20gbW9iaWxlIGRldmlj ZSB0byB0aGUgZU5vZGVCKS4gVGhlDQo+ID4gZGlmZmVyZW5jZSBjb21lIGZyb20gdGhlIGZhY3Qg dGhhdCB0aGUgc2NoZWR1bGVyIGZvciBib3RoIGRvd25saW5rIGFuZA0KPiA+IGluIHVwbGluayBp cyBsb2NhdGVkIGluIHRoZSBlTm9kZUIuIEFzIG9mIG15IHVuZGVyc3RhbmRpbmcgdGhlIHB1cnBv c2UNCj4gPiBvZiB0aGlzIGlzIHRvIHNpbXBsaWZ5IHRvIHRoZSBtYXhpbXVtIHRoZSBkZXZpY2Ug YW5kIHNhdmUgaXRzIGJhdHRlcnkuDQo+ID4gQXMgYSByZXN1bHQgb2YgdGhpcyBkZXNpZ24sIHRo ZSBlTm9kZUIgY2FuIHNlbmQgaW4gZWFjaCBUVEkgdG93YXJkIHRoZQ0KPiA+IG1vYmlsZSBkZXZp Y2Ugd2hlbmV2ZXIgZGF0YSBleGlzdCBpbiBpdHMgYnVmZmVyIChlTm9kZUIgaXMgYSBraW5kIG9m DQo+ID4gSVAgd2lyZWxlc3Mgcm91dGVyKS4gQnV0IGZvciBtb2JpbGUgZGV2aWNlLCBpbiBvcmRl ciB0byBzZW5kIHRoZSBkYXRhDQo+ID4gaW4gdXBsaW5rLCBhZnRlciBhIHBhdXNlLCBpdCBoYXMg Zmlyc3QgdG8gYXNrIHRoZSBuZXR3b3JrIHRvIHNjaGVkdWxlDQo+ID4gaXQuIEFuZCBtb2JpbGUg aGFzIGZvciB0aGlzIHJlcXVlc3QgKGFmdGVyIGEgcGF1c2Ugb2Ygc2VuZGluZykganVzdA0KPiA+ IG9uZSBiaXQsIGFjdHVhbGx5IGlzIG5vdCBldmVuIGEgZGF0YSBiaXQsIGlzIGEgc3BlY2lhbCBy YWRpbyBzaWduYWwNCj4gPiB3aGljaCB0ZWxscyAiSGV5LCBJIGhhdmUgc29tZSBkYXRhIGluIHRo ZSBidWZmZXIgdG8gc2VuZCIuIExldHMgbG9vaw0KPiA+IGF0IGluaXRpYWwgdHJhaW4gb2YgSVcg b2YgMTAgcGFja2V0cy4gSW4gZ29vZCByYWRpbyBjb25kaXRpb25zIHRoZQ0KPiA+IGVOb2RlQiB3 aWxsIGZpdCBhbGwgMTAgcGFja2V0cyBpbiAxIFRUSSBvZiAxIG1zIGFuZCBzZW5kIHRoZW0gYWxs IHRvDQo+ID4gdGhlIGRldmljZS4gRGV2aWNlIHdpbGwgdW5wYWNrIHRoZW0gYW5kIGdlbmVyYXRl IDEwIEFDSyBhbmQgdGhlbiB0ZWxsDQo+ID4gdG8gdGhlIGVOb2RlQiAiSGV5LCBJIGhhdmUgc29t ZSBkYXRhIGluIHRoZSBidWZmZXIgdG8gc2VuZCIuIFRoZQ0KPiA+IGVOb2RlQiBoYXMgbm90IGtu b3dsZWRnZSBob3cgbXVjaCBkYXRhIHRoZSBtb2JpbGUgaGFzIHRvIHNlbmQsDQo+ID4gdGhlcmVm b3Igd2lsbCBhbGxvY2F0ZSBzb21lIG1pbmltdW0NCj4gPiBjYXBhY2l0eTogIldlbGwsIEkgZ2l2 ZSB5b3UgYSBncmFudCB0byBzZW5kIHVwIHRvIDIwMCBCeXRlcyBhbmQgeW91DQo+ID4gbXVzdCBz ZW5kIHRoZW0gZXhhY3RseSBhZnRlciA0IG1zIHlvdSBnZXQgdGhpcyBub3RpZmljYXRpb24iLiBU aGVzZSA0DQo+ID4gbXMgYXJlIGFsbG93YW5jZSBmb3IgbW9iaWxlIGRldmljZSB0byBwcmVwYXJl IGRhdGEgZm9yIHNlbmRpbmcgd2hpY2gNCj4gPiBpcyBhIHZlcnkgY29tcGxpY2F0ZWQgcHJvY2Vz cywgdGhlc2UgNCBtcyBpcyBhIG11c3QgZGVsYXkgYmV0d2VlbiB0aGUgZ3JhbnQNCj4gYW5kIHRo ZSB0cmFuc21pc3Npb24uDQo+ID4gSWYgbW9iaWxlIGhhcyBpbiBpdHMgYnVmZmVyIDEwIEFDSyB0 byBzZW5kIGl0IHdpbGwgc2VuZCBvbmx5IDIgQUNLIGFuZA0KPiA+IHdpbGwgY29tcGxlbWVudCB0 aGVtIHdpdGgsIHNvIGNhbGxlZCwgYnVmZmVyIHN0YXR1cyByZXBvcnQ6ICJJIGRvIGhhdmUNCj4g PiBzdGlsbCA0ODAgYnl0ZXMgdG8gc2VuZCIuIE5vdyBuZXR3b3JrIGhhcyBtb3JlIGluZm9ybWF0 aW9uIGFuZA0KPiA+IGFsbG9jYXRlIGFzDQo+ID4gcmVxdWVzdGVkOiAiV2VsbCwgSSBnaXZlIHlv dSBhIGdyYW50IHRvIHNlbmQgdXAgdG8gMjAwIEJ5dGVzIGFuZCB5b3UNCj4gPiBtdXN0IHNlbmQg dGhlbSBleGFjdGx5IGFmdGVyIDQgbXMgeW91IGdldCB0aGlzIG5vdGlmaWNhdGlvbiIuIEJ1dCwg aW4NCj4gPiB0aGUgbWVhbiB0aW1lIHNvbWUgb3RoZXIgaW5mb3JtYXRpb24gb2YgaGlnaGVyIHBy aW9yaXR5IG1heSBhcHBlYXIgaW4gdGhlDQo+IGRldmljZSAoZS5nLg0KPiA+IHNpZ25hbGluZykg YW5kIHRoZSBkZXZpY2Ugd2lsbCBzZW5kIG9ubHkgNiBvdXQgb2YgOCByZW1haW5pbmcgQUNLDQo+ ID4gdG9nZXRoZXIgd2l0aCBzaWduYWxpbmcgYW5kIGFub3RoZXIgQnVmZmVyIFN0YXR1cyBSZXBv cnQ6ICJIZXksIEkgZG8NCj4gPiBoYXZlIGFub3RoZXIgMTIwIGJ5dGVzIHRvIHNlbmQiLiBBbmQg YWdhaW4gIldlbGwsIEkgZ2l2ZSB5b3UgYSBncmFudA0KPiA+IHRvIHNlbmQgdXAgdG8gMTIwIEJ5 dGVzIGFuZCB5b3UgbXVzdCBzZW5kIHRoZW0gZXhhY3RseSBhZnRlciA0IG1zIHlvdQ0KPiA+IGdl dCB0aGlzIG5vdGlmaWNhdGlvbiIuIEFzIGEgcmVzdWx0IHRoZSBzZXJ2ZXIgd2lsbCByZWNlaXZl IDIgQUNLLA0KPiA+IHRoZW4gYWZ0ZXIgYSBnYXAgb2YgNiBtcyBhbm90aGVyIDYgQUNLLCB0aGVu LCBhZnRlciBhbm90aGVyIDYgbXMsIHRoZQ0KPiA+IHJlbWFpbmluZyAyIEFDSy4gV2l0aCB0aGUg c2NoZWR1bGVyIGluIHRoZSBlTm9kZUIgdGhlc2UgZ2FwcyBhcmUNCj4gPiBpbmV2aXRhYmxlLiBB bmQgb25lIG1vcmUgdGhpbms6IG1vYmlsZSBwYWNrIHRvZ2V0aGVyIHRob3NlIDYgQUNLIGluIDEN Cj4gPiBUVEkgb2YgMSBtcyBhbmQgc2VuZCB0aGVtLCB0aGVuIHRoZSBlTm9kZUIgdW5wYWNrIHRo ZW0gYW5kIHRoZXJlIGlzIG5vDQo+ID4gd2F5IHRvIHJlY292ZXIgdGhlIHNwYWNpbmcgb2YgMC4x MiBtcyBiZXR3ZWVuIHRoZW0uIGVOb2RlQiBzaW1wbHkgc2VuZA0KPiA+IHRoZW0gdG8gdGhlIHNl cnZlciBiYWNrIHRvIGJhY2sgYXQgdGhlIHNwZWVkIG9mIGl0cyBkYXRhIGNhcmQgd2hpY2ggaXMN Cj4gPiB0eXBpY2FsbHkgMSBHYnBzIGFuZCB3ZSBmYWNlIHRoZSBBQ0sgY29tcHJlc3Npb24uIEFm dGVyIHRoZSBsYXN0IDIgQUNLDQo+ID4gd2VyZSBzZW50IHRoZXJlIGlzIG5vdGhpbmcgZWxzZSB0 byBzZW5kLCBzbyB0aGVyZSBpcyBubyBCdWZmZXIgU3RhdHVzDQo+ID4gUmVwb3J0LCB0aGVyZWZv cmUsIGFmdGVyIHBhY2tldHMgb2YgdGhlIG5leHQgdHJhaW4gYXJyaXZlIGFuZCBBQ0sgYXJlDQo+ ID4gZ2VuZXJhdGVkIGV2ZXJ5dGhpbmcgc3RhcnRzIGFnYWluIHdpdGggb25lIGJpdCBvZiBpbmZv cm1hdGlvbiAiSGV5LCBJIGhhdmUNCj4gc29tZSBkYXRhIGluIHRoZSBidWZmZXIgdG8gc2VuZCIu DQo+ID4gVGhhdCdzIHdoeSB0aGVyZSBhcmUgZ2FwcyBhbmQgQUNLIGNvbXByZXNzaW9uLCBsb3Nz LCBvciBtb3JlIGV4YWN0bHksDQo+ID4gbGFjayBvZiBhbnkgdGltaW5nIGluZm9ybWF0aW9uIHdo aWNoIHdvdWxkIGJlIHVzZWZ1bCBmb3IgdGhlIFRDUCBzZWxmLQ0KPiBjbG9ja2luZy4NCj4gPg0K PiA+IFRoYW5rIHlvdSBhbmQgc29ycnkgZm9yIHRha2luZyB5b3VyIHRpbWUuDQo+ID4NCj4gPg0K PiA+IFZlYWNlc2xhdiBSb21hbg0KPiA+DQo+ID4NCj4gPg0KPiA+IC0tLS0tT3JpZ2luYWwgTWVz c2FnZS0tLS0tDQo+ID4gRnJvbTogVmVhY2VzbGF2IFJPTUFODQo+ID4gU2VudDogU2F0dXJkYXks IDA1IFNlcHRlbWJlciAyMDE1IDAxOjEwDQo+ID4gVG86ICdFcmljIER1bWF6ZXQnOyBOZWFsIENh cmR3ZWxsDQo+ID4gQ2M6IHRjcG1AaWV0Zi5vcmc7IFBpZXJzIE8nSGFubG9uOyBTYW5ndGFlIEhh OyBJbmdlbWFyIEpvaGFuc3NvbiBTOw0KPiA+IEVyaWMgRHVtYXpldDsgZW5kMmVuZC1pbnRlcmVz dEBwb3N0ZWwub3JnDQo+ID4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBw YXRjaCBkZXBsb3ltZW50DQo+ID4NCj4gPiBJZiB3ZSBsb29rIGF0IHRoZSBmaXJzdCB0cmFpbjog YWxsIHBhY2tldHMgcmVjZWl2ZWQgaW4gbGVzcyB0aGFuIDEgbXMuDQo+ID4gUHJvYmFibHkgdGhp cyBpcyBvbmx5IGFuIGFwcGVhcmFuY2UgYXMgTFRFIHRyYW5zbWl0cyBpbiwgc28gY2FsbGVkDQo+ ID4gdHJhbnNtaXNzaW9uIHRpbWUgaW50ZXJ2YWwgKFRUSSkgb2YgMSBtcywgYW5kIHdoYXQgd2Ug c2VlIGhlcmUgaXMgdGhhdA0KPiA+IGFsbCAxMCBwYWNrZXRzIG9mIGluaXRpYWwgd2luZG93IGZp dHRlZCBpbiAxIG1zLCBhbmQsIHdoZW4gZGVjb2RlZCwNCj4gPiB3ZXJlIHByZXNlbnRlZCB0byB0 aGUgVENQL0lQIGxheWVyIChhbmQgcGNhcCkgYWxsIGF0IG9uY2UuIEJUVywgOA0KPiA+IHBhY2tl dHMgb2YgMTMwMjIgYnl0ZXMgaW4gMSBtcyBtZWFucyBpbnN0YW50YW5lb3VzIHNwZWVkIG9mIDEw NCBNYnBzLA0KPiA+IGdvb2QgcmFkaW8gY29uZGl0aW9ucy4gVENQIGdlbmVyYXRlcyAxMCBBQ0sg dHJhaW4gb2YgdGhlIGR1cmF0aW9uIG9mDQo+ID4gMS4yIG1zLiBXaWxsIGl0IGJlIGEgRmFzdCBF dGhlcm5ldCBwb3NzaWJseSB3ZSB3b3VsZCBjb25zaWRlciB0aGlzIG5vcm1hbCwNCj4gaXNuJ3Qg aXQgPw0KPiA+IEkgbG9va2VkIGF0IGhvdyBzZXJ2ZXIgcmVwbHkgdG8gdGhlc2UgMTAgQUNLcy4g VGhlcmUgYXJlIDMgZ2FwcyBvZiA0LA0KPiA+IDIgYW5kIDUgbXMgaW4gdGhlIHJlcGx5IHRyYWlu IGFuZCB0aGUgdG90YWwgdHJhaW4gb2YgMTggKD8sIGl0IHNob3VsZA0KPiA+IGJlIDIwKSBwYWNr ZXRzIHJlYWNoZXMgYSBkdXJhdGlvbiBvZiAxNCBtcy4gQUZBSUsgTFRFIG1heSBpbnRyb2R1Y2UN Cj4gPiBnYXBzIGluIEFDSyB0cmFpbiBkdWUgdG8gdXBsaW5rIHNjaGVkdWxpbmcgbWVjaGFuaXNt Lg0KPiA+IE1heSBiZSB0aGVzZSBnYXBzIHRyaWdnZXIgSHlzdGFydCBlYXJseSBleGl0ID8NCj4g Pg0KPiA+IFZlYWNlc2xhdiBSb21hbg0KPiA+IFRlY2huaWNhbCBhbmQgSVQgZGlyZWN0b3INCj4g PiBPcmFuZ2UgTW9sZG92YSBTLkEuDQo+ID4gRml4OiArMzczMjI1NzU0MDANCj4gPiBNb2I6ICsz NzM2OTE5ODQwMA0KPiA+IEZheDogKzM3MzIyNTc1MzA2DQo+ID4NCj4gPiAtLS0tLU9yaWdpbmFs IE1lc3NhZ2UtLS0tLQ0KPiA+IEZyb206IEVyaWMgRHVtYXpldCBbbWFpbHRvOmVyaWMuZHVtYXpl dEBnbWFpbC5jb21dDQo+ID4gU2VudDogRnJpZGF5LCAwNCBTZXB0ZW1iZXIgMjAxNSAxOTo0OA0K PiA+IFRvOiBOZWFsIENhcmR3ZWxsDQo+ID4gQ2M6IFZlYWNlc2xhdiBST01BTjsgdGNwbUBpZXRm Lm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0YWUgSGE7DQo+ID4gSW5nZW1hciBKb2hhbnNzb24g UzsgRXJpYyBEdW1hemV0OyBlbmQyZW5kLWludGVyZXN0QHBvc3RlbC5vcmcNCj4gPiBTdWJqZWN0 OiBSZTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4gPg0KPiA+ IE9uIEZyaSwgMjAxNS0wOS0wNCBhdCAxMTozMiAtMDQwMCwgTmVhbCBDYXJkd2VsbCB3cm90ZToN Cj4gPiA+IEhpIFZlYWNlc2xhdiwNCj4gPiA+DQo+ID4gPiBJIGFncmVlIHRoYXQgaW4geW91ciBM VEUgdHJhY2VzIGl0IGxvb2tzIGxpa2UgQ1VCSUMgSHlzdGFydCBpcw0KPiA+ID4gZXhpdGluZyBz bG93IHN0YXJ0IHRvbyBlYXJseS4NCj4gPiA+DQo+ID4gPiBTSW5jZSB5b3Ugc2VlbSB0byBoYXZl IGEgbmljZSBMVEUgdGVzdGJlZCwgd291bGQgeW91IGJlIGFibGUgdG8gZG8NCj4gPiA+IHNvbWUg ZXhwZXJpbWVudHMgdG8gZmluZCBhIHNldCBvZiBwYXJhbWV0ZXJzIGZvciBIeXN0YXJ0IHRoYXQg d29yaw0KPiA+ID4gYmV0dGVyIGZvciB5b3VyIExURSBlbnZpcm9ubWVudD8gRm9yIGV4YW1wbGUs IHlvdSBtaWdodCB0cnkgdGhlIHR3bw0KPiA+ID4gdmFyaWF0aW9ucyBJIHN1Z2dlc3RlZCBlYXJs aWVyIGluIHRoZSB0aHJlYWQ6DQo+ID4gPg0KPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAg aWYgKGNhLT5jdXJyX3J0dCA+IGNhLT5kZWxheV9taW4gKw0KPiA+ID4gICAgICAgICAgICAgICAg ICAgICAgICAgICAgIEhZU1RBUlRfREVMQVlfVEhSRVNIKGNhLT5kZWxheV9taW4gPj4NCj4gPiA+ IDIpKSB7IG9yDQo+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICBpZiAoY2EtPmN1cnJfcnR0 ID4gY2EtPmRlbGF5X21pbiArDQo+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgSFlT VEFSVF9ERUxBWV9USFJFU0goY2EtPmRlbGF5X21pbiA+Pg0KPiA+ID4gMSkpIHsNCj4gPiA+DQo+ ID4gPiBEbyBhbnkgb2YgdGhvc2UgZ2l2ZSBiZXR0ZXIgcmVzdWx0cyBmb3IgeW91ciB0ZXN0cz8N Cj4gPiA+DQo+ID4gPiBuZWFsDQo+ID4NCj4gPiBBbHNvLCBoeXN0YXJ0IGlzIGZvb2xlZCBieSB0 b28gbWFueSBBQ0sgcmVjZWl2ZWQgaW4gc2hvcnQgcGVyaW9kLg0KPiA+DQo+ID4gVGhpcyBwcm9i bGVtIHdvdWxkIGJlIHNvbHZlZCBpZiBHUk8gd2FzIHVzZWQgYXQgcmVjZWl2ZXIsIGFzIGxlc3Mg QUNLDQo+ID4gd291bGQgYmUgc2VudC4NCj4gPg0KPiA+IFByZXN1bWFibHkgcmVjZWl2ZXIgaXMg bm90IGEgbGludXggVENQIHN0YWNrID8NCj4gPg0KPiA+IE1heWJlIHdlIHNob3VsZCBhZGQgYSBs b2dpYyBpbiBoeXN0YXJ0X3VwZGF0ZSgpIHRvIHRha2Ugb25lIEFDSyBwZXIgbXMuDQo+ID4NCj4g PiAwNTozMjoyNC4xMzQxNzIgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIw NC44MDogRmxhZ3MNCj4gPiBbU10sIHNlcSA0Mjk0NDIzMjE1LCB3aW4gNjU1MzUsIG9wdGlvbnMg W21zcyAxNDYwLHNhY2tPSyxUUyB2YWwNCj4gPiAxMTEyMzUxIGVjciAwLG5vcCx3c2NhbGUgOF0s IGxlbmd0aCAwDQo+ID4gMDU6MzI6MjQuMTYxOTg2IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQu MjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdzDQo+ID4gW1MuXSwgc2VxIDExOTg4Nzc3MjEsIGFjayA0 Mjk0NDIzMjE2LCB3aW4gMjg5NjAsIG9wdGlvbnMgW21zcw0KPiA+IDE0MTYsc2Fja09LLFRTIHZh bA0KPiA+IDI4MDg4MDg2MzQgZWNyIDExMTIzNTEsbm9wLHdzY2FsZSA3XSwgbGVuZ3RoIDANCj4g PiAwNTozMjoyNC4xNjIxMDggSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIw NC44MDogRmxhZ3MNCj4gPiBbLl0sIGFjayAxLCB3aW4gMzQzLCBvcHRpb25zIFtub3Asbm9wLFRT IHZhbCAxMTEyMzU0IGVjciAyODA4ODA4NjM0XSwNCj4gPiBsZW5ndGggMA0KPiA+IDA1OjMyOjI0 LjE2MzA3MiBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFn cw0KPiA+IFtQLl0sIHNlcSAxOjY1OCwgYWNrIDEsIHdpbiAzNDMsIG9wdGlvbnMgW25vcCxub3As VFMgdmFsIDExMTIzNTQgZWNyDQo+ID4gMjgwODgwODYzNF0sIGxlbmd0aCA2NTcNCj4gPiAwNToz MjoyNC4yMDM5ODQgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDog RmxhZ3MNCj4gPiBbLl0sIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFs IDI4MDg4MDg2NzUgZWNyDQo+ID4gMTExMjM1NF0sIGxlbmd0aCAwDQo+ID4gMDU6MzI6MjQuMjEw Mjc1IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdzDQo+ ID4gW1AuXSwgc2VxIDE6Mzg3LCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRT IHZhbCAyODA4ODA4Njc3DQo+ID4gZWNyIDExMTIzNTRdLCBsZW5ndGggMzg2DQo+ID4gMDU6MzI6 MjQuMjEwMzE1IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IEZs YWdzDQo+ID4gWy5dLCBzZXEgMzg3OjE3OTEsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25v cCxub3AsVFMgdmFsDQo+ID4gMjgwODgwODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0aCAxNDA0DQo+ ID4gMDU6MzI6MjQuMjEwMzMyIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYu MzI5MjQ6IEZsYWdzDQo+ID4gWy5dLCBzZXEgMTc5MTozMTk1LCBhY2sgNjU4LCB3aW4gMjM3LCBv cHRpb25zIFtub3Asbm9wLFRTIHZhbA0KPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5n dGggMTQwNA0KPiA+IDA1OjMyOjI0LjIxMDM0NyBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0 My4xMDQuMTU2LjMyOTI0OiBGbGFncw0KPiA+IFsuXSwgc2VxIDMxOTU6NDU5OSwgYWNrIDY1OCwg d2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4gPiAyODA4ODA4Njc3IGVjciAxMTEy MzU0XSwgbGVuZ3RoIDE0MDQNCj4gPiAwNTozMjoyNC4yMTAzNTAgSVAgOTQuMjQzLjEwNC4xNTYu MzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MNCj4gPiBbLl0sIGFjayAzODcsIHdpbiAz NDcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyDQo+ID4gMjgwODgwODY3N10s IGxlbmd0aCAwDQo+ID4gMDU6MzI6MjQuMjEwMzYxIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQu MjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdzDQo+ID4gWy5dLCBzZXEgNDU5OTo2MDAzLCBhY2sgNjU4 LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbA0KPiA+IDI4MDg4MDg2NzcgZWNyIDEx MTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+IDA1OjMyOjI0LjIxMDM3NCBJUCAxOTUuOTUuMTc4LjIw NC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncw0KPiA+IFsuXSwgc2VxIDYwMDM6NzQw NywgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4gPiAyODA4ODA4 Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPiAwNTozMjoyNC4yMTAzODkgSVAgMTk1 Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MNCj4gPiBbLl0sIHNl cSA3NDA3Ojg4MTEsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ ID4gMjgwODgwODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0aCAxNDA0DQo+ID4gMDU6MzI6MjQuMjEw NDAyIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdzDQo+ ID4gWy5dLCBzZXEgODgxMToxMDIxNSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5v cCxUUyB2YWwNCj4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPiAw NTozMjoyNC4yMTA0MTYgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjky NDogRmxhZ3MNCj4gPiBbLl0sIHNlcSAxMDIxNToxMTYxOSwgYWNrIDY1OCwgd2luIDIzNywgb3B0 aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3Ro IDE0MDQNCj4gPiAwNTozMjoyNC4yMTA0MTggSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUu OTUuMTc4LjIwNC44MDogRmxhZ3MNCj4gPiBbLl0sIGFjayAxNzkxLCB3aW4gMzU4LCBvcHRpb25z IFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjcg0KPiA+IDI4MDg4MDg2NzddLCBsZW5ndGggMA0K PiA+IDA1OjMyOjI0LjIxMDQzMSBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2 LjMyOTI0OiBGbGFncw0KPiA+IFsuXSwgc2VxIDExNjE5OjEzMDIzLCBhY2sgNjU4LCB3aW4gMjM3 LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbA0KPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBs ZW5ndGggMTQwNA0KPiA+IDA1OjMyOjI0LjIxMDQ1NSBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+ IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncw0KPiA+IFsuXSwgYWNrIDMxOTUsIHdpbiAzNjksIG9w dGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyDQo+ID4gMjgwODgwODY3N10sIGxlbmd0 aCAwDQo+ID4gMDU6MzI6MjQuMjEwNjg1IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1 LjE3OC4yMDQuODA6IEZsYWdzDQo+ID4gWy5dLCBhY2sgNDU5OSwgd2luIDM4MCwgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3INCj4gPiAyODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4g PiAwNTozMjoyNC4yMTA3MzEgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIw NC44MDogRmxhZ3MNCj4gPiBbLl0sIGFjayA2MDAzLCB3aW4gMzkxLCBvcHRpb25zIFtub3Asbm9w LFRTIHZhbCAxMTEyMzU5IGVjcg0KPiA+IDI4MDg4MDg2NzddLCBsZW5ndGggMA0KPiA+IDA1OjMy OjI0LjIxMDkzNyBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBG bGFncw0KPiA+IFsuXSwgYWNrIDc0MDcsIHdpbiA0MDIsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFs IDExMTIzNTkgZWNyDQo+ID4gMjgwODgwODY3N10sIGxlbmd0aCAwDQo+ID4gMDU6MzI6MjQuMjEx MDc4IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzDQo+ ID4gWy5dLCBhY2sgODgxMSwgd2luIDQxMywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1 OSBlY3INCj4gPiAyODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4gPiAwNTozMjoyNC4yMTEyMjggSVAg OTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MNCj4gPiBbLl0s IGFjayAxMDIxNSwgd2luIDQyNCwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3IN Cj4gPiAyODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4gPiAwNTozMjoyNC4yMTEzNzcgSVAgOTQuMjQz LjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MNCj4gPiBbLl0sIGFjayAx MTYxOSwgd2luIDQzNSwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3INCj4gPiAy ODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4gPiAwNTozMjoyNC4yMTE1NDQgSVAgOTQuMjQzLjEwNC4x NTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MNCj4gPiBbLl0sIGFjayAxMzAyMywg d2luIDQ0Niwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3INCj4gPiAyODA4ODA4 Njc3XSwgbGVuZ3RoIDANCj4gPiAwNTozMjoyNC4yMzc5OTkgSVAgMTk1Ljk1LjE3OC4yMDQuODAg PiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MNCj4gPiBbLl0sIHNlcSAxMzAyMzoxNDQyNywg YWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4gPiAyODA4ODA4NzA5 IGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4gPiAwNTozMjoyNC4yMzgwNTMgSVAgMTk1Ljk1 LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MNCj4gPiBbLl0sIHNlcSAx NDQyNzoxNTgzMSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4g PiAyODA4ODA4NzA5IGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4gPiAwNTozMjoyNC4yMzgw OTAgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MNCj4g PiBbLl0sIGFjayAxNDQyNywgd2luIDQ1Nywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM2 MiBlY3INCj4gPiAyODA4ODA4NzA5XSwgbGVuZ3RoIDANCj4gPiAwNTozMjoyNC4yMzgxNjQgSVAg OTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MNCj4gPiBbLl0s IGFjayAxNTgzMSwgd2luIDQ2OCwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM2MiBlY3IN Cj4gPiAyODA4ODA4NzA5XSwgbGVuZ3RoIDANCj4gPiAwNTozMjoyNC4yNDQwMjQgSVAgMTk1Ljk1 LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MNCj4gPiBbLl0sIHNlcSAx NTgzMToxNzIzNSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4g PiAyODA4ODA4NzEzIGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4gPiAwNTozMjoyNC4yNDQw NjMgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MNCj4g PiBbLl0sIHNlcSAxNzIzNToxODYzOSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5v cCxUUyB2YWwNCj4gPiAyODA4ODA4NzEzIGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4gPg0K DQo= From nobody Thu Oct 1 04:46:04 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 771E81A1F04 for ; Thu, 1 Oct 2015 04:46:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.144 X-Spam-Level: X-Spam-Status: No, score=0.144 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, FRT_BELOW2=2.154, J_CHICKENPOX_36=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AIHmXYzomth6 for ; Thu, 1 Oct 2015 04:45:58 -0700 (PDT) Received: from mailfilter.orange.md (mailfilter.orange.md [94.243.64.204]) by ietfa.amsl.com (Postfix) with ESMTP id 30BA71A1EFC for ; Thu, 1 Oct 2015 04:45:56 -0700 (PDT) Received: from localhost.localdomain (antispam.orange.md [127.0.0.1]) by localhost (Postfix) with ESMTP id 9DF1D27E038; Thu, 1 Oct 2015 14:46:14 +0300 (EEST) Received: from antispam.orange.md by antispam.orange.md with queue id 249680-1; Thu, 01 Oct 2015 11:46:14 GMT Received: from XCHSRV03.main.orange.md (unknown [192.168.200.63]) by mailfilter.orange.md (Postfix) with ESMTP id 3EAD93B26FD; Thu, 1 Oct 2015 14:46:14 +0300 (EEST) Received: from XCHSRV01.main.orange.md ([fe80::685f:aef6:93b0:dea7]) by XCHSRV03.main.orange.md ([fe80::6dbc:28dd:213b:8931%14]) with mapi id 14.02.0328.009; Thu, 1 Oct 2015 14:45:54 +0300 From: Veaceslav ROMAN To: Ingemar Johansson S , Eric Dumazet , Neal Cardwell Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8A== Date: Thu, 1 Oct 2015 11:45:53 +0000 Message-ID: <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> In-Reply-To: <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> Accept-Language: en-US, ro-RO Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.107.165] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Cc: Eric Dumazet , "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2015 11:46:03 -0000 V2VsbCB0aGlzIHNheSANCglJZiBjdXJyX3J0dCBpcyBoaWdoZXIgdGhhbiBkZWxheV9taW4gKyAo MS84ICogZGVsYXlfbWluIG9yIDQgbXMsIHdoaWNoZXZlciBpcyBoaWdoZXIpICB0aGVuIGV4aXQg c2xvdyBzdGFydC4NCg0KV2hlbiBhY3R1YWwgbWluaW11bSBkZWxheSBpcyBsZXNzIHRoYW4gMzIg bXMgdGhlIGRlbGF5X21pbiB3aWxsIGJlIGxlc3MgdGhhbiAzMjw8MywgdGhlbiB0aGUgZGVsYXlf bWluPj4zIHdpbGwgYmUgbGVzcyB0aGFuIDMyIGFuZCB3aWxsIGNvbXBhcmUgd2l0aCA0PDwzIHdo aWNoIGlzIDMyIGFuZCB0aGVyZWZvciB0aGUgdGhyZXNob2xkIHdpbGwgYmUgKHJlYWxfY3Vycl9y dHQ8PDMpID4gKHJlYWxfZGVsYXk8PDMgKyA0PDwzKSBvciAocmVhbF9jdXJyX3J0dCk+KHJlYWxf ZGVsYXkgKzQpDQogDQoNClZlYWNlc2xhdiBSb21hbg0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2Ut LS0tLQ0KRnJvbTogSW5nZW1hciBKb2hhbnNzb24gUyBbbWFpbHRvOmluZ2VtYXIucy5qb2hhbnNz b25AZXJpY3Nzb24uY29tXSANClNlbnQ6IFRodXJzZGF5LCAwMSBPY3RvYmVyIDIwMTUgMTM6MDAN ClRvOiBWZWFjZXNsYXYgUk9NQU47IEVyaWMgRHVtYXpldDsgTmVhbCBDYXJkd2VsbA0KQ2M6IHRj cG1AaWV0Zi5vcmc7IFBpZXJzIE8nSGFubG9uOyBTYW5ndGFlIEhhOyBFcmljIER1bWF6ZXQ7IGVu ZDJlbmQtaW50ZXJlc3RAcG9zdGVsLm9yZzsgSW5nZW1hciBKb2hhbnNzb24gUw0KU3ViamVjdDog UkU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBwYXRjaCBkZXBsb3ltZW50DQoNCkhpDQoNClll cywgdGhlIHZhcmlhYmxlIGRlbGF5X21pbiBpcyBpbiBRMyBmb3IgcmVhc29ucyBvZiBwcmVjaXNp b24gSSBndWVzcyBCdXQgdGhlIGNhbGwgdG8gSFlTVEFSVF9ERUxBWV9USFJFU0ggaXMgd2l0aCBk ZWxheV9taW4gc2hpZnRlZCBkb3duIHRvIFEwDQoNCkhZU1RBUlRfREVMQVlfVEhSRVNIKGNhLT5k ZWxheV9taW4gPj4gMykpICAoc2VlIGh0dHA6Ly9seHIuZnJlZS1lbGVjdHJvbnMuY29tL3NvdXJj ZS9uZXQvaXB2NC90Y3BfY3ViaWMuYyNMNDAzICkgSSBtYXkgYmUgb2ZmIGhlcmUuLiANCg0KUmVn YXJkaW5nIHBhY2luZy4gSSBkb24ndCBoYXZlIHJlYWwgZGF0YSB0byBzaG93IHRoZSBlZmZlY3Rz IG9mIHBhY2luZywgaW4gc2ltdWxhdGlvbnMgaXQgaXMgaG93ZXZlciBwb3NzaWJsZSB0byBzZWUg dGhlIGdhaW5zIGluIHRlcm1zIG9mIGxlc3MgY29hbGVzY2luZywgeWVzIHBhY2luZyBkb2VzIG5v dCBzb2x2ZSB0aGUgQUNLIGNvbXByZXNzaW9uIGJ1dCBpdCBzZWVtIHRvIHNvbHZlIHRoZSBmb2xs b3cgdXAgZWZmZWN0IHRoYXQgQUNLIGNvbXByZXNzaW9uIGdpdmVzLg0KDQovSW5nZW1hcg0KDQo+ IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFZlYWNlc2xhdiBST01BTiBbbWFp bHRvOlZlYWNlc2xhdi5Sb21hbkBvcmFuZ2UubWRdDQo+IFNlbnQ6IGRlbiAxIG9rdG9iZXIgMjAx NSAxMDozMg0KPiBUbzogSW5nZW1hciBKb2hhbnNzb24gUzsgRXJpYyBEdW1hemV0OyBOZWFsIENh cmR3ZWxsDQo+IENjOiB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsg RXJpYyBEdW1hemV0OyBlbmQyZW5kLSANCj4gaW50ZXJlc3RAcG9zdGVsLm9yZw0KPiBTdWJqZWN0 OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4gDQo+IFll cywgSFlTVEFSVF9ERUxBWV9NSU4gICg0VTw8MykgLg0KPiBCdXQgY3Vycl9ydHQgYW5kIGRlbGF5 X21pbiBjb21lIGZyb20gYmljdGNwX2Fja2VkIHdoaWNoIGlzIGFsc28gPDwzIDoNCj4gCUluIHRj cF9jdWJpYy5jLCBmdW5jdGlvbiAgbGluZSA0MzMgKGtlcm5lbCA0LjIpDQo+IAkJZGVsYXkgPSAo cnR0X3VzIDw8IDMpIC8gVVNFQ19QRVJfTVNFQzsgU28sDQo+IDRVPDwzIGlzIGVxdWl2YWxlbnQg dG8gNCBtcyAuDQo+IA0KPiBSZWdhcmRpbmcgcGFjaW5nIEkgYW0gbm90IHN1cmUgd2lsbCBoZWxw IHRoZSBwZXJmb3JtYW5jZTogdGhlcmUgYXJlIA0KPiBnYXBzIGluIEFDSyB0cmFpbnMgb2YgdGhl IG9yZGVyIG9mIG1pbGxpc2Vjb25kcyBhbmQgdGhpcyBpcyBhIGxvc3QgDQo+IHRpbWUgYW5kIHRo ZXJlIGlzIG5vIHdheSB0byBjb21wZW5zYXRlIGl0LiBJZiBpbnRyb2R1Y2UgQUNLIHBhY2luZyBv ciANCj4gZGF0YSBwYWNrZXRzIHBhY2luZyB0aGlzIHdvdWxkIGFkZCB0byB0aGUgb3ZlcmFsbCBs b3N0IHRpbWUuDQo+IFBhY2luZyBjYW4gaGVscCBvbmx5IHRvIGFsbGV2aWF0ZSBhIHBvdGVudGlh bCBwcm9ibGVtIG9mIHNvbWUgDQo+IGRvd25zdHJlYW0gcm91dGVycyB3aXRoIGluc3VmZmljaWVu dCBidWZmZXJzIHdoaWNoIG1heSBpbnRyb2R1Y2UgDQo+IHVuZXhwZWN0ZWQgcGFja2V0cyBkcm9w cyB3aGVuIGxhcmdlIHRyYWluIHNlbnQgYXQgYSBoaWdoIHNwZWVkIGFzIGEgDQo+IHJlcGx5IHRv IGhpZ2hseSBjb21wcmVzc2VkIEFDSyB0cmFpbiwgYnV0LCBvbiBhbm90aGVyIGhhbmRzLg0KPiBC dXQgaG93IG11Y2ggcGFjaW5nIGFuZCBob3cgdG8gYWNjb21tb2RhdGUgdG8gbGFyZ2VseSBjaGFu Z2luZyByYWRpbyANCj4gY29uZGl0aW9ucyA/IEFuZCB3aXRoIHBlcm1hbmVudGx5IGNoYW5naW5n IHRlY2hub2xvZ3kgYW5kIHNwZWVkID8gDQo+IEFueXdheSB0aGUgcGFjaW5nIHdpbGwgYWRkIHRv IHRoZSBwcm9ibGVtIG9mIGdhcHMgYW5kIGJldHRlciBzaXplIA0KPiBwcm9wZXJseSByb3V0ZXJz IGJ1ZmZlcnMuDQo+IA0KPiBJZGVhbGx5IEFDSyB0cmFpbiBpcyBzcGxpdCBpbiAyIHBhcnRzLCBi dXQgaW4gcHJhY3RpY2UgSSBkbyBzZWUgbW9yZS4gDQo+IEJ1dCBhcyB0aGUgc3RyZWFtIHByb2dy ZXNzIHRoZSBsZXNzIGdhcHMuIFRoZSBiaWdnZXIgdGhlIGRvd25saW5rIA0KPiBwcmVzc3VyZSB0 aGUgbW9yZSBBQ0sgYW5kIGxlc3MgY2hhbmNlcyB0byBoYXZlIGEgcGF1c2UgaW4gQUNLIHNlbmRp bmcsIA0KPiB0aGVuIHRoZXJlIHdpbGwgYmUgYSBjb250aW51b3VzIGZsb3cgb2YgQnVmZmVyIFN0 YXR1cyBSZXBvcnRzIGFuZCBsZXNzIA0KPiBvZiB0aGUgU2NoZWR1bGluZyBSZXF1ZXN0IChTUiwg IkhleSwgSSBoYXZlIHNvbWUgZGF0YSB0byBzZW5kIikuIFRoZSANCj4gb25seSBpc3N1ZSBJIGFt IG5vdCBzdXJlIGlzIG9uIHRoZSBwb3RlbnRpYWwgZ2FwcyBpbiBvdGhlciBuZXR3b3Jrcy4gSSAN Cj4gZGlkIG5vdCBwdXQgYmVsbG93IGFsbCBkZXRhaWxzLCBidXQgeW91IHNob3VsZCBrbm93IHRo YXQgZGV2aWNlIGNhbiANCj4gbm90IHNlbmQgU1IgaW4gYXJiaXRyYXJ5IG1vbWVudHMgb2YgdGlt ZSBidXQgaW4gcHJlZGVmaW5lZCBmb3IgaGltIA0KPiBpbnRlcnZhbHMuIEluIG15IG5ldHdvcmsg dGhpcyBwZXJpb2QgaXMgNSBtcywgdGhlcmVmb3JlIHRoZSBSVFQgb2YgdGhlIA0KPiBmaXJzdCBB Q0sgaW4gdGhlIHRyYWluIHdpbGwgaGF2ZSB2YXJpYXRpb25zIG9mIDIuNSBtcy4gQnV0IHN0YW5k YXJkcyANCj4gYWRtaXQgdGhpcyBwZXJpb2QgdG8gYmUgYWxzbyAxMCBtcyAsMjAgbXMsIDQwIG1z LCA4MCBtcyBhbmQgSSBjYW4gbm90IA0KPiBzYXkgd2hhdCBpcyB0aGUgcHJhY3RpY2Ugb2Ygb3Ro ZXJzLiBJIG1heSBvbmx5IGd1ZXNzIHRoYXQgbW9zdCB3aWxsIHRyeSB0byB1c2UgdGhlIGJlc3Qg YW5kIGZhc3Rlc3QsIGkuZS4gNSBtcy4NCj4gQW55d2F5LCB0byBnZXQgYSByZWFsaXN0aWMgbW9k ZWxpbmcgb2YgTFRFIGludGVyYWN0aW9uIHdpdGggVENQIHRoZSANCj4gZGV0YWlsZWQgTUFDIHNj aGVkdWxlciBtb2RlbCBpcyBuZWVkZWQuIFVuZm9ydHVuYXRlbHksIG1vc3Qgb2YgdGhlIA0KPiBs aXRlcmF0dXJlIGZvY3VzIG9uIHNjaGVkdWxpbmcgb2YgdGhlIFBoeXNpY2FsIFJlc291cmNlIEJs b2NrcywgDQo+IEFkYXB0aXZlIE1vZHVsYXRpb24gYW5kIENvZGluZywgTUlNTywgaS5lLiwgZm9j dXNpbmcgb24gdGhlIA0KPiBkaXN0cmlidXRpb24gb2YgcmVzb3VyY2VzIGJldHdlZW4gdXNlcnMs IHdoaWNoIGlzIGdvb2QgYW5kIG5lZWRlZCwgYnV0IA0KPiBkbyBub3QgYWRkcmVzcyB2ZXJ5IG11 Y2ggdGhlIHRpbWluZyBpbnRlcmFjdGlvbiB3aXRoIFRDUC4gIEF0IGxlYXN0LCBJIA0KPiBuZXZl ciBtZXQgaW4gTFRFIHNjaGVkdWxpbmcgbGl0ZXJhdHVyZSBhIHRlcm0gIlRDUCBzZWxmLWNsb2Nr aW5nIi4NCj4gDQo+IFRoZSA4IG1zICBIWVNUQVJUX0RFTEFZX01JTiBjb3VsZCBvbmx5IGJlIGEg c2ltcGxlIHNvbHV0aW9uIGZvciB0aGUgDQo+IHByb2JsZW0gb2YgY291bnRpbmcgb2YgdGhlIHJv dW5kJyBjdXJyX3J0dCBmcm9tIHRoZSBzZWNvbmQgQUNLIG9mIHRoZSANCj4gdHJhaW4gKGFzIHRo ZSBzZWNvbmQgQUNLIFJUVCB3aWxsIGJlLCB0eXBpY2FsbHksIGR1ZSB0byBzY2hlZHVsaW5nLCAN Cj4gNS03IG1zIGxvbmdlciB0aGFuIHRoZSBmaXJzdCBvbmUpLCBidXQgdGhpcyBvbmx5IGlmIFNS IHBlcmlvZGljaXR5IGlzIDUgbXMuDQo+IFRoZSBvdGhlciAyIHByb2JsZW1zIHdpbGwgcmVtYWlu IHVuc29sdmVkLg0KPiANCj4gVmVhY2VzbGF2IFJvbWFuDQo+IA0KPiAtLS0tLU9yaWdpbmFsIE1l c3NhZ2UtLS0tLQ0KPiBGcm9tOiBJbmdlbWFyIEpvaGFuc3NvbiBTIFttYWlsdG86aW5nZW1hci5z LmpvaGFuc3NvbkBlcmljc3Nvbi5jb21dDQo+IFNlbnQ6IFRodXJzZGF5LCAwMSBPY3RvYmVyIDIw MTUgMDg6NTkNCj4gVG86IFZlYWNlc2xhdiBST01BTjsgRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3 ZWxsDQo+IENjOiB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgRXJp YyBEdW1hemV0OyBlbmQyZW5kLSANCj4gaW50ZXJlc3RAcG9zdGVsLm9yZzsgSW5nZW1hciBKb2hh bnNzb24gUw0KPiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRl cGxveW1lbnQNCj4gDQo+IEhpDQo+IA0KPiBUaGFua3MgZm9yIHRoZSBkZXRhaWxlZCByZXBvcnQu IFdoYXQgc2VlbXMgdW5jbGVhciB0byBtZSBpcyB0aGUgdmFsdWUgDQo+IG9mIEhZU1RBUlRfREVM QVlfTUlOLiBZb3UgbWVudGlvbiA4bXMgYnV0IHdoZW4gSSBsb29rIGludG8gdGhlIDQuMiANCj4g a2VybmVsIGNvZGUgSSBnZXQgdGhlIGZlZWxpbmcgdGhhdCBpdCBpcyAzMm1zICggI2RlZmluZSBI WVNUQVJUX0RFTEFZX01JTg0KPiAoNFU8PDMpICAgKS4gSXMgdGhpcyBhIG1pc3Rha2UgZnJvbSBt eSBzaWRlICA/DQo+IEkgaGF2ZSBydW4gTFRFIHNpbXVsYXRpb25zIGFuZCB0cmllZCB0byB3cmlu ZyB0aGlzIEh5U3RhcnQgaXNzdWUgDQo+IGluc2lkZSBvdXQgYW5kIHVwc2lkZSBkb3duIGJ1dCBz b2ZhciBpdCBoYXMgYmVlbiB2ZXJ5IGRpZmZpY3VsdCB0byANCj4gbWFrZSBIeVN0YXJ0IGV4aXQg cHJlbWF0dXJlbHkgYnV0IHRoZW4gb2YgY291cnNlIEkgaGF2ZSBydW4gd2l0aCANCj4gSFlTVEFS VF9ERUxBWV9NSU4gPSAzMm1zLg0KPiANCj4gQXMgcmVnYXJkcyB0byB0aGUgQUNLLWNvbXByZXNz aW9uIHByb2JsZW0sIHllcywgQUNLIGNvbXByZXNzaW9uIGVhc2lseSANCj4gb2NjdXIgaW4gTFRF LCBldmVuIGluIHNpbXVsYXRpb25zLiBXaGF0IEkgaGF2ZSBzZWVuIGlzIHRoYXQgcGFja2V0IA0K PiBwYWNpbmcgaXMgYSB2ZXJ5IGVmZmljaWVudCByZW1lZHksIGZvciBpbnN0YW5jZSB0aGUgcGFj a2V0IGltcGxlbWVudGVkIA0KPiBpbiBRVUlDIHNlZW1zIHRvIHNvbHZlIEFDSyBjb21wcmVzc2lv biBpc3N1ZXMgdmVyeSB3ZWxsLg0KPiANCj4gSXQgaXMgaW50ZXJlc3Rpbmcgd2hhdCB5b3Ugc2F5 IHRoYXQgQUNLIHRyYWlucyBhcmUgc28gbXVjaCBzcGxpdCB1cCwgSSANCj4gY291bGQgdW5kZXJz dGFuZCB0aGF0IHRoZXkgYXJlIHNwbGl0IHVwIGluIHR3byBwYXJ0cyAoaW5pdGlhbCBncmFudCAr IA0KPiBhZGRpdGlvbmFsIGFmdGVyIHJlY2VpdmVkIEJTUikuDQo+IA0KPiAvSW5nZW1hcg0KPiAN Cj4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+IEZyb206IFZlYWNlc2xhdiBST01B TiBbbWFpbHRvOlZlYWNlc2xhdi5Sb21hbkBvcmFuZ2UubWRdDQo+ID4gU2VudDogZGVuIDEgb2t0 b2JlciAyMDE1IDAyOjI1DQo+ID4gVG86IEVyaWMgRHVtYXpldDsgTmVhbCBDYXJkd2VsbA0KPiA+ IENjOiB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgSW5nZW1hciBK b2hhbnNzb24gUzsgDQo+ID4gRXJpYyBEdW1hemV0OyBlbmQyZW5kLWludGVyZXN0QHBvc3RlbC5v cmcNCj4gPiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxv eW1lbnQNCj4gPg0KPiA+IEZpbmFsbHkgY2FuIHNoYXJlIHNvbWUgcmVzdWx0cyBvZiBIeXN0YXJ0 IGludGVyYWN0aW9uIHdpdGggTFRFLg0KPiA+ICBGZXcgd29yZHMgYWJvdXQgdGhlIHRlc3Rpbmcg Y29uZmlndXJhdGlvbjoNCj4gPiAgCURldmljZSBTYW1zdW5nIEdhbGF4eSBOb3RlNExURSAsIHVw IHRvIDE1MCBNYnBzIGNhcGFibGUNCj4gPiAgICAgICAgICAgICAgUmFkaW8gTmV0d29yayBMVEUg MjYwMC8xODAwDQo+ID4gICAgICAgICAgICAgIERpc3RhbmNlIGJldHdlZW4gdGhlIENvcmUgTmV0 d29yayAoRVBDKSBhbmQgQmFzZSBTdGF0aW9uIC0gMTAga20NCj4gPiAgICAgICAgICAgICAgSFRU UCBTZXJ2ZXIgY29ubmVjdGVkIGRpcmVjdGx5IHRvIHRoZSBDb3JlIE5ldHdvcmsgMSBHYnBzDQo+ ID4gICAgICAgICAgICAgIENvcmUgbmV0d29yayBzd2l0Y2hpbmcoRVBDKSAxMCBHYnBzDQo+ID4g ICAgICAgICAgICAgIEJhY2toYXVsIHRyYW5zcG9ydCAgMSBHYnBzLCBmdWxsIElQLCBNUExTDQo+ ID4gICAgICAgICAgICAgIExhc3QgbWlsZSB0cmFuc3BvcnQgZnVsbCBJUCwgTVBMUywgIDMwMCBN YnBzDQo+ID4NCj4gPiAgICAgICAgICAgICBTZXJ2ZXI6IExpbnV4IGs0MHNydiA0LjEuMC0wNDAx MDAtZ2VuZXJpYyAjMjAxNTA3MDMwOTQwIA0KPiA+IFNNUCBGcmkgSnVsIDMNCj4gPiAwOTo0MTo0 NyBVVEMgMjAxNSB4ODZfNjQgeDg2XzY0IHg4Nl82NCBHTlUvTGludXgNCj4gPiAgICAgICAgICAg ICAgICAgICAgICAgICBBcGFjaGUvMi40LjEwIChVYnVudHUpLCBidWlsdDogICBKdWwgMjQgMjAx NSAxNzoyNToxOA0KPiA+DQo+ID4gICAgICAgICAgICBIeXN0YXJ0X2RldGVjdDogMiAoSFlTVEFS VF9ERUxBWSBvbmx5KSwgYXMgc3VnZ2VzdGVkLg0KPiA+ICAgICAgICAgICAgbmV0LmlwdjQudGNw X25vX21ldHJpY3Nfc2F2ZSA9IDEgdG8gYXZvaWQgdGNwIG1ldHJpY3MgDQo+ID4gY2FjaGluZyBp bnRlcmZlcmVuY2UNCj4gPg0KPiA+IAlUZXN0cyB0eXBlczoNCj4gPiAJCTEuIERvd25sb2FkIDMg TUIgZmlsZSwgaW50ZXJjaGFuZ2luZyBIeXN0YXJ0IE9ODQo+IGFuZCBPRkYsIGluDQo+ID4gY29u ZGl0aW9ucyBvZiBMb3cgQmFuZHdpZHRoICgyNS0zNSBNYnBzLCBwb29yIHJhZGlvKSBhbmQgSGln aCANCj4gPiBCYW5kd2lkdGggKDEwMC0xMjAgTWJwcywgZ29vZCByYWRpbykuDQo+ID4gCQkyLiBE b3dubG9hZCAxMCBNQiBmaWxlLCBpbnRlcmNoYW5naW5nIEh5c3RhcnQNCj4gT04gYW5kIE9GRiwg aW4NCj4gPiBjb25kaXRpb25zIG9mIExvdyBCYW5kd2lkdGggKDI1LTM1IE1icHMsIHBvb3IgcmFk aW8pIGFuZCBIaWdoIA0KPiA+IEJhbmR3aWR0aCAoMTAwLTEyMCBNYnBzLCBnb29kIHJhZGlvKS4N Cj4gPg0KPiA+ICAJMTAwIERvd25sb2FkIHRlc3RzIHBlciB0eXBlLg0KPiA+ICAgICAgICAgICAg ICBEb3dubG9hZCBkdXJhdGlvbiBhbmQgYmFuZHdpZHRoIG1lYXN1cmVkIGF0IHRoZSBjbGllbnQg DQo+ID4gc2lkZSBhcyBzaG93biBieSBjdXJsIGZyb20gdGhlIGZpcnN0IHJlY2VpdmVkIGRhdGEg cGFja2V0IHRpbGwgdGhlIA0KPiA+IGVuZCBvZiBzZXNzaW9uICh0aGlzIGV4Y2x1ZGUgRE5TLCAz IHdheSBoYW5kc2hha2UsIGh0dHAgR0VUIGFuZCBpdHMgQUNLKS4NCj4gPiAgICAgICAgICAgICAg SHlzdGFydCBleGl0IGN3bmQgZXh0cmFjdGVkIGZyb20gbnN0YXQgKHRoYW5rIHlvdSBFcmljIA0K PiA+IHRvIHBvaW50aW5nIG91dCB0byBpdCkuDQo+ID4NCj4gPiBPdmVyYWxsIHJlc3VsdHM6DQo+ ID4gCURvd25sb2FkIDNNQiwgTG93IEJhbmR3aWR0aHM6DQo+ID4gCQlIeXN0YXJ0IGF2ZXJhZ2Ug ZG93bmxvYWQgdGltZTogMS40MSBzDQo+ID4gIAkJTm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQg dGltZTogMS4zNiBzDQo+ID4NCj4gPiAJRG93bmxvYWQgM01CLCBIaWdoIEJhbmR3aWR0aHM6DQo+ ID4gCQlIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC42NSBzDQo+ID4gIAkJTm9IeXN0 YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC4zNyAgcw0KPiA+DQo+ID4gCURvd25sb2FkIDEw TUIsIExvdyBCYW5kd2lkdGhzOg0KPiA+IAkJSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6 IDIuNDAgcw0KPiA+ICAJCU5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDIuMTIgcw0K PiA+DQo+ID4gCURvd25sb2FkIDEwTUIsIEhpZ2ggQmFuZHdpZHRoczoNCj4gPiAJCUh5c3RhcnQg YXZlcmFnZSBkb3dubG9hZCB0aW1lOiAxLjQyIHMNCj4gPiAgCQlOb0h5c3RhcnQgYXZlcmFnZSBk b3dubG9hZCB0aW1lOiAwLjg5ICBzDQo+ID4NCj4gPiBUaGVzZSByZXN1bHRzIHNob3cgdGhhdCBI eXN0YXJ0IGhhcyBubyBzaWduaWZpY2FudCBpbXBhY3QgaW4gDQo+ID4gY29uZGl0aW9ucyBvZiBs b3cgYXZhaWxhYmxlIGJhbmR3aWR0aCwgYnV0IGF0IGhpZ2ggYXZhaWxhYmxlIHRoZSANCj4gPiBk ZWNyZWFzZSBvZiBwZXJmb3JtYW5jZSBjYW4gcmVhY2ggNjAtNzAgJS4NCj4gPiBXaXRoIHRoZSBk ZXZlbG9wbWVudCBvZiB0aGUgTFRFLUEgd2hpY2ggdXNlcyBsYXJnZXIgc3BlY3RydW0gdG8gDQo+ ID4gcmVhY2gNCj4gPiAzMDAgTWJwcyBhbmQgbW9yZSB0aGUgaW1wYWN0IG9mIEh5c3RhcnQgd2ls bCBiZWNvbWUgZXZlbiBtb3JlIHZpc2libGUuDQo+ID4NCj4gPiBJIGRvIHNoYXJlIHdpdGggRXJp YyBhbmQgTmVhbCBzb21lIGZpbGVzIGRlc2NyaWJlZCBiZWxvdyBhbmQgYWxzbyANCj4gPiBjYW4g cHJvdmlkZSBsaW5rcyB0byBhbnlvbmUgaW50ZXJlc3RlZC4NCj4gPg0KPiA+IFN1bW1hcnkgb2Yg dGhlc2UgdGVzdHMgcmVzdWx0cyBhcmUgZ2F0aGVyZWQgaW4gdGhlIGZvbGxvd2luZyBFeGNlbCAN Cj4gPiBmaWxlcyAobmFtZXMsIEkgYmVsaWV2ZSwgYXJlIHNlbGYgZXhwbGFuYXRvcnkpOiBUZXN0 XzNNX2xvd19CVy54bHN4LCANCj4gPiBUZXN0XzNNX2hpZ2hfQlcueGxzeCwgVGVzdF8xME1fbG93 X0JXLnhsc3gsDQo+IFRlc3RfMTBNX2hpZ2hfQlcueGxzeC4NCj4gPiBTaW1pbGFybHkgdGNwZHVt cCB0cmFjZXMgYXJlIGNhbGxlZCBUZXN0XzNNX2xvd19CVy5wY2FwLCANCj4gPiBUZXN0XzNNX2hp Z2hfQlcuIHBjYXAsIFRlc3RfMTBNX2xvd19CVy4gcGNhcCwgVGVzdF8xME1faGlnaF9CVy4NCj4g PiBwY2FwIChhYmxlIHRvIGNvbGxlY3Qgb25seSBzZXJ2ZXIgc2lkZSB0cmFjZXMsIGJ1dCB0aGV5 IGFyZSBzZWxmIHN1ZmZpY2llbnQpLg0KPiA+IEZvciB0aGUgMTBNIGRvd25sb2FkcyB0aGVyZSBh cmUgZmV3IG1pc3NpbmcgdHJhY2VzIGFzIHRoZXJlIHRyYWNlIA0KPiA+IGZpbGVzIHdlcmUgcmVj eWNsZWQuIEluIHRyYWNlcyBhcmUgYWxzbyBzc2ggc2Vzc2lvbnMsIHBsZWFzZSwgaWdub3JlIA0K PiA+IHRoZW0sIHRoZXkgd2VyZSB1c2VkIHRvIHB1dCBoeXN0YXJ0IE9OIGFuZCBPRkYgb24gc2Vy dmVyIGFuZCBleHRyYWN0IA0KPiA+IG5zdGF0IHJlc3VsdHMgYmVmb3JlIGFuZCBhZnRlciBlYWNo IGRvd25sb2FkLg0KPiA+DQo+ID4gRXhjZWwgZmlsZXMgaGF2ZSAyIHRhYnM6IGN1cmxfc3VtIGFu ZCBTdW1tYXJ5Lg0KPiA+IEluIHRoZSBjdXJsX3N1bSB0YWIgdGhlIHN0YXQgaXMgY3VtdWxhdGVk IGluIGNocm9ub2xvZ2ljYWwgb3JkZXIgb2YgdGVzdHMuDQo+ID4gSW4gdGhlIFN1bW1hcnlfdGFi ICB0aGUgSHlzdGFydCBhbmQgTm9IeXN0YXJ0IHRlc3RzIGFyZSBzaG93biBzaWRlIA0KPiA+IGJ5 IHNpZGUgKGFzIHNhaWQsIERvd25sb2FkIDEgd2FzIHdpdGggSHlzdGFydCwgRG93bmxvYWQgMiB3 aXRob3V0LCANCj4gPiBEb3dubG9hZCAzIHdpdGggSHlzdGFydCwgRG93bmxvYWQgNCB3aXRob3V0 LCBldGMuKS4gSXQgYWxzbyBpbmNsdWRlIA0KPiA+IGdyYXBocyBvZiBUcmFuc2ZlciB0aW1lIGFu ZCBhIGdyYXBoIHdoaWNoIHNob3dzIHRoZSBkaXN0cmlidXRpb24gaW4gDQo+ID4gJSBvZiBkaWZm ZXJlbnQgSHlzdGFydCBleGl0IHdpbmRvdyBEZWxheUNXTkQuDQo+ID4NCj4gPiBGaWVsZHM6DQo+ ID4gY29tbWVudHMJLSB0ZXN0IGNhc2UNCj4gPiBUcmFuc2ZfVGltZQktIGNhbGN1bGF0ZWQgZnJv bSBjdXJsIG91dHB1dCB0aGUgZHVyYXRpb24NCj4gYmV0d2VlbiBmaXJzdCBkYXRhDQo+ID4gcGFj a2V0IGFuZCBlbmQgb2YgdHJhbnNmZXINCj4gPiBUcmFuc2Zfc3BlZWQJLSBjYWxjdWxhdGVkIGZy b20gY3VybCBvdXRwdXQgdGhlIHJhdGlvIG9mIGZpbGUgc2l6ZQ0KPiBhbmQgZHVyYXRpb24NCj4g PiBiZXR3ZWVuIGZpcnN0IGRhdGEgcGFja2V0IGFuZCBlbmQgb2YgdHJhbnNmZXINCj4gPiBIeXN0 YXJ0CURfRGVsYXkgLSAgaHlzdGFydCBleGl0IG9jY3VycmVuY2UgKGRpZmZlcmVuY2UgYmV0d2Vl bg0KPiA+IFRjcEV4dFRDUEh5c3RhcnREZWxheURldGVjdCBiZWZvcmUgYW5kIGFmdGVyIGRvd25s b2FkKQ0KPiA+IERfRGVsYXlDd25kCSAgLSBIeXN0YXJ0IGV4aXQgY3duZCBmb3IgdGhpcyBkb3du bG9hZA0KPiA+IChkaWZmZXJlbmNlIG9mIFRjcEV4dFRDUEh5c3RhcnREZWxheUN3bmQgYmVmb3Jl IGFuZCBhZnRlciB0aGUNCj4gPiBkb3dubG9hZCkgRGF0ZV9Pbl9TZXJ2ZXIgLSB0aW1lIGp1c3Qg YmVmb3JlIHRoZSBkb3dubG9hZCBzdGFydCwgdG8gDQo+ID4gaGVscCBuYXZpZ2F0ZSB0aGUgcGNh cA0KPiA+DQo+ID4gQWRkaXRpb25hbGx5LCBpbiB0aGUgZmlsZSBUZXN0XzEwTV9sb3dfQlcueGxz eCBpbiB0aGUgdGFiIGN1cmxfc3VtIA0KPiA+IGFyZSBpbmNsdWRlZCBmb3IgZWFjaCBkb3dubG9h ZCB0Y3Auc3RyZWFtIGV4dHJhY3QgZnJvbSBwY2FwIG9mIHRoZSANCj4gPiBmaXJzdA0KPiA+IDEw MCB0Y3AuYW5hbHlzaXMuYWNrX3J0dCB3aGljaCBtYXkgaGVscCBpbiB1bmRlcnN0YW5kaW5nIG9m IHRoZSANCj4gPiBoeXN0YXJ0IGJlaGF2aW9yIChmaWVsZHMgYWNrX3J0dF8xIHRvIGFja19ydHRf MTAwKS4NCj4gPiBJbiB0aGlzIGZpbGUgdGhlcmUgYXJlIGFsc28gZmllbGRzIFVSSV9mcmFtZSwg VVJJX3RpbWUsIFVSSV90aW1lX3JlbGF0aXZlLA0KPiA+IAl0Y3Bfc3RyZWFtLg0KPiA+IFRoZW4g dGhlcmUgYXJlIGNhbGN1bGF0ZWQgZmllbGRzOiBtaW5SVFQgYXMgYSBtaW5pbXVtIG9mIGFsbCAx MDAgQUNLLCByMV81LA0KPiA+IHIyXzgsIHIzXzgsCXI0Xzggd2hpY2ggc2ltdWxhdGUgSHlzdGFy dCBjYWxjdWxhdGlvbnMgb2YgdGhlIHJvdW5kIGN1cnJfcnR0DQo+ID4gYW5kIHIxIG1pbiwgcjIg bWluLCByMyBtaW4sIHI0IG1pbiByZXByZXNlbnQgbWluUlRUIG9mIHRoZSB3aG9sZSByb3VuZC4N Cj4gPiBQbGVhc2UsIGNvcnJlY3QgbWUgaWYgSSBhbSB3cm9uZywgYnV0IGluIG15IHVuZGVyc3Rh bmRpbmcgZm9yIGVhY2ggDQo+ID4gQUNLIHRoZSBoeXN0YXJ0X3VwZGF0ZSBpcyBjYWxsZWQgYmVm b3JlIHRoZSBoeXN0YXJ0X3Jlc2V0LCBhbmQgYXMgYSANCj4gPiByZXN1bHQgdGhlIGNvbXB1dGF0 aW9uIG9mIGN1cl9ydHQgb2YgdGhlIDggcGFja2V0cyBvZiB0aGUgcm91bmQgDQo+ID4gc3RhcnRz IHdpdGggdGhlIDItbmQgcGFja2V0IG9mIHRoZSByb3VuZCwgbm90IHRoZSBmaXJzdCBvbmUuDQo+ ID4gIEkndmUgdXNlIHIxXzUgYXMgYSBtaW4oYWNrX3J0dF83Li4uYWNrX3J0dF8xMSksIGR1ZSB0 byANCj4gPiBoeXN0YXJ0X2xvd193aW5kb3c9MTYsIHIyXzggYXMgYSBtaW4oYWNrX3J0dF8xMiAu Li4gYWNrX3J0dF8xOSksIA0KPiA+IHIzXzggYXMgYQ0KPiA+IG1pbihhY2tfcnR0XzMyIC4uLiBh Y2tfcnR0XzM5KSBhbmQgKSwgcjRfOCBhcyBhIG1pbihhY2tfcnR0XzcyIC4uLg0KPiBhY2tfcnR0 Xzc5KS4NCj4gPiBUbyB1bmRlcnN0YW5kIHRoZSBpbXBhY3Qgb2YgdGhpcyB0aGVyZSBhcmUgYWxz byByMSBIT0wsIHIyIEhPTCwgcjMgDQo+ID4gSE9MLCByNCBIT0wgKEhlYWQgT2YgTGluZSkgd2hp Y2ggY29tcHV0ZSB0aGUgc2FtZSB3aXRoIGZpcnN0IHBhY2tldCANCj4gPiBpbiB0aGUgcm91bmQg aW5jbHVkZWQgaW4gdGhlIHJlbGV2YW50IHJvdW5kOiAgcjFfNSBhcyBhIA0KPiA+IG1pbihhY2tf cnR0XzYuLi5hY2tfcnR0XzEwKSwgcjJfOCBhcyBhIG1pbihhY2tfcnR0XzExIC4uLiANCj4gPiBh Y2tfcnR0XzE4KSwNCj4gPiByM184IGFzIGEgbWluKGFja19ydHRfMzEgLi4uIGFja19ydHRfMzgp IGFuZCApLCByNF84IGFzIGEgbWluKGFja19ydHRfNzEgLi4uDQo+IGFja19ydHRfNzgpLg0KPiA+ DQo+ID4gSWYgbXkgdW5kZXJzdGFuZGluZyBvZiBIeXN0YXJ0IGlzIGNvcnJlY3QgYW5kLCBhc3N1 bWluZyBpbiBlYWNoIA0KPiA+IHRyYWluIHRoZSBmaXJzdCBwYWNrZXQgaGFzIHRoZSBsb3dlc3Qg UlRUIHdoaWNoLCBtb3N0IHByb2JhYmx5IGlzIA0KPiA+IHZhbGlkIGluIGFsbCB0eXBlIG9mIG5l dHdvcmtzLCB0aGVuIHRoZSBleGl0IGZyb20gaHlzdGFydCBpbiB0aGlzIA0KPiA+IGltcGxlbWVu dGF0aW9uIG1heSBoYXBwZW4gYXQgY3duZCAyOSwgNDksIDg5LCAxNjksIGV0Yy4gKGdpdmVuIHRo ZSANCj4gPiBJVyBvZg0KPiAxMCkuDQo+ID4gVGhlc2UgdGVzdHMgc2hvdyB0aGF0IHRoaXMgaXMg dGhlIGNhc2UsIGhvd2V2ZXIgdGhlcmUgYWxzbyANCj4gPiBpbnRlcm1lZGlhdGUgdmFsdWVzLiBI b3dldmVyIHRoZSBjd25kIG9mIDI5IHNoYWxsIGJlIHRoZSBsb3dlc3QgDQo+ID4gcG9zc2libGUg dmFsdWUgYW5kIHRoaXMgaXMgdGhlIGNhc2UgaW4gdGhlc2UgdGVzdHMuIFVuZm9ydHVuYXRlbHks IA0KPiA+IGluIExURSB0aGUgZmlyc3QgcGFja2V0IGluIHRoZSB0cmFpbiBoYXMgY2hhbmNlcyB0 byBhbHdheXMgaGF2ZSB+NiANCj4gPiBtcyBsb3dlciB0aGFuIHRoZSBmb2xsb3dpbmcgb25lcyBh bmQsIGJlY2F1c2UgaXQgaXMgbm90IGNvdW50ZWQgaW4gDQo+ID4gdGhlIDItbmQgdHJhaW4sIHRo ZXJlIGlzIHF1aXRlIGhpZ2ggcGVyY2VudGFnZSBvZiBleGl0IGF0IDI5ICgxNC0zNiUgDQo+ID4g aW4gdGhlc2UgdGVzdHMpLCBldmVuIHRob3VnaCBsYXRlciBvbiBjdWJpYyBpbmNyZWFzZXMgdGhl IGN3bmQgDQo+ID4gd2l0aG91dCBsb3NzZXMgdXAgdG8gbWFueSBodW5kcmVkcy4gRm9yIG1lIHRo aXMgaXMgdG9vIGVhcmx5IGV4aXQgZnJvbSBzbG93IHN0YXJ0Lg0KPiA+IEFzIGl0IGNhbiBiZSBz ZWVuIGZyb20gY29tcGFyaXNvbiBvZiB0aGUgc2ltdWxhdGVkIGNhbGN1bGF0aW9uLCANCj4gPiBp bmNsdXNpb24gb2YgdGhlIGZpcnN0IHBhY2tldCBpbiB0aGUgcm91bmQgY3Vycl9ydHQgY2FsY3Vs YXRpb24gDQo+ID4gd291bGQgZWxpbWluYXRlIHRoZSBjd25kIDI5IGFuZCBzaGlmdCB0byBjd25k IDQ5IG9yIDg5IG9yIGhpZ2hlci4NCj4gPg0KPiA+IEJ1dCB0aGVyZSBhcmUgYWxzbyBpbnRlcm1l ZGlhdGUgdmFsdWVzLCBlLmcuIDY5LiBMb29raW5nIHRocm91Z2ggDQo+ID4gdHJhY2VzIG9uZSBt YXkgc2VlIHRoYXQgdGhpcyBpcyBkdWUgdG8gImNvbXByZXNzZWQgQUNLIiB3aGljaCANCj4gPiBy ZWNlaXZlcyBhIHRpZ2h0bHkgcGFja2VkIHRyYWluIG9mIG1hbnkgQUNLIHNvIHRoYXQgaW4gdHJh Y2VzIHRoZXJlIA0KPiA+IGFyZSBub3QgZGF0YSBwYWNrZXRzIHNlbnQgaW4gdHVybiBieSBzZXJ2 ZXIuIExvb2tzIHRvIG1lLCBiZWNhdXNlIA0KPiA+IGh5c3JhcnRfcmVzZXQgc2V0cyB0aGUgZW5k X3NlcSB0byBzbmRfbmV4dCBhbmQgdGhlIHNlcnZlciBkaWQgbm90IA0KPiA+IG1hbmFnZWQgdG8g c2VuZCB5ZXQgcGFja2V0cyBpbiByZXBseSB0byB0aGlzIGhpZ2ggc3BlZWQgQUNLIHRyYWluIA0K PiA+IHRoZSBzbmRfbmV4dCBwb2ludHMgdG8gbXVjaCBsb3dlciB2YWx1ZSB0aGFuIHdvdWxkIG5v cm1hbGx5IGhhcHBlbiANCj4gPiBhbmQgdGhpcyBhY3R1YWxseSBkZXN0cm95cyB0aGUgY29ycmVj dCBpZGVudGlmaWNhdGlvbiBvZiB0aGUgYm9yZGVycyANCj4gPiBvZiB0aGUgcm91bmQsIHRoZSBy b3VuZCBmYWlscyBzb21ld2hlcmUgaW4gdGhlIG1pZGRsZSBvZiB0aGUgdHJhaW4gDQo+ID4gbm90 IGF0IGl0cyBib3JkZXIsIHRoaXMgbWFrZXMgdmVyeSBsaWtlbHkgaW5jcmVhc2VkIGRlbGF5IGR1 ZSB0byANCj4gPiBxdWV1ZWluZyBhbmQgZWFybGllciBleGl0IGZyb20gc2xvd19zdGFydCBhbmQg aW50ZXJtZWRpYXRlIHZhbHVlcyBvZiANCj4gPiBleGl0IGN3bmQgd2hlcmUNCj4gdGhleSBhcmUg bm90IGV4cGVjdGVkLg0KPiA+DQo+ID4gQW5kIGxhc3QgYnV0IG5vdCBsZWFzdDogdGhlcmUgYXJl IGdhcHMgKDQtNyBtcykgaW4gQUNLIHRyYWlucy4gT24gDQo+ID4gdGhlIHBlcmZlY3RseSBzZW50 IElXIG9mIDEwIGRhdGEgcGFja2V0cyB0cmFpbiB0aGUgdHlwaWNhbCBmb3IgTFRFIA0KPiA+IHdp bGwgYmUgdG8gcmVjZWl2ZSAxIG9yIDIgQUNLLCB0aGVuIGEgZ2FwIG9mIDQtNiBtcyB0aGVuIGFu b3RoZXIgDQo+ID4gY291cGxlIG9mIEFDSywgdGhlbiBhbm90aGVyIGdhcCwgdGhlbiA1IEFDSyBj b21wcmVzc2VkIHRvZ2V0aGVyIGluIGEgDQo+ID4gZnJhY3Rpb24gb2YgbWljcm9zZWNvbmQuIE9m IGNhdXNlLCB0aGUgbmV4dCB0cmFpbiBmcm9tIHRoZSBzZXJ2ZXIgDQo+ID4gd2lsbCBjb250YWlu IHRoZSBzYW1lIGdhcHMuIFZlcnkgcXVpY2tseSwgZS5nLiwgdGhlIEhlYWQgb2YgTGluZSANCj4g PiAoSE9MKSBvZiB0aGUgMy1kIHRyYWluIHdpbGwgZm9sbG93IGltbWVkaWF0ZWx5IHRoZSB0YWls IG9mIHRoZSAyLW5kIA0KPiA+IHRyYWluIHdoaWNoIHdpbGwgbGVhZCB0byBxdWV1ZWluZyBpbiB0 aGUgZW5kIHJvdXRlciAoYWN0dWFsbHkgdGhlIA0KPiA+IHJhZGlvIGJhc2Ugc3RhdGlvbiwgbXVj aCBlYXJsaWVyIHRoYW4gQkRQIHJlYWNoZWQpLCBidXQgdGhpcyB3aWxsIA0KPiA+IGxlYWQgdG8g aW5jcmVhc2Ugb2YgdGhlIHRyYWluIFJUVCBhbmQgdG9vIHllYXJseSBleGl0IGZyb20gc2xvdyAN Cj4gPiBzdGFydC4gV2h5IHRvbyBlYXJseSBleGl0ID8gQmVjYXVzZSwgc2hvdWxkIHNlcnZlciBj b250aW51ZSANCj4gPiBpbmNyZWFzaW5nIHRoZSBzcGVlZCB0aGlzIHdvdWxkIHJlZHVjZSBxdWlj a2VyIHRoZSBnYXBzIGJvdGggaW4gDQo+ID4gZG93bmxpbmsgYW5kIHVwbGluaywgdGhlIHByZWlj ZSBmb3IgdGhpcyBiZWluZyB0aGUgaW5jcmVhc2Ugb2YgdGhlIA0KPiA+IFJUVCAoSSBndWVzcyBk b3VibGUpIGFnYWluc3Qgd2hhdCBvbmUgd291bGQgbm9ybWFsbHkgc2VlDQo+IGluIHRoZSBlbmQt dG8tZW5kIEV0aGVybmV0IG5ldHdvcmtzLg0KPiA+DQo+ID4NCj4gPiBJIGRvIGxpa2UgaWRlYXMg b2YgSHlzdGFydCwgYnV0IEkgIGRvbid0IGtub3cgaG93IGNvdWxkIGl0IGJlIA0KPiA+IHBvc3Np Ymx5IHJlY29uY2lsZWQgd2l0aCBteSBMVEUgcHJvYmxlbXMuICBXaXRoIHRoZSBpbmNyZWFzZSBv ZiBMVEUgDQo+ID4gc3BlZWRzIHRoaXMgZWFybHkgZXhpdCBmcm9tIHNsb3cgc3RhcnQgd2lsbCBi ZWNvbWUgZXZlbiBtb3JlIG9mIHRoZSBwcm9ibGVtLg0KPiA+IEZvciBMVEUsIG5vdCBzdXJlIGhv dyBnb29kIGFuZCBnZW5lcmFsbHkgYWNjZXB0YWJsZSwgc29sdXRpb24gd291bGQgDQo+ID4gYmUs IHBvc3NpYmx5LCB0byBpbmNyZWFzZSB0aGUgSFlTVEFSVF9ERUxBWV9NSU4gdG8gOCBtcywgd2hp Y2gsIGF0IA0KPiA+IGxlYXN0IGluIG15IGNhc2UsIHdvdWxkIGRpbWluaXNoIHRoZSBlYXJsaWVy IGV4aXQgc2lnbmlmaWNhbnRseSwgaWYgDQo+ID4gSSBiZWxpZXZlIGluIHRoZXNlIHRlc3RzIHJl c3VsdHMuDQo+ID4gQW5vdGhlciBzb2x1dGlvbiwgd2hpY2ggSSBkb24ndCBsaWtlIHZlcnkgbXVj aCwgYnV0IGFzIEkga25vdyBzb21lIA0KPiA+IG9wZXJhdG9ycyB1c2UsIHdvdWxkIGJlIHRvIGlu c2VydCBhIGtpbmQgb2YgVENQIGFjY2VsZXJhdG9yIGJldHdlZW4gDQo+ID4gdGhlIEludGVybmV0 IGFuZCB0aGUgbW9iaWxlIG5ldHdvcmsgd2hpY2ggd2lsbCBpbnRlcmNlcHQgYWxsIFRDUCBhbmQg DQo+ID4gd291bGQgc2VuZCBpdCB0byBtb2JpbGUgd2l0aG91dCBIeXN0YXJ0IGJ1dCB0aGlzIHdv dWxkIGtpbGwgYW55IA0KPiA+IGVuZC10by1lbmQgcHJpbmNpcGxlIGFuZCB0aGlzIHdpbGwgbm90 IGJlIHRoZSBJbnRlcm5ldCBhbnltb3JlLg0KPiA+DQo+ID4NCj4gPiBJZiB5b3UgaGF2ZSB0aW1l IGFuZCBuZXZlciBsb29rIGNsb3NlbHkgdG8gTFRFIHRlY2hub2xvZ3kgSSBkbyBhIA0KPiA+IHF1 aWNrIHN1bW1hcnkgYmVsbG93Lg0KPiA+DQo+ID4NCj4gPiBMVEUsIFRDUCBzZWxmLWNsb2NraW5n IGFuZCBBQ0sgY29tcHJlc3Npb24uDQo+ID4NCj4gPiBTcGVudCBzb21lIHRpbWUgdG8gdW5kZXJz dGFuZCB0aGUgYmFzaWNzIG9mIHdoYXQgY291bGQgYmUgdGhlIHJlYXNvbiANCj4gPiBmb3IgYWxs IHRob3NlIHRpbWluZyBlZmZlY3RzIGR1ZSB0byBMVEUuDQo+ID4gCUZpcnN0LCB1bmxpa2Ugd2ly ZSBiYXNlZCB0cmFuc3BvcnQgdGVjaG5vbG9naWVzLCBMVEUgKGJ1dCBhbHNvDQo+IFVNVFMNCj4g PiBhbmQgZnV0dXJlIDVHKSBzZW5kcyBkYXRhLCBib3RoIHVwbGluayBhbmQgZG93bmxpbmsgaW4g cHJlZGVmaW5lZCANCj4gPiB0aW1lc2xvdHMgY2FsbGVkIFRyYW5zbWlzc2lvbiBUaW1lIEludGVy dmFsIChUVEkpLiBJbiBVTVRTIHRoaXMgaXMgDQo+ID4gMm1zLCBpbiBMVEUgaXMNCj4gPiAxIG1z IGFuZCBpdCBzYXlzIHRoYXQgaW4gNUcgaXQgd2lsbCBiZSAwLjUgbXMuDQo+ID4gVGhlc2UgVFRJ IGhhdmUgZXhhY3QgYm9yZGVycyBhbmQgdGlnaHRseSBmb2xsb3cgZWFjaCBvdGhlciBhbmQgDQo+ ID4gcmVxdWlyZSBleGFjdCBzeW5jaHJvbml6YXRpb24gYmV0d2VlbiB0aGUgbW9iaWxlIGRldmlj ZSBhbmQgUmFkaW8gDQo+ID4gQmFzZSBTdGF0aW9uIChjYWxsZWQgZU5vZGVCIGluIExURSkuIFRv IGNvcGUgd2l0aCB2YXJ5aW5nIHJhZGlvIA0KPiA+IGNvbmRpdGlvbnMgdGhlIHNlbmRpbmcgZGF0 YSBiYW5kd2lkdGggaXMgbW9kaWZpZWQgaW4gc3VjaCBhIHdheSB0aGF0IA0KPiA+IHRvIGtlZXAg YSBjb25zdGFudCBwcm9iYWJpbGl0eSBvZiBkYXRhIGxvc3Mgb2YgMTAlLiBCYWQgcmFkaW8gLSBs ZXNzIA0KPiA+IGJhbmR3aWR0aCwgZ29vZA0KPiByYWRpbyAtIG1vcmUgYmFuZHdpZHRoLg0KPiA+ IFRoZSBiYW5kd2lkdGggaXMgbW9kaWZpZWQgYnkgcGFja2luZyBsZXNzIG9yIG1vcmUgYml0cyBp biBhIFRUSSBvZiAxIA0KPiA+IG1zIGluDQo+IExURS4NCj4gPiBCdXQgVFRJIHJlbWFpbiB1bmNo YW5nZWQgb2YgZXhhY3RseSAxIG1zLiBUaGF0IG1lYW5zIHRoYXQsIGF0IHRoZSANCj4gPiBUQ1Av SVAgbGF5ZXIgdGhlIHByZWNpc2lvbiBvZiB0aGUgdGltaW5nIGluZm9ybWF0aW9uIGZyb20gZGV2 aWNlIHRvIA0KPiA+IHRoZSBuZXR3b3JrIGlzIDEgbXMuIEluIHRoZSBjYXNlIG9mIGEgYmFuZHdp ZHRoIG9mIDEgTWJwcyB0aGVyZSB3aWxsIA0KPiA+IGJlIDgzIElQIHBhY2tldHMgcGVyIHNlY29u ZCBvciBvbmUgcGFja2V0IGV2ZXJ5IDEyIG1zLiBJbiB0aGlzIGNhc2UsIA0KPiA+IHBvc3NpYmx5 IHRoZSByZXR1cm4gb2Ygb25lIEFDSyBldmVyeSAxMiBtcyAoZXZlcnkgMTIgVFRJKSB3b3VsZCBi ZSANCj4gPiBtb3JlIG9yIGxlc3Mgc3VmZmljaWVudCB0byBrZWVwIGF0IGEgZ29vZCBsZXZlbCB0 aGUgVENQIA0KPiA+IHNlbGYtY2xvY2tpbmcgbWVjaGFuaXNtLiBCdXQsIHdoZW4gdGFsa2luZyBh Ym91dCAxMDAgTWJwcyAob3IgMzAwIA0KPiA+IE1icHMpIHRoYXQgbWVhbnMgb25lIHBhY2tldCAo YW5kIG9uZSBBQ0spIGV2ZXJ5IDAuMTIgbXMgYW5kIHRoZXJlIGlzIA0KPiA+IG5vIHdheSB0byBw cm92aWRlIGl0IHdpdGggYSBUVEkgb2YgMSBtcy4gVGhpcyBpcyB0aGUgcmVhc29uIGZvciBBQ0sg DQo+ID4gY29tcHJlc3Npb24gaW4gTFRFIGFuZCB0aGUgZnV0dXJlIDVHIHdoaWNoIHByb21pc2Vz IG1vcmUgdGhhbiAxR2JwcyANCj4gPiB3aXRoIGl0cyAwLjUgbXMgVFRJIHdpbGwNCj4gYmUgaW4g YSBiaWcgY29uZmxpY3Qgd2l0aCBUQ1Agc2VsZi1jbG9ja2luZy4NCj4gPiAJQnV0IHRoZSBwcm9i bGVtIGRvIG5vdCBzdG9wcyBoZXJlLiBUaGVyZSBpcyBhIHNpZ25pZmljYW50bHkNCj4gZGlmZmVy ZW50DQo+ID4gbWVjaGFuaXNtcyBvZiB0cmFuc21pc3Npb24gaW4gdGhlIGRvd25saW5rIChmcm9t IGVOb2RlQiB0byBtb2JpbGUNCj4gPiBkZXZpY2UpIGFuZCBpbiB1cGxpbmsgKGZyb20gbW9iaWxl IGRldmljZSB0byB0aGUgZU5vZGVCKS4gVGhlIA0KPiA+IGRpZmZlcmVuY2UgY29tZSBmcm9tIHRo ZSBmYWN0IHRoYXQgdGhlIHNjaGVkdWxlciBmb3IgYm90aCBkb3dubGluayANCj4gPiBhbmQgaW4g dXBsaW5rIGlzIGxvY2F0ZWQgaW4gdGhlIGVOb2RlQi4gQXMgb2YgbXkgdW5kZXJzdGFuZGluZyB0 aGUgDQo+ID4gcHVycG9zZSBvZiB0aGlzIGlzIHRvIHNpbXBsaWZ5IHRvIHRoZSBtYXhpbXVtIHRo ZSBkZXZpY2UgYW5kIHNhdmUgaXRzIGJhdHRlcnkuDQo+ID4gQXMgYSByZXN1bHQgb2YgdGhpcyBk ZXNpZ24sIHRoZSBlTm9kZUIgY2FuIHNlbmQgaW4gZWFjaCBUVEkgdG93YXJkIA0KPiA+IHRoZSBt b2JpbGUgZGV2aWNlIHdoZW5ldmVyIGRhdGEgZXhpc3QgaW4gaXRzIGJ1ZmZlciAoZU5vZGVCIGlz IGEgDQo+ID4ga2luZCBvZiBJUCB3aXJlbGVzcyByb3V0ZXIpLiBCdXQgZm9yIG1vYmlsZSBkZXZp Y2UsIGluIG9yZGVyIHRvIHNlbmQgDQo+ID4gdGhlIGRhdGEgaW4gdXBsaW5rLCBhZnRlciBhIHBh dXNlLCBpdCBoYXMgZmlyc3QgdG8gYXNrIHRoZSBuZXR3b3JrIA0KPiA+IHRvIHNjaGVkdWxlIGl0 LiBBbmQgbW9iaWxlIGhhcyBmb3IgdGhpcyByZXF1ZXN0IChhZnRlciBhIHBhdXNlIG9mIA0KPiA+ IHNlbmRpbmcpIGp1c3Qgb25lIGJpdCwgYWN0dWFsbHkgaXMgbm90IGV2ZW4gYSBkYXRhIGJpdCwg aXMgYSBzcGVjaWFsIA0KPiA+IHJhZGlvIHNpZ25hbCB3aGljaCB0ZWxscyAiSGV5LCBJIGhhdmUg c29tZSBkYXRhIGluIHRoZSBidWZmZXIgdG8gDQo+ID4gc2VuZCIuIExldHMgbG9vayBhdCBpbml0 aWFsIHRyYWluIG9mIElXIG9mIDEwIHBhY2tldHMuIEluIGdvb2QgcmFkaW8gDQo+ID4gY29uZGl0 aW9ucyB0aGUgZU5vZGVCIHdpbGwgZml0IGFsbCAxMCBwYWNrZXRzIGluIDEgVFRJIG9mIDEgbXMg YW5kIA0KPiA+IHNlbmQgdGhlbSBhbGwgdG8gdGhlIGRldmljZS4gRGV2aWNlIHdpbGwgdW5wYWNr IHRoZW0gYW5kIGdlbmVyYXRlIDEwIA0KPiA+IEFDSyBhbmQgdGhlbiB0ZWxsIHRvIHRoZSBlTm9k ZUIgIkhleSwgSSBoYXZlIHNvbWUgZGF0YSBpbiB0aGUgYnVmZmVyIA0KPiA+IHRvIHNlbmQiLiBU aGUgZU5vZGVCIGhhcyBub3Qga25vd2xlZGdlIGhvdyBtdWNoIGRhdGEgdGhlIG1vYmlsZSBoYXMg DQo+ID4gdG8gc2VuZCwgdGhlcmVmb3Igd2lsbCBhbGxvY2F0ZSBzb21lIG1pbmltdW0NCj4gPiBj YXBhY2l0eTogIldlbGwsIEkgZ2l2ZSB5b3UgYSBncmFudCB0byBzZW5kIHVwIHRvIDIwMCBCeXRl cyBhbmQgeW91IA0KPiA+IG11c3Qgc2VuZCB0aGVtIGV4YWN0bHkgYWZ0ZXIgNCBtcyB5b3UgZ2V0 IHRoaXMgbm90aWZpY2F0aW9uIi4gVGhlc2UgDQo+ID4gNCBtcyBhcmUgYWxsb3dhbmNlIGZvciBt b2JpbGUgZGV2aWNlIHRvIHByZXBhcmUgZGF0YSBmb3Igc2VuZGluZyANCj4gPiB3aGljaCBpcyBh IHZlcnkgY29tcGxpY2F0ZWQgcHJvY2VzcywgdGhlc2UgNCBtcyBpcyBhIG11c3QgZGVsYXkgDQo+ ID4gYmV0d2VlbiB0aGUgZ3JhbnQNCj4gYW5kIHRoZSB0cmFuc21pc3Npb24uDQo+ID4gSWYgbW9i aWxlIGhhcyBpbiBpdHMgYnVmZmVyIDEwIEFDSyB0byBzZW5kIGl0IHdpbGwgc2VuZCBvbmx5IDIg QUNLIA0KPiA+IGFuZCB3aWxsIGNvbXBsZW1lbnQgdGhlbSB3aXRoLCBzbyBjYWxsZWQsIGJ1ZmZl ciBzdGF0dXMgcmVwb3J0OiAiSSANCj4gPiBkbyBoYXZlIHN0aWxsIDQ4MCBieXRlcyB0byBzZW5k Ii4gTm93IG5ldHdvcmsgaGFzIG1vcmUgaW5mb3JtYXRpb24gDQo+ID4gYW5kIGFsbG9jYXRlIGFz DQo+ID4gcmVxdWVzdGVkOiAiV2VsbCwgSSBnaXZlIHlvdSBhIGdyYW50IHRvIHNlbmQgdXAgdG8g MjAwIEJ5dGVzIGFuZCB5b3UgDQo+ID4gbXVzdCBzZW5kIHRoZW0gZXhhY3RseSBhZnRlciA0IG1z IHlvdSBnZXQgdGhpcyBub3RpZmljYXRpb24iLiBCdXQsIA0KPiA+IGluIHRoZSBtZWFuIHRpbWUg c29tZSBvdGhlciBpbmZvcm1hdGlvbiBvZiBoaWdoZXIgcHJpb3JpdHkgbWF5IA0KPiA+IGFwcGVh ciBpbiB0aGUNCj4gZGV2aWNlIChlLmcuDQo+ID4gc2lnbmFsaW5nKSBhbmQgdGhlIGRldmljZSB3 aWxsIHNlbmQgb25seSA2IG91dCBvZiA4IHJlbWFpbmluZyBBQ0sgDQo+ID4gdG9nZXRoZXIgd2l0 aCBzaWduYWxpbmcgYW5kIGFub3RoZXIgQnVmZmVyIFN0YXR1cyBSZXBvcnQ6ICJIZXksIEkgZG8g DQo+ID4gaGF2ZSBhbm90aGVyIDEyMCBieXRlcyB0byBzZW5kIi4gQW5kIGFnYWluICJXZWxsLCBJ IGdpdmUgeW91IGEgZ3JhbnQgDQo+ID4gdG8gc2VuZCB1cCB0byAxMjAgQnl0ZXMgYW5kIHlvdSBt dXN0IHNlbmQgdGhlbSBleGFjdGx5IGFmdGVyIDQgbXMgDQo+ID4geW91IGdldCB0aGlzIG5vdGlm aWNhdGlvbiIuIEFzIGEgcmVzdWx0IHRoZSBzZXJ2ZXIgd2lsbCByZWNlaXZlIDIgDQo+ID4gQUNL LCB0aGVuIGFmdGVyIGEgZ2FwIG9mIDYgbXMgYW5vdGhlciA2IEFDSywgdGhlbiwgYWZ0ZXIgYW5v dGhlciA2IA0KPiA+IG1zLCB0aGUgcmVtYWluaW5nIDIgQUNLLiBXaXRoIHRoZSBzY2hlZHVsZXIg aW4gdGhlIGVOb2RlQiB0aGVzZSBnYXBzIA0KPiA+IGFyZSBpbmV2aXRhYmxlLiBBbmQgb25lIG1v cmUgdGhpbms6IG1vYmlsZSBwYWNrIHRvZ2V0aGVyIHRob3NlIDYgQUNLIA0KPiA+IGluIDEgVFRJ IG9mIDEgbXMgYW5kIHNlbmQgdGhlbSwgdGhlbiB0aGUgZU5vZGVCIHVucGFjayB0aGVtIGFuZCAN Cj4gPiB0aGVyZSBpcyBubyB3YXkgdG8gcmVjb3ZlciB0aGUgc3BhY2luZyBvZiAwLjEyIG1zIGJl dHdlZW4gdGhlbS4gDQo+ID4gZU5vZGVCIHNpbXBseSBzZW5kIHRoZW0gdG8gdGhlIHNlcnZlciBi YWNrIHRvIGJhY2sgYXQgdGhlIHNwZWVkIG9mIA0KPiA+IGl0cyBkYXRhIGNhcmQgd2hpY2ggaXMg dHlwaWNhbGx5IDEgR2JwcyBhbmQgd2UgZmFjZSB0aGUgQUNLIA0KPiA+IGNvbXByZXNzaW9uLiBB ZnRlciB0aGUgbGFzdCAyIEFDSyB3ZXJlIHNlbnQgdGhlcmUgaXMgbm90aGluZyBlbHNlIHRvIA0K PiA+IHNlbmQsIHNvIHRoZXJlIGlzIG5vIEJ1ZmZlciBTdGF0dXMgUmVwb3J0LCB0aGVyZWZvcmUs IGFmdGVyIHBhY2tldHMgDQo+ID4gb2YgdGhlIG5leHQgdHJhaW4gYXJyaXZlIGFuZCBBQ0sgYXJl IGdlbmVyYXRlZCBldmVyeXRoaW5nIHN0YXJ0cyANCj4gPiBhZ2FpbiB3aXRoIG9uZSBiaXQgb2Yg aW5mb3JtYXRpb24gIkhleSwgSSBoYXZlDQo+IHNvbWUgZGF0YSBpbiB0aGUgYnVmZmVyIHRvIHNl bmQiLg0KPiA+IFRoYXQncyB3aHkgdGhlcmUgYXJlIGdhcHMgYW5kIEFDSyBjb21wcmVzc2lvbiwg bG9zcywgb3IgbW9yZSANCj4gPiBleGFjdGx5LCBsYWNrIG9mIGFueSB0aW1pbmcgaW5mb3JtYXRp b24gd2hpY2ggd291bGQgYmUgdXNlZnVsIGZvciANCj4gPiB0aGUgVENQIHNlbGYtDQo+IGNsb2Nr aW5nLg0KPiA+DQo+ID4gVGhhbmsgeW91IGFuZCBzb3JyeSBmb3IgdGFraW5nIHlvdXIgdGltZS4N Cj4gPg0KPiA+DQo+ID4gVmVhY2VzbGF2IFJvbWFuDQo+ID4NCj4gPg0KPiA+DQo+ID4gLS0tLS1P cmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiBGcm9tOiBWZWFjZXNsYXYgUk9NQU4NCj4gPiBTZW50 OiBTYXR1cmRheSwgMDUgU2VwdGVtYmVyIDIwMTUgMDE6MTANCj4gPiBUbzogJ0VyaWMgRHVtYXpl dCc7IE5lYWwgQ2FyZHdlbGwNCj4gPiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247 IFNhbmd0YWUgSGE7IEluZ2VtYXIgSm9oYW5zc29uIFM7IA0KPiA+IEVyaWMgRHVtYXpldDsgZW5k MmVuZC1pbnRlcmVzdEBwb3N0ZWwub3JnDQo+ID4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJlXSBU Q1AgSHlTdGFydCBwYXRjaCBkZXBsb3ltZW50DQo+ID4NCj4gPiBJZiB3ZSBsb29rIGF0IHRoZSBm aXJzdCB0cmFpbjogYWxsIHBhY2tldHMgcmVjZWl2ZWQgaW4gbGVzcyB0aGFuIDEgbXMuDQo+ID4g UHJvYmFibHkgdGhpcyBpcyBvbmx5IGFuIGFwcGVhcmFuY2UgYXMgTFRFIHRyYW5zbWl0cyBpbiwg c28gY2FsbGVkIA0KPiA+IHRyYW5zbWlzc2lvbiB0aW1lIGludGVydmFsIChUVEkpIG9mIDEgbXMs IGFuZCB3aGF0IHdlIHNlZSBoZXJlIGlzIA0KPiA+IHRoYXQgYWxsIDEwIHBhY2tldHMgb2YgaW5p dGlhbCB3aW5kb3cgZml0dGVkIGluIDEgbXMsIGFuZCwgd2hlbiANCj4gPiBkZWNvZGVkLCB3ZXJl IHByZXNlbnRlZCB0byB0aGUgVENQL0lQIGxheWVyIChhbmQgcGNhcCkgYWxsIGF0IG9uY2UuIA0K PiA+IEJUVywgOCBwYWNrZXRzIG9mIDEzMDIyIGJ5dGVzIGluIDEgbXMgbWVhbnMgaW5zdGFudGFu ZW91cyBzcGVlZCBvZiANCj4gPiAxMDQgTWJwcywgZ29vZCByYWRpbyBjb25kaXRpb25zLiBUQ1Ag Z2VuZXJhdGVzIDEwIEFDSyB0cmFpbiBvZiB0aGUgDQo+ID4gZHVyYXRpb24gb2YNCj4gPiAxLjIg bXMuIFdpbGwgaXQgYmUgYSBGYXN0IEV0aGVybmV0IHBvc3NpYmx5IHdlIHdvdWxkIGNvbnNpZGVy IHRoaXMgDQo+ID4gbm9ybWFsLA0KPiBpc24ndCBpdCA/DQo+ID4gSSBsb29rZWQgYXQgaG93IHNl cnZlciByZXBseSB0byB0aGVzZSAxMCBBQ0tzLiBUaGVyZSBhcmUgMyBnYXBzIG9mIA0KPiA+IDQs DQo+ID4gMiBhbmQgNSBtcyBpbiB0aGUgcmVwbHkgdHJhaW4gYW5kIHRoZSB0b3RhbCB0cmFpbiBv ZiAxOCAoPywgaXQgDQo+ID4gc2hvdWxkIGJlIDIwKSBwYWNrZXRzIHJlYWNoZXMgYSBkdXJhdGlv biBvZiAxNCBtcy4gQUZBSUsgTFRFIG1heSANCj4gPiBpbnRyb2R1Y2UgZ2FwcyBpbiBBQ0sgdHJh aW4gZHVlIHRvIHVwbGluayBzY2hlZHVsaW5nIG1lY2hhbmlzbS4NCj4gPiBNYXkgYmUgdGhlc2Ug Z2FwcyB0cmlnZ2VyIEh5c3RhcnQgZWFybHkgZXhpdCA/DQo+ID4NCj4gPiBWZWFjZXNsYXYgUm9t YW4NCj4gPiBUZWNobmljYWwgYW5kIElUIGRpcmVjdG9yDQo+ID4gT3JhbmdlIE1vbGRvdmEgUy5B Lg0KPiA+IEZpeDogKzM3MzIyNTc1NDAwDQo+ID4gTW9iOiArMzczNjkxOTg0MDANCj4gPiBGYXg6 ICszNzMyMjU3NTMwNg0KPiA+DQo+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiBG cm9tOiBFcmljIER1bWF6ZXQgW21haWx0bzplcmljLmR1bWF6ZXRAZ21haWwuY29tXQ0KPiA+IFNl bnQ6IEZyaWRheSwgMDQgU2VwdGVtYmVyIDIwMTUgMTk6NDgNCj4gPiBUbzogTmVhbCBDYXJkd2Vs bA0KPiA+IENjOiBWZWFjZXNsYXYgUk9NQU47IHRjcG1AaWV0Zi5vcmc7IFBpZXJzIE8nSGFubG9u OyBTYW5ndGFlIEhhOyANCj4gPiBJbmdlbWFyIEpvaGFuc3NvbiBTOyBFcmljIER1bWF6ZXQ7IGVu ZDJlbmQtaW50ZXJlc3RAcG9zdGVsLm9yZw0KPiA+IFN1YmplY3Q6IFJlOiBbdGNwbV0gW2UyZV0g VENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiA+DQo+ID4gT24gRnJpLCAyMDE1LTA5LTA0 IGF0IDExOjMyIC0wNDAwLCBOZWFsIENhcmR3ZWxsIHdyb3RlOg0KPiA+ID4gSGkgVmVhY2VzbGF2 LA0KPiA+ID4NCj4gPiA+IEkgYWdyZWUgdGhhdCBpbiB5b3VyIExURSB0cmFjZXMgaXQgbG9va3Mg bGlrZSBDVUJJQyBIeXN0YXJ0IGlzIA0KPiA+ID4gZXhpdGluZyBzbG93IHN0YXJ0IHRvbyBlYXJs eS4NCj4gPiA+DQo+ID4gPiBTSW5jZSB5b3Ugc2VlbSB0byBoYXZlIGEgbmljZSBMVEUgdGVzdGJl ZCwgd291bGQgeW91IGJlIGFibGUgdG8gZG8gDQo+ID4gPiBzb21lIGV4cGVyaW1lbnRzIHRvIGZp bmQgYSBzZXQgb2YgcGFyYW1ldGVycyBmb3IgSHlzdGFydCB0aGF0IHdvcmsgDQo+ID4gPiBiZXR0 ZXIgZm9yIHlvdXIgTFRFIGVudmlyb25tZW50PyBGb3IgZXhhbXBsZSwgeW91IG1pZ2h0IHRyeSB0 aGUgDQo+ID4gPiB0d28gdmFyaWF0aW9ucyBJIHN1Z2dlc3RlZCBlYXJsaWVyIGluIHRoZSB0aHJl YWQ6DQo+ID4gPg0KPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgaWYgKGNhLT5jdXJyX3J0 dCA+IGNhLT5kZWxheV9taW4gKw0KPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAgIEhZ U1RBUlRfREVMQVlfVEhSRVNIKGNhLT5kZWxheV9taW4gPj4NCj4gPiA+IDIpKSB7IG9yDQo+ID4g PiAgICAgICAgICAgICAgICAgICAgICAgICBpZiAoY2EtPmN1cnJfcnR0ID4gY2EtPmRlbGF5X21p biArDQo+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgSFlTVEFSVF9ERUxBWV9USFJF U0goY2EtPmRlbGF5X21pbiA+Pg0KPiA+ID4gMSkpIHsNCj4gPiA+DQo+ID4gPiBEbyBhbnkgb2Yg dGhvc2UgZ2l2ZSBiZXR0ZXIgcmVzdWx0cyBmb3IgeW91ciB0ZXN0cz8NCj4gPiA+DQo+ID4gPiBu ZWFsDQo+ID4NCj4gPiBBbHNvLCBoeXN0YXJ0IGlzIGZvb2xlZCBieSB0b28gbWFueSBBQ0sgcmVj ZWl2ZWQgaW4gc2hvcnQgcGVyaW9kLg0KPiA+DQo+ID4gVGhpcyBwcm9ibGVtIHdvdWxkIGJlIHNv bHZlZCBpZiBHUk8gd2FzIHVzZWQgYXQgcmVjZWl2ZXIsIGFzIGxlc3MgDQo+ID4gQUNLIHdvdWxk IGJlIHNlbnQuDQo+ID4NCj4gPiBQcmVzdW1hYmx5IHJlY2VpdmVyIGlzIG5vdCBhIGxpbnV4IFRD UCBzdGFjayA/DQo+ID4NCj4gPiBNYXliZSB3ZSBzaG91bGQgYWRkIGEgbG9naWMgaW4gaHlzdGFy dF91cGRhdGUoKSB0byB0YWtlIG9uZSBBQ0sgcGVyIG1zLg0KPiA+DQo+ID4gMDU6MzI6MjQuMTM0 MTcyIElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzIA0K PiA+IFtTXSwgc2VxIDQyOTQ0MjMyMTUsIHdpbiA2NTUzNSwgb3B0aW9ucyBbbXNzIDE0NjAsc2Fj a09LLFRTIHZhbA0KPiA+IDExMTIzNTEgZWNyIDAsbm9wLHdzY2FsZSA4XSwgbGVuZ3RoIDANCj4g PiAwNTozMjoyNC4xNjE5ODYgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4z MjkyNDogRmxhZ3MgDQo+ID4gW1MuXSwgc2VxIDExOTg4Nzc3MjEsIGFjayA0Mjk0NDIzMjE2LCB3 aW4gMjg5NjAsIG9wdGlvbnMgW21zcyANCj4gPiAxNDE2LHNhY2tPSyxUUyB2YWwNCj4gPiAyODA4 ODA4NjM0IGVjciAxMTEyMzUxLG5vcCx3c2NhbGUgN10sIGxlbmd0aCAwDQo+ID4gMDU6MzI6MjQu MTYyMTA4IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdz IA0KPiA+IFsuXSwgYWNrIDEsIHdpbiAzNDMsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIz NTQgZWNyIA0KPiA+IDI4MDg4MDg2MzRdLCBsZW5ndGggMA0KPiA+IDA1OjMyOjI0LjE2MzA3MiBJ UCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncyANCj4gPiBb UC5dLCBzZXEgMTo2NTgsIGFjayAxLCB3aW4gMzQzLCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAx MTEyMzU0IGVjciANCj4gPiAyODA4ODA4NjM0XSwgbGVuZ3RoIDY1Nw0KPiA+IDA1OjMyOjI0LjIw Mzk4NCBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncyAN Cj4gPiBbLl0sIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDI4MDg4 MDg2NzUgZWNyIA0KPiA+IDExMTIzNTRdLCBsZW5ndGggMA0KPiA+IDA1OjMyOjI0LjIxMDI3NSBJ UCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncyANCj4gPiBb UC5dLCBzZXEgMTozODcsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFs IA0KPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMzg2DQo+ID4gMDU6MzI6MjQu MjEwMzE1IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdz IA0KPiA+IFsuXSwgc2VxIDM4NzoxNzkxLCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3As bm9wLFRTIHZhbA0KPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+ IDA1OjMyOjI0LjIxMDMzMiBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMy OTI0OiBGbGFncyANCj4gPiBbLl0sIHNlcSAxNzkxOjMxOTUsIGFjayA2NTgsIHdpbiAyMzcsIG9w dGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4gMjgwODgwODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0 aCAxNDA0DQo+ID4gMDU6MzI6MjQuMjEwMzQ3IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQz LjEwNC4xNTYuMzI5MjQ6IEZsYWdzIA0KPiA+IFsuXSwgc2VxIDMxOTU6NDU5OSwgYWNrIDY1OCwg d2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4gPiAyODA4ODA4Njc3IGVjciAxMTEy MzU0XSwgbGVuZ3RoIDE0MDQNCj4gPiAwNTozMjoyNC4yMTAzNTAgSVAgOTQuMjQzLjEwNC4xNTYu MzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MgDQo+ID4gWy5dLCBhY2sgMzg3LCB3aW4g MzQ3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjciANCj4gPiAyODA4ODA4Njc3 XSwgbGVuZ3RoIDANCj4gPiAwNTozMjoyNC4yMTAzNjEgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5 NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MgDQo+ID4gWy5dLCBzZXEgNDU5OTo2MDAzLCBhY2sg NjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbA0KPiA+IDI4MDg4MDg2NzcgZWNy IDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+IDA1OjMyOjI0LjIxMDM3NCBJUCAxOTUuOTUuMTc4 LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncyANCj4gPiBbLl0sIHNlcSA2MDAz Ojc0MDcsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4gMjgw ODgwODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0aCAxNDA0DQo+ID4gMDU6MzI6MjQuMjEwMzg5IElQ IDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdzIA0KPiA+IFsu XSwgc2VxIDc0MDc6ODgxMSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2 YWwNCj4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPiAwNTozMjoy NC4yMTA0MDIgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxh Z3MgDQo+ID4gWy5dLCBzZXEgODgxMToxMDIxNSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwNCj4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQN Cj4gPiAwNTozMjoyNC4yMTA0MTYgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1 Ni4zMjkyNDogRmxhZ3MgDQo+ID4gWy5dLCBzZXEgMTAyMTU6MTE2MTksIGFjayA2NTgsIHdpbiAy MzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4gMjgwODgwODY3NyBlY3IgMTExMjM1NF0s IGxlbmd0aCAxNDA0DQo+ID4gMDU6MzI6MjQuMjEwNDE4IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0 ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzIA0KPiA+IFsuXSwgYWNrIDE3OTEsIHdpbiAzNTgs IG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyIA0KPiA+IDI4MDg4MDg2NzddLCBs ZW5ndGggMA0KPiA+IDA1OjMyOjI0LjIxMDQzMSBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0 My4xMDQuMTU2LjMyOTI0OiBGbGFncyANCj4gPiBbLl0sIHNlcSAxMTYxOToxMzAyMywgYWNrIDY1 OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4gPiAyODA4ODA4Njc3IGVjciAx MTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPiAwNTozMjoyNC4yMTA0NTUgSVAgOTQuMjQzLjEwNC4x NTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MgDQo+ID4gWy5dLCBhY2sgMzE5NSwg d2luIDM2OSwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3IgDQo+ID4gMjgwODgw ODY3N10sIGxlbmd0aCAwDQo+ID4gMDU6MzI6MjQuMjEwNjg1IElQIDk0LjI0My4xMDQuMTU2LjMy OTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzIA0KPiA+IFsuXSwgYWNrIDQ1OTksIHdpbiAz ODAsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyIA0KPiA+IDI4MDg4MDg2Nzdd LCBsZW5ndGggMA0KPiA+IDA1OjMyOjI0LjIxMDczMSBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+ IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncyANCj4gPiBbLl0sIGFjayA2MDAzLCB3aW4gMzkxLCBv cHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjciANCj4gPiAyODA4ODA4Njc3XSwgbGVu Z3RoIDANCj4gPiAwNTozMjoyNC4yMTA5MzcgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUu OTUuMTc4LjIwNC44MDogRmxhZ3MgDQo+ID4gWy5dLCBhY2sgNzQwNywgd2luIDQwMiwgb3B0aW9u cyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3IgDQo+ID4gMjgwODgwODY3N10sIGxlbmd0aCAw DQo+ID4gMDU6MzI6MjQuMjExMDc4IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3 OC4yMDQuODA6IEZsYWdzIA0KPiA+IFsuXSwgYWNrIDg4MTEsIHdpbiA0MTMsIG9wdGlvbnMgW25v cCxub3AsVFMgdmFsIDExMTIzNTkgZWNyIA0KPiA+IDI4MDg4MDg2NzddLCBsZW5ndGggMA0KPiA+ IDA1OjMyOjI0LjIxMTIyOCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0 LjgwOiBGbGFncyANCj4gPiBbLl0sIGFjayAxMDIxNSwgd2luIDQyNCwgb3B0aW9ucyBbbm9wLG5v cCxUUyB2YWwgMTExMjM1OSBlY3IgDQo+ID4gMjgwODgwODY3N10sIGxlbmd0aCAwDQo+ID4gMDU6 MzI6MjQuMjExMzc3IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6 IEZsYWdzIA0KPiA+IFsuXSwgYWNrIDExNjE5LCB3aW4gNDM1LCBvcHRpb25zIFtub3Asbm9wLFRT IHZhbCAxMTEyMzU5IGVjciANCj4gPiAyODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4gPiAwNTozMjoy NC4yMTE1NDQgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxh Z3MgDQo+ID4gWy5dLCBhY2sgMTMwMjMsIHdpbiA0NDYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFs IDExMTIzNTkgZWNyIA0KPiA+IDI4MDg4MDg2NzddLCBsZW5ndGggMA0KPiA+IDA1OjMyOjI0LjIz Nzk5OSBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncyAN Cj4gPiBbLl0sIHNlcSAxMzAyMzoxNDQyNywgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9w LG5vcCxUUyB2YWwNCj4gPiAyODA4ODA4NzA5IGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4g PiAwNTozMjoyNC4yMzgwNTMgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4z MjkyNDogRmxhZ3MgDQo+ID4gWy5dLCBzZXEgMTQ0Mjc6MTU4MzEsIGFjayA2NTgsIHdpbiAyMzcs IG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4gMjgwODgwODcwOSBlY3IgMTExMjM1OV0sIGxl bmd0aCAxNDA0DQo+ID4gMDU6MzI6MjQuMjM4MDkwIElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4g MTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzIA0KPiA+IFsuXSwgYWNrIDE0NDI3LCB3aW4gNDU3LCBv cHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzYyIGVjciANCj4gPiAyODA4ODA4NzA5XSwgbGVu Z3RoIDANCj4gPiAwNTozMjoyNC4yMzgxNjQgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUu OTUuMTc4LjIwNC44MDogRmxhZ3MgDQo+ID4gWy5dLCBhY2sgMTU4MzEsIHdpbiA0NjgsIG9wdGlv bnMgW25vcCxub3AsVFMgdmFsIDExMTIzNjIgZWNyIA0KPiA+IDI4MDg4MDg3MDldLCBsZW5ndGgg MA0KPiA+IDA1OjMyOjI0LjI0NDAyNCBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQu MTU2LjMyOTI0OiBGbGFncyANCj4gPiBbLl0sIHNlcSAxNTgzMToxNzIzNSwgYWNrIDY1OCwgd2lu IDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4gPiAyODA4ODA4NzEzIGVjciAxMTEyMzU5 XSwgbGVuZ3RoIDE0MDQNCj4gPiAwNTozMjoyNC4yNDQwNjMgSVAgMTk1Ljk1LjE3OC4yMDQuODAg PiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MgDQo+ID4gWy5dLCBzZXEgMTcyMzU6MTg2Mzks IGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4gMjgwODgwODcx MyBlY3IgMTExMjM1OV0sIGxlbmd0aCAxNDA0DQo+ID4NCg0K From nobody Thu Oct 1 04:57:53 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0DB381A21B5 for ; Thu, 1 Oct 2015 04:57:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.447 X-Spam-Level: X-Spam-Status: No, score=-1.447 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FRT_BELOW2=2.154, J_CHICKENPOX_36=0.6, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zLCkNXO5lAN7 for ; Thu, 1 Oct 2015 04:57:47 -0700 (PDT) Received: from sessmg23.ericsson.net (sessmg23.ericsson.net [193.180.251.45]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CF6A91A21BB for ; Thu, 1 Oct 2015 04:57:45 -0700 (PDT) X-AuditID: c1b4fb2d-f79626d000004282-f4-560d1fb79356 Received: from ESESSHC012.ericsson.se (Unknown_Domain [153.88.253.124]) by sessmg23.ericsson.net (Symantec Mail Security) with SMTP id E8.A8.17026.7BF1D065; Thu, 1 Oct 2015 13:57:43 +0200 (CEST) Received: from ESESSMB205.ericsson.se ([169.254.5.99]) by ESESSHC012.ericsson.se ([153.88.183.54]) with mapi id 14.03.0248.002; Thu, 1 Oct 2015 13:57:41 +0200 From: Ingemar Johansson S To: Veaceslav ROMAN , Eric Dumazet , Neal Cardwell Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8Pq87vsQ Date: Thu, 1 Oct 2015 11:57:40 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> In-Reply-To: <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.147] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprNIsWRmVeSWpSXmKPExsUyM+Jvje52ed4wg2/7NSyeHnvEbrH73FQW i33vz7JZdNzZC2TNmMhoceTKMUaLbSfnM1m8/7+R2YHDY+esu+weCzaVeixZ8pPJY8myp4we Hcc3MAewRnHZpKTmZJalFunbJXBlbN3czFaw7DJTxbHnb1kbGM+cYupi5OSQEDCRuPJ7JTuE LSZx4d56ti5GLg4hgaOMEo+P9TBCOIsYJfp/7mYFqWITsJFYeeg7I4gtIlAnsfTkZyaQImaB i4wST//0gBUJC5hJ3Gp6xwxRZC4x6XQbM0iRiMAqRolzB08AFXFwsAioSCx/Uw9SwyvgKzHn +jImiG3vOCVO9/WCNXMKBErs6vwJNpRRQFbi/vd7LCA2s4C4xK0n86F+EJBYsuc8M4QtKvHy 8T+w+RICShLTtqaBmMwCmhLrd+lDdCpKTOl+yA6xVlDi5MwnLBMYxWYhGToLoWMWko5ZSDoW MLKsYhQtTi0uzk03MtZLLcpMLi7Oz9PLSy3ZxAiMzYNbfuvuYFz92vEQowAHoxIP74LJPGFC rIllxZW5hxilOViUxHlbmB6ECgmkJ5akZqemFqQWxReV5qQWH2Jk4uCUamCcOMNjPVOB2wWx q+1KvTKuBbc8fqmauEjOuyeqa79KRTQotJy5SX2Xweb4tB9fNtZPMPudHfYjbv3Z+TLHkzY1 ebwKam+ydt59ck9m0v/N6ck217NU3B/xbsxeoHmL9bGKi4l85ZxJwr3e7iYtC2c0xxc9mBzl xMA+4Qj3rqT4h0YbN/bu01diKc5INNRiLipOBAB65c+jrgIAAA== Archived-At: Cc: Eric Dumazet , "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2015 11:57:52 -0000 SG1tIA0KWW91IGFyZSByaWdodCAhLiANCldlbGwuLi4gVGhhdCBvZiBjb3Vyc2Ugc2hvdWxkIGV4 cGxhaW4gd2h5IEh5U3RhcnQgcGVyZm9ybXMgYmFkbHkgZm9yIExURS4uLiBBdGxlYXN0IGl0IGlz IHBhcnQgb2YgdGhlIHJlYXNvbi4NCg0KL0luZ2VtYXINCg0KDQoNCj4gLS0tLS1PcmlnaW5hbCBN ZXNzYWdlLS0tLS0NCj4gRnJvbTogVmVhY2VzbGF2IFJPTUFOIFttYWlsdG86VmVhY2VzbGF2LlJv bWFuQG9yYW5nZS5tZF0NCj4gU2VudDogZGVuIDEgb2t0b2JlciAyMDE1IDEzOjQ2DQo+IFRvOiBJ bmdlbWFyIEpvaGFuc3NvbiBTOyBFcmljIER1bWF6ZXQ7IE5lYWwgQ2FyZHdlbGwNCj4gQ2M6IHRj cG1AaWV0Zi5vcmc7IFBpZXJzIE8nSGFubG9uOyBTYW5ndGFlIEhhOyBFcmljIER1bWF6ZXQ7IGVu ZDJlbmQtDQo+IGludGVyZXN0QHBvc3RlbC5vcmcNCj4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJl XSBUQ1AgSHlTdGFydCBwYXRjaCBkZXBsb3ltZW50DQo+IA0KPiBXZWxsIHRoaXMgc2F5DQo+IAlJ ZiBjdXJyX3J0dCBpcyBoaWdoZXIgdGhhbiBkZWxheV9taW4gKyAoMS84ICogZGVsYXlfbWluIG9y IDQgbXMsDQo+IHdoaWNoZXZlciBpcyBoaWdoZXIpICB0aGVuIGV4aXQgc2xvdyBzdGFydC4NCj4g DQo+IFdoZW4gYWN0dWFsIG1pbmltdW0gZGVsYXkgaXMgbGVzcyB0aGFuIDMyIG1zIHRoZSBkZWxh eV9taW4gd2lsbCBiZSBsZXNzIHRoYW4NCj4gMzI8PDMsIHRoZW4gdGhlIGRlbGF5X21pbj4+MyB3 aWxsIGJlIGxlc3MgdGhhbiAzMiBhbmQgd2lsbCBjb21wYXJlIHdpdGgNCj4gNDw8MyB3aGljaCBp cyAzMiBhbmQgdGhlcmVmb3IgdGhlIHRocmVzaG9sZCB3aWxsIGJlIChyZWFsX2N1cnJfcnR0PDwz KSA+DQo+IChyZWFsX2RlbGF5PDwzICsgNDw8Mykgb3IgKHJlYWxfY3Vycl9ydHQpPihyZWFsX2Rl bGF5ICs0KQ0KPiANCj4gDQo+IFZlYWNlc2xhdiBSb21hbg0KPiANCj4gLS0tLS1PcmlnaW5hbCBN ZXNzYWdlLS0tLS0NCj4gRnJvbTogSW5nZW1hciBKb2hhbnNzb24gUyBbbWFpbHRvOmluZ2VtYXIu cy5qb2hhbnNzb25AZXJpY3Nzb24uY29tXQ0KPiBTZW50OiBUaHVyc2RheSwgMDEgT2N0b2JlciAy MDE1IDEzOjAwDQo+IFRvOiBWZWFjZXNsYXYgUk9NQU47IEVyaWMgRHVtYXpldDsgTmVhbCBDYXJk d2VsbA0KPiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0YWUgSGE7IEVy aWMgRHVtYXpldDsgZW5kMmVuZC0NCj4gaW50ZXJlc3RAcG9zdGVsLm9yZzsgSW5nZW1hciBKb2hh bnNzb24gUw0KPiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRl cGxveW1lbnQNCj4gDQo+IEhpDQo+IA0KPiBZZXMsIHRoZSB2YXJpYWJsZSBkZWxheV9taW4gaXMg aW4gUTMgZm9yIHJlYXNvbnMgb2YgcHJlY2lzaW9uIEkgZ3Vlc3MgQnV0IHRoZQ0KPiBjYWxsIHRv IEhZU1RBUlRfREVMQVlfVEhSRVNIIGlzIHdpdGggZGVsYXlfbWluIHNoaWZ0ZWQgZG93biB0byBR MA0KPiANCj4gSFlTVEFSVF9ERUxBWV9USFJFU0goY2EtPmRlbGF5X21pbiA+PiAzKSkgIChzZWUg aHR0cDovL2x4ci5mcmVlLQ0KPiBlbGVjdHJvbnMuY29tL3NvdXJjZS9uZXQvaXB2NC90Y3BfY3Vi aWMuYyNMNDAzICkgSSBtYXkgYmUgb2ZmIGhlcmUuLg0KPiANCj4gUmVnYXJkaW5nIHBhY2luZy4g SSBkb24ndCBoYXZlIHJlYWwgZGF0YSB0byBzaG93IHRoZSBlZmZlY3RzIG9mIHBhY2luZywgaW4N Cj4gc2ltdWxhdGlvbnMgaXQgaXMgaG93ZXZlciBwb3NzaWJsZSB0byBzZWUgdGhlIGdhaW5zIGlu IHRlcm1zIG9mIGxlc3MgY29hbGVzY2luZywNCj4geWVzIHBhY2luZyBkb2VzIG5vdCBzb2x2ZSB0 aGUgQUNLIGNvbXByZXNzaW9uIGJ1dCBpdCBzZWVtIHRvIHNvbHZlIHRoZQ0KPiBmb2xsb3cgdXAg ZWZmZWN0IHRoYXQgQUNLIGNvbXByZXNzaW9uIGdpdmVzLg0KPiANCj4gL0luZ2VtYXINCj4gDQo+ ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiBGcm9tOiBWZWFjZXNsYXYgUk9NQU4g W21haWx0bzpWZWFjZXNsYXYuUm9tYW5Ab3JhbmdlLm1kXQ0KPiA+IFNlbnQ6IGRlbiAxIG9rdG9i ZXIgMjAxNSAxMDozMg0KPiA+IFRvOiBJbmdlbWFyIEpvaGFuc3NvbiBTOyBFcmljIER1bWF6ZXQ7 IE5lYWwgQ2FyZHdlbGwNCj4gPiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNh bmd0YWUgSGE7IEVyaWMgRHVtYXpldDsgZW5kMmVuZC0NCj4gPiBpbnRlcmVzdEBwb3N0ZWwub3Jn DQo+ID4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBwYXRjaCBkZXBsb3lt ZW50DQo+ID4NCj4gPiBZZXMsIEhZU1RBUlRfREVMQVlfTUlOICAoNFU8PDMpIC4NCj4gPiBCdXQg Y3Vycl9ydHQgYW5kIGRlbGF5X21pbiBjb21lIGZyb20gYmljdGNwX2Fja2VkIHdoaWNoIGlzIGFs c28gPDwzIDoNCj4gPiAJSW4gdGNwX2N1YmljLmMsIGZ1bmN0aW9uICBsaW5lIDQzMyAoa2VybmVs IDQuMikNCj4gPiAJCWRlbGF5ID0gKHJ0dF91cyA8PCAzKSAvIFVTRUNfUEVSX01TRUM7IFNvLA0K PiA+IDRVPDwzIGlzIGVxdWl2YWxlbnQgdG8gNCBtcyAuDQo+ID4NCj4gPiBSZWdhcmRpbmcgcGFj aW5nIEkgYW0gbm90IHN1cmUgd2lsbCBoZWxwIHRoZSBwZXJmb3JtYW5jZTogdGhlcmUgYXJlDQo+ ID4gZ2FwcyBpbiBBQ0sgdHJhaW5zIG9mIHRoZSBvcmRlciBvZiBtaWxsaXNlY29uZHMgYW5kIHRo aXMgaXMgYSBsb3N0DQo+ID4gdGltZSBhbmQgdGhlcmUgaXMgbm8gd2F5IHRvIGNvbXBlbnNhdGUg aXQuIElmIGludHJvZHVjZSBBQ0sgcGFjaW5nIG9yDQo+ID4gZGF0YSBwYWNrZXRzIHBhY2luZyB0 aGlzIHdvdWxkIGFkZCB0byB0aGUgb3ZlcmFsbCBsb3N0IHRpbWUuDQo+ID4gUGFjaW5nIGNhbiBo ZWxwIG9ubHkgdG8gYWxsZXZpYXRlIGEgcG90ZW50aWFsIHByb2JsZW0gb2Ygc29tZQ0KPiA+IGRv d25zdHJlYW0gcm91dGVycyB3aXRoIGluc3VmZmljaWVudCBidWZmZXJzIHdoaWNoIG1heSBpbnRy b2R1Y2UNCj4gPiB1bmV4cGVjdGVkIHBhY2tldHMgZHJvcHMgd2hlbiBsYXJnZSB0cmFpbiBzZW50 IGF0IGEgaGlnaCBzcGVlZCBhcyBhDQo+ID4gcmVwbHkgdG8gaGlnaGx5IGNvbXByZXNzZWQgQUNL IHRyYWluLCBidXQsIG9uIGFub3RoZXIgaGFuZHMuDQo+ID4gQnV0IGhvdyBtdWNoIHBhY2luZyBh bmQgaG93IHRvIGFjY29tbW9kYXRlIHRvIGxhcmdlbHkgY2hhbmdpbmcgcmFkaW8NCj4gPiBjb25k aXRpb25zID8gQW5kIHdpdGggcGVybWFuZW50bHkgY2hhbmdpbmcgdGVjaG5vbG9neSBhbmQgc3Bl ZWQgPw0KPiA+IEFueXdheSB0aGUgcGFjaW5nIHdpbGwgYWRkIHRvIHRoZSBwcm9ibGVtIG9mIGdh cHMgYW5kIGJldHRlciBzaXplDQo+ID4gcHJvcGVybHkgcm91dGVycyBidWZmZXJzLg0KPiA+DQo+ ID4gSWRlYWxseSBBQ0sgdHJhaW4gaXMgc3BsaXQgaW4gMiBwYXJ0cywgYnV0IGluIHByYWN0aWNl IEkgZG8gc2VlIG1vcmUuDQo+ID4gQnV0IGFzIHRoZSBzdHJlYW0gcHJvZ3Jlc3MgdGhlIGxlc3Mg Z2Fwcy4gVGhlIGJpZ2dlciB0aGUgZG93bmxpbmsNCj4gPiBwcmVzc3VyZSB0aGUgbW9yZSBBQ0sg YW5kIGxlc3MgY2hhbmNlcyB0byBoYXZlIGEgcGF1c2UgaW4gQUNLIHNlbmRpbmcsDQo+ID4gdGhl biB0aGVyZSB3aWxsIGJlIGEgY29udGludW91cyBmbG93IG9mIEJ1ZmZlciBTdGF0dXMgUmVwb3J0 cyBhbmQgbGVzcw0KPiA+IG9mIHRoZSBTY2hlZHVsaW5nIFJlcXVlc3QgKFNSLCAiSGV5LCBJIGhh dmUgc29tZSBkYXRhIHRvIHNlbmQiKS4gVGhlDQo+ID4gb25seSBpc3N1ZSBJIGFtIG5vdCBzdXJl IGlzIG9uIHRoZSBwb3RlbnRpYWwgZ2FwcyBpbiBvdGhlciBuZXR3b3Jrcy4gSQ0KPiA+IGRpZCBu b3QgcHV0IGJlbGxvdyBhbGwgZGV0YWlscywgYnV0IHlvdSBzaG91bGQga25vdyB0aGF0IGRldmlj ZSBjYW4NCj4gPiBub3Qgc2VuZCBTUiBpbiBhcmJpdHJhcnkgbW9tZW50cyBvZiB0aW1lIGJ1dCBp biBwcmVkZWZpbmVkIGZvciBoaW0NCj4gPiBpbnRlcnZhbHMuIEluIG15IG5ldHdvcmsgdGhpcyBw ZXJpb2QgaXMgNSBtcywgdGhlcmVmb3JlIHRoZSBSVFQgb2YgdGhlDQo+ID4gZmlyc3QgQUNLIGlu IHRoZSB0cmFpbiB3aWxsIGhhdmUgdmFyaWF0aW9ucyBvZiAyLjUgbXMuIEJ1dCBzdGFuZGFyZHMN Cj4gPiBhZG1pdCB0aGlzIHBlcmlvZCB0byBiZSBhbHNvIDEwIG1zICwyMCBtcywgNDAgbXMsIDgw IG1zIGFuZCBJIGNhbiBub3QNCj4gPiBzYXkgd2hhdCBpcyB0aGUgcHJhY3RpY2Ugb2Ygb3RoZXJz LiBJIG1heSBvbmx5IGd1ZXNzIHRoYXQgbW9zdCB3aWxsIHRyeSB0byB1c2UNCj4gdGhlIGJlc3Qg YW5kIGZhc3Rlc3QsIGkuZS4gNSBtcy4NCj4gPiBBbnl3YXksIHRvIGdldCBhIHJlYWxpc3RpYyBt b2RlbGluZyBvZiBMVEUgaW50ZXJhY3Rpb24gd2l0aCBUQ1AgdGhlDQo+ID4gZGV0YWlsZWQgTUFD IHNjaGVkdWxlciBtb2RlbCBpcyBuZWVkZWQuIFVuZm9ydHVuYXRlbHksIG1vc3Qgb2YgdGhlDQo+ ID4gbGl0ZXJhdHVyZSBmb2N1cyBvbiBzY2hlZHVsaW5nIG9mIHRoZSBQaHlzaWNhbCBSZXNvdXJj ZSBCbG9ja3MsDQo+ID4gQWRhcHRpdmUgTW9kdWxhdGlvbiBhbmQgQ29kaW5nLCBNSU1PLCBpLmUu LCBmb2N1c2luZyBvbiB0aGUNCj4gPiBkaXN0cmlidXRpb24gb2YgcmVzb3VyY2VzIGJldHdlZW4g dXNlcnMsIHdoaWNoIGlzIGdvb2QgYW5kIG5lZWRlZCwgYnV0DQo+ID4gZG8gbm90IGFkZHJlc3Mg dmVyeSBtdWNoIHRoZSB0aW1pbmcgaW50ZXJhY3Rpb24gd2l0aCBUQ1AuICBBdCBsZWFzdCwgSQ0K PiA+IG5ldmVyIG1ldCBpbiBMVEUgc2NoZWR1bGluZyBsaXRlcmF0dXJlIGEgdGVybSAiVENQIHNl bGYtY2xvY2tpbmciLg0KPiA+DQo+ID4gVGhlIDggbXMgIEhZU1RBUlRfREVMQVlfTUlOIGNvdWxk IG9ubHkgYmUgYSBzaW1wbGUgc29sdXRpb24gZm9yIHRoZQ0KPiA+IHByb2JsZW0gb2YgY291bnRp bmcgb2YgdGhlIHJvdW5kJyBjdXJyX3J0dCBmcm9tIHRoZSBzZWNvbmQgQUNLIG9mIHRoZQ0KPiA+ IHRyYWluIChhcyB0aGUgc2Vjb25kIEFDSyBSVFQgd2lsbCBiZSwgdHlwaWNhbGx5LCBkdWUgdG8g c2NoZWR1bGluZywNCj4gPiA1LTcgbXMgbG9uZ2VyIHRoYW4gdGhlIGZpcnN0IG9uZSksIGJ1dCB0 aGlzIG9ubHkgaWYgU1IgcGVyaW9kaWNpdHkgaXMgNSBtcy4NCj4gPiBUaGUgb3RoZXIgMiBwcm9i bGVtcyB3aWxsIHJlbWFpbiB1bnNvbHZlZC4NCj4gPg0KPiA+IFZlYWNlc2xhdiBSb21hbg0KPiA+ DQo+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiBGcm9tOiBJbmdlbWFyIEpvaGFu c3NvbiBTIFttYWlsdG86aW5nZW1hci5zLmpvaGFuc3NvbkBlcmljc3Nvbi5jb21dDQo+ID4gU2Vu dDogVGh1cnNkYXksIDAxIE9jdG9iZXIgMjAxNSAwODo1OQ0KPiA+IFRvOiBWZWFjZXNsYXYgUk9N QU47IEVyaWMgRHVtYXpldDsgTmVhbCBDYXJkd2VsbA0KPiA+IENjOiB0Y3BtQGlldGYub3JnOyBQ aWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgRXJpYyBEdW1hemV0OyBlbmQyZW5kLQ0KPiA+IGlu dGVyZXN0QHBvc3RlbC5vcmc7IEluZ2VtYXIgSm9oYW5zc29uIFMNCj4gPiBTdWJqZWN0OiBSRTog W3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4gPg0KPiA+IEhpDQo+ ID4NCj4gPiBUaGFua3MgZm9yIHRoZSBkZXRhaWxlZCByZXBvcnQuIFdoYXQgc2VlbXMgdW5jbGVh ciB0byBtZSBpcyB0aGUgdmFsdWUNCj4gPiBvZiBIWVNUQVJUX0RFTEFZX01JTi4gWW91IG1lbnRp b24gOG1zIGJ1dCB3aGVuIEkgbG9vayBpbnRvIHRoZSA0LjINCj4gPiBrZXJuZWwgY29kZSBJIGdl dCB0aGUgZmVlbGluZyB0aGF0IGl0IGlzIDMybXMgKCAjZGVmaW5lIEhZU1RBUlRfREVMQVlfTUlO DQo+ID4gKDRVPDwzKSAgICkuIElzIHRoaXMgYSBtaXN0YWtlIGZyb20gbXkgc2lkZSAgPw0KPiA+ IEkgaGF2ZSBydW4gTFRFIHNpbXVsYXRpb25zIGFuZCB0cmllZCB0byB3cmluZyB0aGlzIEh5U3Rh cnQgaXNzdWUNCj4gPiBpbnNpZGUgb3V0IGFuZCB1cHNpZGUgZG93biBidXQgc29mYXIgaXQgaGFz IGJlZW4gdmVyeSBkaWZmaWN1bHQgdG8NCj4gPiBtYWtlIEh5U3RhcnQgZXhpdCBwcmVtYXR1cmVs eSBidXQgdGhlbiBvZiBjb3Vyc2UgSSBoYXZlIHJ1biB3aXRoDQo+ID4gSFlTVEFSVF9ERUxBWV9N SU4gPSAzMm1zLg0KPiA+DQo+ID4gQXMgcmVnYXJkcyB0byB0aGUgQUNLLWNvbXByZXNzaW9uIHBy b2JsZW0sIHllcywgQUNLIGNvbXByZXNzaW9uIGVhc2lseQ0KPiA+IG9jY3VyIGluIExURSwgZXZl biBpbiBzaW11bGF0aW9ucy4gV2hhdCBJIGhhdmUgc2VlbiBpcyB0aGF0IHBhY2tldA0KPiA+IHBh Y2luZyBpcyBhIHZlcnkgZWZmaWNpZW50IHJlbWVkeSwgZm9yIGluc3RhbmNlIHRoZSBwYWNrZXQg aW1wbGVtZW50ZWQNCj4gPiBpbiBRVUlDIHNlZW1zIHRvIHNvbHZlIEFDSyBjb21wcmVzc2lvbiBp c3N1ZXMgdmVyeSB3ZWxsLg0KPiA+DQo+ID4gSXQgaXMgaW50ZXJlc3Rpbmcgd2hhdCB5b3Ugc2F5 IHRoYXQgQUNLIHRyYWlucyBhcmUgc28gbXVjaCBzcGxpdCB1cCwgSQ0KPiA+IGNvdWxkIHVuZGVy c3RhbmQgdGhhdCB0aGV5IGFyZSBzcGxpdCB1cCBpbiB0d28gcGFydHMgKGluaXRpYWwgZ3JhbnQg Kw0KPiA+IGFkZGl0aW9uYWwgYWZ0ZXIgcmVjZWl2ZWQgQlNSKS4NCj4gPg0KPiA+IC9JbmdlbWFy DQo+ID4NCj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gPiBGcm9tOiBWZWFj ZXNsYXYgUk9NQU4gW21haWx0bzpWZWFjZXNsYXYuUm9tYW5Ab3JhbmdlLm1kXQ0KPiA+ID4gU2Vu dDogZGVuIDEgb2t0b2JlciAyMDE1IDAyOjI1DQo+ID4gPiBUbzogRXJpYyBEdW1hemV0OyBOZWFs IENhcmR3ZWxsDQo+ID4gPiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0 YWUgSGE7IEluZ2VtYXIgSm9oYW5zc29uIFM7DQo+ID4gPiBFcmljIER1bWF6ZXQ7IGVuZDJlbmQt aW50ZXJlc3RAcG9zdGVsLm9yZw0KPiA+ID4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJlXSBUQ1Ag SHlTdGFydCBwYXRjaCBkZXBsb3ltZW50DQo+ID4gPg0KPiA+ID4gRmluYWxseSBjYW4gc2hhcmUg c29tZSByZXN1bHRzIG9mIEh5c3RhcnQgaW50ZXJhY3Rpb24gd2l0aCBMVEUuDQo+ID4gPiAgRmV3 IHdvcmRzIGFib3V0IHRoZSB0ZXN0aW5nIGNvbmZpZ3VyYXRpb246DQo+ID4gPiAgCURldmljZSBT YW1zdW5nIEdhbGF4eSBOb3RlNExURSAsIHVwIHRvIDE1MCBNYnBzIGNhcGFibGUNCj4gPiA+ICAg ICAgICAgICAgICBSYWRpbyBOZXR3b3JrIExURSAyNjAwLzE4MDANCj4gPiA+ICAgICAgICAgICAg ICBEaXN0YW5jZSBiZXR3ZWVuIHRoZSBDb3JlIE5ldHdvcmsgKEVQQykgYW5kIEJhc2UgU3RhdGlv biAtIDEwDQo+IGttDQo+ID4gPiAgICAgICAgICAgICAgSFRUUCBTZXJ2ZXIgY29ubmVjdGVkIGRp cmVjdGx5IHRvIHRoZSBDb3JlIE5ldHdvcmsgMSBHYnBzDQo+ID4gPiAgICAgICAgICAgICAgQ29y ZSBuZXR3b3JrIHN3aXRjaGluZyhFUEMpIDEwIEdicHMNCj4gPiA+ICAgICAgICAgICAgICBCYWNr aGF1bCB0cmFuc3BvcnQgIDEgR2JwcywgZnVsbCBJUCwgTVBMUw0KPiA+ID4gICAgICAgICAgICAg IExhc3QgbWlsZSB0cmFuc3BvcnQgZnVsbCBJUCwgTVBMUywgIDMwMCBNYnBzDQo+ID4gPg0KPiA+ ID4gICAgICAgICAgICAgU2VydmVyOiBMaW51eCBrNDBzcnYgNC4xLjAtMDQwMTAwLWdlbmVyaWMg IzIwMTUwNzAzMDk0MA0KPiA+ID4gU01QIEZyaSBKdWwgMw0KPiA+ID4gMDk6NDE6NDcgVVRDIDIw MTUgeDg2XzY0IHg4Nl82NCB4ODZfNjQgR05VL0xpbnV4DQo+ID4gPiAgICAgICAgICAgICAgICAg ICAgICAgICBBcGFjaGUvMi40LjEwIChVYnVudHUpLCBidWlsdDogICBKdWwgMjQgMjAxNSAxNzoy NToxOA0KPiA+ID4NCj4gPiA+ICAgICAgICAgICAgSHlzdGFydF9kZXRlY3Q6IDIgKEhZU1RBUlRf REVMQVkgb25seSksIGFzIHN1Z2dlc3RlZC4NCj4gPiA+ICAgICAgICAgICAgbmV0LmlwdjQudGNw X25vX21ldHJpY3Nfc2F2ZSA9IDEgdG8gYXZvaWQgdGNwIG1ldHJpY3MNCj4gPiA+IGNhY2hpbmcg aW50ZXJmZXJlbmNlDQo+ID4gPg0KPiA+ID4gCVRlc3RzIHR5cGVzOg0KPiA+ID4gCQkxLiBEb3du bG9hZCAzIE1CIGZpbGUsIGludGVyY2hhbmdpbmcgSHlzdGFydCBPTg0KPiA+IGFuZCBPRkYsIGlu DQo+ID4gPiBjb25kaXRpb25zIG9mIExvdyBCYW5kd2lkdGggKDI1LTM1IE1icHMsIHBvb3IgcmFk aW8pIGFuZCBIaWdoDQo+ID4gPiBCYW5kd2lkdGggKDEwMC0xMjAgTWJwcywgZ29vZCByYWRpbyku DQo+ID4gPiAJCTIuIERvd25sb2FkIDEwIE1CIGZpbGUsIGludGVyY2hhbmdpbmcgSHlzdGFydA0K PiA+IE9OIGFuZCBPRkYsIGluDQo+ID4gPiBjb25kaXRpb25zIG9mIExvdyBCYW5kd2lkdGggKDI1 LTM1IE1icHMsIHBvb3IgcmFkaW8pIGFuZCBIaWdoDQo+ID4gPiBCYW5kd2lkdGggKDEwMC0xMjAg TWJwcywgZ29vZCByYWRpbykuDQo+ID4gPg0KPiA+ID4gIAkxMDAgRG93bmxvYWQgdGVzdHMgcGVy IHR5cGUuDQo+ID4gPiAgICAgICAgICAgICAgRG93bmxvYWQgZHVyYXRpb24gYW5kIGJhbmR3aWR0 aCBtZWFzdXJlZCBhdCB0aGUgY2xpZW50DQo+ID4gPiBzaWRlIGFzIHNob3duIGJ5IGN1cmwgZnJv bSB0aGUgZmlyc3QgcmVjZWl2ZWQgZGF0YSBwYWNrZXQgdGlsbCB0aGUNCj4gPiA+IGVuZCBvZiBz ZXNzaW9uICh0aGlzIGV4Y2x1ZGUgRE5TLCAzIHdheSBoYW5kc2hha2UsIGh0dHAgR0VUIGFuZCBp dHMNCj4gQUNLKS4NCj4gPiA+ICAgICAgICAgICAgICBIeXN0YXJ0IGV4aXQgY3duZCBleHRyYWN0 ZWQgZnJvbSBuc3RhdCAodGhhbmsgeW91IEVyaWMNCj4gPiA+IHRvIHBvaW50aW5nIG91dCB0byBp dCkuDQo+ID4gPg0KPiA+ID4gT3ZlcmFsbCByZXN1bHRzOg0KPiA+ID4gCURvd25sb2FkIDNNQiwg TG93IEJhbmR3aWR0aHM6DQo+ID4gPiAJCUh5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAx LjQxIHMNCj4gPiA+ICAJCU5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDEuMzYgcw0K PiA+ID4NCj4gPiA+IAlEb3dubG9hZCAzTUIsIEhpZ2ggQmFuZHdpZHRoczoNCj4gPiA+IAkJSHlz dGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDAuNjUgcw0KPiA+ID4gIAkJTm9IeXN0YXJ0IGF2 ZXJhZ2UgZG93bmxvYWQgdGltZTogMC4zNyAgcw0KPiA+ID4NCj4gPiA+IAlEb3dubG9hZCAxME1C LCBMb3cgQmFuZHdpZHRoczoNCj4gPiA+IAkJSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6 IDIuNDAgcw0KPiA+ID4gIAkJTm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMi4xMiBz DQo+ID4gPg0KPiA+ID4gCURvd25sb2FkIDEwTUIsIEhpZ2ggQmFuZHdpZHRoczoNCj4gPiA+IAkJ SHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDEuNDIgcw0KPiA+ID4gIAkJTm9IeXN0YXJ0 IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC44OSAgcw0KPiA+ID4NCj4gPiA+IFRoZXNlIHJlc3Vs dHMgc2hvdyB0aGF0IEh5c3RhcnQgaGFzIG5vIHNpZ25pZmljYW50IGltcGFjdCBpbg0KPiA+ID4g Y29uZGl0aW9ucyBvZiBsb3cgYXZhaWxhYmxlIGJhbmR3aWR0aCwgYnV0IGF0IGhpZ2ggYXZhaWxh YmxlIHRoZQ0KPiA+ID4gZGVjcmVhc2Ugb2YgcGVyZm9ybWFuY2UgY2FuIHJlYWNoIDYwLTcwICUu DQo+ID4gPiBXaXRoIHRoZSBkZXZlbG9wbWVudCBvZiB0aGUgTFRFLUEgd2hpY2ggdXNlcyBsYXJn ZXIgc3BlY3RydW0gdG8NCj4gPiA+IHJlYWNoDQo+ID4gPiAzMDAgTWJwcyBhbmQgbW9yZSB0aGUg aW1wYWN0IG9mIEh5c3RhcnQgd2lsbCBiZWNvbWUgZXZlbiBtb3JlIHZpc2libGUuDQo+ID4gPg0K PiA+ID4gSSBkbyBzaGFyZSB3aXRoIEVyaWMgYW5kIE5lYWwgc29tZSBmaWxlcyBkZXNjcmliZWQg YmVsb3cgYW5kIGFsc28NCj4gPiA+IGNhbiBwcm92aWRlIGxpbmtzIHRvIGFueW9uZSBpbnRlcmVz dGVkLg0KPiA+ID4NCj4gPiA+IFN1bW1hcnkgb2YgdGhlc2UgdGVzdHMgcmVzdWx0cyBhcmUgZ2F0 aGVyZWQgaW4gdGhlIGZvbGxvd2luZyBFeGNlbA0KPiA+ID4gZmlsZXMgKG5hbWVzLCBJIGJlbGll dmUsIGFyZSBzZWxmIGV4cGxhbmF0b3J5KTogVGVzdF8zTV9sb3dfQlcueGxzeCwNCj4gPiA+IFRl c3RfM01faGlnaF9CVy54bHN4LCBUZXN0XzEwTV9sb3dfQlcueGxzeCwNCj4gPiBUZXN0XzEwTV9o aWdoX0JXLnhsc3guDQo+ID4gPiBTaW1pbGFybHkgdGNwZHVtcCB0cmFjZXMgYXJlIGNhbGxlZCBU ZXN0XzNNX2xvd19CVy5wY2FwLA0KPiA+ID4gVGVzdF8zTV9oaWdoX0JXLiBwY2FwLCBUZXN0XzEw TV9sb3dfQlcuIHBjYXAsDQo+IFRlc3RfMTBNX2hpZ2hfQlcuDQo+ID4gPiBwY2FwIChhYmxlIHRv IGNvbGxlY3Qgb25seSBzZXJ2ZXIgc2lkZSB0cmFjZXMsIGJ1dCB0aGV5IGFyZSBzZWxmIHN1ZmZp Y2llbnQpLg0KPiA+ID4gRm9yIHRoZSAxME0gZG93bmxvYWRzIHRoZXJlIGFyZSBmZXcgbWlzc2lu ZyB0cmFjZXMgYXMgdGhlcmUgdHJhY2UNCj4gPiA+IGZpbGVzIHdlcmUgcmVjeWNsZWQuIEluIHRy YWNlcyBhcmUgYWxzbyBzc2ggc2Vzc2lvbnMsIHBsZWFzZSwgaWdub3JlDQo+ID4gPiB0aGVtLCB0 aGV5IHdlcmUgdXNlZCB0byBwdXQgaHlzdGFydCBPTiBhbmQgT0ZGIG9uIHNlcnZlciBhbmQgZXh0 cmFjdA0KPiA+ID4gbnN0YXQgcmVzdWx0cyBiZWZvcmUgYW5kIGFmdGVyIGVhY2ggZG93bmxvYWQu DQo+ID4gPg0KPiA+ID4gRXhjZWwgZmlsZXMgaGF2ZSAyIHRhYnM6IGN1cmxfc3VtIGFuZCBTdW1t YXJ5Lg0KPiA+ID4gSW4gdGhlIGN1cmxfc3VtIHRhYiB0aGUgc3RhdCBpcyBjdW11bGF0ZWQgaW4g Y2hyb25vbG9naWNhbCBvcmRlciBvZiB0ZXN0cy4NCj4gPiA+IEluIHRoZSBTdW1tYXJ5X3RhYiAg dGhlIEh5c3RhcnQgYW5kIE5vSHlzdGFydCB0ZXN0cyBhcmUgc2hvd24gc2lkZQ0KPiA+ID4gYnkg c2lkZSAoYXMgc2FpZCwgRG93bmxvYWQgMSB3YXMgd2l0aCBIeXN0YXJ0LCBEb3dubG9hZCAyIHdp dGhvdXQsDQo+ID4gPiBEb3dubG9hZCAzIHdpdGggSHlzdGFydCwgRG93bmxvYWQgNCB3aXRob3V0 LCBldGMuKS4gSXQgYWxzbyBpbmNsdWRlDQo+ID4gPiBncmFwaHMgb2YgVHJhbnNmZXIgdGltZSBh bmQgYSBncmFwaCB3aGljaCBzaG93cyB0aGUgZGlzdHJpYnV0aW9uIGluDQo+ID4gPiAlIG9mIGRp ZmZlcmVudCBIeXN0YXJ0IGV4aXQgd2luZG93IERlbGF5Q1dORC4NCj4gPiA+DQo+ID4gPiBGaWVs ZHM6DQo+ID4gPiBjb21tZW50cwktIHRlc3QgY2FzZQ0KPiA+ID4gVHJhbnNmX1RpbWUJLSBjYWxj dWxhdGVkIGZyb20gY3VybCBvdXRwdXQgdGhlIGR1cmF0aW9uDQo+ID4gYmV0d2VlbiBmaXJzdCBk YXRhDQo+ID4gPiBwYWNrZXQgYW5kIGVuZCBvZiB0cmFuc2Zlcg0KPiA+ID4gVHJhbnNmX3NwZWVk CS0gY2FsY3VsYXRlZCBmcm9tIGN1cmwgb3V0cHV0IHRoZSByYXRpbyBvZiBmaWxlIHNpemUNCj4g PiBhbmQgZHVyYXRpb24NCj4gPiA+IGJldHdlZW4gZmlyc3QgZGF0YSBwYWNrZXQgYW5kIGVuZCBv ZiB0cmFuc2Zlcg0KPiA+ID4gSHlzdGFydAlEX0RlbGF5IC0gIGh5c3RhcnQgZXhpdCBvY2N1cnJl bmNlIChkaWZmZXJlbmNlIGJldHdlZW4NCj4gPiA+IFRjcEV4dFRDUEh5c3RhcnREZWxheURldGVj dCBiZWZvcmUgYW5kIGFmdGVyIGRvd25sb2FkKQ0KPiA+ID4gRF9EZWxheUN3bmQJICAtIEh5c3Rh cnQgZXhpdCBjd25kIGZvciB0aGlzIGRvd25sb2FkDQo+ID4gPiAoZGlmZmVyZW5jZSBvZiBUY3BF eHRUQ1BIeXN0YXJ0RGVsYXlDd25kIGJlZm9yZSBhbmQgYWZ0ZXIgdGhlDQo+ID4gPiBkb3dubG9h ZCkgRGF0ZV9Pbl9TZXJ2ZXIgLSB0aW1lIGp1c3QgYmVmb3JlIHRoZSBkb3dubG9hZCBzdGFydCwg dG8NCj4gPiA+IGhlbHAgbmF2aWdhdGUgdGhlIHBjYXANCj4gPiA+DQo+ID4gPiBBZGRpdGlvbmFs bHksIGluIHRoZSBmaWxlIFRlc3RfMTBNX2xvd19CVy54bHN4IGluIHRoZSB0YWIgY3VybF9zdW0N Cj4gPiA+IGFyZSBpbmNsdWRlZCBmb3IgZWFjaCBkb3dubG9hZCB0Y3Auc3RyZWFtIGV4dHJhY3Qg ZnJvbSBwY2FwIG9mIHRoZQ0KPiA+ID4gZmlyc3QNCj4gPiA+IDEwMCB0Y3AuYW5hbHlzaXMuYWNr X3J0dCB3aGljaCBtYXkgaGVscCBpbiB1bmRlcnN0YW5kaW5nIG9mIHRoZQ0KPiA+ID4gaHlzdGFy dCBiZWhhdmlvciAoZmllbGRzIGFja19ydHRfMSB0byBhY2tfcnR0XzEwMCkuDQo+ID4gPiBJbiB0 aGlzIGZpbGUgdGhlcmUgYXJlIGFsc28gZmllbGRzIFVSSV9mcmFtZSwgVVJJX3RpbWUsIFVSSV90 aW1lX3JlbGF0aXZlLA0KPiA+ID4gCXRjcF9zdHJlYW0uDQo+ID4gPiBUaGVuIHRoZXJlIGFyZSBj YWxjdWxhdGVkIGZpZWxkczogbWluUlRUIGFzIGEgbWluaW11bSBvZiBhbGwgMTAwIEFDSywNCj4g cjFfNSwNCj4gPiA+IHIyXzgsIHIzXzgsCXI0Xzggd2hpY2ggc2ltdWxhdGUgSHlzdGFydCBjYWxj dWxhdGlvbnMgb2YgdGhlDQo+IHJvdW5kIGN1cnJfcnR0DQo+ID4gPiBhbmQgcjEgbWluLCByMiBt aW4sIHIzIG1pbiwgcjQgbWluIHJlcHJlc2VudCBtaW5SVFQgb2YgdGhlIHdob2xlIHJvdW5kLg0K PiA+ID4gUGxlYXNlLCBjb3JyZWN0IG1lIGlmIEkgYW0gd3JvbmcsIGJ1dCBpbiBteSB1bmRlcnN0 YW5kaW5nIGZvciBlYWNoDQo+ID4gPiBBQ0sgdGhlIGh5c3RhcnRfdXBkYXRlIGlzIGNhbGxlZCBi ZWZvcmUgdGhlIGh5c3RhcnRfcmVzZXQsIGFuZCBhcyBhDQo+ID4gPiByZXN1bHQgdGhlIGNvbXB1 dGF0aW9uIG9mIGN1cl9ydHQgb2YgdGhlIDggcGFja2V0cyBvZiB0aGUgcm91bmQNCj4gPiA+IHN0 YXJ0cyB3aXRoIHRoZSAyLW5kIHBhY2tldCBvZiB0aGUgcm91bmQsIG5vdCB0aGUgZmlyc3Qgb25l Lg0KPiA+ID4gIEkndmUgdXNlIHIxXzUgYXMgYSBtaW4oYWNrX3J0dF83Li4uYWNrX3J0dF8xMSks IGR1ZSB0bw0KPiA+ID4gaHlzdGFydF9sb3dfd2luZG93PTE2LCByMl84IGFzIGEgbWluKGFja19y dHRfMTIgLi4uIGFja19ydHRfMTkpLA0KPiA+ID4gcjNfOCBhcyBhDQo+ID4gPiBtaW4oYWNrX3J0 dF8zMiAuLi4gYWNrX3J0dF8zOSkgYW5kICksIHI0XzggYXMgYSBtaW4oYWNrX3J0dF83MiAuLi4N Cj4gPiBhY2tfcnR0Xzc5KS4NCj4gPiA+IFRvIHVuZGVyc3RhbmQgdGhlIGltcGFjdCBvZiB0aGlz IHRoZXJlIGFyZSBhbHNvIHIxIEhPTCwgcjIgSE9MLCByMw0KPiA+ID4gSE9MLCByNCBIT0wgKEhl YWQgT2YgTGluZSkgd2hpY2ggY29tcHV0ZSB0aGUgc2FtZSB3aXRoIGZpcnN0IHBhY2tldA0KPiA+ ID4gaW4gdGhlIHJvdW5kIGluY2x1ZGVkIGluIHRoZSByZWxldmFudCByb3VuZDogIHIxXzUgYXMg YQ0KPiA+ID4gbWluKGFja19ydHRfNi4uLmFja19ydHRfMTApLCByMl84IGFzIGEgbWluKGFja19y dHRfMTEgLi4uDQo+ID4gPiBhY2tfcnR0XzE4KSwNCj4gPiA+IHIzXzggYXMgYSBtaW4oYWNrX3J0 dF8zMSAuLi4gYWNrX3J0dF8zOCkgYW5kICksIHI0XzggYXMgYSBtaW4oYWNrX3J0dF83MSAuLi4N Cj4gPiBhY2tfcnR0Xzc4KS4NCj4gPiA+DQo+ID4gPiBJZiBteSB1bmRlcnN0YW5kaW5nIG9mIEh5 c3RhcnQgaXMgY29ycmVjdCBhbmQsIGFzc3VtaW5nIGluIGVhY2gNCj4gPiA+IHRyYWluIHRoZSBm aXJzdCBwYWNrZXQgaGFzIHRoZSBsb3dlc3QgUlRUIHdoaWNoLCBtb3N0IHByb2JhYmx5IGlzDQo+ ID4gPiB2YWxpZCBpbiBhbGwgdHlwZSBvZiBuZXR3b3JrcywgdGhlbiB0aGUgZXhpdCBmcm9tIGh5 c3RhcnQgaW4gdGhpcw0KPiA+ID4gaW1wbGVtZW50YXRpb24gbWF5IGhhcHBlbiBhdCBjd25kIDI5 LCA0OSwgODksIDE2OSwgZXRjLiAoZ2l2ZW4gdGhlDQo+ID4gPiBJVyBvZg0KPiA+IDEwKS4NCj4g PiA+IFRoZXNlIHRlc3RzIHNob3cgdGhhdCB0aGlzIGlzIHRoZSBjYXNlLCBob3dldmVyIHRoZXJl IGFsc28NCj4gPiA+IGludGVybWVkaWF0ZSB2YWx1ZXMuIEhvd2V2ZXIgdGhlIGN3bmQgb2YgMjkg c2hhbGwgYmUgdGhlIGxvd2VzdA0KPiA+ID4gcG9zc2libGUgdmFsdWUgYW5kIHRoaXMgaXMgdGhl IGNhc2UgaW4gdGhlc2UgdGVzdHMuIFVuZm9ydHVuYXRlbHksDQo+ID4gPiBpbiBMVEUgdGhlIGZp cnN0IHBhY2tldCBpbiB0aGUgdHJhaW4gaGFzIGNoYW5jZXMgdG8gYWx3YXlzIGhhdmUgfjYNCj4g PiA+IG1zIGxvd2VyIHRoYW4gdGhlIGZvbGxvd2luZyBvbmVzIGFuZCwgYmVjYXVzZSBpdCBpcyBu b3QgY291bnRlZCBpbg0KPiA+ID4gdGhlIDItbmQgdHJhaW4sIHRoZXJlIGlzIHF1aXRlIGhpZ2gg cGVyY2VudGFnZSBvZiBleGl0IGF0IDI5ICgxNC0zNiUNCj4gPiA+IGluIHRoZXNlIHRlc3RzKSwg ZXZlbiB0aG91Z2ggbGF0ZXIgb24gY3ViaWMgaW5jcmVhc2VzIHRoZSBjd25kDQo+ID4gPiB3aXRo b3V0IGxvc3NlcyB1cCB0byBtYW55IGh1bmRyZWRzLiBGb3IgbWUgdGhpcyBpcyB0b28gZWFybHkg ZXhpdCBmcm9tDQo+IHNsb3cgc3RhcnQuDQo+ID4gPiBBcyBpdCBjYW4gYmUgc2VlbiBmcm9tIGNv bXBhcmlzb24gb2YgdGhlIHNpbXVsYXRlZCBjYWxjdWxhdGlvbiwNCj4gPiA+IGluY2x1c2lvbiBv ZiB0aGUgZmlyc3QgcGFja2V0IGluIHRoZSByb3VuZCBjdXJyX3J0dCBjYWxjdWxhdGlvbg0KPiA+ ID4gd291bGQgZWxpbWluYXRlIHRoZSBjd25kIDI5IGFuZCBzaGlmdCB0byBjd25kIDQ5IG9yIDg5 IG9yIGhpZ2hlci4NCj4gPiA+DQo+ID4gPiBCdXQgdGhlcmUgYXJlIGFsc28gaW50ZXJtZWRpYXRl IHZhbHVlcywgZS5nLiA2OS4gTG9va2luZyB0aHJvdWdoDQo+ID4gPiB0cmFjZXMgb25lIG1heSBz ZWUgdGhhdCB0aGlzIGlzIGR1ZSB0byAiY29tcHJlc3NlZCBBQ0siIHdoaWNoDQo+ID4gPiByZWNl aXZlcyBhIHRpZ2h0bHkgcGFja2VkIHRyYWluIG9mIG1hbnkgQUNLIHNvIHRoYXQgaW4gdHJhY2Vz IHRoZXJlDQo+ID4gPiBhcmUgbm90IGRhdGEgcGFja2V0cyBzZW50IGluIHR1cm4gYnkgc2VydmVy LiBMb29rcyB0byBtZSwgYmVjYXVzZQ0KPiA+ID4gaHlzcmFydF9yZXNldCBzZXRzIHRoZSBlbmRf c2VxIHRvIHNuZF9uZXh0IGFuZCB0aGUgc2VydmVyIGRpZCBub3QNCj4gPiA+IG1hbmFnZWQgdG8g c2VuZCB5ZXQgcGFja2V0cyBpbiByZXBseSB0byB0aGlzIGhpZ2ggc3BlZWQgQUNLIHRyYWluDQo+ ID4gPiB0aGUgc25kX25leHQgcG9pbnRzIHRvIG11Y2ggbG93ZXIgdmFsdWUgdGhhbiB3b3VsZCBu b3JtYWxseSBoYXBwZW4NCj4gPiA+IGFuZCB0aGlzIGFjdHVhbGx5IGRlc3Ryb3lzIHRoZSBjb3Jy ZWN0IGlkZW50aWZpY2F0aW9uIG9mIHRoZSBib3JkZXJzDQo+ID4gPiBvZiB0aGUgcm91bmQsIHRo ZSByb3VuZCBmYWlscyBzb21ld2hlcmUgaW4gdGhlIG1pZGRsZSBvZiB0aGUgdHJhaW4NCj4gPiA+ IG5vdCBhdCBpdHMgYm9yZGVyLCB0aGlzIG1ha2VzIHZlcnkgbGlrZWx5IGluY3JlYXNlZCBkZWxh eSBkdWUgdG8NCj4gPiA+IHF1ZXVlaW5nIGFuZCBlYXJsaWVyIGV4aXQgZnJvbSBzbG93X3N0YXJ0 IGFuZCBpbnRlcm1lZGlhdGUgdmFsdWVzIG9mDQo+ID4gPiBleGl0IGN3bmQgd2hlcmUNCj4gPiB0 aGV5IGFyZSBub3QgZXhwZWN0ZWQuDQo+ID4gPg0KPiA+ID4gQW5kIGxhc3QgYnV0IG5vdCBsZWFz dDogdGhlcmUgYXJlIGdhcHMgKDQtNyBtcykgaW4gQUNLIHRyYWlucy4gT24NCj4gPiA+IHRoZSBw ZXJmZWN0bHkgc2VudCBJVyBvZiAxMCBkYXRhIHBhY2tldHMgdHJhaW4gdGhlIHR5cGljYWwgZm9y IExURQ0KPiA+ID4gd2lsbCBiZSB0byByZWNlaXZlIDEgb3IgMiBBQ0ssIHRoZW4gYSBnYXAgb2Yg NC02IG1zIHRoZW4gYW5vdGhlcg0KPiA+ID4gY291cGxlIG9mIEFDSywgdGhlbiBhbm90aGVyIGdh cCwgdGhlbiA1IEFDSyBjb21wcmVzc2VkIHRvZ2V0aGVyIGluIGENCj4gPiA+IGZyYWN0aW9uIG9m IG1pY3Jvc2Vjb25kLiBPZiBjYXVzZSwgdGhlIG5leHQgdHJhaW4gZnJvbSB0aGUgc2VydmVyDQo+ ID4gPiB3aWxsIGNvbnRhaW4gdGhlIHNhbWUgZ2Fwcy4gVmVyeSBxdWlja2x5LCBlLmcuLCB0aGUg SGVhZCBvZiBMaW5lDQo+ID4gPiAoSE9MKSBvZiB0aGUgMy1kIHRyYWluIHdpbGwgZm9sbG93IGlt bWVkaWF0ZWx5IHRoZSB0YWlsIG9mIHRoZSAyLW5kDQo+ID4gPiB0cmFpbiB3aGljaCB3aWxsIGxl YWQgdG8gcXVldWVpbmcgaW4gdGhlIGVuZCByb3V0ZXIgKGFjdHVhbGx5IHRoZQ0KPiA+ID4gcmFk aW8gYmFzZSBzdGF0aW9uLCBtdWNoIGVhcmxpZXIgdGhhbiBCRFAgcmVhY2hlZCksIGJ1dCB0aGlz IHdpbGwNCj4gPiA+IGxlYWQgdG8gaW5jcmVhc2Ugb2YgdGhlIHRyYWluIFJUVCBhbmQgdG9vIHll YXJseSBleGl0IGZyb20gc2xvdw0KPiA+ID4gc3RhcnQuIFdoeSB0b28gZWFybHkgZXhpdCA/IEJl Y2F1c2UsIHNob3VsZCBzZXJ2ZXIgY29udGludWUNCj4gPiA+IGluY3JlYXNpbmcgdGhlIHNwZWVk IHRoaXMgd291bGQgcmVkdWNlIHF1aWNrZXIgdGhlIGdhcHMgYm90aCBpbg0KPiA+ID4gZG93bmxp bmsgYW5kIHVwbGluaywgdGhlIHByZWljZSBmb3IgdGhpcyBiZWluZyB0aGUgaW5jcmVhc2Ugb2Yg dGhlDQo+ID4gPiBSVFQgKEkgZ3Vlc3MgZG91YmxlKSBhZ2FpbnN0IHdoYXQgb25lIHdvdWxkIG5v cm1hbGx5IHNlZQ0KPiA+IGluIHRoZSBlbmQtdG8tZW5kIEV0aGVybmV0IG5ldHdvcmtzLg0KPiA+ ID4NCj4gPiA+DQo+ID4gPiBJIGRvIGxpa2UgaWRlYXMgb2YgSHlzdGFydCwgYnV0IEkgIGRvbid0 IGtub3cgaG93IGNvdWxkIGl0IGJlDQo+ID4gPiBwb3NzaWJseSByZWNvbmNpbGVkIHdpdGggbXkg TFRFIHByb2JsZW1zLiAgV2l0aCB0aGUgaW5jcmVhc2Ugb2YgTFRFDQo+ID4gPiBzcGVlZHMgdGhp cyBlYXJseSBleGl0IGZyb20gc2xvdyBzdGFydCB3aWxsIGJlY29tZSBldmVuIG1vcmUgb2YgdGhl DQo+IHByb2JsZW0uDQo+ID4gPiBGb3IgTFRFLCBub3Qgc3VyZSBob3cgZ29vZCBhbmQgZ2VuZXJh bGx5IGFjY2VwdGFibGUsIHNvbHV0aW9uIHdvdWxkDQo+ID4gPiBiZSwgcG9zc2libHksIHRvIGlu Y3JlYXNlIHRoZSBIWVNUQVJUX0RFTEFZX01JTiB0byA4IG1zLCB3aGljaCwgYXQNCj4gPiA+IGxl YXN0IGluIG15IGNhc2UsIHdvdWxkIGRpbWluaXNoIHRoZSBlYXJsaWVyIGV4aXQgc2lnbmlmaWNh bnRseSwgaWYNCj4gPiA+IEkgYmVsaWV2ZSBpbiB0aGVzZSB0ZXN0cyByZXN1bHRzLg0KPiA+ID4g QW5vdGhlciBzb2x1dGlvbiwgd2hpY2ggSSBkb24ndCBsaWtlIHZlcnkgbXVjaCwgYnV0IGFzIEkg a25vdyBzb21lDQo+ID4gPiBvcGVyYXRvcnMgdXNlLCB3b3VsZCBiZSB0byBpbnNlcnQgYSBraW5k IG9mIFRDUCBhY2NlbGVyYXRvciBiZXR3ZWVuDQo+ID4gPiB0aGUgSW50ZXJuZXQgYW5kIHRoZSBt b2JpbGUgbmV0d29yayB3aGljaCB3aWxsIGludGVyY2VwdCBhbGwgVENQIGFuZA0KPiA+ID4gd291 bGQgc2VuZCBpdCB0byBtb2JpbGUgd2l0aG91dCBIeXN0YXJ0IGJ1dCB0aGlzIHdvdWxkIGtpbGwg YW55DQo+ID4gPiBlbmQtdG8tZW5kIHByaW5jaXBsZSBhbmQgdGhpcyB3aWxsIG5vdCBiZSB0aGUg SW50ZXJuZXQgYW55bW9yZS4NCj4gPiA+DQo+ID4gPg0KPiA+ID4gSWYgeW91IGhhdmUgdGltZSBh bmQgbmV2ZXIgbG9vayBjbG9zZWx5IHRvIExURSB0ZWNobm9sb2d5IEkgZG8gYQ0KPiA+ID4gcXVp Y2sgc3VtbWFyeSBiZWxsb3cuDQo+ID4gPg0KPiA+ID4NCj4gPiA+IExURSwgVENQIHNlbGYtY2xv Y2tpbmcgYW5kIEFDSyBjb21wcmVzc2lvbi4NCj4gPiA+DQo+ID4gPiBTcGVudCBzb21lIHRpbWUg dG8gdW5kZXJzdGFuZCB0aGUgYmFzaWNzIG9mIHdoYXQgY291bGQgYmUgdGhlIHJlYXNvbg0KPiA+ ID4gZm9yIGFsbCB0aG9zZSB0aW1pbmcgZWZmZWN0cyBkdWUgdG8gTFRFLg0KPiA+ID4gCUZpcnN0 LCB1bmxpa2Ugd2lyZSBiYXNlZCB0cmFuc3BvcnQgdGVjaG5vbG9naWVzLCBMVEUgKGJ1dCBhbHNv DQo+ID4gVU1UUw0KPiA+ID4gYW5kIGZ1dHVyZSA1Rykgc2VuZHMgZGF0YSwgYm90aCB1cGxpbmsg YW5kIGRvd25saW5rIGluIHByZWRlZmluZWQNCj4gPiA+IHRpbWVzbG90cyBjYWxsZWQgVHJhbnNt aXNzaW9uIFRpbWUgSW50ZXJ2YWwgKFRUSSkuIEluIFVNVFMgdGhpcyBpcw0KPiA+ID4gMm1zLCBp biBMVEUgaXMNCj4gPiA+IDEgbXMgYW5kIGl0IHNheXMgdGhhdCBpbiA1RyBpdCB3aWxsIGJlIDAu NSBtcy4NCj4gPiA+IFRoZXNlIFRUSSBoYXZlIGV4YWN0IGJvcmRlcnMgYW5kIHRpZ2h0bHkgZm9s bG93IGVhY2ggb3RoZXIgYW5kDQo+ID4gPiByZXF1aXJlIGV4YWN0IHN5bmNocm9uaXphdGlvbiBi ZXR3ZWVuIHRoZSBtb2JpbGUgZGV2aWNlIGFuZCBSYWRpbw0KPiA+ID4gQmFzZSBTdGF0aW9uIChj YWxsZWQgZU5vZGVCIGluIExURSkuIFRvIGNvcGUgd2l0aCB2YXJ5aW5nIHJhZGlvDQo+ID4gPiBj b25kaXRpb25zIHRoZSBzZW5kaW5nIGRhdGEgYmFuZHdpZHRoIGlzIG1vZGlmaWVkIGluIHN1Y2gg YSB3YXkgdGhhdA0KPiA+ID4gdG8ga2VlcCBhIGNvbnN0YW50IHByb2JhYmlsaXR5IG9mIGRhdGEg bG9zcyBvZiAxMCUuIEJhZCByYWRpbyAtIGxlc3MNCj4gPiA+IGJhbmR3aWR0aCwgZ29vZA0KPiA+ IHJhZGlvIC0gbW9yZSBiYW5kd2lkdGguDQo+ID4gPiBUaGUgYmFuZHdpZHRoIGlzIG1vZGlmaWVk IGJ5IHBhY2tpbmcgbGVzcyBvciBtb3JlIGJpdHMgaW4gYSBUVEkgb2YgMQ0KPiA+ID4gbXMgaW4N Cj4gPiBMVEUuDQo+ID4gPiBCdXQgVFRJIHJlbWFpbiB1bmNoYW5nZWQgb2YgZXhhY3RseSAxIG1z LiBUaGF0IG1lYW5zIHRoYXQsIGF0IHRoZQ0KPiA+ID4gVENQL0lQIGxheWVyIHRoZSBwcmVjaXNp b24gb2YgdGhlIHRpbWluZyBpbmZvcm1hdGlvbiBmcm9tIGRldmljZSB0bw0KPiA+ID4gdGhlIG5l dHdvcmsgaXMgMSBtcy4gSW4gdGhlIGNhc2Ugb2YgYSBiYW5kd2lkdGggb2YgMSBNYnBzIHRoZXJl IHdpbGwNCj4gPiA+IGJlIDgzIElQIHBhY2tldHMgcGVyIHNlY29uZCBvciBvbmUgcGFja2V0IGV2 ZXJ5IDEyIG1zLiBJbiB0aGlzIGNhc2UsDQo+ID4gPiBwb3NzaWJseSB0aGUgcmV0dXJuIG9mIG9u ZSBBQ0sgZXZlcnkgMTIgbXMgKGV2ZXJ5IDEyIFRUSSkgd291bGQgYmUNCj4gPiA+IG1vcmUgb3Ig bGVzcyBzdWZmaWNpZW50IHRvIGtlZXAgYXQgYSBnb29kIGxldmVsIHRoZSBUQ1ANCj4gPiA+IHNl bGYtY2xvY2tpbmcgbWVjaGFuaXNtLiBCdXQsIHdoZW4gdGFsa2luZyBhYm91dCAxMDAgTWJwcyAo b3IgMzAwDQo+ID4gPiBNYnBzKSB0aGF0IG1lYW5zIG9uZSBwYWNrZXQgKGFuZCBvbmUgQUNLKSBl dmVyeSAwLjEyIG1zIGFuZCB0aGVyZSBpcw0KPiA+ID4gbm8gd2F5IHRvIHByb3ZpZGUgaXQgd2l0 aCBhIFRUSSBvZiAxIG1zLiBUaGlzIGlzIHRoZSByZWFzb24gZm9yIEFDSw0KPiA+ID4gY29tcHJl c3Npb24gaW4gTFRFIGFuZCB0aGUgZnV0dXJlIDVHIHdoaWNoIHByb21pc2VzIG1vcmUgdGhhbiAx R2Jwcw0KPiA+ID4gd2l0aCBpdHMgMC41IG1zIFRUSSB3aWxsDQo+ID4gYmUgaW4gYSBiaWcgY29u ZmxpY3Qgd2l0aCBUQ1Agc2VsZi1jbG9ja2luZy4NCj4gPiA+IAlCdXQgdGhlIHByb2JsZW0gZG8g bm90IHN0b3BzIGhlcmUuIFRoZXJlIGlzIGEgc2lnbmlmaWNhbnRseQ0KPiA+IGRpZmZlcmVudA0K PiA+ID4gbWVjaGFuaXNtcyBvZiB0cmFuc21pc3Npb24gaW4gdGhlIGRvd25saW5rIChmcm9tIGVO b2RlQiB0byBtb2JpbGUNCj4gPiA+IGRldmljZSkgYW5kIGluIHVwbGluayAoZnJvbSBtb2JpbGUg ZGV2aWNlIHRvIHRoZSBlTm9kZUIpLiBUaGUNCj4gPiA+IGRpZmZlcmVuY2UgY29tZSBmcm9tIHRo ZSBmYWN0IHRoYXQgdGhlIHNjaGVkdWxlciBmb3IgYm90aCBkb3dubGluaw0KPiA+ID4gYW5kIGlu IHVwbGluayBpcyBsb2NhdGVkIGluIHRoZSBlTm9kZUIuIEFzIG9mIG15IHVuZGVyc3RhbmRpbmcg dGhlDQo+ID4gPiBwdXJwb3NlIG9mIHRoaXMgaXMgdG8gc2ltcGxpZnkgdG8gdGhlIG1heGltdW0g dGhlIGRldmljZSBhbmQgc2F2ZSBpdHMNCj4gYmF0dGVyeS4NCj4gPiA+IEFzIGEgcmVzdWx0IG9m IHRoaXMgZGVzaWduLCB0aGUgZU5vZGVCIGNhbiBzZW5kIGluIGVhY2ggVFRJIHRvd2FyZA0KPiA+ ID4gdGhlIG1vYmlsZSBkZXZpY2Ugd2hlbmV2ZXIgZGF0YSBleGlzdCBpbiBpdHMgYnVmZmVyIChl Tm9kZUIgaXMgYQ0KPiA+ID4ga2luZCBvZiBJUCB3aXJlbGVzcyByb3V0ZXIpLiBCdXQgZm9yIG1v YmlsZSBkZXZpY2UsIGluIG9yZGVyIHRvIHNlbmQNCj4gPiA+IHRoZSBkYXRhIGluIHVwbGluaywg YWZ0ZXIgYSBwYXVzZSwgaXQgaGFzIGZpcnN0IHRvIGFzayB0aGUgbmV0d29yaw0KPiA+ID4gdG8g c2NoZWR1bGUgaXQuIEFuZCBtb2JpbGUgaGFzIGZvciB0aGlzIHJlcXVlc3QgKGFmdGVyIGEgcGF1 c2Ugb2YNCj4gPiA+IHNlbmRpbmcpIGp1c3Qgb25lIGJpdCwgYWN0dWFsbHkgaXMgbm90IGV2ZW4g YSBkYXRhIGJpdCwgaXMgYSBzcGVjaWFsDQo+ID4gPiByYWRpbyBzaWduYWwgd2hpY2ggdGVsbHMg IkhleSwgSSBoYXZlIHNvbWUgZGF0YSBpbiB0aGUgYnVmZmVyIHRvDQo+ID4gPiBzZW5kIi4gTGV0 cyBsb29rIGF0IGluaXRpYWwgdHJhaW4gb2YgSVcgb2YgMTAgcGFja2V0cy4gSW4gZ29vZCByYWRp bw0KPiA+ID4gY29uZGl0aW9ucyB0aGUgZU5vZGVCIHdpbGwgZml0IGFsbCAxMCBwYWNrZXRzIGlu IDEgVFRJIG9mIDEgbXMgYW5kDQo+ID4gPiBzZW5kIHRoZW0gYWxsIHRvIHRoZSBkZXZpY2UuIERl dmljZSB3aWxsIHVucGFjayB0aGVtIGFuZCBnZW5lcmF0ZSAxMA0KPiA+ID4gQUNLIGFuZCB0aGVu IHRlbGwgdG8gdGhlIGVOb2RlQiAiSGV5LCBJIGhhdmUgc29tZSBkYXRhIGluIHRoZSBidWZmZXIN Cj4gPiA+IHRvIHNlbmQiLiBUaGUgZU5vZGVCIGhhcyBub3Qga25vd2xlZGdlIGhvdyBtdWNoIGRh dGEgdGhlIG1vYmlsZSBoYXMNCj4gPiA+IHRvIHNlbmQsIHRoZXJlZm9yIHdpbGwgYWxsb2NhdGUg c29tZSBtaW5pbXVtDQo+ID4gPiBjYXBhY2l0eTogIldlbGwsIEkgZ2l2ZSB5b3UgYSBncmFudCB0 byBzZW5kIHVwIHRvIDIwMCBCeXRlcyBhbmQgeW91DQo+ID4gPiBtdXN0IHNlbmQgdGhlbSBleGFj dGx5IGFmdGVyIDQgbXMgeW91IGdldCB0aGlzIG5vdGlmaWNhdGlvbiIuIFRoZXNlDQo+ID4gPiA0 IG1zIGFyZSBhbGxvd2FuY2UgZm9yIG1vYmlsZSBkZXZpY2UgdG8gcHJlcGFyZSBkYXRhIGZvciBz ZW5kaW5nDQo+ID4gPiB3aGljaCBpcyBhIHZlcnkgY29tcGxpY2F0ZWQgcHJvY2VzcywgdGhlc2Ug NCBtcyBpcyBhIG11c3QgZGVsYXkNCj4gPiA+IGJldHdlZW4gdGhlIGdyYW50DQo+ID4gYW5kIHRo ZSB0cmFuc21pc3Npb24uDQo+ID4gPiBJZiBtb2JpbGUgaGFzIGluIGl0cyBidWZmZXIgMTAgQUNL IHRvIHNlbmQgaXQgd2lsbCBzZW5kIG9ubHkgMiBBQ0sNCj4gPiA+IGFuZCB3aWxsIGNvbXBsZW1l bnQgdGhlbSB3aXRoLCBzbyBjYWxsZWQsIGJ1ZmZlciBzdGF0dXMgcmVwb3J0OiAiSQ0KPiA+ID4g ZG8gaGF2ZSBzdGlsbCA0ODAgYnl0ZXMgdG8gc2VuZCIuIE5vdyBuZXR3b3JrIGhhcyBtb3JlIGlu Zm9ybWF0aW9uDQo+ID4gPiBhbmQgYWxsb2NhdGUgYXMNCj4gPiA+IHJlcXVlc3RlZDogIldlbGws IEkgZ2l2ZSB5b3UgYSBncmFudCB0byBzZW5kIHVwIHRvIDIwMCBCeXRlcyBhbmQgeW91DQo+ID4g PiBtdXN0IHNlbmQgdGhlbSBleGFjdGx5IGFmdGVyIDQgbXMgeW91IGdldCB0aGlzIG5vdGlmaWNh dGlvbiIuIEJ1dCwNCj4gPiA+IGluIHRoZSBtZWFuIHRpbWUgc29tZSBvdGhlciBpbmZvcm1hdGlv biBvZiBoaWdoZXIgcHJpb3JpdHkgbWF5DQo+ID4gPiBhcHBlYXIgaW4gdGhlDQo+ID4gZGV2aWNl IChlLmcuDQo+ID4gPiBzaWduYWxpbmcpIGFuZCB0aGUgZGV2aWNlIHdpbGwgc2VuZCBvbmx5IDYg b3V0IG9mIDggcmVtYWluaW5nIEFDSw0KPiA+ID4gdG9nZXRoZXIgd2l0aCBzaWduYWxpbmcgYW5k IGFub3RoZXIgQnVmZmVyIFN0YXR1cyBSZXBvcnQ6ICJIZXksIEkgZG8NCj4gPiA+IGhhdmUgYW5v dGhlciAxMjAgYnl0ZXMgdG8gc2VuZCIuIEFuZCBhZ2FpbiAiV2VsbCwgSSBnaXZlIHlvdSBhIGdy YW50DQo+ID4gPiB0byBzZW5kIHVwIHRvIDEyMCBCeXRlcyBhbmQgeW91IG11c3Qgc2VuZCB0aGVt IGV4YWN0bHkgYWZ0ZXIgNCBtcw0KPiA+ID4geW91IGdldCB0aGlzIG5vdGlmaWNhdGlvbiIuIEFz IGEgcmVzdWx0IHRoZSBzZXJ2ZXIgd2lsbCByZWNlaXZlIDINCj4gPiA+IEFDSywgdGhlbiBhZnRl ciBhIGdhcCBvZiA2IG1zIGFub3RoZXIgNiBBQ0ssIHRoZW4sIGFmdGVyIGFub3RoZXIgNg0KPiA+ ID4gbXMsIHRoZSByZW1haW5pbmcgMiBBQ0suIFdpdGggdGhlIHNjaGVkdWxlciBpbiB0aGUgZU5v ZGVCIHRoZXNlIGdhcHMNCj4gPiA+IGFyZSBpbmV2aXRhYmxlLiBBbmQgb25lIG1vcmUgdGhpbms6 IG1vYmlsZSBwYWNrIHRvZ2V0aGVyIHRob3NlIDYgQUNLDQo+ID4gPiBpbiAxIFRUSSBvZiAxIG1z IGFuZCBzZW5kIHRoZW0sIHRoZW4gdGhlIGVOb2RlQiB1bnBhY2sgdGhlbSBhbmQNCj4gPiA+IHRo ZXJlIGlzIG5vIHdheSB0byByZWNvdmVyIHRoZSBzcGFjaW5nIG9mIDAuMTIgbXMgYmV0d2VlbiB0 aGVtLg0KPiA+ID4gZU5vZGVCIHNpbXBseSBzZW5kIHRoZW0gdG8gdGhlIHNlcnZlciBiYWNrIHRv IGJhY2sgYXQgdGhlIHNwZWVkIG9mDQo+ID4gPiBpdHMgZGF0YSBjYXJkIHdoaWNoIGlzIHR5cGlj YWxseSAxIEdicHMgYW5kIHdlIGZhY2UgdGhlIEFDSw0KPiA+ID4gY29tcHJlc3Npb24uIEFmdGVy IHRoZSBsYXN0IDIgQUNLIHdlcmUgc2VudCB0aGVyZSBpcyBub3RoaW5nIGVsc2UgdG8NCj4gPiA+ IHNlbmQsIHNvIHRoZXJlIGlzIG5vIEJ1ZmZlciBTdGF0dXMgUmVwb3J0LCB0aGVyZWZvcmUsIGFm dGVyIHBhY2tldHMNCj4gPiA+IG9mIHRoZSBuZXh0IHRyYWluIGFycml2ZSBhbmQgQUNLIGFyZSBn ZW5lcmF0ZWQgZXZlcnl0aGluZyBzdGFydHMNCj4gPiA+IGFnYWluIHdpdGggb25lIGJpdCBvZiBp bmZvcm1hdGlvbiAiSGV5LCBJIGhhdmUNCj4gPiBzb21lIGRhdGEgaW4gdGhlIGJ1ZmZlciB0byBz ZW5kIi4NCj4gPiA+IFRoYXQncyB3aHkgdGhlcmUgYXJlIGdhcHMgYW5kIEFDSyBjb21wcmVzc2lv biwgbG9zcywgb3IgbW9yZQ0KPiA+ID4gZXhhY3RseSwgbGFjayBvZiBhbnkgdGltaW5nIGluZm9y bWF0aW9uIHdoaWNoIHdvdWxkIGJlIHVzZWZ1bCBmb3INCj4gPiA+IHRoZSBUQ1Agc2VsZi0NCj4g PiBjbG9ja2luZy4NCj4gPiA+DQo+ID4gPiBUaGFuayB5b3UgYW5kIHNvcnJ5IGZvciB0YWtpbmcg eW91ciB0aW1lLg0KPiA+ID4NCj4gPiA+DQo+ID4gPiBWZWFjZXNsYXYgUm9tYW4NCj4gPiA+DQo+ ID4gPg0KPiA+ID4NCj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gPiBGcm9t OiBWZWFjZXNsYXYgUk9NQU4NCj4gPiA+IFNlbnQ6IFNhdHVyZGF5LCAwNSBTZXB0ZW1iZXIgMjAx NSAwMToxMA0KPiA+ID4gVG86ICdFcmljIER1bWF6ZXQnOyBOZWFsIENhcmR3ZWxsDQo+ID4gPiBD YzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0YWUgSGE7IEluZ2VtYXIgSm9o YW5zc29uIFM7DQo+ID4gPiBFcmljIER1bWF6ZXQ7IGVuZDJlbmQtaW50ZXJlc3RAcG9zdGVsLm9y Zw0KPiA+ID4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBwYXRjaCBkZXBs b3ltZW50DQo+ID4gPg0KPiA+ID4gSWYgd2UgbG9vayBhdCB0aGUgZmlyc3QgdHJhaW46IGFsbCBw YWNrZXRzIHJlY2VpdmVkIGluIGxlc3MgdGhhbiAxIG1zLg0KPiA+ID4gUHJvYmFibHkgdGhpcyBp cyBvbmx5IGFuIGFwcGVhcmFuY2UgYXMgTFRFIHRyYW5zbWl0cyBpbiwgc28gY2FsbGVkDQo+ID4g PiB0cmFuc21pc3Npb24gdGltZSBpbnRlcnZhbCAoVFRJKSBvZiAxIG1zLCBhbmQgd2hhdCB3ZSBz ZWUgaGVyZSBpcw0KPiA+ID4gdGhhdCBhbGwgMTAgcGFja2V0cyBvZiBpbml0aWFsIHdpbmRvdyBm aXR0ZWQgaW4gMSBtcywgYW5kLCB3aGVuDQo+ID4gPiBkZWNvZGVkLCB3ZXJlIHByZXNlbnRlZCB0 byB0aGUgVENQL0lQIGxheWVyIChhbmQgcGNhcCkgYWxsIGF0IG9uY2UuDQo+ID4gPiBCVFcsIDgg cGFja2V0cyBvZiAxMzAyMiBieXRlcyBpbiAxIG1zIG1lYW5zIGluc3RhbnRhbmVvdXMgc3BlZWQg b2YNCj4gPiA+IDEwNCBNYnBzLCBnb29kIHJhZGlvIGNvbmRpdGlvbnMuIFRDUCBnZW5lcmF0ZXMg MTAgQUNLIHRyYWluIG9mIHRoZQ0KPiA+ID4gZHVyYXRpb24gb2YNCj4gPiA+IDEuMiBtcy4gV2ls bCBpdCBiZSBhIEZhc3QgRXRoZXJuZXQgcG9zc2libHkgd2Ugd291bGQgY29uc2lkZXIgdGhpcw0K PiA+ID4gbm9ybWFsLA0KPiA+IGlzbid0IGl0ID8NCj4gPiA+IEkgbG9va2VkIGF0IGhvdyBzZXJ2 ZXIgcmVwbHkgdG8gdGhlc2UgMTAgQUNLcy4gVGhlcmUgYXJlIDMgZ2FwcyBvZg0KPiA+ID4gNCwN Cj4gPiA+IDIgYW5kIDUgbXMgaW4gdGhlIHJlcGx5IHRyYWluIGFuZCB0aGUgdG90YWwgdHJhaW4g b2YgMTggKD8sIGl0DQo+ID4gPiBzaG91bGQgYmUgMjApIHBhY2tldHMgcmVhY2hlcyBhIGR1cmF0 aW9uIG9mIDE0IG1zLiBBRkFJSyBMVEUgbWF5DQo+ID4gPiBpbnRyb2R1Y2UgZ2FwcyBpbiBBQ0sg dHJhaW4gZHVlIHRvIHVwbGluayBzY2hlZHVsaW5nIG1lY2hhbmlzbS4NCj4gPiA+IE1heSBiZSB0 aGVzZSBnYXBzIHRyaWdnZXIgSHlzdGFydCBlYXJseSBleGl0ID8NCj4gPiA+DQo+ID4gPiBWZWFj ZXNsYXYgUm9tYW4NCj4gPiA+IFRlY2huaWNhbCBhbmQgSVQgZGlyZWN0b3INCj4gPiA+IE9yYW5n ZSBNb2xkb3ZhIFMuQS4NCj4gPiA+IEZpeDogKzM3MzIyNTc1NDAwDQo+ID4gPiBNb2I6ICszNzM2 OTE5ODQwMA0KPiA+ID4gRmF4OiArMzczMjI1NzUzMDYNCj4gPiA+DQo+ID4gPiAtLS0tLU9yaWdp bmFsIE1lc3NhZ2UtLS0tLQ0KPiA+ID4gRnJvbTogRXJpYyBEdW1hemV0IFttYWlsdG86ZXJpYy5k dW1hemV0QGdtYWlsLmNvbV0NCj4gPiA+IFNlbnQ6IEZyaWRheSwgMDQgU2VwdGVtYmVyIDIwMTUg MTk6NDgNCj4gPiA+IFRvOiBOZWFsIENhcmR3ZWxsDQo+ID4gPiBDYzogVmVhY2VzbGF2IFJPTUFO OyB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsNCj4gPiA+IEluZ2Vt YXIgSm9oYW5zc29uIFM7IEVyaWMgRHVtYXpldDsgZW5kMmVuZC1pbnRlcmVzdEBwb3N0ZWwub3Jn DQo+ID4gPiBTdWJqZWN0OiBSZTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxv eW1lbnQNCj4gPiA+DQo+ID4gPiBPbiBGcmksIDIwMTUtMDktMDQgYXQgMTE6MzIgLTA0MDAsIE5l YWwgQ2FyZHdlbGwgd3JvdGU6DQo+ID4gPiA+IEhpIFZlYWNlc2xhdiwNCj4gPiA+ID4NCj4gPiA+ ID4gSSBhZ3JlZSB0aGF0IGluIHlvdXIgTFRFIHRyYWNlcyBpdCBsb29rcyBsaWtlIENVQklDIEh5 c3RhcnQgaXMNCj4gPiA+ID4gZXhpdGluZyBzbG93IHN0YXJ0IHRvbyBlYXJseS4NCj4gPiA+ID4N Cj4gPiA+ID4gU0luY2UgeW91IHNlZW0gdG8gaGF2ZSBhIG5pY2UgTFRFIHRlc3RiZWQsIHdvdWxk IHlvdSBiZSBhYmxlIHRvIGRvDQo+ID4gPiA+IHNvbWUgZXhwZXJpbWVudHMgdG8gZmluZCBhIHNl dCBvZiBwYXJhbWV0ZXJzIGZvciBIeXN0YXJ0IHRoYXQgd29yaw0KPiA+ID4gPiBiZXR0ZXIgZm9y IHlvdXIgTFRFIGVudmlyb25tZW50PyBGb3IgZXhhbXBsZSwgeW91IG1pZ2h0IHRyeSB0aGUNCj4g PiA+ID4gdHdvIHZhcmlhdGlvbnMgSSBzdWdnZXN0ZWQgZWFybGllciBpbiB0aGUgdGhyZWFkOg0K PiA+ID4gPg0KPiA+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICBpZiAoY2EtPmN1cnJfcnR0 ID4gY2EtPmRlbGF5X21pbiArDQo+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBI WVNUQVJUX0RFTEFZX1RIUkVTSChjYS0+ZGVsYXlfbWluID4+DQo+ID4gPiA+IDIpKSB7IG9yDQo+ ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgIGlmIChjYS0+Y3Vycl9ydHQgPiBjYS0+ZGVs YXlfbWluICsNCj4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAgIEhZU1RBUlRfREVM QVlfVEhSRVNIKGNhLT5kZWxheV9taW4gPj4NCj4gPiA+ID4gMSkpIHsNCj4gPiA+ID4NCj4gPiA+ ID4gRG8gYW55IG9mIHRob3NlIGdpdmUgYmV0dGVyIHJlc3VsdHMgZm9yIHlvdXIgdGVzdHM/DQo+ ID4gPiA+DQo+ID4gPiA+IG5lYWwNCj4gPiA+DQo+ID4gPiBBbHNvLCBoeXN0YXJ0IGlzIGZvb2xl ZCBieSB0b28gbWFueSBBQ0sgcmVjZWl2ZWQgaW4gc2hvcnQgcGVyaW9kLg0KPiA+ID4NCj4gPiA+ IFRoaXMgcHJvYmxlbSB3b3VsZCBiZSBzb2x2ZWQgaWYgR1JPIHdhcyB1c2VkIGF0IHJlY2VpdmVy LCBhcyBsZXNzDQo+ID4gPiBBQ0sgd291bGQgYmUgc2VudC4NCj4gPiA+DQo+ID4gPiBQcmVzdW1h Ymx5IHJlY2VpdmVyIGlzIG5vdCBhIGxpbnV4IFRDUCBzdGFjayA/DQo+ID4gPg0KPiA+ID4gTWF5 YmUgd2Ugc2hvdWxkIGFkZCBhIGxvZ2ljIGluIGh5c3RhcnRfdXBkYXRlKCkgdG8gdGFrZSBvbmUg QUNLIHBlciBtcy4NCj4gPiA+DQo+ID4gPiAwNTozMjoyNC4xMzQxNzIgSVAgOTQuMjQzLjEwNC4x NTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MNCj4gPiA+IFtTXSwgc2VxIDQyOTQ0 MjMyMTUsIHdpbiA2NTUzNSwgb3B0aW9ucyBbbXNzIDE0NjAsc2Fja09LLFRTIHZhbA0KPiA+ID4g MTExMjM1MSBlY3IgMCxub3Asd3NjYWxlIDhdLCBsZW5ndGggMA0KPiA+ID4gMDU6MzI6MjQuMTYx OTg2IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdzDQo+ ID4gPiBbUy5dLCBzZXEgMTE5ODg3NzcyMSwgYWNrIDQyOTQ0MjMyMTYsIHdpbiAyODk2MCwgb3B0 aW9ucyBbbXNzDQo+ID4gPiAxNDE2LHNhY2tPSyxUUyB2YWwNCj4gPiA+IDI4MDg4MDg2MzQgZWNy IDExMTIzNTEsbm9wLHdzY2FsZSA3XSwgbGVuZ3RoIDANCj4gPiA+IDA1OjMyOjI0LjE2MjEwOCBJ UCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncw0KPiA+ID4g Wy5dLCBhY2sgMSwgd2luIDM0Mywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1NCBlY3IN Cj4gPiA+IDI4MDg4MDg2MzRdLCBsZW5ndGggMA0KPiA+ID4gMDU6MzI6MjQuMTYzMDcyIElQIDk0 LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzDQo+ID4gPiBbUC5d LCBzZXEgMTo2NTgsIGFjayAxLCB3aW4gMzQzLCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEy MzU0IGVjcg0KPiA+ID4gMjgwODgwODYzNF0sIGxlbmd0aCA2NTcNCj4gPiA+IDA1OjMyOjI0LjIw Mzk4NCBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncw0K PiA+ID4gWy5dLCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAyODA4 ODA4Njc1IGVjcg0KPiA+ID4gMTExMjM1NF0sIGxlbmd0aCAwDQo+ID4gPiAwNTozMjoyNC4yMTAy NzUgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MNCj4g PiA+IFtQLl0sIHNlcSAxOjM4NywgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxU UyB2YWwNCj4gPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMzg2DQo+ID4gPiAw NTozMjoyNC4yMTAzMTUgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjky NDogRmxhZ3MNCj4gPiA+IFsuXSwgc2VxIDM4NzoxNzkxLCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRp b25zIFtub3Asbm9wLFRTIHZhbA0KPiA+ID4gMjgwODgwODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0 aCAxNDA0DQo+ID4gPiAwNTozMjoyNC4yMTAzMzIgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4y NDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MNCj4gPiA+IFsuXSwgc2VxIDE3OTE6MzE5NSwgYWNrIDY1 OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4gPiA+IDI4MDg4MDg2NzcgZWNy IDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+ID4gMDU6MzI6MjQuMjEwMzQ3IElQIDE5NS45NS4x NzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IEZsYWdzDQo+ID4gPiBbLl0sIHNlcSAz MTk1OjQ1OTksIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4g PiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPiA+IDA1OjMyOjI0LjIx MDM1MCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncw0K PiA+ID4gWy5dLCBhY2sgMzg3LCB3aW4gMzQ3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEy MzU5IGVjcg0KPiA+ID4gMjgwODgwODY3N10sIGxlbmd0aCAwDQo+ID4gPiAwNTozMjoyNC4yMTAz NjEgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MNCj4g PiA+IFsuXSwgc2VxIDQ1OTk6NjAwMywgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5v cCxUUyB2YWwNCj4gPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+ ID4gMDU6MzI6MjQuMjEwMzc0IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYu MzI5MjQ6IEZsYWdzDQo+ID4gPiBbLl0sIHNlcSA2MDAzOjc0MDcsIGFjayA2NTgsIHdpbiAyMzcs IG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwg bGVuZ3RoIDE0MDQNCj4gPiA+IDA1OjMyOjI0LjIxMDM4OSBJUCAxOTUuOTUuMTc4LjIwNC44MCA+ IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncw0KPiA+ID4gWy5dLCBzZXEgNzQwNzo4ODExLCBh Y2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbA0KPiA+ID4gMjgwODgwODY3 NyBlY3IgMTExMjM1NF0sIGxlbmd0aCAxNDA0DQo+ID4gPiAwNTozMjoyNC4yMTA0MDIgSVAgMTk1 Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogRmxhZ3MNCj4gPiA+IFsuXSwg c2VxIDg4MTE6MTAyMTUsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFs DQo+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPiA+IDA1OjMy OjI0LjIxMDQxNiBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBG bGFncw0KPiA+ID4gWy5dLCBzZXEgMTAyMTU6MTE2MTksIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlv bnMgW25vcCxub3AsVFMgdmFsDQo+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3Ro IDE0MDQNCj4gPiA+IDA1OjMyOjI0LjIxMDQxOCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5 NS45NS4xNzguMjA0LjgwOiBGbGFncw0KPiA+ID4gWy5dLCBhY2sgMTc5MSwgd2luIDM1OCwgb3B0 aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3INCj4gPiA+IDI4MDg4MDg2NzddLCBsZW5n dGggMA0KPiA+ID4gMDU6MzI6MjQuMjEwNDMxIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQz LjEwNC4xNTYuMzI5MjQ6IEZsYWdzDQo+ID4gPiBbLl0sIHNlcSAxMTYxOToxMzAyMywgYWNrIDY1 OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwNCj4gPiA+IDI4MDg4MDg2NzcgZWNy IDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+ID4gMDU6MzI6MjQuMjEwNDU1IElQIDk0LjI0My4x MDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzDQo+ID4gPiBbLl0sIGFjayAz MTk1LCB3aW4gMzY5LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjcg0KPiA+ID4g MjgwODgwODY3N10sIGxlbmd0aCAwDQo+ID4gPiAwNTozMjoyNC4yMTA2ODUgSVAgOTQuMjQzLjEw NC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MNCj4gPiA+IFsuXSwgYWNrIDQ1 OTksIHdpbiAzODAsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyDQo+ID4gPiAy ODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4gPiA+IDA1OjMyOjI0LjIxMDczMSBJUCA5NC4yNDMuMTA0 LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncw0KPiA+ID4gWy5dLCBhY2sgNjAw Mywgd2luIDM5MSwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3INCj4gPiA+IDI4 MDg4MDg2NzddLCBsZW5ndGggMA0KPiA+ID4gMDU6MzI6MjQuMjEwOTM3IElQIDk0LjI0My4xMDQu MTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IEZsYWdzDQo+ID4gPiBbLl0sIGFjayA3NDA3 LCB3aW4gNDAyLCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjcg0KPiA+ID4gMjgw ODgwODY3N10sIGxlbmd0aCAwDQo+ID4gPiAwNTozMjoyNC4yMTEwNzggSVAgOTQuMjQzLjEwNC4x NTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogRmxhZ3MNCj4gPiA+IFsuXSwgYWNrIDg4MTEs IHdpbiA0MTMsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyDQo+ID4gPiAyODA4 ODA4Njc3XSwgbGVuZ3RoIDANCj4gPiA+IDA1OjMyOjI0LjIxMTIyOCBJUCA5NC4yNDMuMTA0LjE1 Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncw0KPiA+ID4gWy5dLCBhY2sgMTAyMTUs IHdpbiA0MjQsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyDQo+ID4gPiAyODA4 ODA4Njc3XSwgbGVuZ3RoIDANCj4gPiA+IDA1OjMyOjI0LjIxMTM3NyBJUCA5NC4yNDMuMTA0LjE1 Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncw0KPiA+ID4gWy5dLCBhY2sgMTE2MTks IHdpbiA0MzUsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyDQo+ID4gPiAyODA4 ODA4Njc3XSwgbGVuZ3RoIDANCj4gPiA+IDA1OjMyOjI0LjIxMTU0NCBJUCA5NC4yNDMuMTA0LjE1 Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiBGbGFncw0KPiA+ID4gWy5dLCBhY2sgMTMwMjMs IHdpbiA0NDYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyDQo+ID4gPiAyODA4 ODA4Njc3XSwgbGVuZ3RoIDANCj4gPiA+IDA1OjMyOjI0LjIzNzk5OSBJUCAxOTUuOTUuMTc4LjIw NC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncw0KPiA+ID4gWy5dLCBzZXEgMTMwMjM6 MTQ0MjcsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4gPiAy ODA4ODA4NzA5IGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4gPiA+IDA1OjMyOjI0LjIzODA1 MyBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncw0KPiA+ ID4gWy5dLCBzZXEgMTQ0Mjc6MTU4MzEsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxu b3AsVFMgdmFsDQo+ID4gPiAyODA4ODA4NzA5IGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4g PiA+IDA1OjMyOjI0LjIzODA5MCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzgu MjA0LjgwOiBGbGFncw0KPiA+ID4gWy5dLCBhY2sgMTQ0MjcsIHdpbiA0NTcsIG9wdGlvbnMgW25v cCxub3AsVFMgdmFsIDExMTIzNjIgZWNyDQo+ID4gPiAyODA4ODA4NzA5XSwgbGVuZ3RoIDANCj4g PiA+IDA1OjMyOjI0LjIzODE2NCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzgu MjA0LjgwOiBGbGFncw0KPiA+ID4gWy5dLCBhY2sgMTU4MzEsIHdpbiA0NjgsIG9wdGlvbnMgW25v cCxub3AsVFMgdmFsIDExMTIzNjIgZWNyDQo+ID4gPiAyODA4ODA4NzA5XSwgbGVuZ3RoIDANCj4g PiA+IDA1OjMyOjI0LjI0NDAyNCBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2 LjMyOTI0OiBGbGFncw0KPiA+ID4gWy5dLCBzZXEgMTU4MzE6MTcyMzUsIGFjayA2NTgsIHdpbiAy MzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4gPiAyODA4ODA4NzEzIGVjciAxMTEyMzU5 XSwgbGVuZ3RoIDE0MDQNCj4gPiA+IDA1OjMyOjI0LjI0NDA2MyBJUCAxOTUuOTUuMTc4LjIwNC44 MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiBGbGFncw0KPiA+ID4gWy5dLCBzZXEgMTcyMzU6MTg2 MzksIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4gPiAyODA4 ODA4NzEzIGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4gPiA+DQoNCg== From nobody Thu Oct 1 08:17:21 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0CBA51B2D14 for ; Thu, 1 Oct 2015 08:17:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.144 X-Spam-Level: X-Spam-Status: No, score=0.144 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, FRT_BELOW2=2.154, J_CHICKENPOX_36=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8ZBHrGJtcbj0 for ; Thu, 1 Oct 2015 08:17:15 -0700 (PDT) Received: from mailfilter.orange.md (mailfilter.orange.md [94.243.64.204]) by ietfa.amsl.com (Postfix) with ESMTP id 124D51B2D11 for ; Thu, 1 Oct 2015 08:17:13 -0700 (PDT) Received: from localhost.localdomain (antispam.orange.md [127.0.0.1]) by localhost (Postfix) with ESMTP id B8E5327E038; Thu, 1 Oct 2015 18:17:27 +0300 (EEST) Received: from antispam.orange.md by antispam.orange.md with queue id 249713-1; Thu, 01 Oct 2015 15:17:27 GMT Received: from XCHSRV03.main.orange.md (unknown [192.168.200.63]) by mailfilter.orange.md (Postfix) with ESMTP id 941393B27D4; Thu, 1 Oct 2015 18:17:27 +0300 (EEST) Received: from XCHSRV01.main.orange.md ([fe80::685f:aef6:93b0:dea7]) by XCHSRV03.main.orange.md ([fe80::6dbc:28dd:213b:8931%14]) with mapi id 14.02.0328.009; Thu, 1 Oct 2015 18:17:07 +0300 From: Veaceslav ROMAN To: Eric Dumazet , Ingemar Johansson S Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8Pq87vsQ9XoBmgDq86x2kA== Date: Thu, 1 Oct 2015 15:17:06 +0000 Message-ID: <7DBBB686E19D2049ADAACD210B474BB10166E863AD@XCHSRV01.main.orange.md> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se> In-Reply-To: Accept-Language: en-US, ro-RO Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.107.165] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2015 15:17:21 -0000 U3VyZSAxMCBtcyB3aWxsIGJlIGJldHRlciBhcyBtb3JlIG9mIHRoZSBMVEUgdGVjaG5vbG9naWNh bCBqdW1wcyB3aWxsIGJlIGNvdmVyZWQgYW5kLCBhdCBsZWFzdCwgd2lsbCBoZWxwIGdldCByaWQg b2YgZXhpdCBhdCB0aGUgbWluaW11bSBleGl0IGN3bmQgKDI5LCB3aGljaCBhdCAyMCBtcyBSVFQg aXMgfjE2TWJwcykgYW5kIHdpbGwgc2hpZnQgdG8gNDkgKH4yOCBNYnBzKSBvciB0byA4OSAofiA1 MCBNYnBzKS4gIA0KV2UgaGF2ZSBhbGwgbmVjZXNzYXJ5IHRvIHJlcGVhdCB0aG9zZSB0ZXN0cy4g IA0KDQpWZWFjZXNsYXYgUm9tYW4NCg0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJv bTogRXJpYyBEdW1hemV0IFttYWlsdG86ZWR1bWF6ZXRAZ29vZ2xlLmNvbV0gDQpTZW50OiBUaHVy c2RheSwgMDEgT2N0b2JlciAyMDE1IDE1OjQ0DQpUbzogSW5nZW1hciBKb2hhbnNzb24gUw0KQ2M6 IFZlYWNlc2xhdiBST01BTjsgRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsOyB0Y3BtQGlldGYu b3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgZW5kMmVuZC1pbnRlcmVzdEBwb3N0ZWwu b3JnDQpTdWJqZWN0OiBSZTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1l bnQNCg0KWWVzLCB0aGlzIDRtcyB2YWx1ZSBwcm9ibGVtIGlzIGV2ZW4gbW9yZSB2aXNpYmxlIHdp dGggVENQIHBhY2luZyAoYXMgaW1wbGVtZW50ZWQgd2l0aCBsaW51eCBzY2hfZnEgcGFja2V0IHNj aGVkdWxlcikgYmVjYXVzZSBvZiBhcHBhcmVudCBkZWxheXMgY2F1c2VkIGJ5IHNvam91cm4gdGlt ZSBpbiBwYWNrZXQgc2NoZWR1bGVyIGl0c2VsZi4NClRDUCBkb2VzIG5vdCBvZmZzZXQgdGhpcyBz b2pvdXJuIHRpbWUgKGFzIFJUVCBpcyBhbHNvIHVzZWQgZm9yIFJUTyBkZXRlcm1pbmF0aW9uLCBi ZXR0ZXIgaGF2ZSBhbiBpbmZsYXRlZCB2aWV3KQ0KDQpJcyBzd2l0Y2hpbmcgdG8gMTAgbXMgd291 bGQgYmUgYmV0dGVyIGluIHlvdXIgTFRFIHRlc3RzID8NCg0KDQoNCk9uIFRodSwgT2N0IDEsIDIw MTUgYXQgNDo1NyBBTSwgSW5nZW1hciBKb2hhbnNzb24gUyA8aW5nZW1hci5zLmpvaGFuc3NvbkBl cmljc3Nvbi5jb20+IHdyb3RlOg0KPiBIbW0NCj4gWW91IGFyZSByaWdodCAhLg0KPiBXZWxsLi4u IFRoYXQgb2YgY291cnNlIHNob3VsZCBleHBsYWluIHdoeSBIeVN0YXJ0IHBlcmZvcm1zIGJhZGx5 IGZvciBMVEUuLi4gQXRsZWFzdCBpdCBpcyBwYXJ0IG9mIHRoZSByZWFzb24uDQo+DQo+IC9Jbmdl bWFyDQo+DQo+DQo+DQo+PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPj4gRnJvbTogVmVh Y2VzbGF2IFJPTUFOIFttYWlsdG86VmVhY2VzbGF2LlJvbWFuQG9yYW5nZS5tZF0NCj4+IFNlbnQ6 IGRlbiAxIG9rdG9iZXIgMjAxNSAxMzo0Ng0KPj4gVG86IEluZ2VtYXIgSm9oYW5zc29uIFM7IEVy aWMgRHVtYXpldDsgTmVhbCBDYXJkd2VsbA0KPj4gQ2M6IHRjcG1AaWV0Zi5vcmc7IFBpZXJzIE8n SGFubG9uOyBTYW5ndGFlIEhhOyBFcmljIER1bWF6ZXQ7IGVuZDJlbmQtIA0KPj4gaW50ZXJlc3RA cG9zdGVsLm9yZw0KPj4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBwYXRj aCBkZXBsb3ltZW50DQo+Pg0KPj4gV2VsbCB0aGlzIHNheQ0KPj4gICAgICAgSWYgY3Vycl9ydHQg aXMgaGlnaGVyIHRoYW4gZGVsYXlfbWluICsgKDEvOCAqIGRlbGF5X21pbiBvciA0IA0KPj4gbXMs IHdoaWNoZXZlciBpcyBoaWdoZXIpICB0aGVuIGV4aXQgc2xvdyBzdGFydC4NCj4+DQo+PiBXaGVu IGFjdHVhbCBtaW5pbXVtIGRlbGF5IGlzIGxlc3MgdGhhbiAzMiBtcyB0aGUgZGVsYXlfbWluIHdp bGwgYmUgDQo+PiBsZXNzIHRoYW4gMzI8PDMsIHRoZW4gdGhlIGRlbGF5X21pbj4+MyB3aWxsIGJl IGxlc3MgdGhhbiAzMiBhbmQgd2lsbCANCj4+IGNvbXBhcmUgd2l0aA0KPj4gNDw8MyB3aGljaCBp cyAzMiBhbmQgdGhlcmVmb3IgdGhlIHRocmVzaG9sZCB3aWxsIGJlIA0KPj4gKHJlYWxfY3Vycl9y dHQ8PDMpID4NCj4+IChyZWFsX2RlbGF5PDwzICsgNDw8Mykgb3IgKHJlYWxfY3Vycl9ydHQpPihy ZWFsX2RlbGF5ICs0KQ0KPj4NCj4+DQo+PiBWZWFjZXNsYXYgUm9tYW4NCj4+DQo+PiAtLS0tLU9y aWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPj4gRnJvbTogSW5nZW1hciBKb2hhbnNzb24gUyBbbWFpbHRv OmluZ2VtYXIucy5qb2hhbnNzb25AZXJpY3Nzb24uY29tXQ0KPj4gU2VudDogVGh1cnNkYXksIDAx IE9jdG9iZXIgMjAxNSAxMzowMA0KPj4gVG86IFZlYWNlc2xhdiBST01BTjsgRXJpYyBEdW1hemV0 OyBOZWFsIENhcmR3ZWxsDQo+PiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNh bmd0YWUgSGE7IEVyaWMgRHVtYXpldDsgZW5kMmVuZC0gDQo+PiBpbnRlcmVzdEBwb3N0ZWwub3Jn OyBJbmdlbWFyIEpvaGFuc3NvbiBTDQo+PiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBI eVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4+DQo+PiBIaQ0KPj4NCj4+IFllcywgdGhlIHZhcmlh YmxlIGRlbGF5X21pbiBpcyBpbiBRMyBmb3IgcmVhc29ucyBvZiBwcmVjaXNpb24gSSBndWVzcyAN Cj4+IEJ1dCB0aGUgY2FsbCB0byBIWVNUQVJUX0RFTEFZX1RIUkVTSCBpcyB3aXRoIGRlbGF5X21p biBzaGlmdGVkIGRvd24gDQo+PiB0byBRMA0KPj4NCj4+IEhZU1RBUlRfREVMQVlfVEhSRVNIKGNh LT5kZWxheV9taW4gPj4gMykpICAoc2VlIGh0dHA6Ly9seHIuZnJlZS0NCj4+IGVsZWN0cm9ucy5j b20vc291cmNlL25ldC9pcHY0L3RjcF9jdWJpYy5jI0w0MDMgKSBJIG1heSBiZSBvZmYgaGVyZS4u DQo+Pg0KPj4gUmVnYXJkaW5nIHBhY2luZy4gSSBkb24ndCBoYXZlIHJlYWwgZGF0YSB0byBzaG93 IHRoZSBlZmZlY3RzIG9mIA0KPj4gcGFjaW5nLCBpbiBzaW11bGF0aW9ucyBpdCBpcyBob3dldmVy IHBvc3NpYmxlIHRvIHNlZSB0aGUgZ2FpbnMgaW4gDQo+PiB0ZXJtcyBvZiBsZXNzIGNvYWxlc2Np bmcsIHllcyBwYWNpbmcgZG9lcyBub3Qgc29sdmUgdGhlIEFDSyANCj4+IGNvbXByZXNzaW9uIGJ1 dCBpdCBzZWVtIHRvIHNvbHZlIHRoZSBmb2xsb3cgdXAgZWZmZWN0IHRoYXQgQUNLIGNvbXByZXNz aW9uIGdpdmVzLg0KPj4NCj4+IC9JbmdlbWFyDQo+Pg0KPj4gPiAtLS0tLU9yaWdpbmFsIE1lc3Nh Z2UtLS0tLQ0KPj4gPiBGcm9tOiBWZWFjZXNsYXYgUk9NQU4gW21haWx0bzpWZWFjZXNsYXYuUm9t YW5Ab3JhbmdlLm1kXQ0KPj4gPiBTZW50OiBkZW4gMSBva3RvYmVyIDIwMTUgMTA6MzINCj4+ID4g VG86IEluZ2VtYXIgSm9oYW5zc29uIFM7IEVyaWMgRHVtYXpldDsgTmVhbCBDYXJkd2VsbA0KPj4g PiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0YWUgSGE7IEVyaWMgRHVt YXpldDsgDQo+PiA+IGVuZDJlbmQtIGludGVyZXN0QHBvc3RlbC5vcmcNCj4+ID4gU3ViamVjdDog UkU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBwYXRjaCBkZXBsb3ltZW50DQo+PiA+DQo+PiA+ IFllcywgSFlTVEFSVF9ERUxBWV9NSU4gICg0VTw8MykgLg0KPj4gPiBCdXQgY3Vycl9ydHQgYW5k IGRlbGF5X21pbiBjb21lIGZyb20gYmljdGNwX2Fja2VkIHdoaWNoIGlzIGFsc28gPDwzIDoNCj4+ ID4gICAgIEluIHRjcF9jdWJpYy5jLCBmdW5jdGlvbiAgbGluZSA0MzMgKGtlcm5lbCA0LjIpDQo+ PiA+ICAgICAgICAgICAgIGRlbGF5ID0gKHJ0dF91cyA8PCAzKSAvIFVTRUNfUEVSX01TRUM7IFNv LA0KPj4gPiA0VTw8MyBpcyBlcXVpdmFsZW50IHRvIDQgbXMgLg0KPj4gPg0KPj4gPiBSZWdhcmRp bmcgcGFjaW5nIEkgYW0gbm90IHN1cmUgd2lsbCBoZWxwIHRoZSBwZXJmb3JtYW5jZTogdGhlcmUg YXJlIA0KPj4gPiBnYXBzIGluIEFDSyB0cmFpbnMgb2YgdGhlIG9yZGVyIG9mIG1pbGxpc2Vjb25k cyBhbmQgdGhpcyBpcyBhIGxvc3QgDQo+PiA+IHRpbWUgYW5kIHRoZXJlIGlzIG5vIHdheSB0byBj b21wZW5zYXRlIGl0LiBJZiBpbnRyb2R1Y2UgQUNLIHBhY2luZyANCj4+ID4gb3IgZGF0YSBwYWNr ZXRzIHBhY2luZyB0aGlzIHdvdWxkIGFkZCB0byB0aGUgb3ZlcmFsbCBsb3N0IHRpbWUuDQo+PiA+ IFBhY2luZyBjYW4gaGVscCBvbmx5IHRvIGFsbGV2aWF0ZSBhIHBvdGVudGlhbCBwcm9ibGVtIG9m IHNvbWUgDQo+PiA+IGRvd25zdHJlYW0gcm91dGVycyB3aXRoIGluc3VmZmljaWVudCBidWZmZXJz IHdoaWNoIG1heSBpbnRyb2R1Y2UgDQo+PiA+IHVuZXhwZWN0ZWQgcGFja2V0cyBkcm9wcyB3aGVu IGxhcmdlIHRyYWluIHNlbnQgYXQgYSBoaWdoIHNwZWVkIGFzIGEgDQo+PiA+IHJlcGx5IHRvIGhp Z2hseSBjb21wcmVzc2VkIEFDSyB0cmFpbiwgYnV0LCBvbiBhbm90aGVyIGhhbmRzLg0KPj4gPiBC dXQgaG93IG11Y2ggcGFjaW5nIGFuZCBob3cgdG8gYWNjb21tb2RhdGUgdG8gbGFyZ2VseSBjaGFu Z2luZyANCj4+ID4gcmFkaW8gY29uZGl0aW9ucyA/IEFuZCB3aXRoIHBlcm1hbmVudGx5IGNoYW5n aW5nIHRlY2hub2xvZ3kgYW5kIHNwZWVkID8NCj4+ID4gQW55d2F5IHRoZSBwYWNpbmcgd2lsbCBh ZGQgdG8gdGhlIHByb2JsZW0gb2YgZ2FwcyBhbmQgYmV0dGVyIHNpemUgDQo+PiA+IHByb3Blcmx5 IHJvdXRlcnMgYnVmZmVycy4NCj4+ID4NCj4+ID4gSWRlYWxseSBBQ0sgdHJhaW4gaXMgc3BsaXQg aW4gMiBwYXJ0cywgYnV0IGluIHByYWN0aWNlIEkgZG8gc2VlIG1vcmUuDQo+PiA+IEJ1dCBhcyB0 aGUgc3RyZWFtIHByb2dyZXNzIHRoZSBsZXNzIGdhcHMuIFRoZSBiaWdnZXIgdGhlIGRvd25saW5r IA0KPj4gPiBwcmVzc3VyZSB0aGUgbW9yZSBBQ0sgYW5kIGxlc3MgY2hhbmNlcyB0byBoYXZlIGEg cGF1c2UgaW4gQUNLIA0KPj4gPiBzZW5kaW5nLCB0aGVuIHRoZXJlIHdpbGwgYmUgYSBjb250aW51 b3VzIGZsb3cgb2YgQnVmZmVyIFN0YXR1cyANCj4+ID4gUmVwb3J0cyBhbmQgbGVzcyBvZiB0aGUg U2NoZWR1bGluZyBSZXF1ZXN0IChTUiwgIkhleSwgSSBoYXZlIHNvbWUgDQo+PiA+IGRhdGEgdG8g c2VuZCIpLiBUaGUgb25seSBpc3N1ZSBJIGFtIG5vdCBzdXJlIGlzIG9uIHRoZSBwb3RlbnRpYWwg DQo+PiA+IGdhcHMgaW4gb3RoZXIgbmV0d29ya3MuIEkgZGlkIG5vdCBwdXQgYmVsbG93IGFsbCBk ZXRhaWxzLCBidXQgeW91IA0KPj4gPiBzaG91bGQga25vdyB0aGF0IGRldmljZSBjYW4gbm90IHNl bmQgU1IgaW4gYXJiaXRyYXJ5IG1vbWVudHMgb2YgDQo+PiA+IHRpbWUgYnV0IGluIHByZWRlZmlu ZWQgZm9yIGhpbSBpbnRlcnZhbHMuIEluIG15IG5ldHdvcmsgdGhpcyBwZXJpb2QgDQo+PiA+IGlz IDUgbXMsIHRoZXJlZm9yZSB0aGUgUlRUIG9mIHRoZSBmaXJzdCBBQ0sgaW4gdGhlIHRyYWluIHdp bGwgaGF2ZSANCj4+ID4gdmFyaWF0aW9ucyBvZiAyLjUgbXMuIEJ1dCBzdGFuZGFyZHMgYWRtaXQg dGhpcyBwZXJpb2QgdG8gYmUgYWxzbyAxMCANCj4+ID4gbXMgLDIwIG1zLCA0MCBtcywgODAgbXMg YW5kIEkgY2FuIG5vdCBzYXkgd2hhdCBpcyB0aGUgcHJhY3RpY2Ugb2YgDQo+PiA+IG90aGVycy4g SSBtYXkgb25seSBndWVzcyB0aGF0IG1vc3Qgd2lsbCB0cnkgdG8gdXNlDQo+PiB0aGUgYmVzdCBh bmQgZmFzdGVzdCwgaS5lLiA1IG1zLg0KPj4gPiBBbnl3YXksIHRvIGdldCBhIHJlYWxpc3RpYyBt b2RlbGluZyBvZiBMVEUgaW50ZXJhY3Rpb24gd2l0aCBUQ1AgdGhlIA0KPj4gPiBkZXRhaWxlZCBN QUMgc2NoZWR1bGVyIG1vZGVsIGlzIG5lZWRlZC4gVW5mb3J0dW5hdGVseSwgbW9zdCBvZiB0aGUg DQo+PiA+IGxpdGVyYXR1cmUgZm9jdXMgb24gc2NoZWR1bGluZyBvZiB0aGUgUGh5c2ljYWwgUmVz b3VyY2UgQmxvY2tzLCANCj4+ID4gQWRhcHRpdmUgTW9kdWxhdGlvbiBhbmQgQ29kaW5nLCBNSU1P LCBpLmUuLCBmb2N1c2luZyBvbiB0aGUgDQo+PiA+IGRpc3RyaWJ1dGlvbiBvZiByZXNvdXJjZXMg YmV0d2VlbiB1c2Vycywgd2hpY2ggaXMgZ29vZCBhbmQgbmVlZGVkLCANCj4+ID4gYnV0IGRvIG5v dCBhZGRyZXNzIHZlcnkgbXVjaCB0aGUgdGltaW5nIGludGVyYWN0aW9uIHdpdGggVENQLiAgQXQg DQo+PiA+IGxlYXN0LCBJIG5ldmVyIG1ldCBpbiBMVEUgc2NoZWR1bGluZyBsaXRlcmF0dXJlIGEg dGVybSAiVENQIHNlbGYtY2xvY2tpbmciLg0KPj4gPg0KPj4gPiBUaGUgOCBtcyAgSFlTVEFSVF9E RUxBWV9NSU4gY291bGQgb25seSBiZSBhIHNpbXBsZSBzb2x1dGlvbiBmb3IgdGhlIA0KPj4gPiBw cm9ibGVtIG9mIGNvdW50aW5nIG9mIHRoZSByb3VuZCcgY3Vycl9ydHQgZnJvbSB0aGUgc2Vjb25k IEFDSyBvZiANCj4+ID4gdGhlIHRyYWluIChhcyB0aGUgc2Vjb25kIEFDSyBSVFQgd2lsbCBiZSwg dHlwaWNhbGx5LCBkdWUgdG8gDQo+PiA+IHNjaGVkdWxpbmcsDQo+PiA+IDUtNyBtcyBsb25nZXIg dGhhbiB0aGUgZmlyc3Qgb25lKSwgYnV0IHRoaXMgb25seSBpZiBTUiBwZXJpb2RpY2l0eSBpcyA1 IG1zLg0KPj4gPiBUaGUgb3RoZXIgMiBwcm9ibGVtcyB3aWxsIHJlbWFpbiB1bnNvbHZlZC4NCj4+ ID4NCj4+ID4gVmVhY2VzbGF2IFJvbWFuDQo+PiA+DQo+PiA+IC0tLS0tT3JpZ2luYWwgTWVzc2Fn ZS0tLS0tDQo+PiA+IEZyb206IEluZ2VtYXIgSm9oYW5zc29uIFMgW21haWx0bzppbmdlbWFyLnMu am9oYW5zc29uQGVyaWNzc29uLmNvbV0NCj4+ID4gU2VudDogVGh1cnNkYXksIDAxIE9jdG9iZXIg MjAxNSAwODo1OQ0KPj4gPiBUbzogVmVhY2VzbGF2IFJPTUFOOyBFcmljIER1bWF6ZXQ7IE5lYWwg Q2FyZHdlbGwNCj4+ID4gQ2M6IHRjcG1AaWV0Zi5vcmc7IFBpZXJzIE8nSGFubG9uOyBTYW5ndGFl IEhhOyBFcmljIER1bWF6ZXQ7IA0KPj4gPiBlbmQyZW5kLSBpbnRlcmVzdEBwb3N0ZWwub3JnOyBJ bmdlbWFyIEpvaGFuc3NvbiBTDQo+PiA+IFN1YmplY3Q6IFJFOiBbdGNwbV0gW2UyZV0gVENQIEh5 U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPj4gPg0KPj4gPiBIaQ0KPj4gPg0KPj4gPiBUaGFua3Mg Zm9yIHRoZSBkZXRhaWxlZCByZXBvcnQuIFdoYXQgc2VlbXMgdW5jbGVhciB0byBtZSBpcyB0aGUg DQo+PiA+IHZhbHVlIG9mIEhZU1RBUlRfREVMQVlfTUlOLiBZb3UgbWVudGlvbiA4bXMgYnV0IHdo ZW4gSSBsb29rIGludG8gDQo+PiA+IHRoZSA0LjIga2VybmVsIGNvZGUgSSBnZXQgdGhlIGZlZWxp bmcgdGhhdCBpdCBpcyAzMm1zICggI2RlZmluZSBIWVNUQVJUX0RFTEFZX01JTg0KPj4gPiAoNFU8 PDMpICAgKS4gSXMgdGhpcyBhIG1pc3Rha2UgZnJvbSBteSBzaWRlICA/DQo+PiA+IEkgaGF2ZSBy dW4gTFRFIHNpbXVsYXRpb25zIGFuZCB0cmllZCB0byB3cmluZyB0aGlzIEh5U3RhcnQgaXNzdWUg DQo+PiA+IGluc2lkZSBvdXQgYW5kIHVwc2lkZSBkb3duIGJ1dCBzb2ZhciBpdCBoYXMgYmVlbiB2 ZXJ5IGRpZmZpY3VsdCB0byANCj4+ID4gbWFrZSBIeVN0YXJ0IGV4aXQgcHJlbWF0dXJlbHkgYnV0 IHRoZW4gb2YgY291cnNlIEkgaGF2ZSBydW4gd2l0aCANCj4+ID4gSFlTVEFSVF9ERUxBWV9NSU4g PSAzMm1zLg0KPj4gPg0KPj4gPiBBcyByZWdhcmRzIHRvIHRoZSBBQ0stY29tcHJlc3Npb24gcHJv YmxlbSwgeWVzLCBBQ0sgY29tcHJlc3Npb24gDQo+PiA+IGVhc2lseSBvY2N1ciBpbiBMVEUsIGV2 ZW4gaW4gc2ltdWxhdGlvbnMuIFdoYXQgSSBoYXZlIHNlZW4gaXMgdGhhdCANCj4+ID4gcGFja2V0 IHBhY2luZyBpcyBhIHZlcnkgZWZmaWNpZW50IHJlbWVkeSwgZm9yIGluc3RhbmNlIHRoZSBwYWNr ZXQgDQo+PiA+IGltcGxlbWVudGVkIGluIFFVSUMgc2VlbXMgdG8gc29sdmUgQUNLIGNvbXByZXNz aW9uIGlzc3VlcyB2ZXJ5IHdlbGwuDQo+PiA+DQo+PiA+IEl0IGlzIGludGVyZXN0aW5nIHdoYXQg eW91IHNheSB0aGF0IEFDSyB0cmFpbnMgYXJlIHNvIG11Y2ggc3BsaXQgDQo+PiA+IHVwLCBJIGNv dWxkIHVuZGVyc3RhbmQgdGhhdCB0aGV5IGFyZSBzcGxpdCB1cCBpbiB0d28gcGFydHMgKGluaXRp YWwgDQo+PiA+IGdyYW50ICsgYWRkaXRpb25hbCBhZnRlciByZWNlaXZlZCBCU1IpLg0KPj4gPg0K Pj4gPiAvSW5nZW1hcg0KPj4gPg0KPj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ PiA+ID4gRnJvbTogVmVhY2VzbGF2IFJPTUFOIFttYWlsdG86VmVhY2VzbGF2LlJvbWFuQG9yYW5n ZS5tZF0NCj4+ID4gPiBTZW50OiBkZW4gMSBva3RvYmVyIDIwMTUgMDI6MjUNCj4+ID4gPiBUbzog RXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsDQo+PiA+ID4gQ2M6IHRjcG1AaWV0Zi5vcmc7IFBp ZXJzIE8nSGFubG9uOyBTYW5ndGFlIEhhOyBJbmdlbWFyIEpvaGFuc3NvbiANCj4+ID4gPiBTOyBF cmljIER1bWF6ZXQ7IGVuZDJlbmQtaW50ZXJlc3RAcG9zdGVsLm9yZw0KPj4gPiA+IFN1YmplY3Q6 IFJFOiBbdGNwbV0gW2UyZV0gVENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPj4gPiA+DQo+ PiA+ID4gRmluYWxseSBjYW4gc2hhcmUgc29tZSByZXN1bHRzIG9mIEh5c3RhcnQgaW50ZXJhY3Rp b24gd2l0aCBMVEUuDQo+PiA+ID4gIEZldyB3b3JkcyBhYm91dCB0aGUgdGVzdGluZyBjb25maWd1 cmF0aW9uOg0KPj4gPiA+ICAgRGV2aWNlIFNhbXN1bmcgR2FsYXh5IE5vdGU0TFRFICwgdXAgdG8g MTUwIE1icHMgY2FwYWJsZQ0KPj4gPiA+ICAgICAgICAgICAgICBSYWRpbyBOZXR3b3JrIExURSAy NjAwLzE4MDANCj4+ID4gPiAgICAgICAgICAgICAgRGlzdGFuY2UgYmV0d2VlbiB0aGUgQ29yZSBO ZXR3b3JrIChFUEMpIGFuZCBCYXNlIA0KPj4gPiA+IFN0YXRpb24gLSAxMA0KPj4ga20NCj4+ID4g PiAgICAgICAgICAgICAgSFRUUCBTZXJ2ZXIgY29ubmVjdGVkIGRpcmVjdGx5IHRvIHRoZSBDb3Jl IE5ldHdvcmsgMSBHYnBzDQo+PiA+ID4gICAgICAgICAgICAgIENvcmUgbmV0d29yayBzd2l0Y2hp bmcoRVBDKSAxMCBHYnBzDQo+PiA+ID4gICAgICAgICAgICAgIEJhY2toYXVsIHRyYW5zcG9ydCAg MSBHYnBzLCBmdWxsIElQLCBNUExTDQo+PiA+ID4gICAgICAgICAgICAgIExhc3QgbWlsZSB0cmFu c3BvcnQgZnVsbCBJUCwgTVBMUywgIDMwMCBNYnBzDQo+PiA+ID4NCj4+ID4gPiAgICAgICAgICAg ICBTZXJ2ZXI6IExpbnV4IGs0MHNydiA0LjEuMC0wNDAxMDAtZ2VuZXJpYyANCj4+ID4gPiAjMjAx NTA3MDMwOTQwIFNNUCBGcmkgSnVsIDMNCj4+ID4gPiAwOTo0MTo0NyBVVEMgMjAxNSB4ODZfNjQg eDg2XzY0IHg4Nl82NCBHTlUvTGludXgNCj4+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICBB cGFjaGUvMi40LjEwIChVYnVudHUpLCBidWlsdDogICBKdWwgMjQgMjAxNSAxNzoyNToxOA0KPj4g PiA+DQo+PiA+ID4gICAgICAgICAgICBIeXN0YXJ0X2RldGVjdDogMiAoSFlTVEFSVF9ERUxBWSBv bmx5KSwgYXMgc3VnZ2VzdGVkLg0KPj4gPiA+ICAgICAgICAgICAgbmV0LmlwdjQudGNwX25vX21l dHJpY3Nfc2F2ZSA9IDEgdG8gYXZvaWQgdGNwIG1ldHJpY3MgDQo+PiA+ID4gY2FjaGluZyBpbnRl cmZlcmVuY2UNCj4+ID4gPg0KPj4gPiA+ICAgVGVzdHMgdHlwZXM6DQo+PiA+ID4gICAgICAgICAg IDEuIERvd25sb2FkIDMgTUIgZmlsZSwgaW50ZXJjaGFuZ2luZyBIeXN0YXJ0IE9ODQo+PiA+IGFu ZCBPRkYsIGluDQo+PiA+ID4gY29uZGl0aW9ucyBvZiBMb3cgQmFuZHdpZHRoICgyNS0zNSBNYnBz LCBwb29yIHJhZGlvKSBhbmQgSGlnaCANCj4+ID4gPiBCYW5kd2lkdGggKDEwMC0xMjAgTWJwcywg Z29vZCByYWRpbykuDQo+PiA+ID4gICAgICAgICAgIDIuIERvd25sb2FkIDEwIE1CIGZpbGUsIGlu dGVyY2hhbmdpbmcgSHlzdGFydA0KPj4gPiBPTiBhbmQgT0ZGLCBpbg0KPj4gPiA+IGNvbmRpdGlv bnMgb2YgTG93IEJhbmR3aWR0aCAoMjUtMzUgTWJwcywgcG9vciByYWRpbykgYW5kIEhpZ2ggDQo+ PiA+ID4gQmFuZHdpZHRoICgxMDAtMTIwIE1icHMsIGdvb2QgcmFkaW8pLg0KPj4gPiA+DQo+PiA+ ID4gICAxMDAgRG93bmxvYWQgdGVzdHMgcGVyIHR5cGUuDQo+PiA+ID4gICAgICAgICAgICAgIERv d25sb2FkIGR1cmF0aW9uIGFuZCBiYW5kd2lkdGggbWVhc3VyZWQgYXQgdGhlIA0KPj4gPiA+IGNs aWVudCBzaWRlIGFzIHNob3duIGJ5IGN1cmwgZnJvbSB0aGUgZmlyc3QgcmVjZWl2ZWQgZGF0YSBw YWNrZXQgDQo+PiA+ID4gdGlsbCB0aGUgZW5kIG9mIHNlc3Npb24gKHRoaXMgZXhjbHVkZSBETlMs IDMgd2F5IGhhbmRzaGFrZSwgaHR0cCANCj4+ID4gPiBHRVQgYW5kIGl0cw0KPj4gQUNLKS4NCj4+ ID4gPiAgICAgICAgICAgICAgSHlzdGFydCBleGl0IGN3bmQgZXh0cmFjdGVkIGZyb20gbnN0YXQg KHRoYW5rIHlvdSANCj4+ID4gPiBFcmljIHRvIHBvaW50aW5nIG91dCB0byBpdCkuDQo+PiA+ID4N Cj4+ID4gPiBPdmVyYWxsIHJlc3VsdHM6DQo+PiA+ID4gICBEb3dubG9hZCAzTUIsIExvdyBCYW5k d2lkdGhzOg0KPj4gPiA+ICAgICAgICAgICBIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTog MS40MSBzDQo+PiA+ID4gICAgICAgICAgIE5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6 IDEuMzYgcw0KPj4gPiA+DQo+PiA+ID4gICBEb3dubG9hZCAzTUIsIEhpZ2ggQmFuZHdpZHRoczoN Cj4+ID4gPiAgICAgICAgICAgSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDAuNjUgcw0K Pj4gPiA+ICAgICAgICAgICBOb0h5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAwLjM3ICBz DQo+PiA+ID4NCj4+ID4gPiAgIERvd25sb2FkIDEwTUIsIExvdyBCYW5kd2lkdGhzOg0KPj4gPiA+ ICAgICAgICAgICBIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMi40MCBzDQo+PiA+ID4g ICAgICAgICAgIE5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDIuMTIgcw0KPj4gPiA+ DQo+PiA+ID4gICBEb3dubG9hZCAxME1CLCBIaWdoIEJhbmR3aWR0aHM6DQo+PiA+ID4gICAgICAg ICAgIEh5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAxLjQyIHMNCj4+ID4gPiAgICAgICAg ICAgTm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC44OSAgcw0KPj4gPiA+DQo+PiA+ ID4gVGhlc2UgcmVzdWx0cyBzaG93IHRoYXQgSHlzdGFydCBoYXMgbm8gc2lnbmlmaWNhbnQgaW1w YWN0IGluIA0KPj4gPiA+IGNvbmRpdGlvbnMgb2YgbG93IGF2YWlsYWJsZSBiYW5kd2lkdGgsIGJ1 dCBhdCBoaWdoIGF2YWlsYWJsZSB0aGUgDQo+PiA+ID4gZGVjcmVhc2Ugb2YgcGVyZm9ybWFuY2Ug Y2FuIHJlYWNoIDYwLTcwICUuDQo+PiA+ID4gV2l0aCB0aGUgZGV2ZWxvcG1lbnQgb2YgdGhlIExU RS1BIHdoaWNoIHVzZXMgbGFyZ2VyIHNwZWN0cnVtIHRvIA0KPj4gPiA+IHJlYWNoDQo+PiA+ID4g MzAwIE1icHMgYW5kIG1vcmUgdGhlIGltcGFjdCBvZiBIeXN0YXJ0IHdpbGwgYmVjb21lIGV2ZW4g bW9yZSB2aXNpYmxlLg0KPj4gPiA+DQo+PiA+ID4gSSBkbyBzaGFyZSB3aXRoIEVyaWMgYW5kIE5l YWwgc29tZSBmaWxlcyBkZXNjcmliZWQgYmVsb3cgYW5kIGFsc28gDQo+PiA+ID4gY2FuIHByb3Zp ZGUgbGlua3MgdG8gYW55b25lIGludGVyZXN0ZWQuDQo+PiA+ID4NCj4+ID4gPiBTdW1tYXJ5IG9m IHRoZXNlIHRlc3RzIHJlc3VsdHMgYXJlIGdhdGhlcmVkIGluIHRoZSBmb2xsb3dpbmcgDQo+PiA+ ID4gRXhjZWwgZmlsZXMgKG5hbWVzLCBJIGJlbGlldmUsIGFyZSBzZWxmIGV4cGxhbmF0b3J5KTog DQo+PiA+ID4gVGVzdF8zTV9sb3dfQlcueGxzeCwgVGVzdF8zTV9oaWdoX0JXLnhsc3gsIFRlc3Rf MTBNX2xvd19CVy54bHN4LA0KPj4gPiBUZXN0XzEwTV9oaWdoX0JXLnhsc3guDQo+PiA+ID4gU2lt aWxhcmx5IHRjcGR1bXAgdHJhY2VzIGFyZSBjYWxsZWQgVGVzdF8zTV9sb3dfQlcucGNhcCwgDQo+ PiA+ID4gVGVzdF8zTV9oaWdoX0JXLiBwY2FwLCBUZXN0XzEwTV9sb3dfQlcuIHBjYXAsDQo+PiBU ZXN0XzEwTV9oaWdoX0JXLg0KPj4gPiA+IHBjYXAgKGFibGUgdG8gY29sbGVjdCBvbmx5IHNlcnZl ciBzaWRlIHRyYWNlcywgYnV0IHRoZXkgYXJlIHNlbGYgc3VmZmljaWVudCkuDQo+PiA+ID4gRm9y IHRoZSAxME0gZG93bmxvYWRzIHRoZXJlIGFyZSBmZXcgbWlzc2luZyB0cmFjZXMgYXMgdGhlcmUg dHJhY2UgDQo+PiA+ID4gZmlsZXMgd2VyZSByZWN5Y2xlZC4gSW4gdHJhY2VzIGFyZSBhbHNvIHNz aCBzZXNzaW9ucywgcGxlYXNlLCANCj4+ID4gPiBpZ25vcmUgdGhlbSwgdGhleSB3ZXJlIHVzZWQg dG8gcHV0IGh5c3RhcnQgT04gYW5kIE9GRiBvbiBzZXJ2ZXIgDQo+PiA+ID4gYW5kIGV4dHJhY3Qg bnN0YXQgcmVzdWx0cyBiZWZvcmUgYW5kIGFmdGVyIGVhY2ggZG93bmxvYWQuDQo+PiA+ID4NCj4+ ID4gPiBFeGNlbCBmaWxlcyBoYXZlIDIgdGFiczogY3VybF9zdW0gYW5kIFN1bW1hcnkuDQo+PiA+ ID4gSW4gdGhlIGN1cmxfc3VtIHRhYiB0aGUgc3RhdCBpcyBjdW11bGF0ZWQgaW4gY2hyb25vbG9n aWNhbCBvcmRlciBvZiB0ZXN0cy4NCj4+ID4gPiBJbiB0aGUgU3VtbWFyeV90YWIgIHRoZSBIeXN0 YXJ0IGFuZCBOb0h5c3RhcnQgdGVzdHMgYXJlIHNob3duIA0KPj4gPiA+IHNpZGUgYnkgc2lkZSAo YXMgc2FpZCwgRG93bmxvYWQgMSB3YXMgd2l0aCBIeXN0YXJ0LCBEb3dubG9hZCAyIA0KPj4gPiA+ IHdpdGhvdXQsIERvd25sb2FkIDMgd2l0aCBIeXN0YXJ0LCBEb3dubG9hZCA0IHdpdGhvdXQsIGV0 Yy4pLiBJdCANCj4+ID4gPiBhbHNvIGluY2x1ZGUgZ3JhcGhzIG9mIFRyYW5zZmVyIHRpbWUgYW5k IGEgZ3JhcGggd2hpY2ggc2hvd3MgdGhlIA0KPj4gPiA+IGRpc3RyaWJ1dGlvbiBpbiAlIG9mIGRp ZmZlcmVudCBIeXN0YXJ0IGV4aXQgd2luZG93IERlbGF5Q1dORC4NCj4+ID4gPg0KPj4gPiA+IEZp ZWxkczoNCj4+ID4gPiBjb21tZW50cyAgLSB0ZXN0IGNhc2UNCj4+ID4gPiBUcmFuc2ZfVGltZSAg ICAgICAtIGNhbGN1bGF0ZWQgZnJvbSBjdXJsIG91dHB1dCB0aGUgZHVyYXRpb24NCj4+ID4gYmV0 d2VlbiBmaXJzdCBkYXRhDQo+PiA+ID4gcGFja2V0IGFuZCBlbmQgb2YgdHJhbnNmZXINCj4+ID4g PiBUcmFuc2Zfc3BlZWQgICAgICAtIGNhbGN1bGF0ZWQgZnJvbSBjdXJsIG91dHB1dCB0aGUgcmF0 aW8gb2YgZmlsZSBzaXplDQo+PiA+IGFuZCBkdXJhdGlvbg0KPj4gPiA+IGJldHdlZW4gZmlyc3Qg ZGF0YSBwYWNrZXQgYW5kIGVuZCBvZiB0cmFuc2Zlcg0KPj4gPiA+IEh5c3RhcnQgICBEX0RlbGF5 IC0gIGh5c3RhcnQgZXhpdCBvY2N1cnJlbmNlIChkaWZmZXJlbmNlIGJldHdlZW4NCj4+ID4gPiBU Y3BFeHRUQ1BIeXN0YXJ0RGVsYXlEZXRlY3QgYmVmb3JlIGFuZCBhZnRlciBkb3dubG9hZCkNCj4+ ID4gPiBEX0RlbGF5Q3duZCAgICAgICAgIC0gSHlzdGFydCBleGl0IGN3bmQgZm9yIHRoaXMgZG93 bmxvYWQNCj4+ID4gPiAoZGlmZmVyZW5jZSBvZiBUY3BFeHRUQ1BIeXN0YXJ0RGVsYXlDd25kIGJl Zm9yZSBhbmQgYWZ0ZXIgdGhlDQo+PiA+ID4gZG93bmxvYWQpIERhdGVfT25fU2VydmVyIC0gdGlt ZSBqdXN0IGJlZm9yZSB0aGUgZG93bmxvYWQgc3RhcnQsIA0KPj4gPiA+IHRvIGhlbHAgbmF2aWdh dGUgdGhlIHBjYXANCj4+ID4gPg0KPj4gPiA+IEFkZGl0aW9uYWxseSwgaW4gdGhlIGZpbGUgVGVz dF8xME1fbG93X0JXLnhsc3ggaW4gdGhlIHRhYiANCj4+ID4gPiBjdXJsX3N1bSBhcmUgaW5jbHVk ZWQgZm9yIGVhY2ggZG93bmxvYWQgdGNwLnN0cmVhbSBleHRyYWN0IGZyb20gDQo+PiA+ID4gcGNh cCBvZiB0aGUgZmlyc3QNCj4+ID4gPiAxMDAgdGNwLmFuYWx5c2lzLmFja19ydHQgd2hpY2ggbWF5 IGhlbHAgaW4gdW5kZXJzdGFuZGluZyBvZiB0aGUgDQo+PiA+ID4gaHlzdGFydCBiZWhhdmlvciAo ZmllbGRzIGFja19ydHRfMSB0byBhY2tfcnR0XzEwMCkuDQo+PiA+ID4gSW4gdGhpcyBmaWxlIHRo ZXJlIGFyZSBhbHNvIGZpZWxkcyBVUklfZnJhbWUsIFVSSV90aW1lLCBVUklfdGltZV9yZWxhdGl2 ZSwNCj4+ID4gPiAgIHRjcF9zdHJlYW0uDQo+PiA+ID4gVGhlbiB0aGVyZSBhcmUgY2FsY3VsYXRl ZCBmaWVsZHM6IG1pblJUVCBhcyBhIG1pbmltdW0gb2YgYWxsIDEwMCANCj4+ID4gPiBBQ0ssDQo+ PiByMV81LA0KPj4gPiA+IHIyXzgsIHIzXzgsICAgICAgIHI0Xzggd2hpY2ggc2ltdWxhdGUgSHlz dGFydCBjYWxjdWxhdGlvbnMgb2YgdGhlDQo+PiByb3VuZCBjdXJyX3J0dA0KPj4gPiA+IGFuZCBy MSBtaW4sIHIyIG1pbiwgcjMgbWluLCByNCBtaW4gcmVwcmVzZW50IG1pblJUVCBvZiB0aGUgd2hv bGUgcm91bmQuDQo+PiA+ID4gUGxlYXNlLCBjb3JyZWN0IG1lIGlmIEkgYW0gd3JvbmcsIGJ1dCBp biBteSB1bmRlcnN0YW5kaW5nIGZvciANCj4+ID4gPiBlYWNoIEFDSyB0aGUgaHlzdGFydF91cGRh dGUgaXMgY2FsbGVkIGJlZm9yZSB0aGUgaHlzdGFydF9yZXNldCwgDQo+PiA+ID4gYW5kIGFzIGEg cmVzdWx0IHRoZSBjb21wdXRhdGlvbiBvZiBjdXJfcnR0IG9mIHRoZSA4IHBhY2tldHMgb2YgDQo+ PiA+ID4gdGhlIHJvdW5kIHN0YXJ0cyB3aXRoIHRoZSAyLW5kIHBhY2tldCBvZiB0aGUgcm91bmQs IG5vdCB0aGUgZmlyc3Qgb25lLg0KPj4gPiA+ICBJJ3ZlIHVzZSByMV81IGFzIGEgbWluKGFja19y dHRfNy4uLmFja19ydHRfMTEpLCBkdWUgdG8gDQo+PiA+ID4gaHlzdGFydF9sb3dfd2luZG93PTE2 LCByMl84IGFzIGEgbWluKGFja19ydHRfMTIgLi4uIGFja19ydHRfMTkpLA0KPj4gPiA+IHIzXzgg YXMgYQ0KPj4gPiA+IG1pbihhY2tfcnR0XzMyIC4uLiBhY2tfcnR0XzM5KSBhbmQgKSwgcjRfOCBh cyBhIG1pbihhY2tfcnR0XzcyIC4uLg0KPj4gPiBhY2tfcnR0Xzc5KS4NCj4+ID4gPiBUbyB1bmRl cnN0YW5kIHRoZSBpbXBhY3Qgb2YgdGhpcyB0aGVyZSBhcmUgYWxzbyByMSBIT0wsIHIyIEhPTCwg DQo+PiA+ID4gcjMgSE9MLCByNCBIT0wgKEhlYWQgT2YgTGluZSkgd2hpY2ggY29tcHV0ZSB0aGUg c2FtZSB3aXRoIGZpcnN0IA0KPj4gPiA+IHBhY2tldCBpbiB0aGUgcm91bmQgaW5jbHVkZWQgaW4g dGhlIHJlbGV2YW50IHJvdW5kOiAgcjFfNSBhcyBhIA0KPj4gPiA+IG1pbihhY2tfcnR0XzYuLi5h Y2tfcnR0XzEwKSwgcjJfOCBhcyBhIG1pbihhY2tfcnR0XzExIC4uLg0KPj4gPiA+IGFja19ydHRf MTgpLA0KPj4gPiA+IHIzXzggYXMgYSBtaW4oYWNrX3J0dF8zMSAuLi4gYWNrX3J0dF8zOCkgYW5k ICksIHI0XzggYXMgYSBtaW4oYWNrX3J0dF83MSAuLi4NCj4+ID4gYWNrX3J0dF83OCkuDQo+PiA+ ID4NCj4+ID4gPiBJZiBteSB1bmRlcnN0YW5kaW5nIG9mIEh5c3RhcnQgaXMgY29ycmVjdCBhbmQs IGFzc3VtaW5nIGluIGVhY2ggDQo+PiA+ID4gdHJhaW4gdGhlIGZpcnN0IHBhY2tldCBoYXMgdGhl IGxvd2VzdCBSVFQgd2hpY2gsIG1vc3QgcHJvYmFibHkgaXMgDQo+PiA+ID4gdmFsaWQgaW4gYWxs IHR5cGUgb2YgbmV0d29ya3MsIHRoZW4gdGhlIGV4aXQgZnJvbSBoeXN0YXJ0IGluIHRoaXMgDQo+ PiA+ID4gaW1wbGVtZW50YXRpb24gbWF5IGhhcHBlbiBhdCBjd25kIDI5LCA0OSwgODksIDE2OSwg ZXRjLiAoZ2l2ZW4gDQo+PiA+ID4gdGhlIElXIG9mDQo+PiA+IDEwKS4NCj4+ID4gPiBUaGVzZSB0 ZXN0cyBzaG93IHRoYXQgdGhpcyBpcyB0aGUgY2FzZSwgaG93ZXZlciB0aGVyZSBhbHNvIA0KPj4g PiA+IGludGVybWVkaWF0ZSB2YWx1ZXMuIEhvd2V2ZXIgdGhlIGN3bmQgb2YgMjkgc2hhbGwgYmUg dGhlIGxvd2VzdCANCj4+ID4gPiBwb3NzaWJsZSB2YWx1ZSBhbmQgdGhpcyBpcyB0aGUgY2FzZSBp biB0aGVzZSB0ZXN0cy4gDQo+PiA+ID4gVW5mb3J0dW5hdGVseSwgaW4gTFRFIHRoZSBmaXJzdCBw YWNrZXQgaW4gdGhlIHRyYWluIGhhcyBjaGFuY2VzIA0KPj4gPiA+IHRvIGFsd2F5cyBoYXZlIH42 IG1zIGxvd2VyIHRoYW4gdGhlIGZvbGxvd2luZyBvbmVzIGFuZCwgYmVjYXVzZSANCj4+ID4gPiBp dCBpcyBub3QgY291bnRlZCBpbiB0aGUgMi1uZCB0cmFpbiwgdGhlcmUgaXMgcXVpdGUgaGlnaCAN Cj4+ID4gPiBwZXJjZW50YWdlIG9mIGV4aXQgYXQgMjkgKDE0LTM2JSBpbiB0aGVzZSB0ZXN0cyks IGV2ZW4gdGhvdWdoIA0KPj4gPiA+IGxhdGVyIG9uIGN1YmljIGluY3JlYXNlcyB0aGUgY3duZCB3 aXRob3V0IGxvc3NlcyB1cCB0byBtYW55IA0KPj4gPiA+IGh1bmRyZWRzLiBGb3IgbWUgdGhpcyBp cyB0b28gZWFybHkgZXhpdCBmcm9tDQo+PiBzbG93IHN0YXJ0Lg0KPj4gPiA+IEFzIGl0IGNhbiBi ZSBzZWVuIGZyb20gY29tcGFyaXNvbiBvZiB0aGUgc2ltdWxhdGVkIGNhbGN1bGF0aW9uLCANCj4+ ID4gPiBpbmNsdXNpb24gb2YgdGhlIGZpcnN0IHBhY2tldCBpbiB0aGUgcm91bmQgY3Vycl9ydHQg Y2FsY3VsYXRpb24gDQo+PiA+ID4gd291bGQgZWxpbWluYXRlIHRoZSBjd25kIDI5IGFuZCBzaGlm dCB0byBjd25kIDQ5IG9yIDg5IG9yIGhpZ2hlci4NCj4+ID4gPg0KPj4gPiA+IEJ1dCB0aGVyZSBh cmUgYWxzbyBpbnRlcm1lZGlhdGUgdmFsdWVzLCBlLmcuIDY5LiBMb29raW5nIHRocm91Z2ggDQo+ PiA+ID4gdHJhY2VzIG9uZSBtYXkgc2VlIHRoYXQgdGhpcyBpcyBkdWUgdG8gImNvbXByZXNzZWQg QUNLIiB3aGljaCANCj4+ID4gPiByZWNlaXZlcyBhIHRpZ2h0bHkgcGFja2VkIHRyYWluIG9mIG1h bnkgQUNLIHNvIHRoYXQgaW4gdHJhY2VzIA0KPj4gPiA+IHRoZXJlIGFyZSBub3QgZGF0YSBwYWNr ZXRzIHNlbnQgaW4gdHVybiBieSBzZXJ2ZXIuIExvb2tzIHRvIG1lLCANCj4+ID4gPiBiZWNhdXNl IGh5c3JhcnRfcmVzZXQgc2V0cyB0aGUgZW5kX3NlcSB0byBzbmRfbmV4dCBhbmQgdGhlIHNlcnZl ciANCj4+ID4gPiBkaWQgbm90IG1hbmFnZWQgdG8gc2VuZCB5ZXQgcGFja2V0cyBpbiByZXBseSB0 byB0aGlzIGhpZ2ggc3BlZWQgDQo+PiA+ID4gQUNLIHRyYWluIHRoZSBzbmRfbmV4dCBwb2ludHMg dG8gbXVjaCBsb3dlciB2YWx1ZSB0aGFuIHdvdWxkIA0KPj4gPiA+IG5vcm1hbGx5IGhhcHBlbiBh bmQgdGhpcyBhY3R1YWxseSBkZXN0cm95cyB0aGUgY29ycmVjdCANCj4+ID4gPiBpZGVudGlmaWNh dGlvbiBvZiB0aGUgYm9yZGVycyBvZiB0aGUgcm91bmQsIHRoZSByb3VuZCBmYWlscyANCj4+ID4g PiBzb21ld2hlcmUgaW4gdGhlIG1pZGRsZSBvZiB0aGUgdHJhaW4gbm90IGF0IGl0cyBib3JkZXIs IHRoaXMgDQo+PiA+ID4gbWFrZXMgdmVyeSBsaWtlbHkgaW5jcmVhc2VkIGRlbGF5IGR1ZSB0byBx dWV1ZWluZyBhbmQgZWFybGllciANCj4+ID4gPiBleGl0IGZyb20gc2xvd19zdGFydCBhbmQgaW50 ZXJtZWRpYXRlIHZhbHVlcyBvZiBleGl0IGN3bmQgd2hlcmUNCj4+ID4gdGhleSBhcmUgbm90IGV4 cGVjdGVkLg0KPj4gPiA+DQo+PiA+ID4gQW5kIGxhc3QgYnV0IG5vdCBsZWFzdDogdGhlcmUgYXJl IGdhcHMgKDQtNyBtcykgaW4gQUNLIHRyYWlucy4gT24gDQo+PiA+ID4gdGhlIHBlcmZlY3RseSBz ZW50IElXIG9mIDEwIGRhdGEgcGFja2V0cyB0cmFpbiB0aGUgdHlwaWNhbCBmb3IgDQo+PiA+ID4g TFRFIHdpbGwgYmUgdG8gcmVjZWl2ZSAxIG9yIDIgQUNLLCB0aGVuIGEgZ2FwIG9mIDQtNiBtcyB0 aGVuIA0KPj4gPiA+IGFub3RoZXIgY291cGxlIG9mIEFDSywgdGhlbiBhbm90aGVyIGdhcCwgdGhl biA1IEFDSyBjb21wcmVzc2VkIA0KPj4gPiA+IHRvZ2V0aGVyIGluIGEgZnJhY3Rpb24gb2YgbWlj cm9zZWNvbmQuIE9mIGNhdXNlLCB0aGUgbmV4dCB0cmFpbiANCj4+ID4gPiBmcm9tIHRoZSBzZXJ2 ZXIgd2lsbCBjb250YWluIHRoZSBzYW1lIGdhcHMuIFZlcnkgcXVpY2tseSwgZS5nLiwgDQo+PiA+ ID4gdGhlIEhlYWQgb2YgTGluZQ0KPj4gPiA+IChIT0wpIG9mIHRoZSAzLWQgdHJhaW4gd2lsbCBm b2xsb3cgaW1tZWRpYXRlbHkgdGhlIHRhaWwgb2YgdGhlIA0KPj4gPiA+IDItbmQgdHJhaW4gd2hp Y2ggd2lsbCBsZWFkIHRvIHF1ZXVlaW5nIGluIHRoZSBlbmQgcm91dGVyIA0KPj4gPiA+IChhY3R1 YWxseSB0aGUgcmFkaW8gYmFzZSBzdGF0aW9uLCBtdWNoIGVhcmxpZXIgdGhhbiBCRFAgcmVhY2hl ZCksIA0KPj4gPiA+IGJ1dCB0aGlzIHdpbGwgbGVhZCB0byBpbmNyZWFzZSBvZiB0aGUgdHJhaW4g UlRUIGFuZCB0b28geWVhcmx5IA0KPj4gPiA+IGV4aXQgZnJvbSBzbG93IHN0YXJ0LiBXaHkgdG9v IGVhcmx5IGV4aXQgPyBCZWNhdXNlLCBzaG91bGQgc2VydmVyIA0KPj4gPiA+IGNvbnRpbnVlIGlu Y3JlYXNpbmcgdGhlIHNwZWVkIHRoaXMgd291bGQgcmVkdWNlIHF1aWNrZXIgdGhlIGdhcHMgDQo+ PiA+ID4gYm90aCBpbiBkb3dubGluayBhbmQgdXBsaW5rLCB0aGUgcHJlaWNlIGZvciB0aGlzIGJl aW5nIHRoZSANCj4+ID4gPiBpbmNyZWFzZSBvZiB0aGUgUlRUIChJIGd1ZXNzIGRvdWJsZSkgYWdh aW5zdCB3aGF0IG9uZSB3b3VsZCANCj4+ID4gPiBub3JtYWxseSBzZWUNCj4+ID4gaW4gdGhlIGVu ZC10by1lbmQgRXRoZXJuZXQgbmV0d29ya3MuDQo+PiA+ID4NCj4+ID4gPg0KPj4gPiA+IEkgZG8g bGlrZSBpZGVhcyBvZiBIeXN0YXJ0LCBidXQgSSAgZG9uJ3Qga25vdyBob3cgY291bGQgaXQgYmUg DQo+PiA+ID4gcG9zc2libHkgcmVjb25jaWxlZCB3aXRoIG15IExURSBwcm9ibGVtcy4gIFdpdGgg dGhlIGluY3JlYXNlIG9mIA0KPj4gPiA+IExURSBzcGVlZHMgdGhpcyBlYXJseSBleGl0IGZyb20g c2xvdyBzdGFydCB3aWxsIGJlY29tZSBldmVuIG1vcmUgDQo+PiA+ID4gb2YgdGhlDQo+PiBwcm9i bGVtLg0KPj4gPiA+IEZvciBMVEUsIG5vdCBzdXJlIGhvdyBnb29kIGFuZCBnZW5lcmFsbHkgYWNj ZXB0YWJsZSwgc29sdXRpb24gDQo+PiA+ID4gd291bGQgYmUsIHBvc3NpYmx5LCB0byBpbmNyZWFz ZSB0aGUgSFlTVEFSVF9ERUxBWV9NSU4gdG8gOCBtcywgDQo+PiA+ID4gd2hpY2gsIGF0IGxlYXN0 IGluIG15IGNhc2UsIHdvdWxkIGRpbWluaXNoIHRoZSBlYXJsaWVyIGV4aXQgDQo+PiA+ID4gc2ln bmlmaWNhbnRseSwgaWYgSSBiZWxpZXZlIGluIHRoZXNlIHRlc3RzIHJlc3VsdHMuDQo+PiA+ID4g QW5vdGhlciBzb2x1dGlvbiwgd2hpY2ggSSBkb24ndCBsaWtlIHZlcnkgbXVjaCwgYnV0IGFzIEkg a25vdyANCj4+ID4gPiBzb21lIG9wZXJhdG9ycyB1c2UsIHdvdWxkIGJlIHRvIGluc2VydCBhIGtp bmQgb2YgVENQIGFjY2VsZXJhdG9yIA0KPj4gPiA+IGJldHdlZW4gdGhlIEludGVybmV0IGFuZCB0 aGUgbW9iaWxlIG5ldHdvcmsgd2hpY2ggd2lsbCBpbnRlcmNlcHQgDQo+PiA+ID4gYWxsIFRDUCBh bmQgd291bGQgc2VuZCBpdCB0byBtb2JpbGUgd2l0aG91dCBIeXN0YXJ0IGJ1dCB0aGlzIA0KPj4g PiA+IHdvdWxkIGtpbGwgYW55IGVuZC10by1lbmQgcHJpbmNpcGxlIGFuZCB0aGlzIHdpbGwgbm90 IGJlIHRoZSBJbnRlcm5ldCBhbnltb3JlLg0KPj4gPiA+DQo+PiA+ID4NCj4+ID4gPiBJZiB5b3Ug aGF2ZSB0aW1lIGFuZCBuZXZlciBsb29rIGNsb3NlbHkgdG8gTFRFIHRlY2hub2xvZ3kgSSBkbyBh IA0KPj4gPiA+IHF1aWNrIHN1bW1hcnkgYmVsbG93Lg0KPj4gPiA+DQo+PiA+ID4NCj4+ID4gPiBM VEUsIFRDUCBzZWxmLWNsb2NraW5nIGFuZCBBQ0sgY29tcHJlc3Npb24uDQo+PiA+ID4NCj4+ID4g PiBTcGVudCBzb21lIHRpbWUgdG8gdW5kZXJzdGFuZCB0aGUgYmFzaWNzIG9mIHdoYXQgY291bGQg YmUgdGhlIA0KPj4gPiA+IHJlYXNvbiBmb3IgYWxsIHRob3NlIHRpbWluZyBlZmZlY3RzIGR1ZSB0 byBMVEUuDQo+PiA+ID4gICBGaXJzdCwgdW5saWtlIHdpcmUgYmFzZWQgdHJhbnNwb3J0IHRlY2hu b2xvZ2llcywgTFRFIChidXQgYWxzbw0KPj4gPiBVTVRTDQo+PiA+ID4gYW5kIGZ1dHVyZSA1Rykg c2VuZHMgZGF0YSwgYm90aCB1cGxpbmsgYW5kIGRvd25saW5rIGluIHByZWRlZmluZWQgDQo+PiA+ ID4gdGltZXNsb3RzIGNhbGxlZCBUcmFuc21pc3Npb24gVGltZSBJbnRlcnZhbCAoVFRJKS4gSW4g VU1UUyB0aGlzIA0KPj4gPiA+IGlzIDJtcywgaW4gTFRFIGlzDQo+PiA+ID4gMSBtcyBhbmQgaXQg c2F5cyB0aGF0IGluIDVHIGl0IHdpbGwgYmUgMC41IG1zLg0KPj4gPiA+IFRoZXNlIFRUSSBoYXZl IGV4YWN0IGJvcmRlcnMgYW5kIHRpZ2h0bHkgZm9sbG93IGVhY2ggb3RoZXIgYW5kIA0KPj4gPiA+ IHJlcXVpcmUgZXhhY3Qgc3luY2hyb25pemF0aW9uIGJldHdlZW4gdGhlIG1vYmlsZSBkZXZpY2Ug YW5kIFJhZGlvIA0KPj4gPiA+IEJhc2UgU3RhdGlvbiAoY2FsbGVkIGVOb2RlQiBpbiBMVEUpLiBU byBjb3BlIHdpdGggdmFyeWluZyByYWRpbyANCj4+ID4gPiBjb25kaXRpb25zIHRoZSBzZW5kaW5n IGRhdGEgYmFuZHdpZHRoIGlzIG1vZGlmaWVkIGluIHN1Y2ggYSB3YXkgDQo+PiA+ID4gdGhhdCB0 byBrZWVwIGEgY29uc3RhbnQgcHJvYmFiaWxpdHkgb2YgZGF0YSBsb3NzIG9mIDEwJS4gQmFkIA0K Pj4gPiA+IHJhZGlvIC0gbGVzcyBiYW5kd2lkdGgsIGdvb2QNCj4+ID4gcmFkaW8gLSBtb3JlIGJh bmR3aWR0aC4NCj4+ID4gPiBUaGUgYmFuZHdpZHRoIGlzIG1vZGlmaWVkIGJ5IHBhY2tpbmcgbGVz cyBvciBtb3JlIGJpdHMgaW4gYSBUVEkgDQo+PiA+ID4gb2YgMSBtcyBpbg0KPj4gPiBMVEUuDQo+ PiA+ID4gQnV0IFRUSSByZW1haW4gdW5jaGFuZ2VkIG9mIGV4YWN0bHkgMSBtcy4gVGhhdCBtZWFu cyB0aGF0LCBhdCB0aGUgDQo+PiA+ID4gVENQL0lQIGxheWVyIHRoZSBwcmVjaXNpb24gb2YgdGhl IHRpbWluZyBpbmZvcm1hdGlvbiBmcm9tIGRldmljZSANCj4+ID4gPiB0byB0aGUgbmV0d29yayBp cyAxIG1zLiBJbiB0aGUgY2FzZSBvZiBhIGJhbmR3aWR0aCBvZiAxIE1icHMgDQo+PiA+ID4gdGhl cmUgd2lsbCBiZSA4MyBJUCBwYWNrZXRzIHBlciBzZWNvbmQgb3Igb25lIHBhY2tldCBldmVyeSAx MiBtcy4gDQo+PiA+ID4gSW4gdGhpcyBjYXNlLCBwb3NzaWJseSB0aGUgcmV0dXJuIG9mIG9uZSBB Q0sgZXZlcnkgMTIgbXMgKGV2ZXJ5IA0KPj4gPiA+IDEyIFRUSSkgd291bGQgYmUgbW9yZSBvciBs ZXNzIHN1ZmZpY2llbnQgdG8ga2VlcCBhdCBhIGdvb2QgbGV2ZWwgDQo+PiA+ID4gdGhlIFRDUCBz ZWxmLWNsb2NraW5nIG1lY2hhbmlzbS4gQnV0LCB3aGVuIHRhbGtpbmcgYWJvdXQgMTAwIE1icHMg DQo+PiA+ID4gKG9yIDMwMA0KPj4gPiA+IE1icHMpIHRoYXQgbWVhbnMgb25lIHBhY2tldCAoYW5k IG9uZSBBQ0spIGV2ZXJ5IDAuMTIgbXMgYW5kIHRoZXJlIA0KPj4gPiA+IGlzIG5vIHdheSB0byBw cm92aWRlIGl0IHdpdGggYSBUVEkgb2YgMSBtcy4gVGhpcyBpcyB0aGUgcmVhc29uIA0KPj4gPiA+ IGZvciBBQ0sgY29tcHJlc3Npb24gaW4gTFRFIGFuZCB0aGUgZnV0dXJlIDVHIHdoaWNoIHByb21p c2VzIG1vcmUgDQo+PiA+ID4gdGhhbiAxR2JwcyB3aXRoIGl0cyAwLjUgbXMgVFRJIHdpbGwNCj4+ ID4gYmUgaW4gYSBiaWcgY29uZmxpY3Qgd2l0aCBUQ1Agc2VsZi1jbG9ja2luZy4NCj4+ID4gPiAg IEJ1dCB0aGUgcHJvYmxlbSBkbyBub3Qgc3RvcHMgaGVyZS4gVGhlcmUgaXMgYSBzaWduaWZpY2Fu dGx5DQo+PiA+IGRpZmZlcmVudA0KPj4gPiA+IG1lY2hhbmlzbXMgb2YgdHJhbnNtaXNzaW9uIGlu IHRoZSBkb3dubGluayAoZnJvbSBlTm9kZUIgdG8gbW9iaWxlDQo+PiA+ID4gZGV2aWNlKSBhbmQg aW4gdXBsaW5rIChmcm9tIG1vYmlsZSBkZXZpY2UgdG8gdGhlIGVOb2RlQikuIFRoZSANCj4+ID4g PiBkaWZmZXJlbmNlIGNvbWUgZnJvbSB0aGUgZmFjdCB0aGF0IHRoZSBzY2hlZHVsZXIgZm9yIGJv dGggDQo+PiA+ID4gZG93bmxpbmsgYW5kIGluIHVwbGluayBpcyBsb2NhdGVkIGluIHRoZSBlTm9k ZUIuIEFzIG9mIG15IA0KPj4gPiA+IHVuZGVyc3RhbmRpbmcgdGhlIHB1cnBvc2Ugb2YgdGhpcyBp cyB0byBzaW1wbGlmeSB0byB0aGUgbWF4aW11bSANCj4+ID4gPiB0aGUgZGV2aWNlIGFuZCBzYXZl IGl0cw0KPj4gYmF0dGVyeS4NCj4+ID4gPiBBcyBhIHJlc3VsdCBvZiB0aGlzIGRlc2lnbiwgdGhl IGVOb2RlQiBjYW4gc2VuZCBpbiBlYWNoIFRUSSANCj4+ID4gPiB0b3dhcmQgdGhlIG1vYmlsZSBk ZXZpY2Ugd2hlbmV2ZXIgZGF0YSBleGlzdCBpbiBpdHMgYnVmZmVyIA0KPj4gPiA+IChlTm9kZUIg aXMgYSBraW5kIG9mIElQIHdpcmVsZXNzIHJvdXRlcikuIEJ1dCBmb3IgbW9iaWxlIGRldmljZSwg DQo+PiA+ID4gaW4gb3JkZXIgdG8gc2VuZCB0aGUgZGF0YSBpbiB1cGxpbmssIGFmdGVyIGEgcGF1 c2UsIGl0IGhhcyBmaXJzdCANCj4+ID4gPiB0byBhc2sgdGhlIG5ldHdvcmsgdG8gc2NoZWR1bGUg aXQuIEFuZCBtb2JpbGUgaGFzIGZvciB0aGlzIA0KPj4gPiA+IHJlcXVlc3QgKGFmdGVyIGEgcGF1 c2Ugb2YNCj4+ID4gPiBzZW5kaW5nKSBqdXN0IG9uZSBiaXQsIGFjdHVhbGx5IGlzIG5vdCBldmVu IGEgZGF0YSBiaXQsIGlzIGEgDQo+PiA+ID4gc3BlY2lhbCByYWRpbyBzaWduYWwgd2hpY2ggdGVs bHMgIkhleSwgSSBoYXZlIHNvbWUgZGF0YSBpbiB0aGUgDQo+PiA+ID4gYnVmZmVyIHRvIHNlbmQi LiBMZXRzIGxvb2sgYXQgaW5pdGlhbCB0cmFpbiBvZiBJVyBvZiAxMCBwYWNrZXRzLiANCj4+ID4g PiBJbiBnb29kIHJhZGlvIGNvbmRpdGlvbnMgdGhlIGVOb2RlQiB3aWxsIGZpdCBhbGwgMTAgcGFj a2V0cyBpbiAxIA0KPj4gPiA+IFRUSSBvZiAxIG1zIGFuZCBzZW5kIHRoZW0gYWxsIHRvIHRoZSBk ZXZpY2UuIERldmljZSB3aWxsIHVucGFjayANCj4+ID4gPiB0aGVtIGFuZCBnZW5lcmF0ZSAxMCBB Q0sgYW5kIHRoZW4gdGVsbCB0byB0aGUgZU5vZGVCICJIZXksIEkgaGF2ZSANCj4+ID4gPiBzb21l IGRhdGEgaW4gdGhlIGJ1ZmZlciB0byBzZW5kIi4gVGhlIGVOb2RlQiBoYXMgbm90IGtub3dsZWRn ZSANCj4+ID4gPiBob3cgbXVjaCBkYXRhIHRoZSBtb2JpbGUgaGFzIHRvIHNlbmQsIHRoZXJlZm9y IHdpbGwgYWxsb2NhdGUgc29tZSANCj4+ID4gPiBtaW5pbXVtDQo+PiA+ID4gY2FwYWNpdHk6ICJX ZWxsLCBJIGdpdmUgeW91IGEgZ3JhbnQgdG8gc2VuZCB1cCB0byAyMDAgQnl0ZXMgYW5kIA0KPj4g PiA+IHlvdSBtdXN0IHNlbmQgdGhlbSBleGFjdGx5IGFmdGVyIDQgbXMgeW91IGdldCB0aGlzIG5v dGlmaWNhdGlvbiIuIA0KPj4gPiA+IFRoZXNlDQo+PiA+ID4gNCBtcyBhcmUgYWxsb3dhbmNlIGZv ciBtb2JpbGUgZGV2aWNlIHRvIHByZXBhcmUgZGF0YSBmb3Igc2VuZGluZyANCj4+ID4gPiB3aGlj aCBpcyBhIHZlcnkgY29tcGxpY2F0ZWQgcHJvY2VzcywgdGhlc2UgNCBtcyBpcyBhIG11c3QgZGVs YXkgDQo+PiA+ID4gYmV0d2VlbiB0aGUgZ3JhbnQNCj4+ID4gYW5kIHRoZSB0cmFuc21pc3Npb24u DQo+PiA+ID4gSWYgbW9iaWxlIGhhcyBpbiBpdHMgYnVmZmVyIDEwIEFDSyB0byBzZW5kIGl0IHdp bGwgc2VuZCBvbmx5IDIgDQo+PiA+ID4gQUNLIGFuZCB3aWxsIGNvbXBsZW1lbnQgdGhlbSB3aXRo LCBzbyBjYWxsZWQsIGJ1ZmZlciBzdGF0dXMgDQo+PiA+ID4gcmVwb3J0OiAiSSBkbyBoYXZlIHN0 aWxsIDQ4MCBieXRlcyB0byBzZW5kIi4gTm93IG5ldHdvcmsgaGFzIG1vcmUgDQo+PiA+ID4gaW5m b3JtYXRpb24gYW5kIGFsbG9jYXRlIGFzDQo+PiA+ID4gcmVxdWVzdGVkOiAiV2VsbCwgSSBnaXZl IHlvdSBhIGdyYW50IHRvIHNlbmQgdXAgdG8gMjAwIEJ5dGVzIGFuZCANCj4+ID4gPiB5b3UgbXVz dCBzZW5kIHRoZW0gZXhhY3RseSBhZnRlciA0IG1zIHlvdSBnZXQgdGhpcyBub3RpZmljYXRpb24i LiANCj4+ID4gPiBCdXQsIGluIHRoZSBtZWFuIHRpbWUgc29tZSBvdGhlciBpbmZvcm1hdGlvbiBv ZiBoaWdoZXIgcHJpb3JpdHkgDQo+PiA+ID4gbWF5IGFwcGVhciBpbiB0aGUNCj4+ID4gZGV2aWNl IChlLmcuDQo+PiA+ID4gc2lnbmFsaW5nKSBhbmQgdGhlIGRldmljZSB3aWxsIHNlbmQgb25seSA2 IG91dCBvZiA4IHJlbWFpbmluZyBBQ0sgDQo+PiA+ID4gdG9nZXRoZXIgd2l0aCBzaWduYWxpbmcg YW5kIGFub3RoZXIgQnVmZmVyIFN0YXR1cyBSZXBvcnQ6ICJIZXksIEkgDQo+PiA+ID4gZG8gaGF2 ZSBhbm90aGVyIDEyMCBieXRlcyB0byBzZW5kIi4gQW5kIGFnYWluICJXZWxsLCBJIGdpdmUgeW91 IGEgDQo+PiA+ID4gZ3JhbnQgdG8gc2VuZCB1cCB0byAxMjAgQnl0ZXMgYW5kIHlvdSBtdXN0IHNl bmQgdGhlbSBleGFjdGx5IA0KPj4gPiA+IGFmdGVyIDQgbXMgeW91IGdldCB0aGlzIG5vdGlmaWNh dGlvbiIuIEFzIGEgcmVzdWx0IHRoZSBzZXJ2ZXIgDQo+PiA+ID4gd2lsbCByZWNlaXZlIDIgQUNL LCB0aGVuIGFmdGVyIGEgZ2FwIG9mIDYgbXMgYW5vdGhlciA2IEFDSywgdGhlbiwgDQo+PiA+ID4g YWZ0ZXIgYW5vdGhlciA2IG1zLCB0aGUgcmVtYWluaW5nIDIgQUNLLiBXaXRoIHRoZSBzY2hlZHVs ZXIgaW4gDQo+PiA+ID4gdGhlIGVOb2RlQiB0aGVzZSBnYXBzIGFyZSBpbmV2aXRhYmxlLiBBbmQg b25lIG1vcmUgdGhpbms6IG1vYmlsZSANCj4+ID4gPiBwYWNrIHRvZ2V0aGVyIHRob3NlIDYgQUNL IGluIDEgVFRJIG9mIDEgbXMgYW5kIHNlbmQgdGhlbSwgdGhlbiANCj4+ID4gPiB0aGUgZU5vZGVC IHVucGFjayB0aGVtIGFuZCB0aGVyZSBpcyBubyB3YXkgdG8gcmVjb3ZlciB0aGUgc3BhY2luZyBv ZiAwLjEyIG1zIGJldHdlZW4gdGhlbS4NCj4+ID4gPiBlTm9kZUIgc2ltcGx5IHNlbmQgdGhlbSB0 byB0aGUgc2VydmVyIGJhY2sgdG8gYmFjayBhdCB0aGUgc3BlZWQgDQo+PiA+ID4gb2YgaXRzIGRh dGEgY2FyZCB3aGljaCBpcyB0eXBpY2FsbHkgMSBHYnBzIGFuZCB3ZSBmYWNlIHRoZSBBQ0sgDQo+ PiA+ID4gY29tcHJlc3Npb24uIEFmdGVyIHRoZSBsYXN0IDIgQUNLIHdlcmUgc2VudCB0aGVyZSBp cyBub3RoaW5nIGVsc2UgDQo+PiA+ID4gdG8gc2VuZCwgc28gdGhlcmUgaXMgbm8gQnVmZmVyIFN0 YXR1cyBSZXBvcnQsIHRoZXJlZm9yZSwgYWZ0ZXIgDQo+PiA+ID4gcGFja2V0cyBvZiB0aGUgbmV4 dCB0cmFpbiBhcnJpdmUgYW5kIEFDSyBhcmUgZ2VuZXJhdGVkIGV2ZXJ5dGhpbmcgDQo+PiA+ID4g c3RhcnRzIGFnYWluIHdpdGggb25lIGJpdCBvZiBpbmZvcm1hdGlvbiAiSGV5LCBJIGhhdmUNCj4+ ID4gc29tZSBkYXRhIGluIHRoZSBidWZmZXIgdG8gc2VuZCIuDQo+PiA+ID4gVGhhdCdzIHdoeSB0 aGVyZSBhcmUgZ2FwcyBhbmQgQUNLIGNvbXByZXNzaW9uLCBsb3NzLCBvciBtb3JlIA0KPj4gPiA+ IGV4YWN0bHksIGxhY2sgb2YgYW55IHRpbWluZyBpbmZvcm1hdGlvbiB3aGljaCB3b3VsZCBiZSB1 c2VmdWwgZm9yIA0KPj4gPiA+IHRoZSBUQ1Agc2VsZi0NCj4+ID4gY2xvY2tpbmcuDQo+PiA+ID4N Cj4+ID4gPiBUaGFuayB5b3UgYW5kIHNvcnJ5IGZvciB0YWtpbmcgeW91ciB0aW1lLg0KPj4gPiA+ DQo+PiA+ID4NCj4+ID4gPiBWZWFjZXNsYXYgUm9tYW4NCj4+ID4gPg0KPj4gPiA+DQo+PiA+ID4N Cj4+ID4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPj4gPiA+IEZyb206IFZlYWNlc2xh diBST01BTg0KPj4gPiA+IFNlbnQ6IFNhdHVyZGF5LCAwNSBTZXB0ZW1iZXIgMjAxNSAwMToxMA0K Pj4gPiA+IFRvOiAnRXJpYyBEdW1hemV0JzsgTmVhbCBDYXJkd2VsbA0KPj4gPiA+IENjOiB0Y3Bt QGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgSW5nZW1hciBKb2hhbnNzb24g DQo+PiA+ID4gUzsgRXJpYyBEdW1hemV0OyBlbmQyZW5kLWludGVyZXN0QHBvc3RlbC5vcmcNCj4+ ID4gPiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1l bnQNCj4+ID4gPg0KPj4gPiA+IElmIHdlIGxvb2sgYXQgdGhlIGZpcnN0IHRyYWluOiBhbGwgcGFj a2V0cyByZWNlaXZlZCBpbiBsZXNzIHRoYW4gMSBtcy4NCj4+ID4gPiBQcm9iYWJseSB0aGlzIGlz IG9ubHkgYW4gYXBwZWFyYW5jZSBhcyBMVEUgdHJhbnNtaXRzIGluLCBzbyANCj4+ID4gPiBjYWxs ZWQgdHJhbnNtaXNzaW9uIHRpbWUgaW50ZXJ2YWwgKFRUSSkgb2YgMSBtcywgYW5kIHdoYXQgd2Ug c2VlIA0KPj4gPiA+IGhlcmUgaXMgdGhhdCBhbGwgMTAgcGFja2V0cyBvZiBpbml0aWFsIHdpbmRv dyBmaXR0ZWQgaW4gMSBtcywgDQo+PiA+ID4gYW5kLCB3aGVuIGRlY29kZWQsIHdlcmUgcHJlc2Vu dGVkIHRvIHRoZSBUQ1AvSVAgbGF5ZXIgKGFuZCBwY2FwKSBhbGwgYXQgb25jZS4NCj4+ID4gPiBC VFcsIDggcGFja2V0cyBvZiAxMzAyMiBieXRlcyBpbiAxIG1zIG1lYW5zIGluc3RhbnRhbmVvdXMg c3BlZWQgDQo+PiA+ID4gb2YNCj4+ID4gPiAxMDQgTWJwcywgZ29vZCByYWRpbyBjb25kaXRpb25z LiBUQ1AgZ2VuZXJhdGVzIDEwIEFDSyB0cmFpbiBvZiANCj4+ID4gPiB0aGUgZHVyYXRpb24gb2YN Cj4+ID4gPiAxLjIgbXMuIFdpbGwgaXQgYmUgYSBGYXN0IEV0aGVybmV0IHBvc3NpYmx5IHdlIHdv dWxkIGNvbnNpZGVyIA0KPj4gPiA+IHRoaXMgbm9ybWFsLA0KPj4gPiBpc24ndCBpdCA/DQo+PiA+ ID4gSSBsb29rZWQgYXQgaG93IHNlcnZlciByZXBseSB0byB0aGVzZSAxMCBBQ0tzLiBUaGVyZSBh cmUgMyBnYXBzIA0KPj4gPiA+IG9mIDQsDQo+PiA+ID4gMiBhbmQgNSBtcyBpbiB0aGUgcmVwbHkg dHJhaW4gYW5kIHRoZSB0b3RhbCB0cmFpbiBvZiAxOCAoPywgaXQgDQo+PiA+ID4gc2hvdWxkIGJl IDIwKSBwYWNrZXRzIHJlYWNoZXMgYSBkdXJhdGlvbiBvZiAxNCBtcy4gQUZBSUsgTFRFIG1heSAN Cj4+ID4gPiBpbnRyb2R1Y2UgZ2FwcyBpbiBBQ0sgdHJhaW4gZHVlIHRvIHVwbGluayBzY2hlZHVs aW5nIG1lY2hhbmlzbS4NCj4+ID4gPiBNYXkgYmUgdGhlc2UgZ2FwcyB0cmlnZ2VyIEh5c3RhcnQg ZWFybHkgZXhpdCA/DQo+PiA+ID4NCj4+ID4gPiBWZWFjZXNsYXYgUm9tYW4NCj4+ID4gPiBUZWNo bmljYWwgYW5kIElUIGRpcmVjdG9yDQo+PiA+ID4gT3JhbmdlIE1vbGRvdmEgUy5BLg0KPj4gPiA+ IEZpeDogKzM3MzIyNTc1NDAwDQo+PiA+ID4gTW9iOiArMzczNjkxOTg0MDANCj4+ID4gPiBGYXg6 ICszNzMyMjU3NTMwNg0KPj4gPiA+DQo+PiA+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0N Cj4+ID4gPiBGcm9tOiBFcmljIER1bWF6ZXQgW21haWx0bzplcmljLmR1bWF6ZXRAZ21haWwuY29t XQ0KPj4gPiA+IFNlbnQ6IEZyaWRheSwgMDQgU2VwdGVtYmVyIDIwMTUgMTk6NDgNCj4+ID4gPiBU bzogTmVhbCBDYXJkd2VsbA0KPj4gPiA+IENjOiBWZWFjZXNsYXYgUk9NQU47IHRjcG1AaWV0Zi5v cmc7IFBpZXJzIE8nSGFubG9uOyBTYW5ndGFlIEhhOyANCj4+ID4gPiBJbmdlbWFyIEpvaGFuc3Nv biBTOyBFcmljIER1bWF6ZXQ7IGVuZDJlbmQtaW50ZXJlc3RAcG9zdGVsLm9yZw0KPj4gPiA+IFN1 YmplY3Q6IFJlOiBbdGNwbV0gW2UyZV0gVENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPj4g PiA+DQo+PiA+ID4gT24gRnJpLCAyMDE1LTA5LTA0IGF0IDExOjMyIC0wNDAwLCBOZWFsIENhcmR3 ZWxsIHdyb3RlOg0KPj4gPiA+ID4gSGkgVmVhY2VzbGF2LA0KPj4gPiA+ID4NCj4+ID4gPiA+IEkg YWdyZWUgdGhhdCBpbiB5b3VyIExURSB0cmFjZXMgaXQgbG9va3MgbGlrZSBDVUJJQyBIeXN0YXJ0 IGlzIA0KPj4gPiA+ID4gZXhpdGluZyBzbG93IHN0YXJ0IHRvbyBlYXJseS4NCj4+ID4gPiA+DQo+ PiA+ID4gPiBTSW5jZSB5b3Ugc2VlbSB0byBoYXZlIGEgbmljZSBMVEUgdGVzdGJlZCwgd291bGQg eW91IGJlIGFibGUgdG8gDQo+PiA+ID4gPiBkbyBzb21lIGV4cGVyaW1lbnRzIHRvIGZpbmQgYSBz ZXQgb2YgcGFyYW1ldGVycyBmb3IgSHlzdGFydCANCj4+ID4gPiA+IHRoYXQgd29yayBiZXR0ZXIg Zm9yIHlvdXIgTFRFIGVudmlyb25tZW50PyBGb3IgZXhhbXBsZSwgeW91IA0KPj4gPiA+ID4gbWln aHQgdHJ5IHRoZSB0d28gdmFyaWF0aW9ucyBJIHN1Z2dlc3RlZCBlYXJsaWVyIGluIHRoZSB0aHJl YWQ6DQo+PiA+ID4gPg0KPj4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgaWYgKGNhLT5j dXJyX3J0dCA+IGNhLT5kZWxheV9taW4gKw0KPj4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAg ICAgICAgIEhZU1RBUlRfREVMQVlfVEhSRVNIKGNhLT5kZWxheV9taW4gDQo+PiA+ID4gPiA+Pg0K Pj4gPiA+ID4gMikpIHsgb3INCj4+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgIGlmIChj YS0+Y3Vycl9ydHQgPiBjYS0+ZGVsYXlfbWluICsNCj4+ID4gPiA+ICAgICAgICAgICAgICAgICAg ICAgICAgICAgICBIWVNUQVJUX0RFTEFZX1RIUkVTSChjYS0+ZGVsYXlfbWluIA0KPj4gPiA+ID4g Pj4NCj4+ID4gPiA+IDEpKSB7DQo+PiA+ID4gPg0KPj4gPiA+ID4gRG8gYW55IG9mIHRob3NlIGdp dmUgYmV0dGVyIHJlc3VsdHMgZm9yIHlvdXIgdGVzdHM/DQo+PiA+ID4gPg0KPj4gPiA+ID4gbmVh bA0KPj4gPiA+DQo+PiA+ID4gQWxzbywgaHlzdGFydCBpcyBmb29sZWQgYnkgdG9vIG1hbnkgQUNL IHJlY2VpdmVkIGluIHNob3J0IHBlcmlvZC4NCj4+ID4gPg0KPj4gPiA+IFRoaXMgcHJvYmxlbSB3 b3VsZCBiZSBzb2x2ZWQgaWYgR1JPIHdhcyB1c2VkIGF0IHJlY2VpdmVyLCBhcyBsZXNzIA0KPj4g PiA+IEFDSyB3b3VsZCBiZSBzZW50Lg0KPj4gPiA+DQo+PiA+ID4gUHJlc3VtYWJseSByZWNlaXZl ciBpcyBub3QgYSBsaW51eCBUQ1Agc3RhY2sgPw0KPj4gPiA+DQo+PiA+ID4gTWF5YmUgd2Ugc2hv dWxkIGFkZCBhIGxvZ2ljIGluIGh5c3RhcnRfdXBkYXRlKCkgdG8gdGFrZSBvbmUgQUNLIHBlciBt cy4NCj4+ID4gPg0KPj4gPiA+IDA1OjMyOjI0LjEzNDE3MiBJUCA5NC4yNDMuMTA0LjE1Ni4zMjky NCA+IDE5NS45NS4xNzguMjA0LjgwOiANCj4+ID4gPiBGbGFncyBbU10sIHNlcSA0Mjk0NDIzMjE1 LCB3aW4gNjU1MzUsIG9wdGlvbnMgW21zcyAxNDYwLHNhY2tPSyxUUyANCj4+ID4gPiB2YWwNCj4+ ID4gPiAxMTEyMzUxIGVjciAwLG5vcCx3c2NhbGUgOF0sIGxlbmd0aCAwDQo+PiA+ID4gMDU6MzI6 MjQuMTYxOTg2IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IA0K Pj4gPiA+IEZsYWdzIFtTLl0sIHNlcSAxMTk4ODc3NzIxLCBhY2sgNDI5NDQyMzIxNiwgd2luIDI4 OTYwLCBvcHRpb25zIA0KPj4gPiA+IFttc3MgMTQxNixzYWNrT0ssVFMgdmFsDQo+PiA+ID4gMjgw ODgwODYzNCBlY3IgMTExMjM1MSxub3Asd3NjYWxlIDddLCBsZW5ndGggMA0KPj4gPiA+IDA1OjMy OjI0LjE2MjEwOCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiAN Cj4+ID4gPiBGbGFncyBbLl0sIGFjayAxLCB3aW4gMzQzLCBvcHRpb25zIFtub3Asbm9wLFRTIHZh bCAxMTEyMzU0IGVjciANCj4+ID4gPiAyODA4ODA4NjM0XSwgbGVuZ3RoIDANCj4+ID4gPiAwNToz MjoyNC4xNjMwNzIgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDog DQo+PiA+ID4gRmxhZ3MgW1AuXSwgc2VxIDE6NjU4LCBhY2sgMSwgd2luIDM0Mywgb3B0aW9ucyBb bm9wLG5vcCxUUyB2YWwgDQo+PiA+ID4gMTExMjM1NCBlY3IgMjgwODgwODYzNF0sIGxlbmd0aCA2 NTcNCj4+ID4gPiAwNTozMjoyNC4yMDM5ODQgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMu MTA0LjE1Ni4zMjkyNDogDQo+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRp b25zIFtub3Asbm9wLFRTIHZhbCAyODA4ODA4Njc1IA0KPj4gPiA+IGVjciAxMTEyMzU0XSwgbGVu Z3RoIDANCj4+ID4gPiAwNTozMjoyNC4yMTAyNzUgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4y NDMuMTA0LjE1Ni4zMjkyNDogDQo+PiA+ID4gRmxhZ3MgW1AuXSwgc2VxIDE6Mzg3LCBhY2sgNjU4 LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbA0KPj4gPiA+IDI4MDg4MDg2NzcgZWNy IDExMTIzNTRdLCBsZW5ndGggMzg2DQo+PiA+ID4gMDU6MzI6MjQuMjEwMzE1IElQIDE5NS45NS4x NzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IA0KPj4gPiA+IEZsYWdzIFsuXSwgc2Vx IDM4NzoxNzkxLCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIA0KPj4gPiA+ IHZhbA0KPj4gPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPj4gPiA+ IDA1OjMyOjI0LjIxMDMzMiBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMy OTI0OiANCj4+ID4gPiBGbGFncyBbLl0sIHNlcSAxNzkxOjMxOTUsIGFjayA2NTgsIHdpbiAyMzcs IG9wdGlvbnMgW25vcCxub3AsVFMgDQo+PiA+ID4gdmFsDQo+PiA+ID4gMjgwODgwODY3NyBlY3Ig MTExMjM1NF0sIGxlbmd0aCAxNDA0DQo+PiA+ID4gMDU6MzI6MjQuMjEwMzQ3IElQIDE5NS45NS4x NzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6IA0KPj4gPiA+IEZsYWdzIFsuXSwgc2Vx IDMxOTU6NDU5OSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyANCj4+ID4g PiB2YWwNCj4+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4+ID4g PiAwNTozMjoyNC4yMTAzNTAgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIw NC44MDogDQo+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgMzg3LCB3aW4gMzQ3LCBvcHRpb25zIFtub3As bm9wLFRTIHZhbCAxMTEyMzU5IGVjciANCj4+ID4gPiAyODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4+ ID4gPiAwNTozMjoyNC4yMTAzNjEgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1 Ni4zMjkyNDogDQo+PiA+ID4gRmxhZ3MgWy5dLCBzZXEgNDU5OTo2MDAzLCBhY2sgNjU4LCB3aW4g MjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIA0KPj4gPiA+IHZhbA0KPj4gPiA+IDI4MDg4MDg2Nzcg ZWNyIDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPj4gPiA+IDA1OjMyOjI0LjIxMDM3NCBJUCAxOTUu OTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiANCj4+ID4gPiBGbGFncyBbLl0s IHNlcSA2MDAzOjc0MDcsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgDQo+ PiA+ID4gdmFsDQo+PiA+ID4gMjgwODgwODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0aCAxNDA0DQo+ PiA+ID4gMDU6MzI6MjQuMjEwMzg5IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4x NTYuMzI5MjQ6IA0KPj4gPiA+IEZsYWdzIFsuXSwgc2VxIDc0MDc6ODgxMSwgYWNrIDY1OCwgd2lu IDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyANCj4+ID4gPiB2YWwNCj4+ID4gPiAyODA4ODA4Njc3 IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4+ID4gPiAwNTozMjoyNC4yMTA0MDIgSVAgMTk1 Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogDQo+PiA+ID4gRmxhZ3MgWy5d LCBzZXEgODgxMToxMDIxNSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyAN Cj4+ID4gPiB2YWwNCj4+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQN Cj4+ID4gPiAwNTozMjoyNC4yMTA0MTYgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0 LjE1Ni4zMjkyNDogDQo+PiA+ID4gRmxhZ3MgWy5dLCBzZXEgMTAyMTU6MTE2MTksIGFjayA2NTgs IHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgDQo+PiA+ID4gdmFsDQo+PiA+ID4gMjgwODgw ODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0aCAxNDA0DQo+PiA+ID4gMDU6MzI6MjQuMjEwNDE4IElQ IDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IA0KPj4gPiA+IEZsYWdz IFsuXSwgYWNrIDE3OTEsIHdpbiAzNTgsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkg ZWNyIA0KPj4gPiA+IDI4MDg4MDg2NzddLCBsZW5ndGggMA0KPj4gPiA+IDA1OjMyOjI0LjIxMDQz MSBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiANCj4+ID4gPiBG bGFncyBbLl0sIHNlcSAxMTYxOToxMzAyMywgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9w LG5vcCxUUyANCj4+ID4gPiB2YWwNCj4+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVu Z3RoIDE0MDQNCj4+ID4gPiAwNTozMjoyNC4yMTA0NTUgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQg PiAxOTUuOTUuMTc4LjIwNC44MDogDQo+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgMzE5NSwgd2luIDM2 OSwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3IgDQo+PiA+ID4gMjgwODgwODY3 N10sIGxlbmd0aCAwDQo+PiA+ID4gMDU6MzI6MjQuMjEwNjg1IElQIDk0LjI0My4xMDQuMTU2LjMy OTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IA0KPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDQ1OTksIHdp biAzODAsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyIA0KPj4gPiA+IDI4MDg4 MDg2NzddLCBsZW5ndGggMA0KPj4gPiA+IDA1OjMyOjI0LjIxMDczMSBJUCA5NC4yNDMuMTA0LjE1 Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiANCj4+ID4gPiBGbGFncyBbLl0sIGFjayA2MDAz LCB3aW4gMzkxLCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjciANCj4+ID4gPiAy ODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4+ID4gPiAwNTozMjoyNC4yMTA5MzcgSVAgOTQuMjQzLjEw NC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogDQo+PiA+ID4gRmxhZ3MgWy5dLCBhY2sg NzQwNywgd2luIDQwMiwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3IgDQo+PiA+ ID4gMjgwODgwODY3N10sIGxlbmd0aCAwDQo+PiA+ID4gMDU6MzI6MjQuMjExMDc4IElQIDk0LjI0 My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IA0KPj4gPiA+IEZsYWdzIFsuXSwg YWNrIDg4MTEsIHdpbiA0MTMsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNyIA0K Pj4gPiA+IDI4MDg4MDg2NzddLCBsZW5ndGggMA0KPj4gPiA+IDA1OjMyOjI0LjIxMTIyOCBJUCA5 NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOiANCj4+ID4gPiBGbGFncyBb Ll0sIGFjayAxMDIxNSwgd2luIDQyNCwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSAN Cj4+ID4gPiBlY3IgMjgwODgwODY3N10sIGxlbmd0aCAwDQo+PiA+ID4gMDU6MzI6MjQuMjExMzc3 IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6IA0KPj4gPiA+IEZs YWdzIFsuXSwgYWNrIDExNjE5LCB3aW4gNDM1LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEy MzU5IA0KPj4gPiA+IGVjciAyODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4+ID4gPiAwNTozMjoyNC4y MTE1NDQgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDogDQo+PiA+ ID4gRmxhZ3MgWy5dLCBhY2sgMTMwMjMsIHdpbiA0NDYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFs IDExMTIzNTkgDQo+PiA+ID4gZWNyIDI4MDg4MDg2NzddLCBsZW5ndGggMA0KPj4gPiA+IDA1OjMy OjI0LjIzNzk5OSBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0OiAN Cj4+ID4gPiBGbGFncyBbLl0sIHNlcSAxMzAyMzoxNDQyNywgYWNrIDY1OCwgd2luIDIzNywgb3B0 aW9ucyBbbm9wLG5vcCxUUyANCj4+ID4gPiB2YWwNCj4+ID4gPiAyODA4ODA4NzA5IGVjciAxMTEy MzU5XSwgbGVuZ3RoIDE0MDQNCj4+ID4gPiAwNTozMjoyNC4yMzgwNTMgSVAgMTk1Ljk1LjE3OC4y MDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogDQo+PiA+ID4gRmxhZ3MgWy5dLCBzZXEgMTQ0 Mjc6MTU4MzEsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgDQo+PiA+ID4g dmFsDQo+PiA+ID4gMjgwODgwODcwOSBlY3IgMTExMjM1OV0sIGxlbmd0aCAxNDA0DQo+PiA+ID4g MDU6MzI6MjQuMjM4MDkwIElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQu ODA6IA0KPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDE0NDI3LCB3aW4gNDU3LCBvcHRpb25zIFtub3As bm9wLFRTIHZhbCAxMTEyMzYyIA0KPj4gPiA+IGVjciAyODA4ODA4NzA5XSwgbGVuZ3RoIDANCj4+ ID4gPiAwNTozMjoyNC4yMzgxNjQgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4 LjIwNC44MDogDQo+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgMTU4MzEsIHdpbiA0NjgsIG9wdGlvbnMg W25vcCxub3AsVFMgdmFsIDExMTIzNjIgDQo+PiA+ID4gZWNyIDI4MDg4MDg3MDldLCBsZW5ndGgg MA0KPj4gPiA+IDA1OjMyOjI0LjI0NDAyNCBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4x MDQuMTU2LjMyOTI0OiANCj4+ID4gPiBGbGFncyBbLl0sIHNlcSAxNTgzMToxNzIzNSwgYWNrIDY1 OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUyANCj4+ID4gPiB2YWwNCj4+ID4gPiAyODA4 ODA4NzEzIGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4+ID4gPiAwNTozMjoyNC4yNDQwNjMg SVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDogDQo+PiA+ID4gRmxh Z3MgWy5dLCBzZXEgMTcyMzU6MTg2MzksIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxu b3AsVFMgDQo+PiA+ID4gdmFsDQo+PiA+ID4gMjgwODgwODcxMyBlY3IgMTExMjM1OV0sIGxlbmd0 aCAxNDA0DQo+PiA+ID4NCj4NCg== From nobody Thu Oct 1 10:02:20 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2D2C71B2DF1 for ; Thu, 1 Oct 2015 10:02:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.447 X-Spam-Level: X-Spam-Status: No, score=-1.447 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FRT_BELOW2=2.154, J_CHICKENPOX_36=0.6, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JUIU7vt7eGHL for ; Thu, 1 Oct 2015 10:02:13 -0700 (PDT) Received: from sesbmg23.ericsson.net (sesbmg23.ericsson.net [193.180.251.37]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4136B1B2DEE for ; Thu, 1 Oct 2015 10:02:12 -0700 (PDT) X-AuditID: c1b4fb25-f79a26d00000149a-09-560d6712d830 Received: from ESESSHC012.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg23.ericsson.net (Symantec Mail Security) with SMTP id 83.B2.05274.2176D065; Thu, 1 Oct 2015 19:02:10 +0200 (CEST) Received: from ESESSMB205.ericsson.se ([169.254.5.99]) by ESESSHC012.ericsson.se ([153.88.183.54]) with mapi id 14.03.0248.002; Thu, 1 Oct 2015 19:02:09 +0200 From: Ingemar Johansson S To: Veaceslav ROMAN , Eric Dumazet Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8Pq87vsQ9Xnw1wDq87cGANXnMWyg Date: Thu, 1 Oct 2015 17:02:08 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA34BD89C3@ESESSMB205.ericsson.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E863AD@XCHSRV01.main.orange.md> In-Reply-To: <7DBBB686E19D2049ADAACD210B474BB10166E863AD@XCHSRV01.main.orange.md> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.154] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsUyM+Jvja5QOm+YweSlchZPjz1it9h9biqL xb73Z9ksOu7sBbJmTGS0OHLlGKPFtpPzmSze/9/I7MDhsXPWXXaPBZtKPZYs+cnksWTZU0aP juMbmANYo7hsUlJzMstSi/TtErgy1j6ey1hwrYm5YuL/ZywNjA9+MHUxcnJICJhITD08AcoW k7hwbz1bFyMXh5DAUUaJtftOMEI4ixglnh09DFbFJmAjsfLQd0YQW0QgVOLgtMVgHcwCO5gk Nh/cwAqSEBYwk7jV9I4ZoshcYtLpNmaQIhGBfYwSU4+/AkuwCKhI/H08DWwSr4CvxJ8HU5gg 1m3hlrh0aiI7SIJTIFDi+fSfbCA2o4CsxP3v91hAbGYBcYlbT+ZDHS4gsWTPeWYIW1Ti5eN/ rBC2ksSi25+BajiA6jUl1u/Sh2hVlJjS/ZAdYq+gxMmZT1gmMIrNQjJ1FkLHLCQds5B0LGBk WcUoWpxanJSbbmSsl1qUmVxcnJ+nl5dasokRGJ8Ht/xW3cF4+Y3jIUYBDkYlHt4Fk3nChFgT y4orcw8xSnOwKInzNjM9CBUSSE8sSc1OTS1ILYovKs1JLT7EyMTBKdXAyN1/zTC11OhgxtbL 9Ze+iX8QVb5dZF6+sT1+as95FpEbm7b2czW5BHKEW3WnxKlFz+54zrTurOoTj77TG37MqV+V LNE8t5bPNlbGX+RfuNo1CXf/bfr8x3l2vGxSkg1d/NWIrVZk8pz1olObGSpk2+Y+nf3n7muf c/P2h6XsWR976viBE9e2KbEUZyQaajEXFScCAC/Jy3GwAgAA Archived-At: Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , Ingemar Johansson S , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2015 17:02:19 -0000 SGkNCg0KQW4gaW5jcmVhc2VkIHRocmVzaG9sZCB3b3VsZCBwcm9iYWJseSBzb2x2ZSB0aGUgcHJv YmxlbSBhbmQgaXMgcG9zc2libHkgdGhlIHF1aWNrIGZpeC4gDQpJIHdpbGwgYW55d2F5IG5lZWQg dG8gcmVydW4gcGFydHMgb2YgbXkgc2ltdWxhdGlvbnMgYmVjYXVzZSBvZiBteSBvd24gMzJtcyBt aXN0YWtlLCBpdCBpcyBxdWl0ZSBzdHJhaWdodGZvcndhcmQgdG8gYWRkIGEgZmV3IGFsdGVybmF0 aXZlIHRocmVzaG9sZCB2YWx1ZXMgaW4gdGhlIHNhbWUgd29yay4NCg0KV29uZGVyIHRob3VnaCBp ZiBvbmUgY2FuIGRvIGl0IHNtYXJ0ZXI/Lg0KRm9yIGluc3RhbmNlLCBhc3N1bWluZyB0aGF0IHRo ZSB0aW1lc3RhbXAgb3B0aW9uIGlzIGVuYWJsZWQsIGl0IHNob3VsZCB0aGVyZSBiZSBwb3NzaWJs ZSB0byBpbmZlciBpZiBBQ0tzIGJlbG9uZyB0byB0aGUgc2FtZSBIeVN0YXJ0IHJvdW5kLCBiYXNl ZCBvbiBUU3ZhbC4gDQpJdCBvZiBjb3Vyc2UgYWRkcyBzb21lIGV4dHJhIGxpbmVzIG9mIGNvZGUg dG8gdGhlIEh5U3RhcnQgYWxnb3JpdGhtIGJ1dCB0aGVuIG9uZSBjYW4gcGVyaGFwcyBrZWVwIHRo ZSBjdXJyZW50IHRocmVzaG9sZHMgPy4NCg0KL0luZ2VtYXINCg0KPiAtLS0tLU9yaWdpbmFsIE1l c3NhZ2UtLS0tLQ0KPiBGcm9tOiBWZWFjZXNsYXYgUk9NQU4gW21haWx0bzpWZWFjZXNsYXYuUm9t YW5Ab3JhbmdlLm1kXQ0KPiBTZW50OiBkZW4gMSBva3RvYmVyIDIwMTUgMTc6MTcNCj4gVG86IEVy aWMgRHVtYXpldDsgSW5nZW1hciBKb2hhbnNzb24gUw0KPiBDYzogRXJpYyBEdW1hemV0OyBOZWFs IENhcmR3ZWxsOyB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsNCj4g ZW5kMmVuZC1pbnRlcmVzdEBwb3N0ZWwub3JnDQo+IFN1YmplY3Q6IFJFOiBbdGNwbV0gW2UyZV0g VENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiANCj4gU3VyZSAxMCBtcyB3aWxsIGJlIGJl dHRlciBhcyBtb3JlIG9mIHRoZSBMVEUgdGVjaG5vbG9naWNhbCBqdW1wcyB3aWxsIGJlDQo+IGNv dmVyZWQgYW5kLCBhdCBsZWFzdCwgd2lsbCBoZWxwIGdldCByaWQgb2YgZXhpdCBhdCB0aGUgbWlu aW11bSBleGl0IGN3bmQgKDI5LA0KPiB3aGljaCBhdCAyMCBtcyBSVFQgaXMgfjE2TWJwcykgYW5k IHdpbGwgc2hpZnQgdG8gNDkgKH4yOCBNYnBzKSBvciB0byA4OSAofiA1MA0KPiBNYnBzKS4NCj4g V2UgaGF2ZSBhbGwgbmVjZXNzYXJ5IHRvIHJlcGVhdCB0aG9zZSB0ZXN0cy4NCj4gDQo+IFZlYWNl c2xhdiBSb21hbg0KPiANCj4gDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206 IEVyaWMgRHVtYXpldCBbbWFpbHRvOmVkdW1hemV0QGdvb2dsZS5jb21dDQo+IFNlbnQ6IFRodXJz ZGF5LCAwMSBPY3RvYmVyIDIwMTUgMTU6NDQNCj4gVG86IEluZ2VtYXIgSm9oYW5zc29uIFMNCj4g Q2M6IFZlYWNlc2xhdiBST01BTjsgRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsOyB0Y3BtQGll dGYub3JnOyBQaWVycw0KPiBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgZW5kMmVuZC1pbnRlcmVzdEBw b3N0ZWwub3JnDQo+IFN1YmplY3Q6IFJlOiBbdGNwbV0gW2UyZV0gVENQIEh5U3RhcnQgcGF0Y2gg ZGVwbG95bWVudA0KPiANCj4gWWVzLCB0aGlzIDRtcyB2YWx1ZSBwcm9ibGVtIGlzIGV2ZW4gbW9y ZSB2aXNpYmxlIHdpdGggVENQIHBhY2luZyAoYXMNCj4gaW1wbGVtZW50ZWQgd2l0aCBsaW51eCBz Y2hfZnEgcGFja2V0IHNjaGVkdWxlcikgYmVjYXVzZSBvZiBhcHBhcmVudA0KPiBkZWxheXMgY2F1 c2VkIGJ5IHNvam91cm4gdGltZSBpbiBwYWNrZXQgc2NoZWR1bGVyIGl0c2VsZi4NCj4gVENQIGRv ZXMgbm90IG9mZnNldCB0aGlzIHNvam91cm4gdGltZSAoYXMgUlRUIGlzIGFsc28gdXNlZCBmb3Ig UlRPDQo+IGRldGVybWluYXRpb24sIGJldHRlciBoYXZlIGFuIGluZmxhdGVkIHZpZXcpDQo+IA0K PiBJcyBzd2l0Y2hpbmcgdG8gMTAgbXMgd291bGQgYmUgYmV0dGVyIGluIHlvdXIgTFRFIHRlc3Rz ID8NCj4gDQo+IA0KPiANCj4gT24gVGh1LCBPY3QgMSwgMjAxNSBhdCA0OjU3IEFNLCBJbmdlbWFy IEpvaGFuc3NvbiBTDQo+IDxpbmdlbWFyLnMuam9oYW5zc29uQGVyaWNzc29uLmNvbT4gd3JvdGU6 DQo+ID4gSG1tDQo+ID4gWW91IGFyZSByaWdodCAhLg0KPiA+IFdlbGwuLi4gVGhhdCBvZiBjb3Vy c2Ugc2hvdWxkIGV4cGxhaW4gd2h5IEh5U3RhcnQgcGVyZm9ybXMgYmFkbHkgZm9yIExURS4uLg0K PiBBdGxlYXN0IGl0IGlzIHBhcnQgb2YgdGhlIHJlYXNvbi4NCj4gPg0KPiA+IC9JbmdlbWFyDQo+ ID4NCj4gPg0KPiA+DQo+ID4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+IEZyb206 IFZlYWNlc2xhdiBST01BTiBbbWFpbHRvOlZlYWNlc2xhdi5Sb21hbkBvcmFuZ2UubWRdDQo+ID4+ IFNlbnQ6IGRlbiAxIG9rdG9iZXIgMjAxNSAxMzo0Ng0KPiA+PiBUbzogSW5nZW1hciBKb2hhbnNz b24gUzsgRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsDQo+ID4+IENjOiB0Y3BtQGlldGYub3Jn OyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgRXJpYyBEdW1hemV0OyBlbmQyZW5kLQ0KPiA+ PiBpbnRlcmVzdEBwb3N0ZWwub3JnDQo+ID4+IFN1YmplY3Q6IFJFOiBbdGNwbV0gW2UyZV0gVENQ IEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiA+Pg0KPiA+PiBXZWxsIHRoaXMgc2F5DQo+ID4+ ICAgICAgIElmIGN1cnJfcnR0IGlzIGhpZ2hlciB0aGFuIGRlbGF5X21pbiArICgxLzggKiBkZWxh eV9taW4gb3IgNA0KPiA+PiBtcywgd2hpY2hldmVyIGlzIGhpZ2hlcikgIHRoZW4gZXhpdCBzbG93 IHN0YXJ0Lg0KPiA+Pg0KPiA+PiBXaGVuIGFjdHVhbCBtaW5pbXVtIGRlbGF5IGlzIGxlc3MgdGhh biAzMiBtcyB0aGUgZGVsYXlfbWluIHdpbGwgYmUNCj4gPj4gbGVzcyB0aGFuIDMyPDwzLCB0aGVu IHRoZSBkZWxheV9taW4+PjMgd2lsbCBiZSBsZXNzIHRoYW4gMzIgYW5kIHdpbGwNCj4gPj4gY29t cGFyZSB3aXRoDQo+ID4+IDQ8PDMgd2hpY2ggaXMgMzIgYW5kIHRoZXJlZm9yIHRoZSB0aHJlc2hv bGQgd2lsbCBiZQ0KPiA+PiAocmVhbF9jdXJyX3J0dDw8MykgPg0KPiA+PiAocmVhbF9kZWxheTw8 MyArIDQ8PDMpIG9yIChyZWFsX2N1cnJfcnR0KT4ocmVhbF9kZWxheSArNCkNCj4gPj4NCj4gPj4N Cj4gPj4gVmVhY2VzbGF2IFJvbWFuDQo+ID4+DQo+ID4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0t LS0tDQo+ID4+IEZyb206IEluZ2VtYXIgSm9oYW5zc29uIFMgW21haWx0bzppbmdlbWFyLnMuam9o YW5zc29uQGVyaWNzc29uLmNvbV0NCj4gPj4gU2VudDogVGh1cnNkYXksIDAxIE9jdG9iZXIgMjAx NSAxMzowMA0KPiA+PiBUbzogVmVhY2VzbGF2IFJPTUFOOyBFcmljIER1bWF6ZXQ7IE5lYWwgQ2Fy ZHdlbGwNCj4gPj4gQ2M6IHRjcG1AaWV0Zi5vcmc7IFBpZXJzIE8nSGFubG9uOyBTYW5ndGFlIEhh OyBFcmljIER1bWF6ZXQ7IGVuZDJlbmQtDQo+ID4+IGludGVyZXN0QHBvc3RlbC5vcmc7IEluZ2Vt YXIgSm9oYW5zc29uIFMNCj4gPj4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFy dCBwYXRjaCBkZXBsb3ltZW50DQo+ID4+DQo+ID4+IEhpDQo+ID4+DQo+ID4+IFllcywgdGhlIHZh cmlhYmxlIGRlbGF5X21pbiBpcyBpbiBRMyBmb3IgcmVhc29ucyBvZiBwcmVjaXNpb24gSSBndWVz cw0KPiA+PiBCdXQgdGhlIGNhbGwgdG8gSFlTVEFSVF9ERUxBWV9USFJFU0ggaXMgd2l0aCBkZWxh eV9taW4gc2hpZnRlZCBkb3duDQo+ID4+IHRvIFEwDQo+ID4+DQo+ID4+IEhZU1RBUlRfREVMQVlf VEhSRVNIKGNhLT5kZWxheV9taW4gPj4gMykpICAoc2VlIGh0dHA6Ly9seHIuZnJlZS0NCj4gPj4g ZWxlY3Ryb25zLmNvbS9zb3VyY2UvbmV0L2lwdjQvdGNwX2N1YmljLmMjTDQwMyApIEkgbWF5IGJl IG9mZiBoZXJlLi4NCj4gPj4NCj4gPj4gUmVnYXJkaW5nIHBhY2luZy4gSSBkb24ndCBoYXZlIHJl YWwgZGF0YSB0byBzaG93IHRoZSBlZmZlY3RzIG9mDQo+ID4+IHBhY2luZywgaW4gc2ltdWxhdGlv bnMgaXQgaXMgaG93ZXZlciBwb3NzaWJsZSB0byBzZWUgdGhlIGdhaW5zIGluDQo+ID4+IHRlcm1z IG9mIGxlc3MgY29hbGVzY2luZywgeWVzIHBhY2luZyBkb2VzIG5vdCBzb2x2ZSB0aGUgQUNLDQo+ ID4+IGNvbXByZXNzaW9uIGJ1dCBpdCBzZWVtIHRvIHNvbHZlIHRoZSBmb2xsb3cgdXAgZWZmZWN0 IHRoYXQgQUNLDQo+IGNvbXByZXNzaW9uIGdpdmVzLg0KPiA+Pg0KPiA+PiAvSW5nZW1hcg0KPiA+ Pg0KPiA+PiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+ID4gRnJvbTogVmVhY2Vz bGF2IFJPTUFOIFttYWlsdG86VmVhY2VzbGF2LlJvbWFuQG9yYW5nZS5tZF0NCj4gPj4gPiBTZW50 OiBkZW4gMSBva3RvYmVyIDIwMTUgMTA6MzINCj4gPj4gPiBUbzogSW5nZW1hciBKb2hhbnNzb24g UzsgRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsDQo+ID4+ID4gQ2M6IHRjcG1AaWV0Zi5vcmc7 IFBpZXJzIE8nSGFubG9uOyBTYW5ndGFlIEhhOyBFcmljIER1bWF6ZXQ7DQo+ID4+ID4gZW5kMmVu ZC0gaW50ZXJlc3RAcG9zdGVsLm9yZw0KPiA+PiA+IFN1YmplY3Q6IFJFOiBbdGNwbV0gW2UyZV0g VENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiA+PiA+DQo+ID4+ID4gWWVzLCBIWVNUQVJU X0RFTEFZX01JTiAgKDRVPDwzKSAuDQo+ID4+ID4gQnV0IGN1cnJfcnR0IGFuZCBkZWxheV9taW4g Y29tZSBmcm9tIGJpY3RjcF9hY2tlZCB3aGljaCBpcyBhbHNvIDw8MyA6DQo+ID4+ID4gICAgIElu IHRjcF9jdWJpYy5jLCBmdW5jdGlvbiAgbGluZSA0MzMgKGtlcm5lbCA0LjIpDQo+ID4+ID4gICAg ICAgICAgICAgZGVsYXkgPSAocnR0X3VzIDw8IDMpIC8gVVNFQ19QRVJfTVNFQzsgU28sDQo+ID4+ ID4gNFU8PDMgaXMgZXF1aXZhbGVudCB0byA0IG1zIC4NCj4gPj4gPg0KPiA+PiA+IFJlZ2FyZGlu ZyBwYWNpbmcgSSBhbSBub3Qgc3VyZSB3aWxsIGhlbHAgdGhlIHBlcmZvcm1hbmNlOiB0aGVyZSBh cmUNCj4gPj4gPiBnYXBzIGluIEFDSyB0cmFpbnMgb2YgdGhlIG9yZGVyIG9mIG1pbGxpc2Vjb25k cyBhbmQgdGhpcyBpcyBhIGxvc3QNCj4gPj4gPiB0aW1lIGFuZCB0aGVyZSBpcyBubyB3YXkgdG8g Y29tcGVuc2F0ZSBpdC4gSWYgaW50cm9kdWNlIEFDSyBwYWNpbmcNCj4gPj4gPiBvciBkYXRhIHBh Y2tldHMgcGFjaW5nIHRoaXMgd291bGQgYWRkIHRvIHRoZSBvdmVyYWxsIGxvc3QgdGltZS4NCj4g Pj4gPiBQYWNpbmcgY2FuIGhlbHAgb25seSB0byBhbGxldmlhdGUgYSBwb3RlbnRpYWwgcHJvYmxl bSBvZiBzb21lDQo+ID4+ID4gZG93bnN0cmVhbSByb3V0ZXJzIHdpdGggaW5zdWZmaWNpZW50IGJ1 ZmZlcnMgd2hpY2ggbWF5IGludHJvZHVjZQ0KPiA+PiA+IHVuZXhwZWN0ZWQgcGFja2V0cyBkcm9w cyB3aGVuIGxhcmdlIHRyYWluIHNlbnQgYXQgYSBoaWdoIHNwZWVkIGFzIGENCj4gPj4gPiByZXBs eSB0byBoaWdobHkgY29tcHJlc3NlZCBBQ0sgdHJhaW4sIGJ1dCwgb24gYW5vdGhlciBoYW5kcy4N Cj4gPj4gPiBCdXQgaG93IG11Y2ggcGFjaW5nIGFuZCBob3cgdG8gYWNjb21tb2RhdGUgdG8gbGFy Z2VseSBjaGFuZ2luZw0KPiA+PiA+IHJhZGlvIGNvbmRpdGlvbnMgPyBBbmQgd2l0aCBwZXJtYW5l bnRseSBjaGFuZ2luZyB0ZWNobm9sb2d5IGFuZA0KPiBzcGVlZCA/DQo+ID4+ID4gQW55d2F5IHRo ZSBwYWNpbmcgd2lsbCBhZGQgdG8gdGhlIHByb2JsZW0gb2YgZ2FwcyBhbmQgYmV0dGVyIHNpemUN Cj4gPj4gPiBwcm9wZXJseSByb3V0ZXJzIGJ1ZmZlcnMuDQo+ID4+ID4NCj4gPj4gPiBJZGVhbGx5 IEFDSyB0cmFpbiBpcyBzcGxpdCBpbiAyIHBhcnRzLCBidXQgaW4gcHJhY3RpY2UgSSBkbyBzZWUg bW9yZS4NCj4gPj4gPiBCdXQgYXMgdGhlIHN0cmVhbSBwcm9ncmVzcyB0aGUgbGVzcyBnYXBzLiBU aGUgYmlnZ2VyIHRoZSBkb3dubGluaw0KPiA+PiA+IHByZXNzdXJlIHRoZSBtb3JlIEFDSyBhbmQg bGVzcyBjaGFuY2VzIHRvIGhhdmUgYSBwYXVzZSBpbiBBQ0sNCj4gPj4gPiBzZW5kaW5nLCB0aGVu IHRoZXJlIHdpbGwgYmUgYSBjb250aW51b3VzIGZsb3cgb2YgQnVmZmVyIFN0YXR1cw0KPiA+PiA+ IFJlcG9ydHMgYW5kIGxlc3Mgb2YgdGhlIFNjaGVkdWxpbmcgUmVxdWVzdCAoU1IsICJIZXksIEkg aGF2ZSBzb21lDQo+ID4+ID4gZGF0YSB0byBzZW5kIikuIFRoZSBvbmx5IGlzc3VlIEkgYW0gbm90 IHN1cmUgaXMgb24gdGhlIHBvdGVudGlhbA0KPiA+PiA+IGdhcHMgaW4gb3RoZXIgbmV0d29ya3Mu IEkgZGlkIG5vdCBwdXQgYmVsbG93IGFsbCBkZXRhaWxzLCBidXQgeW91DQo+ID4+ID4gc2hvdWxk IGtub3cgdGhhdCBkZXZpY2UgY2FuIG5vdCBzZW5kIFNSIGluIGFyYml0cmFyeSBtb21lbnRzIG9m DQo+ID4+ID4gdGltZSBidXQgaW4gcHJlZGVmaW5lZCBmb3IgaGltIGludGVydmFscy4gSW4gbXkg bmV0d29yayB0aGlzIHBlcmlvZA0KPiA+PiA+IGlzIDUgbXMsIHRoZXJlZm9yZSB0aGUgUlRUIG9m IHRoZSBmaXJzdCBBQ0sgaW4gdGhlIHRyYWluIHdpbGwgaGF2ZQ0KPiA+PiA+IHZhcmlhdGlvbnMg b2YgMi41IG1zLiBCdXQgc3RhbmRhcmRzIGFkbWl0IHRoaXMgcGVyaW9kIHRvIGJlIGFsc28gMTAN Cj4gPj4gPiBtcyAsMjAgbXMsIDQwIG1zLCA4MCBtcyBhbmQgSSBjYW4gbm90IHNheSB3aGF0IGlz IHRoZSBwcmFjdGljZSBvZg0KPiA+PiA+IG90aGVycy4gSSBtYXkgb25seSBndWVzcyB0aGF0IG1v c3Qgd2lsbCB0cnkgdG8gdXNlDQo+ID4+IHRoZSBiZXN0IGFuZCBmYXN0ZXN0LCBpLmUuIDUgbXMu DQo+ID4+ID4gQW55d2F5LCB0byBnZXQgYSByZWFsaXN0aWMgbW9kZWxpbmcgb2YgTFRFIGludGVy YWN0aW9uIHdpdGggVENQIHRoZQ0KPiA+PiA+IGRldGFpbGVkIE1BQyBzY2hlZHVsZXIgbW9kZWwg aXMgbmVlZGVkLiBVbmZvcnR1bmF0ZWx5LCBtb3N0IG9mIHRoZQ0KPiA+PiA+IGxpdGVyYXR1cmUg Zm9jdXMgb24gc2NoZWR1bGluZyBvZiB0aGUgUGh5c2ljYWwgUmVzb3VyY2UgQmxvY2tzLA0KPiA+ PiA+IEFkYXB0aXZlIE1vZHVsYXRpb24gYW5kIENvZGluZywgTUlNTywgaS5lLiwgZm9jdXNpbmcg b24gdGhlDQo+ID4+ID4gZGlzdHJpYnV0aW9uIG9mIHJlc291cmNlcyBiZXR3ZWVuIHVzZXJzLCB3 aGljaCBpcyBnb29kIGFuZCBuZWVkZWQsDQo+ID4+ID4gYnV0IGRvIG5vdCBhZGRyZXNzIHZlcnkg bXVjaCB0aGUgdGltaW5nIGludGVyYWN0aW9uIHdpdGggVENQLiAgQXQNCj4gPj4gPiBsZWFzdCwg SSBuZXZlciBtZXQgaW4gTFRFIHNjaGVkdWxpbmcgbGl0ZXJhdHVyZSBhIHRlcm0gIlRDUCBzZWxm LWNsb2NraW5nIi4NCj4gPj4gPg0KPiA+PiA+IFRoZSA4IG1zICBIWVNUQVJUX0RFTEFZX01JTiBj b3VsZCBvbmx5IGJlIGEgc2ltcGxlIHNvbHV0aW9uIGZvciB0aGUNCj4gPj4gPiBwcm9ibGVtIG9m IGNvdW50aW5nIG9mIHRoZSByb3VuZCcgY3Vycl9ydHQgZnJvbSB0aGUgc2Vjb25kIEFDSyBvZg0K PiA+PiA+IHRoZSB0cmFpbiAoYXMgdGhlIHNlY29uZCBBQ0sgUlRUIHdpbGwgYmUsIHR5cGljYWxs eSwgZHVlIHRvDQo+ID4+ID4gc2NoZWR1bGluZywNCj4gPj4gPiA1LTcgbXMgbG9uZ2VyIHRoYW4g dGhlIGZpcnN0IG9uZSksIGJ1dCB0aGlzIG9ubHkgaWYgU1IgcGVyaW9kaWNpdHkgaXMgNSBtcy4N Cj4gPj4gPiBUaGUgb3RoZXIgMiBwcm9ibGVtcyB3aWxsIHJlbWFpbiB1bnNvbHZlZC4NCj4gPj4g Pg0KPiA+PiA+IFZlYWNlc2xhdiBSb21hbg0KPiA+PiA+DQo+ID4+ID4gLS0tLS1PcmlnaW5hbCBN ZXNzYWdlLS0tLS0NCj4gPj4gPiBGcm9tOiBJbmdlbWFyIEpvaGFuc3NvbiBTDQo+IFttYWlsdG86 aW5nZW1hci5zLmpvaGFuc3NvbkBlcmljc3Nvbi5jb21dDQo+ID4+ID4gU2VudDogVGh1cnNkYXks IDAxIE9jdG9iZXIgMjAxNSAwODo1OQ0KPiA+PiA+IFRvOiBWZWFjZXNsYXYgUk9NQU47IEVyaWMg RHVtYXpldDsgTmVhbCBDYXJkd2VsbA0KPiA+PiA+IENjOiB0Y3BtQGlldGYub3JnOyBQaWVycyBP J0hhbmxvbjsgU2FuZ3RhZSBIYTsgRXJpYyBEdW1hemV0Ow0KPiA+PiA+IGVuZDJlbmQtIGludGVy ZXN0QHBvc3RlbC5vcmc7IEluZ2VtYXIgSm9oYW5zc29uIFMNCj4gPj4gPiBTdWJqZWN0OiBSRTog W3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4gPj4gPg0KPiA+PiA+ IEhpDQo+ID4+ID4NCj4gPj4gPiBUaGFua3MgZm9yIHRoZSBkZXRhaWxlZCByZXBvcnQuIFdoYXQg c2VlbXMgdW5jbGVhciB0byBtZSBpcyB0aGUNCj4gPj4gPiB2YWx1ZSBvZiBIWVNUQVJUX0RFTEFZ X01JTi4gWW91IG1lbnRpb24gOG1zIGJ1dCB3aGVuIEkgbG9vayBpbnRvDQo+ID4+ID4gdGhlIDQu MiBrZXJuZWwgY29kZSBJIGdldCB0aGUgZmVlbGluZyB0aGF0IGl0IGlzIDMybXMgKCAjZGVmaW5l DQo+IEhZU1RBUlRfREVMQVlfTUlODQo+ID4+ID4gKDRVPDwzKSAgICkuIElzIHRoaXMgYSBtaXN0 YWtlIGZyb20gbXkgc2lkZSAgPw0KPiA+PiA+IEkgaGF2ZSBydW4gTFRFIHNpbXVsYXRpb25zIGFu ZCB0cmllZCB0byB3cmluZyB0aGlzIEh5U3RhcnQgaXNzdWUNCj4gPj4gPiBpbnNpZGUgb3V0IGFu ZCB1cHNpZGUgZG93biBidXQgc29mYXIgaXQgaGFzIGJlZW4gdmVyeSBkaWZmaWN1bHQgdG8NCj4g Pj4gPiBtYWtlIEh5U3RhcnQgZXhpdCBwcmVtYXR1cmVseSBidXQgdGhlbiBvZiBjb3Vyc2UgSSBo YXZlIHJ1biB3aXRoDQo+ID4+ID4gSFlTVEFSVF9ERUxBWV9NSU4gPSAzMm1zLg0KPiA+PiA+DQo+ ID4+ID4gQXMgcmVnYXJkcyB0byB0aGUgQUNLLWNvbXByZXNzaW9uIHByb2JsZW0sIHllcywgQUNL IGNvbXByZXNzaW9uDQo+ID4+ID4gZWFzaWx5IG9jY3VyIGluIExURSwgZXZlbiBpbiBzaW11bGF0 aW9ucy4gV2hhdCBJIGhhdmUgc2VlbiBpcyB0aGF0DQo+ID4+ID4gcGFja2V0IHBhY2luZyBpcyBh IHZlcnkgZWZmaWNpZW50IHJlbWVkeSwgZm9yIGluc3RhbmNlIHRoZSBwYWNrZXQNCj4gPj4gPiBp bXBsZW1lbnRlZCBpbiBRVUlDIHNlZW1zIHRvIHNvbHZlIEFDSyBjb21wcmVzc2lvbiBpc3N1ZXMg dmVyeSB3ZWxsLg0KPiA+PiA+DQo+ID4+ID4gSXQgaXMgaW50ZXJlc3Rpbmcgd2hhdCB5b3Ugc2F5 IHRoYXQgQUNLIHRyYWlucyBhcmUgc28gbXVjaCBzcGxpdA0KPiA+PiA+IHVwLCBJIGNvdWxkIHVu ZGVyc3RhbmQgdGhhdCB0aGV5IGFyZSBzcGxpdCB1cCBpbiB0d28gcGFydHMgKGluaXRpYWwNCj4g Pj4gPiBncmFudCArIGFkZGl0aW9uYWwgYWZ0ZXIgcmVjZWl2ZWQgQlNSKS4NCj4gPj4gPg0KPiA+ PiA+IC9JbmdlbWFyDQo+ID4+ID4NCj4gPj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0t DQo+ID4+ID4gPiBGcm9tOiBWZWFjZXNsYXYgUk9NQU4gW21haWx0bzpWZWFjZXNsYXYuUm9tYW5A b3JhbmdlLm1kXQ0KPiA+PiA+ID4gU2VudDogZGVuIDEgb2t0b2JlciAyMDE1IDAyOjI1DQo+ID4+ ID4gPiBUbzogRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsDQo+ID4+ID4gPiBDYzogdGNwbUBp ZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0YWUgSGE7IEluZ2VtYXIgSm9oYW5zc29uDQo+ ID4+ID4gPiBTOyBFcmljIER1bWF6ZXQ7IGVuZDJlbmQtaW50ZXJlc3RAcG9zdGVsLm9yZw0KPiA+ PiA+ID4gU3ViamVjdDogUkU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBwYXRjaCBkZXBsb3lt ZW50DQo+ID4+ID4gPg0KPiA+PiA+ID4gRmluYWxseSBjYW4gc2hhcmUgc29tZSByZXN1bHRzIG9m IEh5c3RhcnQgaW50ZXJhY3Rpb24gd2l0aCBMVEUuDQo+ID4+ID4gPiAgRmV3IHdvcmRzIGFib3V0 IHRoZSB0ZXN0aW5nIGNvbmZpZ3VyYXRpb246DQo+ID4+ID4gPiAgIERldmljZSBTYW1zdW5nIEdh bGF4eSBOb3RlNExURSAsIHVwIHRvIDE1MCBNYnBzIGNhcGFibGUNCj4gPj4gPiA+ICAgICAgICAg ICAgICBSYWRpbyBOZXR3b3JrIExURSAyNjAwLzE4MDANCj4gPj4gPiA+ICAgICAgICAgICAgICBE aXN0YW5jZSBiZXR3ZWVuIHRoZSBDb3JlIE5ldHdvcmsgKEVQQykgYW5kIEJhc2UNCj4gPj4gPiA+ IFN0YXRpb24gLSAxMA0KPiA+PiBrbQ0KPiA+PiA+ID4gICAgICAgICAgICAgIEhUVFAgU2VydmVy IGNvbm5lY3RlZCBkaXJlY3RseSB0byB0aGUgQ29yZSBOZXR3b3JrIDEgR2Jwcw0KPiA+PiA+ID4g ICAgICAgICAgICAgIENvcmUgbmV0d29yayBzd2l0Y2hpbmcoRVBDKSAxMCBHYnBzDQo+ID4+ID4g PiAgICAgICAgICAgICAgQmFja2hhdWwgdHJhbnNwb3J0ICAxIEdicHMsIGZ1bGwgSVAsIE1QTFMN Cj4gPj4gPiA+ICAgICAgICAgICAgICBMYXN0IG1pbGUgdHJhbnNwb3J0IGZ1bGwgSVAsIE1QTFMs ICAzMDAgTWJwcw0KPiA+PiA+ID4NCj4gPj4gPiA+ICAgICAgICAgICAgIFNlcnZlcjogTGludXgg azQwc3J2IDQuMS4wLTA0MDEwMC1nZW5lcmljDQo+ID4+ID4gPiAjMjAxNTA3MDMwOTQwIFNNUCBG cmkgSnVsIDMNCj4gPj4gPiA+IDA5OjQxOjQ3IFVUQyAyMDE1IHg4Nl82NCB4ODZfNjQgeDg2XzY0 IEdOVS9MaW51eA0KPiA+PiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgQXBhY2hlLzIuNC4x MCAoVWJ1bnR1KSwgYnVpbHQ6ICAgSnVsIDI0IDIwMTUgMTc6MjU6MTgNCj4gPj4gPiA+DQo+ID4+ ID4gPiAgICAgICAgICAgIEh5c3RhcnRfZGV0ZWN0OiAyIChIWVNUQVJUX0RFTEFZIG9ubHkpLCBh cyBzdWdnZXN0ZWQuDQo+ID4+ID4gPiAgICAgICAgICAgIG5ldC5pcHY0LnRjcF9ub19tZXRyaWNz X3NhdmUgPSAxIHRvIGF2b2lkIHRjcCBtZXRyaWNzDQo+ID4+ID4gPiBjYWNoaW5nIGludGVyZmVy ZW5jZQ0KPiA+PiA+ID4NCj4gPj4gPiA+ICAgVGVzdHMgdHlwZXM6DQo+ID4+ID4gPiAgICAgICAg ICAgMS4gRG93bmxvYWQgMyBNQiBmaWxlLCBpbnRlcmNoYW5naW5nIEh5c3RhcnQgT04NCj4gPj4g PiBhbmQgT0ZGLCBpbg0KPiA+PiA+ID4gY29uZGl0aW9ucyBvZiBMb3cgQmFuZHdpZHRoICgyNS0z NSBNYnBzLCBwb29yIHJhZGlvKSBhbmQgSGlnaA0KPiA+PiA+ID4gQmFuZHdpZHRoICgxMDAtMTIw IE1icHMsIGdvb2QgcmFkaW8pLg0KPiA+PiA+ID4gICAgICAgICAgIDIuIERvd25sb2FkIDEwIE1C IGZpbGUsIGludGVyY2hhbmdpbmcgSHlzdGFydA0KPiA+PiA+IE9OIGFuZCBPRkYsIGluDQo+ID4+ ID4gPiBjb25kaXRpb25zIG9mIExvdyBCYW5kd2lkdGggKDI1LTM1IE1icHMsIHBvb3IgcmFkaW8p IGFuZCBIaWdoDQo+ID4+ID4gPiBCYW5kd2lkdGggKDEwMC0xMjAgTWJwcywgZ29vZCByYWRpbyku DQo+ID4+ID4gPg0KPiA+PiA+ID4gICAxMDAgRG93bmxvYWQgdGVzdHMgcGVyIHR5cGUuDQo+ID4+ ID4gPiAgICAgICAgICAgICAgRG93bmxvYWQgZHVyYXRpb24gYW5kIGJhbmR3aWR0aCBtZWFzdXJl ZCBhdCB0aGUNCj4gPj4gPiA+IGNsaWVudCBzaWRlIGFzIHNob3duIGJ5IGN1cmwgZnJvbSB0aGUg Zmlyc3QgcmVjZWl2ZWQgZGF0YSBwYWNrZXQNCj4gPj4gPiA+IHRpbGwgdGhlIGVuZCBvZiBzZXNz aW9uICh0aGlzIGV4Y2x1ZGUgRE5TLCAzIHdheSBoYW5kc2hha2UsIGh0dHANCj4gPj4gPiA+IEdF VCBhbmQgaXRzDQo+ID4+IEFDSykuDQo+ID4+ID4gPiAgICAgICAgICAgICAgSHlzdGFydCBleGl0 IGN3bmQgZXh0cmFjdGVkIGZyb20gbnN0YXQgKHRoYW5rIHlvdQ0KPiA+PiA+ID4gRXJpYyB0byBw b2ludGluZyBvdXQgdG8gaXQpLg0KPiA+PiA+ID4NCj4gPj4gPiA+IE92ZXJhbGwgcmVzdWx0czoN Cj4gPj4gPiA+ICAgRG93bmxvYWQgM01CLCBMb3cgQmFuZHdpZHRoczoNCj4gPj4gPiA+ICAgICAg ICAgICBIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS40MSBzDQo+ID4+ID4gPiAgICAg ICAgICAgTm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS4zNiBzDQo+ID4+ID4gPg0K PiA+PiA+ID4gICBEb3dubG9hZCAzTUIsIEhpZ2ggQmFuZHdpZHRoczoNCj4gPj4gPiA+ICAgICAg ICAgICBIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC42NSBzDQo+ID4+ID4gPiAgICAg ICAgICAgTm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC4zNyAgcw0KPiA+PiA+ID4N Cj4gPj4gPiA+ICAgRG93bmxvYWQgMTBNQiwgTG93IEJhbmR3aWR0aHM6DQo+ID4+ID4gPiAgICAg ICAgICAgSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDIuNDAgcw0KPiA+PiA+ID4gICAg ICAgICAgIE5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDIuMTIgcw0KPiA+PiA+ID4N Cj4gPj4gPiA+ICAgRG93bmxvYWQgMTBNQiwgSGlnaCBCYW5kd2lkdGhzOg0KPiA+PiA+ID4gICAg ICAgICAgIEh5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAxLjQyIHMNCj4gPj4gPiA+ICAg ICAgICAgICBOb0h5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAwLjg5ICBzDQo+ID4+ID4g Pg0KPiA+PiA+ID4gVGhlc2UgcmVzdWx0cyBzaG93IHRoYXQgSHlzdGFydCBoYXMgbm8gc2lnbmlm aWNhbnQgaW1wYWN0IGluDQo+ID4+ID4gPiBjb25kaXRpb25zIG9mIGxvdyBhdmFpbGFibGUgYmFu ZHdpZHRoLCBidXQgYXQgaGlnaCBhdmFpbGFibGUgdGhlDQo+ID4+ID4gPiBkZWNyZWFzZSBvZiBw ZXJmb3JtYW5jZSBjYW4gcmVhY2ggNjAtNzAgJS4NCj4gPj4gPiA+IFdpdGggdGhlIGRldmVsb3Bt ZW50IG9mIHRoZSBMVEUtQSB3aGljaCB1c2VzIGxhcmdlciBzcGVjdHJ1bSB0bw0KPiA+PiA+ID4g cmVhY2gNCj4gPj4gPiA+IDMwMCBNYnBzIGFuZCBtb3JlIHRoZSBpbXBhY3Qgb2YgSHlzdGFydCB3 aWxsIGJlY29tZSBldmVuIG1vcmUNCj4gdmlzaWJsZS4NCj4gPj4gPiA+DQo+ID4+ID4gPiBJIGRv IHNoYXJlIHdpdGggRXJpYyBhbmQgTmVhbCBzb21lIGZpbGVzIGRlc2NyaWJlZCBiZWxvdyBhbmQg YWxzbw0KPiA+PiA+ID4gY2FuIHByb3ZpZGUgbGlua3MgdG8gYW55b25lIGludGVyZXN0ZWQuDQo+ ID4+ID4gPg0KPiA+PiA+ID4gU3VtbWFyeSBvZiB0aGVzZSB0ZXN0cyByZXN1bHRzIGFyZSBnYXRo ZXJlZCBpbiB0aGUgZm9sbG93aW5nDQo+ID4+ID4gPiBFeGNlbCBmaWxlcyAobmFtZXMsIEkgYmVs aWV2ZSwgYXJlIHNlbGYgZXhwbGFuYXRvcnkpOg0KPiA+PiA+ID4gVGVzdF8zTV9sb3dfQlcueGxz eCwgVGVzdF8zTV9oaWdoX0JXLnhsc3gsDQo+IFRlc3RfMTBNX2xvd19CVy54bHN4LA0KPiA+PiA+ IFRlc3RfMTBNX2hpZ2hfQlcueGxzeC4NCj4gPj4gPiA+IFNpbWlsYXJseSB0Y3BkdW1wIHRyYWNl cyBhcmUgY2FsbGVkIFRlc3RfM01fbG93X0JXLnBjYXAsDQo+ID4+ID4gPiBUZXN0XzNNX2hpZ2hf QlcuIHBjYXAsIFRlc3RfMTBNX2xvd19CVy4gcGNhcCwNCj4gPj4gVGVzdF8xME1faGlnaF9CVy4N Cj4gPj4gPiA+IHBjYXAgKGFibGUgdG8gY29sbGVjdCBvbmx5IHNlcnZlciBzaWRlIHRyYWNlcywg YnV0IHRoZXkgYXJlIHNlbGYNCj4gc3VmZmljaWVudCkuDQo+ID4+ID4gPiBGb3IgdGhlIDEwTSBk b3dubG9hZHMgdGhlcmUgYXJlIGZldyBtaXNzaW5nIHRyYWNlcyBhcyB0aGVyZSB0cmFjZQ0KPiA+ PiA+ID4gZmlsZXMgd2VyZSByZWN5Y2xlZC4gSW4gdHJhY2VzIGFyZSBhbHNvIHNzaCBzZXNzaW9u cywgcGxlYXNlLA0KPiA+PiA+ID4gaWdub3JlIHRoZW0sIHRoZXkgd2VyZSB1c2VkIHRvIHB1dCBo eXN0YXJ0IE9OIGFuZCBPRkYgb24gc2VydmVyDQo+ID4+ID4gPiBhbmQgZXh0cmFjdCBuc3RhdCBy ZXN1bHRzIGJlZm9yZSBhbmQgYWZ0ZXIgZWFjaCBkb3dubG9hZC4NCj4gPj4gPiA+DQo+ID4+ID4g PiBFeGNlbCBmaWxlcyBoYXZlIDIgdGFiczogY3VybF9zdW0gYW5kIFN1bW1hcnkuDQo+ID4+ID4g PiBJbiB0aGUgY3VybF9zdW0gdGFiIHRoZSBzdGF0IGlzIGN1bXVsYXRlZCBpbiBjaHJvbm9sb2dp Y2FsIG9yZGVyIG9mDQo+IHRlc3RzLg0KPiA+PiA+ID4gSW4gdGhlIFN1bW1hcnlfdGFiICB0aGUg SHlzdGFydCBhbmQgTm9IeXN0YXJ0IHRlc3RzIGFyZSBzaG93bg0KPiA+PiA+ID4gc2lkZSBieSBz aWRlIChhcyBzYWlkLCBEb3dubG9hZCAxIHdhcyB3aXRoIEh5c3RhcnQsIERvd25sb2FkIDINCj4g Pj4gPiA+IHdpdGhvdXQsIERvd25sb2FkIDMgd2l0aCBIeXN0YXJ0LCBEb3dubG9hZCA0IHdpdGhv dXQsIGV0Yy4pLiBJdA0KPiA+PiA+ID4gYWxzbyBpbmNsdWRlIGdyYXBocyBvZiBUcmFuc2ZlciB0 aW1lIGFuZCBhIGdyYXBoIHdoaWNoIHNob3dzIHRoZQ0KPiA+PiA+ID4gZGlzdHJpYnV0aW9uIGlu ICUgb2YgZGlmZmVyZW50IEh5c3RhcnQgZXhpdCB3aW5kb3cgRGVsYXlDV05ELg0KPiA+PiA+ID4N Cj4gPj4gPiA+IEZpZWxkczoNCj4gPj4gPiA+IGNvbW1lbnRzICAtIHRlc3QgY2FzZQ0KPiA+PiA+ ID4gVHJhbnNmX1RpbWUgICAgICAgLSBjYWxjdWxhdGVkIGZyb20gY3VybCBvdXRwdXQgdGhlIGR1 cmF0aW9uDQo+ID4+ID4gYmV0d2VlbiBmaXJzdCBkYXRhDQo+ID4+ID4gPiBwYWNrZXQgYW5kIGVu ZCBvZiB0cmFuc2Zlcg0KPiA+PiA+ID4gVHJhbnNmX3NwZWVkICAgICAgLSBjYWxjdWxhdGVkIGZy b20gY3VybCBvdXRwdXQgdGhlIHJhdGlvIG9mIGZpbGUgc2l6ZQ0KPiA+PiA+IGFuZCBkdXJhdGlv bg0KPiA+PiA+ID4gYmV0d2VlbiBmaXJzdCBkYXRhIHBhY2tldCBhbmQgZW5kIG9mIHRyYW5zZmVy DQo+ID4+ID4gPiBIeXN0YXJ0ICAgRF9EZWxheSAtICBoeXN0YXJ0IGV4aXQgb2NjdXJyZW5jZSAo ZGlmZmVyZW5jZSBiZXR3ZWVuDQo+ID4+ID4gPiBUY3BFeHRUQ1BIeXN0YXJ0RGVsYXlEZXRlY3Qg YmVmb3JlIGFuZCBhZnRlciBkb3dubG9hZCkNCj4gPj4gPiA+IERfRGVsYXlDd25kICAgICAgICAg LSBIeXN0YXJ0IGV4aXQgY3duZCBmb3IgdGhpcyBkb3dubG9hZA0KPiA+PiA+ID4gKGRpZmZlcmVu Y2Ugb2YgVGNwRXh0VENQSHlzdGFydERlbGF5Q3duZCBiZWZvcmUgYW5kIGFmdGVyIHRoZQ0KPiA+ PiA+ID4gZG93bmxvYWQpIERhdGVfT25fU2VydmVyIC0gdGltZSBqdXN0IGJlZm9yZSB0aGUgZG93 bmxvYWQgc3RhcnQsDQo+ID4+ID4gPiB0byBoZWxwIG5hdmlnYXRlIHRoZSBwY2FwDQo+ID4+ID4g Pg0KPiA+PiA+ID4gQWRkaXRpb25hbGx5LCBpbiB0aGUgZmlsZSBUZXN0XzEwTV9sb3dfQlcueGxz eCBpbiB0aGUgdGFiDQo+ID4+ID4gPiBjdXJsX3N1bSBhcmUgaW5jbHVkZWQgZm9yIGVhY2ggZG93 bmxvYWQgdGNwLnN0cmVhbSBleHRyYWN0IGZyb20NCj4gPj4gPiA+IHBjYXAgb2YgdGhlIGZpcnN0 DQo+ID4+ID4gPiAxMDAgdGNwLmFuYWx5c2lzLmFja19ydHQgd2hpY2ggbWF5IGhlbHAgaW4gdW5k ZXJzdGFuZGluZyBvZiB0aGUNCj4gPj4gPiA+IGh5c3RhcnQgYmVoYXZpb3IgKGZpZWxkcyBhY2tf cnR0XzEgdG8gYWNrX3J0dF8xMDApLg0KPiA+PiA+ID4gSW4gdGhpcyBmaWxlIHRoZXJlIGFyZSBh bHNvIGZpZWxkcyBVUklfZnJhbWUsIFVSSV90aW1lLA0KPiBVUklfdGltZV9yZWxhdGl2ZSwNCj4g Pj4gPiA+ICAgdGNwX3N0cmVhbS4NCj4gPj4gPiA+IFRoZW4gdGhlcmUgYXJlIGNhbGN1bGF0ZWQg ZmllbGRzOiBtaW5SVFQgYXMgYSBtaW5pbXVtIG9mIGFsbCAxMDANCj4gPj4gPiA+IEFDSywNCj4g Pj4gcjFfNSwNCj4gPj4gPiA+IHIyXzgsIHIzXzgsICAgICAgIHI0Xzggd2hpY2ggc2ltdWxhdGUg SHlzdGFydCBjYWxjdWxhdGlvbnMgb2YgdGhlDQo+ID4+IHJvdW5kIGN1cnJfcnR0DQo+ID4+ID4g PiBhbmQgcjEgbWluLCByMiBtaW4sIHIzIG1pbiwgcjQgbWluIHJlcHJlc2VudCBtaW5SVFQgb2Yg dGhlIHdob2xlDQo+IHJvdW5kLg0KPiA+PiA+ID4gUGxlYXNlLCBjb3JyZWN0IG1lIGlmIEkgYW0g d3JvbmcsIGJ1dCBpbiBteSB1bmRlcnN0YW5kaW5nIGZvcg0KPiA+PiA+ID4gZWFjaCBBQ0sgdGhl IGh5c3RhcnRfdXBkYXRlIGlzIGNhbGxlZCBiZWZvcmUgdGhlIGh5c3RhcnRfcmVzZXQsDQo+ID4+ ID4gPiBhbmQgYXMgYSByZXN1bHQgdGhlIGNvbXB1dGF0aW9uIG9mIGN1cl9ydHQgb2YgdGhlIDgg cGFja2V0cyBvZg0KPiA+PiA+ID4gdGhlIHJvdW5kIHN0YXJ0cyB3aXRoIHRoZSAyLW5kIHBhY2tl dCBvZiB0aGUgcm91bmQsIG5vdCB0aGUgZmlyc3Qgb25lLg0KPiA+PiA+ID4gIEkndmUgdXNlIHIx XzUgYXMgYSBtaW4oYWNrX3J0dF83Li4uYWNrX3J0dF8xMSksIGR1ZSB0bw0KPiA+PiA+ID4gaHlz dGFydF9sb3dfd2luZG93PTE2LCByMl84IGFzIGEgbWluKGFja19ydHRfMTIgLi4uIGFja19ydHRf MTkpLA0KPiA+PiA+ID4gcjNfOCBhcyBhDQo+ID4+ID4gPiBtaW4oYWNrX3J0dF8zMiAuLi4gYWNr X3J0dF8zOSkgYW5kICksIHI0XzggYXMgYSBtaW4oYWNrX3J0dF83MiAuLi4NCj4gPj4gPiBhY2tf cnR0Xzc5KS4NCj4gPj4gPiA+IFRvIHVuZGVyc3RhbmQgdGhlIGltcGFjdCBvZiB0aGlzIHRoZXJl IGFyZSBhbHNvIHIxIEhPTCwgcjIgSE9MLA0KPiA+PiA+ID4gcjMgSE9MLCByNCBIT0wgKEhlYWQg T2YgTGluZSkgd2hpY2ggY29tcHV0ZSB0aGUgc2FtZSB3aXRoIGZpcnN0DQo+ID4+ID4gPiBwYWNr ZXQgaW4gdGhlIHJvdW5kIGluY2x1ZGVkIGluIHRoZSByZWxldmFudCByb3VuZDogIHIxXzUgYXMg YQ0KPiA+PiA+ID4gbWluKGFja19ydHRfNi4uLmFja19ydHRfMTApLCByMl84IGFzIGEgbWluKGFj a19ydHRfMTEgLi4uDQo+ID4+ID4gPiBhY2tfcnR0XzE4KSwNCj4gPj4gPiA+IHIzXzggYXMgYSBt aW4oYWNrX3J0dF8zMSAuLi4gYWNrX3J0dF8zOCkgYW5kICksIHI0XzggYXMgYSBtaW4oYWNrX3J0 dF83MQ0KPiAuLi4NCj4gPj4gPiBhY2tfcnR0Xzc4KS4NCj4gPj4gPiA+DQo+ID4+ID4gPiBJZiBt eSB1bmRlcnN0YW5kaW5nIG9mIEh5c3RhcnQgaXMgY29ycmVjdCBhbmQsIGFzc3VtaW5nIGluIGVh Y2gNCj4gPj4gPiA+IHRyYWluIHRoZSBmaXJzdCBwYWNrZXQgaGFzIHRoZSBsb3dlc3QgUlRUIHdo aWNoLCBtb3N0IHByb2JhYmx5IGlzDQo+ID4+ID4gPiB2YWxpZCBpbiBhbGwgdHlwZSBvZiBuZXR3 b3JrcywgdGhlbiB0aGUgZXhpdCBmcm9tIGh5c3RhcnQgaW4gdGhpcw0KPiA+PiA+ID4gaW1wbGVt ZW50YXRpb24gbWF5IGhhcHBlbiBhdCBjd25kIDI5LCA0OSwgODksIDE2OSwgZXRjLiAoZ2l2ZW4N Cj4gPj4gPiA+IHRoZSBJVyBvZg0KPiA+PiA+IDEwKS4NCj4gPj4gPiA+IFRoZXNlIHRlc3RzIHNo b3cgdGhhdCB0aGlzIGlzIHRoZSBjYXNlLCBob3dldmVyIHRoZXJlIGFsc28NCj4gPj4gPiA+IGlu dGVybWVkaWF0ZSB2YWx1ZXMuIEhvd2V2ZXIgdGhlIGN3bmQgb2YgMjkgc2hhbGwgYmUgdGhlIGxv d2VzdA0KPiA+PiA+ID4gcG9zc2libGUgdmFsdWUgYW5kIHRoaXMgaXMgdGhlIGNhc2UgaW4gdGhl c2UgdGVzdHMuDQo+ID4+ID4gPiBVbmZvcnR1bmF0ZWx5LCBpbiBMVEUgdGhlIGZpcnN0IHBhY2tl dCBpbiB0aGUgdHJhaW4gaGFzIGNoYW5jZXMNCj4gPj4gPiA+IHRvIGFsd2F5cyBoYXZlIH42IG1z IGxvd2VyIHRoYW4gdGhlIGZvbGxvd2luZyBvbmVzIGFuZCwgYmVjYXVzZQ0KPiA+PiA+ID4gaXQg aXMgbm90IGNvdW50ZWQgaW4gdGhlIDItbmQgdHJhaW4sIHRoZXJlIGlzIHF1aXRlIGhpZ2gNCj4g Pj4gPiA+IHBlcmNlbnRhZ2Ugb2YgZXhpdCBhdCAyOSAoMTQtMzYlIGluIHRoZXNlIHRlc3RzKSwg ZXZlbiB0aG91Z2gNCj4gPj4gPiA+IGxhdGVyIG9uIGN1YmljIGluY3JlYXNlcyB0aGUgY3duZCB3 aXRob3V0IGxvc3NlcyB1cCB0byBtYW55DQo+ID4+ID4gPiBodW5kcmVkcy4gRm9yIG1lIHRoaXMg aXMgdG9vIGVhcmx5IGV4aXQgZnJvbQ0KPiA+PiBzbG93IHN0YXJ0Lg0KPiA+PiA+ID4gQXMgaXQg Y2FuIGJlIHNlZW4gZnJvbSBjb21wYXJpc29uIG9mIHRoZSBzaW11bGF0ZWQgY2FsY3VsYXRpb24s DQo+ID4+ID4gPiBpbmNsdXNpb24gb2YgdGhlIGZpcnN0IHBhY2tldCBpbiB0aGUgcm91bmQgY3Vy cl9ydHQgY2FsY3VsYXRpb24NCj4gPj4gPiA+IHdvdWxkIGVsaW1pbmF0ZSB0aGUgY3duZCAyOSBh bmQgc2hpZnQgdG8gY3duZCA0OSBvciA4OSBvciBoaWdoZXIuDQo+ID4+ID4gPg0KPiA+PiA+ID4g QnV0IHRoZXJlIGFyZSBhbHNvIGludGVybWVkaWF0ZSB2YWx1ZXMsIGUuZy4gNjkuIExvb2tpbmcg dGhyb3VnaA0KPiA+PiA+ID4gdHJhY2VzIG9uZSBtYXkgc2VlIHRoYXQgdGhpcyBpcyBkdWUgdG8g ImNvbXByZXNzZWQgQUNLIiB3aGljaA0KPiA+PiA+ID4gcmVjZWl2ZXMgYSB0aWdodGx5IHBhY2tl ZCB0cmFpbiBvZiBtYW55IEFDSyBzbyB0aGF0IGluIHRyYWNlcw0KPiA+PiA+ID4gdGhlcmUgYXJl IG5vdCBkYXRhIHBhY2tldHMgc2VudCBpbiB0dXJuIGJ5IHNlcnZlci4gTG9va3MgdG8gbWUsDQo+ ID4+ID4gPiBiZWNhdXNlIGh5c3JhcnRfcmVzZXQgc2V0cyB0aGUgZW5kX3NlcSB0byBzbmRfbmV4 dCBhbmQgdGhlIHNlcnZlcg0KPiA+PiA+ID4gZGlkIG5vdCBtYW5hZ2VkIHRvIHNlbmQgeWV0IHBh Y2tldHMgaW4gcmVwbHkgdG8gdGhpcyBoaWdoIHNwZWVkDQo+ID4+ID4gPiBBQ0sgdHJhaW4gdGhl IHNuZF9uZXh0IHBvaW50cyB0byBtdWNoIGxvd2VyIHZhbHVlIHRoYW4gd291bGQNCj4gPj4gPiA+ IG5vcm1hbGx5IGhhcHBlbiBhbmQgdGhpcyBhY3R1YWxseSBkZXN0cm95cyB0aGUgY29ycmVjdA0K PiA+PiA+ID4gaWRlbnRpZmljYXRpb24gb2YgdGhlIGJvcmRlcnMgb2YgdGhlIHJvdW5kLCB0aGUg cm91bmQgZmFpbHMNCj4gPj4gPiA+IHNvbWV3aGVyZSBpbiB0aGUgbWlkZGxlIG9mIHRoZSB0cmFp biBub3QgYXQgaXRzIGJvcmRlciwgdGhpcw0KPiA+PiA+ID4gbWFrZXMgdmVyeSBsaWtlbHkgaW5j cmVhc2VkIGRlbGF5IGR1ZSB0byBxdWV1ZWluZyBhbmQgZWFybGllcg0KPiA+PiA+ID4gZXhpdCBm cm9tIHNsb3dfc3RhcnQgYW5kIGludGVybWVkaWF0ZSB2YWx1ZXMgb2YgZXhpdCBjd25kIHdoZXJl DQo+ID4+ID4gdGhleSBhcmUgbm90IGV4cGVjdGVkLg0KPiA+PiA+ID4NCj4gPj4gPiA+IEFuZCBs YXN0IGJ1dCBub3QgbGVhc3Q6IHRoZXJlIGFyZSBnYXBzICg0LTcgbXMpIGluIEFDSyB0cmFpbnMu IE9uDQo+ID4+ID4gPiB0aGUgcGVyZmVjdGx5IHNlbnQgSVcgb2YgMTAgZGF0YSBwYWNrZXRzIHRy YWluIHRoZSB0eXBpY2FsIGZvcg0KPiA+PiA+ID4gTFRFIHdpbGwgYmUgdG8gcmVjZWl2ZSAxIG9y IDIgQUNLLCB0aGVuIGEgZ2FwIG9mIDQtNiBtcyB0aGVuDQo+ID4+ID4gPiBhbm90aGVyIGNvdXBs ZSBvZiBBQ0ssIHRoZW4gYW5vdGhlciBnYXAsIHRoZW4gNSBBQ0sgY29tcHJlc3NlZA0KPiA+PiA+ ID4gdG9nZXRoZXIgaW4gYSBmcmFjdGlvbiBvZiBtaWNyb3NlY29uZC4gT2YgY2F1c2UsIHRoZSBu ZXh0IHRyYWluDQo+ID4+ID4gPiBmcm9tIHRoZSBzZXJ2ZXIgd2lsbCBjb250YWluIHRoZSBzYW1l IGdhcHMuIFZlcnkgcXVpY2tseSwgZS5nLiwNCj4gPj4gPiA+IHRoZSBIZWFkIG9mIExpbmUNCj4g Pj4gPiA+IChIT0wpIG9mIHRoZSAzLWQgdHJhaW4gd2lsbCBmb2xsb3cgaW1tZWRpYXRlbHkgdGhl IHRhaWwgb2YgdGhlDQo+ID4+ID4gPiAyLW5kIHRyYWluIHdoaWNoIHdpbGwgbGVhZCB0byBxdWV1 ZWluZyBpbiB0aGUgZW5kIHJvdXRlcg0KPiA+PiA+ID4gKGFjdHVhbGx5IHRoZSByYWRpbyBiYXNl IHN0YXRpb24sIG11Y2ggZWFybGllciB0aGFuIEJEUCByZWFjaGVkKSwNCj4gPj4gPiA+IGJ1dCB0 aGlzIHdpbGwgbGVhZCB0byBpbmNyZWFzZSBvZiB0aGUgdHJhaW4gUlRUIGFuZCB0b28geWVhcmx5 DQo+ID4+ID4gPiBleGl0IGZyb20gc2xvdyBzdGFydC4gV2h5IHRvbyBlYXJseSBleGl0ID8gQmVj YXVzZSwgc2hvdWxkIHNlcnZlcg0KPiA+PiA+ID4gY29udGludWUgaW5jcmVhc2luZyB0aGUgc3Bl ZWQgdGhpcyB3b3VsZCByZWR1Y2UgcXVpY2tlciB0aGUgZ2Fwcw0KPiA+PiA+ID4gYm90aCBpbiBk b3dubGluayBhbmQgdXBsaW5rLCB0aGUgcHJlaWNlIGZvciB0aGlzIGJlaW5nIHRoZQ0KPiA+PiA+ ID4gaW5jcmVhc2Ugb2YgdGhlIFJUVCAoSSBndWVzcyBkb3VibGUpIGFnYWluc3Qgd2hhdCBvbmUg d291bGQNCj4gPj4gPiA+IG5vcm1hbGx5IHNlZQ0KPiA+PiA+IGluIHRoZSBlbmQtdG8tZW5kIEV0 aGVybmV0IG5ldHdvcmtzLg0KPiA+PiA+ID4NCj4gPj4gPiA+DQo+ID4+ID4gPiBJIGRvIGxpa2Ug aWRlYXMgb2YgSHlzdGFydCwgYnV0IEkgIGRvbid0IGtub3cgaG93IGNvdWxkIGl0IGJlDQo+ID4+ ID4gPiBwb3NzaWJseSByZWNvbmNpbGVkIHdpdGggbXkgTFRFIHByb2JsZW1zLiAgV2l0aCB0aGUg aW5jcmVhc2Ugb2YNCj4gPj4gPiA+IExURSBzcGVlZHMgdGhpcyBlYXJseSBleGl0IGZyb20gc2xv dyBzdGFydCB3aWxsIGJlY29tZSBldmVuIG1vcmUNCj4gPj4gPiA+IG9mIHRoZQ0KPiA+PiBwcm9i bGVtLg0KPiA+PiA+ID4gRm9yIExURSwgbm90IHN1cmUgaG93IGdvb2QgYW5kIGdlbmVyYWxseSBh Y2NlcHRhYmxlLCBzb2x1dGlvbg0KPiA+PiA+ID4gd291bGQgYmUsIHBvc3NpYmx5LCB0byBpbmNy ZWFzZSB0aGUgSFlTVEFSVF9ERUxBWV9NSU4gdG8gOCBtcywNCj4gPj4gPiA+IHdoaWNoLCBhdCBs ZWFzdCBpbiBteSBjYXNlLCB3b3VsZCBkaW1pbmlzaCB0aGUgZWFybGllciBleGl0DQo+ID4+ID4g PiBzaWduaWZpY2FudGx5LCBpZiBJIGJlbGlldmUgaW4gdGhlc2UgdGVzdHMgcmVzdWx0cy4NCj4g Pj4gPiA+IEFub3RoZXIgc29sdXRpb24sIHdoaWNoIEkgZG9uJ3QgbGlrZSB2ZXJ5IG11Y2gsIGJ1 dCBhcyBJIGtub3cNCj4gPj4gPiA+IHNvbWUgb3BlcmF0b3JzIHVzZSwgd291bGQgYmUgdG8gaW5z ZXJ0IGEga2luZCBvZiBUQ1AgYWNjZWxlcmF0b3INCj4gPj4gPiA+IGJldHdlZW4gdGhlIEludGVy bmV0IGFuZCB0aGUgbW9iaWxlIG5ldHdvcmsgd2hpY2ggd2lsbCBpbnRlcmNlcHQNCj4gPj4gPiA+ IGFsbCBUQ1AgYW5kIHdvdWxkIHNlbmQgaXQgdG8gbW9iaWxlIHdpdGhvdXQgSHlzdGFydCBidXQg dGhpcw0KPiA+PiA+ID4gd291bGQga2lsbCBhbnkgZW5kLXRvLWVuZCBwcmluY2lwbGUgYW5kIHRo aXMgd2lsbCBub3QgYmUgdGhlIEludGVybmV0DQo+IGFueW1vcmUuDQo+ID4+ID4gPg0KPiA+PiA+ ID4NCj4gPj4gPiA+IElmIHlvdSBoYXZlIHRpbWUgYW5kIG5ldmVyIGxvb2sgY2xvc2VseSB0byBM VEUgdGVjaG5vbG9neSBJIGRvIGENCj4gPj4gPiA+IHF1aWNrIHN1bW1hcnkgYmVsbG93Lg0KPiA+ PiA+ID4NCj4gPj4gPiA+DQo+ID4+ID4gPiBMVEUsIFRDUCBzZWxmLWNsb2NraW5nIGFuZCBBQ0sg Y29tcHJlc3Npb24uDQo+ID4+ID4gPg0KPiA+PiA+ID4gU3BlbnQgc29tZSB0aW1lIHRvIHVuZGVy c3RhbmQgdGhlIGJhc2ljcyBvZiB3aGF0IGNvdWxkIGJlIHRoZQ0KPiA+PiA+ID4gcmVhc29uIGZv ciBhbGwgdGhvc2UgdGltaW5nIGVmZmVjdHMgZHVlIHRvIExURS4NCj4gPj4gPiA+ICAgRmlyc3Qs IHVubGlrZSB3aXJlIGJhc2VkIHRyYW5zcG9ydCB0ZWNobm9sb2dpZXMsIExURSAoYnV0IGFsc28N Cj4gPj4gPiBVTVRTDQo+ID4+ID4gPiBhbmQgZnV0dXJlIDVHKSBzZW5kcyBkYXRhLCBib3RoIHVw bGluayBhbmQgZG93bmxpbmsgaW4gcHJlZGVmaW5lZA0KPiA+PiA+ID4gdGltZXNsb3RzIGNhbGxl ZCBUcmFuc21pc3Npb24gVGltZSBJbnRlcnZhbCAoVFRJKS4gSW4gVU1UUyB0aGlzDQo+ID4+ID4g PiBpcyAybXMsIGluIExURSBpcw0KPiA+PiA+ID4gMSBtcyBhbmQgaXQgc2F5cyB0aGF0IGluIDVH IGl0IHdpbGwgYmUgMC41IG1zLg0KPiA+PiA+ID4gVGhlc2UgVFRJIGhhdmUgZXhhY3QgYm9yZGVy cyBhbmQgdGlnaHRseSBmb2xsb3cgZWFjaCBvdGhlciBhbmQNCj4gPj4gPiA+IHJlcXVpcmUgZXhh Y3Qgc3luY2hyb25pemF0aW9uIGJldHdlZW4gdGhlIG1vYmlsZSBkZXZpY2UgYW5kIFJhZGlvDQo+ ID4+ID4gPiBCYXNlIFN0YXRpb24gKGNhbGxlZCBlTm9kZUIgaW4gTFRFKS4gVG8gY29wZSB3aXRo IHZhcnlpbmcgcmFkaW8NCj4gPj4gPiA+IGNvbmRpdGlvbnMgdGhlIHNlbmRpbmcgZGF0YSBiYW5k d2lkdGggaXMgbW9kaWZpZWQgaW4gc3VjaCBhIHdheQ0KPiA+PiA+ID4gdGhhdCB0byBrZWVwIGEg Y29uc3RhbnQgcHJvYmFiaWxpdHkgb2YgZGF0YSBsb3NzIG9mIDEwJS4gQmFkDQo+ID4+ID4gPiBy YWRpbyAtIGxlc3MgYmFuZHdpZHRoLCBnb29kDQo+ID4+ID4gcmFkaW8gLSBtb3JlIGJhbmR3aWR0 aC4NCj4gPj4gPiA+IFRoZSBiYW5kd2lkdGggaXMgbW9kaWZpZWQgYnkgcGFja2luZyBsZXNzIG9y IG1vcmUgYml0cyBpbiBhIFRUSQ0KPiA+PiA+ID4gb2YgMSBtcyBpbg0KPiA+PiA+IExURS4NCj4g Pj4gPiA+IEJ1dCBUVEkgcmVtYWluIHVuY2hhbmdlZCBvZiBleGFjdGx5IDEgbXMuIFRoYXQgbWVh bnMgdGhhdCwgYXQgdGhlDQo+ID4+ID4gPiBUQ1AvSVAgbGF5ZXIgdGhlIHByZWNpc2lvbiBvZiB0 aGUgdGltaW5nIGluZm9ybWF0aW9uIGZyb20gZGV2aWNlDQo+ID4+ID4gPiB0byB0aGUgbmV0d29y ayBpcyAxIG1zLiBJbiB0aGUgY2FzZSBvZiBhIGJhbmR3aWR0aCBvZiAxIE1icHMNCj4gPj4gPiA+ IHRoZXJlIHdpbGwgYmUgODMgSVAgcGFja2V0cyBwZXIgc2Vjb25kIG9yIG9uZSBwYWNrZXQgZXZl cnkgMTIgbXMuDQo+ID4+ID4gPiBJbiB0aGlzIGNhc2UsIHBvc3NpYmx5IHRoZSByZXR1cm4gb2Yg b25lIEFDSyBldmVyeSAxMiBtcyAoZXZlcnkNCj4gPj4gPiA+IDEyIFRUSSkgd291bGQgYmUgbW9y ZSBvciBsZXNzIHN1ZmZpY2llbnQgdG8ga2VlcCBhdCBhIGdvb2QgbGV2ZWwNCj4gPj4gPiA+IHRo ZSBUQ1Agc2VsZi1jbG9ja2luZyBtZWNoYW5pc20uIEJ1dCwgd2hlbiB0YWxraW5nIGFib3V0IDEw MCBNYnBzDQo+ID4+ID4gPiAob3IgMzAwDQo+ID4+ID4gPiBNYnBzKSB0aGF0IG1lYW5zIG9uZSBw YWNrZXQgKGFuZCBvbmUgQUNLKSBldmVyeSAwLjEyIG1zIGFuZCB0aGVyZQ0KPiA+PiA+ID4gaXMg bm8gd2F5IHRvIHByb3ZpZGUgaXQgd2l0aCBhIFRUSSBvZiAxIG1zLiBUaGlzIGlzIHRoZSByZWFz b24NCj4gPj4gPiA+IGZvciBBQ0sgY29tcHJlc3Npb24gaW4gTFRFIGFuZCB0aGUgZnV0dXJlIDVH IHdoaWNoIHByb21pc2VzIG1vcmUNCj4gPj4gPiA+IHRoYW4gMUdicHMgd2l0aCBpdHMgMC41IG1z IFRUSSB3aWxsDQo+ID4+ID4gYmUgaW4gYSBiaWcgY29uZmxpY3Qgd2l0aCBUQ1Agc2VsZi1jbG9j a2luZy4NCj4gPj4gPiA+ICAgQnV0IHRoZSBwcm9ibGVtIGRvIG5vdCBzdG9wcyBoZXJlLiBUaGVy ZSBpcyBhIHNpZ25pZmljYW50bHkNCj4gPj4gPiBkaWZmZXJlbnQNCj4gPj4gPiA+IG1lY2hhbmlz bXMgb2YgdHJhbnNtaXNzaW9uIGluIHRoZSBkb3dubGluayAoZnJvbSBlTm9kZUIgdG8gbW9iaWxl DQo+ID4+ID4gPiBkZXZpY2UpIGFuZCBpbiB1cGxpbmsgKGZyb20gbW9iaWxlIGRldmljZSB0byB0 aGUgZU5vZGVCKS4gVGhlDQo+ID4+ID4gPiBkaWZmZXJlbmNlIGNvbWUgZnJvbSB0aGUgZmFjdCB0 aGF0IHRoZSBzY2hlZHVsZXIgZm9yIGJvdGgNCj4gPj4gPiA+IGRvd25saW5rIGFuZCBpbiB1cGxp bmsgaXMgbG9jYXRlZCBpbiB0aGUgZU5vZGVCLiBBcyBvZiBteQ0KPiA+PiA+ID4gdW5kZXJzdGFu ZGluZyB0aGUgcHVycG9zZSBvZiB0aGlzIGlzIHRvIHNpbXBsaWZ5IHRvIHRoZSBtYXhpbXVtDQo+ ID4+ID4gPiB0aGUgZGV2aWNlIGFuZCBzYXZlIGl0cw0KPiA+PiBiYXR0ZXJ5Lg0KPiA+PiA+ID4g QXMgYSByZXN1bHQgb2YgdGhpcyBkZXNpZ24sIHRoZSBlTm9kZUIgY2FuIHNlbmQgaW4gZWFjaCBU VEkNCj4gPj4gPiA+IHRvd2FyZCB0aGUgbW9iaWxlIGRldmljZSB3aGVuZXZlciBkYXRhIGV4aXN0 IGluIGl0cyBidWZmZXINCj4gPj4gPiA+IChlTm9kZUIgaXMgYSBraW5kIG9mIElQIHdpcmVsZXNz IHJvdXRlcikuIEJ1dCBmb3IgbW9iaWxlIGRldmljZSwNCj4gPj4gPiA+IGluIG9yZGVyIHRvIHNl bmQgdGhlIGRhdGEgaW4gdXBsaW5rLCBhZnRlciBhIHBhdXNlLCBpdCBoYXMgZmlyc3QNCj4gPj4g PiA+IHRvIGFzayB0aGUgbmV0d29yayB0byBzY2hlZHVsZSBpdC4gQW5kIG1vYmlsZSBoYXMgZm9y IHRoaXMNCj4gPj4gPiA+IHJlcXVlc3QgKGFmdGVyIGEgcGF1c2Ugb2YNCj4gPj4gPiA+IHNlbmRp bmcpIGp1c3Qgb25lIGJpdCwgYWN0dWFsbHkgaXMgbm90IGV2ZW4gYSBkYXRhIGJpdCwgaXMgYQ0K PiA+PiA+ID4gc3BlY2lhbCByYWRpbyBzaWduYWwgd2hpY2ggdGVsbHMgIkhleSwgSSBoYXZlIHNv bWUgZGF0YSBpbiB0aGUNCj4gPj4gPiA+IGJ1ZmZlciB0byBzZW5kIi4gTGV0cyBsb29rIGF0IGlu aXRpYWwgdHJhaW4gb2YgSVcgb2YgMTAgcGFja2V0cy4NCj4gPj4gPiA+IEluIGdvb2QgcmFkaW8g Y29uZGl0aW9ucyB0aGUgZU5vZGVCIHdpbGwgZml0IGFsbCAxMCBwYWNrZXRzIGluIDENCj4gPj4g PiA+IFRUSSBvZiAxIG1zIGFuZCBzZW5kIHRoZW0gYWxsIHRvIHRoZSBkZXZpY2UuIERldmljZSB3 aWxsIHVucGFjaw0KPiA+PiA+ID4gdGhlbSBhbmQgZ2VuZXJhdGUgMTAgQUNLIGFuZCB0aGVuIHRl bGwgdG8gdGhlIGVOb2RlQiAiSGV5LCBJIGhhdmUNCj4gPj4gPiA+IHNvbWUgZGF0YSBpbiB0aGUg YnVmZmVyIHRvIHNlbmQiLiBUaGUgZU5vZGVCIGhhcyBub3Qga25vd2xlZGdlDQo+ID4+ID4gPiBo b3cgbXVjaCBkYXRhIHRoZSBtb2JpbGUgaGFzIHRvIHNlbmQsIHRoZXJlZm9yIHdpbGwgYWxsb2Nh dGUgc29tZQ0KPiA+PiA+ID4gbWluaW11bQ0KPiA+PiA+ID4gY2FwYWNpdHk6ICJXZWxsLCBJIGdp dmUgeW91IGEgZ3JhbnQgdG8gc2VuZCB1cCB0byAyMDAgQnl0ZXMgYW5kDQo+ID4+ID4gPiB5b3Ug bXVzdCBzZW5kIHRoZW0gZXhhY3RseSBhZnRlciA0IG1zIHlvdSBnZXQgdGhpcyBub3RpZmljYXRp b24iLg0KPiA+PiA+ID4gVGhlc2UNCj4gPj4gPiA+IDQgbXMgYXJlIGFsbG93YW5jZSBmb3IgbW9i aWxlIGRldmljZSB0byBwcmVwYXJlIGRhdGEgZm9yIHNlbmRpbmcNCj4gPj4gPiA+IHdoaWNoIGlz IGEgdmVyeSBjb21wbGljYXRlZCBwcm9jZXNzLCB0aGVzZSA0IG1zIGlzIGEgbXVzdCBkZWxheQ0K PiA+PiA+ID4gYmV0d2VlbiB0aGUgZ3JhbnQNCj4gPj4gPiBhbmQgdGhlIHRyYW5zbWlzc2lvbi4N Cj4gPj4gPiA+IElmIG1vYmlsZSBoYXMgaW4gaXRzIGJ1ZmZlciAxMCBBQ0sgdG8gc2VuZCBpdCB3 aWxsIHNlbmQgb25seSAyDQo+ID4+ID4gPiBBQ0sgYW5kIHdpbGwgY29tcGxlbWVudCB0aGVtIHdp dGgsIHNvIGNhbGxlZCwgYnVmZmVyIHN0YXR1cw0KPiA+PiA+ID4gcmVwb3J0OiAiSSBkbyBoYXZl IHN0aWxsIDQ4MCBieXRlcyB0byBzZW5kIi4gTm93IG5ldHdvcmsgaGFzIG1vcmUNCj4gPj4gPiA+ IGluZm9ybWF0aW9uIGFuZCBhbGxvY2F0ZSBhcw0KPiA+PiA+ID4gcmVxdWVzdGVkOiAiV2VsbCwg SSBnaXZlIHlvdSBhIGdyYW50IHRvIHNlbmQgdXAgdG8gMjAwIEJ5dGVzIGFuZA0KPiA+PiA+ID4g eW91IG11c3Qgc2VuZCB0aGVtIGV4YWN0bHkgYWZ0ZXIgNCBtcyB5b3UgZ2V0IHRoaXMgbm90aWZp Y2F0aW9uIi4NCj4gPj4gPiA+IEJ1dCwgaW4gdGhlIG1lYW4gdGltZSBzb21lIG90aGVyIGluZm9y bWF0aW9uIG9mIGhpZ2hlciBwcmlvcml0eQ0KPiA+PiA+ID4gbWF5IGFwcGVhciBpbiB0aGUNCj4g Pj4gPiBkZXZpY2UgKGUuZy4NCj4gPj4gPiA+IHNpZ25hbGluZykgYW5kIHRoZSBkZXZpY2Ugd2ls bCBzZW5kIG9ubHkgNiBvdXQgb2YgOCByZW1haW5pbmcgQUNLDQo+ID4+ID4gPiB0b2dldGhlciB3 aXRoIHNpZ25hbGluZyBhbmQgYW5vdGhlciBCdWZmZXIgU3RhdHVzIFJlcG9ydDogIkhleSwgSQ0K PiA+PiA+ID4gZG8gaGF2ZSBhbm90aGVyIDEyMCBieXRlcyB0byBzZW5kIi4gQW5kIGFnYWluICJX ZWxsLCBJIGdpdmUgeW91IGENCj4gPj4gPiA+IGdyYW50IHRvIHNlbmQgdXAgdG8gMTIwIEJ5dGVz IGFuZCB5b3UgbXVzdCBzZW5kIHRoZW0gZXhhY3RseQ0KPiA+PiA+ID4gYWZ0ZXIgNCBtcyB5b3Ug Z2V0IHRoaXMgbm90aWZpY2F0aW9uIi4gQXMgYSByZXN1bHQgdGhlIHNlcnZlcg0KPiA+PiA+ID4g d2lsbCByZWNlaXZlIDIgQUNLLCB0aGVuIGFmdGVyIGEgZ2FwIG9mIDYgbXMgYW5vdGhlciA2IEFD SywgdGhlbiwNCj4gPj4gPiA+IGFmdGVyIGFub3RoZXIgNiBtcywgdGhlIHJlbWFpbmluZyAyIEFD Sy4gV2l0aCB0aGUgc2NoZWR1bGVyIGluDQo+ID4+ID4gPiB0aGUgZU5vZGVCIHRoZXNlIGdhcHMg YXJlIGluZXZpdGFibGUuIEFuZCBvbmUgbW9yZSB0aGluazogbW9iaWxlDQo+ID4+ID4gPiBwYWNr IHRvZ2V0aGVyIHRob3NlIDYgQUNLIGluIDEgVFRJIG9mIDEgbXMgYW5kIHNlbmQgdGhlbSwgdGhl bg0KPiA+PiA+ID4gdGhlIGVOb2RlQiB1bnBhY2sgdGhlbSBhbmQgdGhlcmUgaXMgbm8gd2F5IHRv IHJlY292ZXIgdGhlIHNwYWNpbmcNCj4gb2YgMC4xMiBtcyBiZXR3ZWVuIHRoZW0uDQo+ID4+ID4g PiBlTm9kZUIgc2ltcGx5IHNlbmQgdGhlbSB0byB0aGUgc2VydmVyIGJhY2sgdG8gYmFjayBhdCB0 aGUgc3BlZWQNCj4gPj4gPiA+IG9mIGl0cyBkYXRhIGNhcmQgd2hpY2ggaXMgdHlwaWNhbGx5IDEg R2JwcyBhbmQgd2UgZmFjZSB0aGUgQUNLDQo+ID4+ID4gPiBjb21wcmVzc2lvbi4gQWZ0ZXIgdGhl IGxhc3QgMiBBQ0sgd2VyZSBzZW50IHRoZXJlIGlzIG5vdGhpbmcgZWxzZQ0KPiA+PiA+ID4gdG8g c2VuZCwgc28gdGhlcmUgaXMgbm8gQnVmZmVyIFN0YXR1cyBSZXBvcnQsIHRoZXJlZm9yZSwgYWZ0 ZXINCj4gPj4gPiA+IHBhY2tldHMgb2YgdGhlIG5leHQgdHJhaW4gYXJyaXZlIGFuZCBBQ0sgYXJl IGdlbmVyYXRlZCBldmVyeXRoaW5nDQo+ID4+ID4gPiBzdGFydHMgYWdhaW4gd2l0aCBvbmUgYml0 IG9mIGluZm9ybWF0aW9uICJIZXksIEkgaGF2ZQ0KPiA+PiA+IHNvbWUgZGF0YSBpbiB0aGUgYnVm ZmVyIHRvIHNlbmQiLg0KPiA+PiA+ID4gVGhhdCdzIHdoeSB0aGVyZSBhcmUgZ2FwcyBhbmQgQUNL IGNvbXByZXNzaW9uLCBsb3NzLCBvciBtb3JlDQo+ID4+ID4gPiBleGFjdGx5LCBsYWNrIG9mIGFu eSB0aW1pbmcgaW5mb3JtYXRpb24gd2hpY2ggd291bGQgYmUgdXNlZnVsIGZvcg0KPiA+PiA+ID4g dGhlIFRDUCBzZWxmLQ0KPiA+PiA+IGNsb2NraW5nLg0KPiA+PiA+ID4NCj4gPj4gPiA+IFRoYW5r IHlvdSBhbmQgc29ycnkgZm9yIHRha2luZyB5b3VyIHRpbWUuDQo+ID4+ID4gPg0KPiA+PiA+ID4N Cj4gPj4gPiA+IFZlYWNlc2xhdiBSb21hbg0KPiA+PiA+ID4NCj4gPj4gPiA+DQo+ID4+ID4gPg0K PiA+PiA+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4gPiA+IEZyb206IFZlYWNl c2xhdiBST01BTg0KPiA+PiA+ID4gU2VudDogU2F0dXJkYXksIDA1IFNlcHRlbWJlciAyMDE1IDAx OjEwDQo+ID4+ID4gPiBUbzogJ0VyaWMgRHVtYXpldCc7IE5lYWwgQ2FyZHdlbGwNCj4gPj4gPiA+ IENjOiB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgSW5nZW1hciBK b2hhbnNzb24NCj4gPj4gPiA+IFM7IEVyaWMgRHVtYXpldDsgZW5kMmVuZC1pbnRlcmVzdEBwb3N0 ZWwub3JnDQo+ID4+ID4gPiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBh dGNoIGRlcGxveW1lbnQNCj4gPj4gPiA+DQo+ID4+ID4gPiBJZiB3ZSBsb29rIGF0IHRoZSBmaXJz dCB0cmFpbjogYWxsIHBhY2tldHMgcmVjZWl2ZWQgaW4gbGVzcyB0aGFuIDEgbXMuDQo+ID4+ID4g PiBQcm9iYWJseSB0aGlzIGlzIG9ubHkgYW4gYXBwZWFyYW5jZSBhcyBMVEUgdHJhbnNtaXRzIGlu LCBzbw0KPiA+PiA+ID4gY2FsbGVkIHRyYW5zbWlzc2lvbiB0aW1lIGludGVydmFsIChUVEkpIG9m IDEgbXMsIGFuZCB3aGF0IHdlIHNlZQ0KPiA+PiA+ID4gaGVyZSBpcyB0aGF0IGFsbCAxMCBwYWNr ZXRzIG9mIGluaXRpYWwgd2luZG93IGZpdHRlZCBpbiAxIG1zLA0KPiA+PiA+ID4gYW5kLCB3aGVu IGRlY29kZWQsIHdlcmUgcHJlc2VudGVkIHRvIHRoZSBUQ1AvSVAgbGF5ZXIgKGFuZCBwY2FwKSBh bGwNCj4gYXQgb25jZS4NCj4gPj4gPiA+IEJUVywgOCBwYWNrZXRzIG9mIDEzMDIyIGJ5dGVzIGlu IDEgbXMgbWVhbnMgaW5zdGFudGFuZW91cyBzcGVlZA0KPiA+PiA+ID4gb2YNCj4gPj4gPiA+IDEw NCBNYnBzLCBnb29kIHJhZGlvIGNvbmRpdGlvbnMuIFRDUCBnZW5lcmF0ZXMgMTAgQUNLIHRyYWlu IG9mDQo+ID4+ID4gPiB0aGUgZHVyYXRpb24gb2YNCj4gPj4gPiA+IDEuMiBtcy4gV2lsbCBpdCBi ZSBhIEZhc3QgRXRoZXJuZXQgcG9zc2libHkgd2Ugd291bGQgY29uc2lkZXINCj4gPj4gPiA+IHRo aXMgbm9ybWFsLA0KPiA+PiA+IGlzbid0IGl0ID8NCj4gPj4gPiA+IEkgbG9va2VkIGF0IGhvdyBz ZXJ2ZXIgcmVwbHkgdG8gdGhlc2UgMTAgQUNLcy4gVGhlcmUgYXJlIDMgZ2Fwcw0KPiA+PiA+ID4g b2YgNCwNCj4gPj4gPiA+IDIgYW5kIDUgbXMgaW4gdGhlIHJlcGx5IHRyYWluIGFuZCB0aGUgdG90 YWwgdHJhaW4gb2YgMTggKD8sIGl0DQo+ID4+ID4gPiBzaG91bGQgYmUgMjApIHBhY2tldHMgcmVh Y2hlcyBhIGR1cmF0aW9uIG9mIDE0IG1zLiBBRkFJSyBMVEUgbWF5DQo+ID4+ID4gPiBpbnRyb2R1 Y2UgZ2FwcyBpbiBBQ0sgdHJhaW4gZHVlIHRvIHVwbGluayBzY2hlZHVsaW5nIG1lY2hhbmlzbS4N Cj4gPj4gPiA+IE1heSBiZSB0aGVzZSBnYXBzIHRyaWdnZXIgSHlzdGFydCBlYXJseSBleGl0ID8N Cj4gPj4gPiA+DQo+ID4+ID4gPiBWZWFjZXNsYXYgUm9tYW4NCj4gPj4gPiA+IFRlY2huaWNhbCBh bmQgSVQgZGlyZWN0b3INCj4gPj4gPiA+IE9yYW5nZSBNb2xkb3ZhIFMuQS4NCj4gPj4gPiA+IEZp eDogKzM3MzIyNTc1NDAwDQo+ID4+ID4gPiBNb2I6ICszNzM2OTE5ODQwMA0KPiA+PiA+ID4gRmF4 OiArMzczMjI1NzUzMDYNCj4gPj4gPiA+DQo+ID4+ID4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2Ut LS0tLQ0KPiA+PiA+ID4gRnJvbTogRXJpYyBEdW1hemV0IFttYWlsdG86ZXJpYy5kdW1hemV0QGdt YWlsLmNvbV0NCj4gPj4gPiA+IFNlbnQ6IEZyaWRheSwgMDQgU2VwdGVtYmVyIDIwMTUgMTk6NDgN Cj4gPj4gPiA+IFRvOiBOZWFsIENhcmR3ZWxsDQo+ID4+ID4gPiBDYzogVmVhY2VzbGF2IFJPTUFO OyB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsNCj4gPj4gPiA+IElu Z2VtYXIgSm9oYW5zc29uIFM7IEVyaWMgRHVtYXpldDsgZW5kMmVuZC1pbnRlcmVzdEBwb3N0ZWwu b3JnDQo+ID4+ID4gPiBTdWJqZWN0OiBSZTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNo IGRlcGxveW1lbnQNCj4gPj4gPiA+DQo+ID4+ID4gPiBPbiBGcmksIDIwMTUtMDktMDQgYXQgMTE6 MzIgLTA0MDAsIE5lYWwgQ2FyZHdlbGwgd3JvdGU6DQo+ID4+ID4gPiA+IEhpIFZlYWNlc2xhdiwN Cj4gPj4gPiA+ID4NCj4gPj4gPiA+ID4gSSBhZ3JlZSB0aGF0IGluIHlvdXIgTFRFIHRyYWNlcyBp dCBsb29rcyBsaWtlIENVQklDIEh5c3RhcnQgaXMNCj4gPj4gPiA+ID4gZXhpdGluZyBzbG93IHN0 YXJ0IHRvbyBlYXJseS4NCj4gPj4gPiA+ID4NCj4gPj4gPiA+ID4gU0luY2UgeW91IHNlZW0gdG8g aGF2ZSBhIG5pY2UgTFRFIHRlc3RiZWQsIHdvdWxkIHlvdSBiZSBhYmxlIHRvDQo+ID4+ID4gPiA+ IGRvIHNvbWUgZXhwZXJpbWVudHMgdG8gZmluZCBhIHNldCBvZiBwYXJhbWV0ZXJzIGZvciBIeXN0 YXJ0DQo+ID4+ID4gPiA+IHRoYXQgd29yayBiZXR0ZXIgZm9yIHlvdXIgTFRFIGVudmlyb25tZW50 PyBGb3IgZXhhbXBsZSwgeW91DQo+ID4+ID4gPiA+IG1pZ2h0IHRyeSB0aGUgdHdvIHZhcmlhdGlv bnMgSSBzdWdnZXN0ZWQgZWFybGllciBpbiB0aGUgdGhyZWFkOg0KPiA+PiA+ID4gPg0KPiA+PiA+ ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICBpZiAoY2EtPmN1cnJfcnR0ID4gY2EtPmRlbGF5 X21pbiArDQo+ID4+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBIWVNUQVJUX0RF TEFZX1RIUkVTSChjYS0+ZGVsYXlfbWluDQo+ID4+ID4gPiA+ID4+DQo+ID4+ID4gPiA+IDIpKSB7 IG9yDQo+ID4+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgIGlmIChjYS0+Y3Vycl9ydHQg PiBjYS0+ZGVsYXlfbWluICsNCj4gPj4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAg IEhZU1RBUlRfREVMQVlfVEhSRVNIKGNhLT5kZWxheV9taW4NCj4gPj4gPiA+ID4gPj4NCj4gPj4g PiA+ID4gMSkpIHsNCj4gPj4gPiA+ID4NCj4gPj4gPiA+ID4gRG8gYW55IG9mIHRob3NlIGdpdmUg YmV0dGVyIHJlc3VsdHMgZm9yIHlvdXIgdGVzdHM/DQo+ID4+ID4gPiA+DQo+ID4+ID4gPiA+IG5l YWwNCj4gPj4gPiA+DQo+ID4+ID4gPiBBbHNvLCBoeXN0YXJ0IGlzIGZvb2xlZCBieSB0b28gbWFu eSBBQ0sgcmVjZWl2ZWQgaW4gc2hvcnQgcGVyaW9kLg0KPiA+PiA+ID4NCj4gPj4gPiA+IFRoaXMg cHJvYmxlbSB3b3VsZCBiZSBzb2x2ZWQgaWYgR1JPIHdhcyB1c2VkIGF0IHJlY2VpdmVyLCBhcyBs ZXNzDQo+ID4+ID4gPiBBQ0sgd291bGQgYmUgc2VudC4NCj4gPj4gPiA+DQo+ID4+ID4gPiBQcmVz dW1hYmx5IHJlY2VpdmVyIGlzIG5vdCBhIGxpbnV4IFRDUCBzdGFjayA/DQo+ID4+ID4gPg0KPiA+ PiA+ID4gTWF5YmUgd2Ugc2hvdWxkIGFkZCBhIGxvZ2ljIGluIGh5c3RhcnRfdXBkYXRlKCkgdG8g dGFrZSBvbmUgQUNLIHBlcg0KPiBtcy4NCj4gPj4gPiA+DQo+ID4+ID4gPiAwNTozMjoyNC4xMzQx NzIgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+ IEZsYWdzIFtTXSwgc2VxIDQyOTQ0MjMyMTUsIHdpbiA2NTUzNSwgb3B0aW9ucyBbbXNzIDE0NjAs c2Fja09LLFRTDQo+ID4+ID4gPiB2YWwNCj4gPj4gPiA+IDExMTIzNTEgZWNyIDAsbm9wLHdzY2Fs ZSA4XSwgbGVuZ3RoIDANCj4gPj4gPiA+IDA1OjMyOjI0LjE2MTk4NiBJUCAxOTUuOTUuMTc4LjIw NC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0Og0KPiA+PiA+ID4gRmxhZ3MgW1MuXSwgc2VxIDEx OTg4Nzc3MjEsIGFjayA0Mjk0NDIzMjE2LCB3aW4gMjg5NjAsIG9wdGlvbnMNCj4gPj4gPiA+IFtt c3MgMTQxNixzYWNrT0ssVFMgdmFsDQo+ID4+ID4gPiAyODA4ODA4NjM0IGVjciAxMTEyMzUxLG5v cCx3c2NhbGUgN10sIGxlbmd0aCAwDQo+ID4+ID4gPiAwNTozMjoyNC4xNjIxMDggSVAgOTQuMjQz LjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwg YWNrIDEsIHdpbiAzNDMsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTQgZWNyDQo+ID4+ ID4gPiAyODA4ODA4NjM0XSwgbGVuZ3RoIDANCj4gPj4gPiA+IDA1OjMyOjI0LjE2MzA3MiBJUCA5 NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOg0KPiA+PiA+ID4gRmxhZ3Mg W1AuXSwgc2VxIDE6NjU4LCBhY2sgMSwgd2luIDM0Mywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwN Cj4gPj4gPiA+IDExMTIzNTQgZWNyIDI4MDg4MDg2MzRdLCBsZW5ndGggNjU3DQo+ID4+ID4gPiAw NTozMjoyNC4yMDM5ODQgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjky NDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5v cCxUUyB2YWwgMjgwODgwODY3NQ0KPiA+PiA+ID4gZWNyIDExMTIzNTRdLCBsZW5ndGggMA0KPiA+ PiA+ID4gMDU6MzI6MjQuMjEwMjc1IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4x NTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbUC5dLCBzZXEgMTozODcsIGFjayA2NTgsIHdpbiAy MzcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsDQo+ID4+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEy MzU0XSwgbGVuZ3RoIDM4Ng0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwMzE1IElQIDE5NS45NS4xNzgu MjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbLl0sIHNlcSAz ODc6MTc5MSwgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUw0KPiA+PiA+ID4g dmFsDQo+ID4+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPj4g PiA+IDA1OjMyOjI0LjIxMDMzMiBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2 LjMyOTI0Og0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBzZXEgMTc5MTozMTk1LCBhY2sgNjU4LCB3aW4g MjM3LCBvcHRpb25zIFtub3Asbm9wLFRTDQo+ID4+ID4gPiB2YWwNCj4gPj4gPiA+IDI4MDg4MDg2 NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwMzQ3IElQ IDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFn cyBbLl0sIHNlcSAzMTk1OjQ1OTksIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3As VFMNCj4gPj4gPiA+IHZhbA0KPiA+PiA+ID4gMjgwODgwODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0 aCAxNDA0DQo+ID4+ID4gPiAwNTozMjoyNC4yMTAzNTAgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQg PiAxOTUuOTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDM4Nywgd2luIDM0 Nywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3INCj4gPj4gPiA+IDI4MDg4MDg2 NzddLCBsZW5ndGggMA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwMzYxIElQIDE5NS45NS4xNzguMjA0 LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbLl0sIHNlcSA0NTk5 OjYwMDMsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMNCj4gPj4gPiA+IHZh bA0KPiA+PiA+ID4gMjgwODgwODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0aCAxNDA0DQo+ID4+ID4g PiAwNTozMjoyNC4yMTAzNzQgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4z MjkyNDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgc2VxIDYwMDM6NzQwNywgYWNrIDY1OCwgd2luIDIz Nywgb3B0aW9ucyBbbm9wLG5vcCxUUw0KPiA+PiA+ID4gdmFsDQo+ID4+ID4gPiAyODA4ODA4Njc3 IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPj4gPiA+IDA1OjMyOjI0LjIxMDM4OSBJUCAx OTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0Og0KPiA+PiA+ID4gRmxhZ3Mg Wy5dLCBzZXEgNzQwNzo4ODExLCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRT DQo+ID4+ID4gPiB2YWwNCj4gPj4gPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGgg MTQwNA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwNDAyIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQu MjQzLjEwNC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbLl0sIHNlcSA4ODExOjEwMjE1LCBh Y2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTDQo+ID4+ID4gPiB2YWwNCj4gPj4g PiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+PiA+ID4gMDU6MzI6 MjQuMjEwNDE2IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6DQo+ ID4+ID4gPiBGbGFncyBbLl0sIHNlcSAxMDIxNToxMTYxOSwgYWNrIDY1OCwgd2luIDIzNywgb3B0 aW9ucyBbbm9wLG5vcCxUUw0KPiA+PiA+ID4gdmFsDQo+ID4+ID4gPiAyODA4ODA4Njc3IGVjciAx MTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPj4gPiA+IDA1OjMyOjI0LjIxMDQxOCBJUCA5NC4yNDMu MTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOg0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBh Y2sgMTc5MSwgd2luIDM1OCwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3INCj4g Pj4gPiA+IDI4MDg4MDg2NzddLCBsZW5ndGggMA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwNDMxIElQ IDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFn cyBbLl0sIHNlcSAxMTYxOToxMzAyMywgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5v cCxUUw0KPiA+PiA+ID4gdmFsDQo+ID4+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVu Z3RoIDE0MDQNCj4gPj4gPiA+IDA1OjMyOjI0LjIxMDQ1NSBJUCA5NC4yNDMuMTA0LjE1Ni4zMjky NCA+IDE5NS45NS4xNzguMjA0LjgwOg0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgMzE5NSwgd2lu IDM2OSwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSBlY3INCj4gPj4gPiA+IDI4MDg4 MDg2NzddLCBsZW5ndGggMA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwNjg1IElQIDk0LjI0My4xMDQu MTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6DQo+ID4+ID4gPiBGbGFncyBbLl0sIGFjayA0 NTk5LCB3aW4gMzgwLCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IGVjcg0KPiA+PiA+ ID4gMjgwODgwODY3N10sIGxlbmd0aCAwDQo+ID4+ID4gPiAwNTozMjoyNC4yMTA3MzEgSVAgOTQu MjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+IEZsYWdzIFsu XSwgYWNrIDYwMDMsIHdpbiAzOTEsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgZWNy DQo+ID4+ID4gPiAyODA4ODA4Njc3XSwgbGVuZ3RoIDANCj4gPj4gPiA+IDA1OjMyOjI0LjIxMDkz NyBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOg0KPiA+PiA+ID4g RmxhZ3MgWy5dLCBhY2sgNzQwNywgd2luIDQwMiwgb3B0aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTEx MjM1OSBlY3INCj4gPj4gPiA+IDI4MDg4MDg2NzddLCBsZW5ndGggMA0KPiA+PiA+ID4gMDU6MzI6 MjQuMjExMDc4IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6DQo+ ID4+ID4gPiBGbGFncyBbLl0sIGFjayA4ODExLCB3aW4gNDEzLCBvcHRpb25zIFtub3Asbm9wLFRT IHZhbCAxMTEyMzU5IGVjcg0KPiA+PiA+ID4gMjgwODgwODY3N10sIGxlbmd0aCAwDQo+ID4+ID4g PiAwNTozMjoyNC4yMTEyMjggSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIw NC44MDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDEwMjE1LCB3aW4gNDI0LCBvcHRpb25zIFtu b3Asbm9wLFRTIHZhbCAxMTEyMzU5DQo+ID4+ID4gPiBlY3IgMjgwODgwODY3N10sIGxlbmd0aCAw DQo+ID4+ID4gPiAwNTozMjoyNC4yMTEzNzcgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUu OTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDExNjE5LCB3aW4gNDM1LCBv cHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5DQo+ID4+ID4gPiBlY3IgMjgwODgwODY3N10s IGxlbmd0aCAwDQo+ID4+ID4gPiAwNTozMjoyNC4yMTE1NDQgSVAgOTQuMjQzLjEwNC4xNTYuMzI5 MjQgPiAxOTUuOTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDEzMDIzLCB3 aW4gNDQ2LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5DQo+ID4+ID4gPiBlY3IgMjgw ODgwODY3N10sIGxlbmd0aCAwDQo+ID4+ID4gPiAwNTozMjoyNC4yMzc5OTkgSVAgMTk1Ljk1LjE3 OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgc2Vx IDEzMDIzOjE0NDI3LCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTDQo+ID4+ ID4gPiB2YWwNCj4gPj4gPiA+IDI4MDg4MDg3MDkgZWNyIDExMTIzNTldLCBsZW5ndGggMTQwNA0K PiA+PiA+ID4gMDU6MzI6MjQuMjM4MDUzIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEw NC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbLl0sIHNlcSAxNDQyNzoxNTgzMSwgYWNrIDY1 OCwgd2luIDIzNywgb3B0aW9ucyBbbm9wLG5vcCxUUw0KPiA+PiA+ID4gdmFsDQo+ID4+ID4gPiAy ODA4ODA4NzA5IGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4gPj4gPiA+IDA1OjMyOjI0LjIz ODA5MCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0LjgwOg0KPiA+PiA+ ID4gRmxhZ3MgWy5dLCBhY2sgMTQ0MjcsIHdpbiA0NTcsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFs IDExMTIzNjINCj4gPj4gPiA+IGVjciAyODA4ODA4NzA5XSwgbGVuZ3RoIDANCj4gPj4gPiA+IDA1 OjMyOjI0LjIzODE2NCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0Ljgw Og0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgMTU4MzEsIHdpbiA0NjgsIG9wdGlvbnMgW25vcCxu b3AsVFMgdmFsIDExMTIzNjINCj4gPj4gPiA+IGVjciAyODA4ODA4NzA5XSwgbGVuZ3RoIDANCj4g Pj4gPiA+IDA1OjMyOjI0LjI0NDAyNCBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQu MTU2LjMyOTI0Og0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBzZXEgMTU4MzE6MTcyMzUsIGFjayA2NTgs IHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMNCj4gPj4gPiA+IHZhbA0KPiA+PiA+ID4gMjgw ODgwODcxMyBlY3IgMTExMjM1OV0sIGxlbmd0aCAxNDA0DQo+ID4+ID4gPiAwNTozMjoyNC4yNDQw NjMgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDoNCj4gPj4gPiA+ IEZsYWdzIFsuXSwgc2VxIDE3MjM1OjE4NjM5LCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtu b3Asbm9wLFRTDQo+ID4+ID4gPiB2YWwNCj4gPj4gPiA+IDI4MDg4MDg3MTMgZWNyIDExMTIzNTld LCBsZW5ndGggMTQwNA0KPiA+PiA+ID4NCj4gPg0K From nobody Fri Oct 2 04:53:41 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 537101B2ACC for ; Fri, 2 Oct 2015 04:53:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.144 X-Spam-Level: X-Spam-Status: No, score=0.144 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, FRT_BELOW2=2.154, J_CHICKENPOX_36=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dq6f2LAZrCA2 for ; Fri, 2 Oct 2015 04:53:32 -0700 (PDT) Received: from mailfilter.orange.md (mailfilter.orange.md [94.243.64.204]) by ietfa.amsl.com (Postfix) with ESMTP id E4BE81B2AC5 for ; Fri, 2 Oct 2015 04:53:31 -0700 (PDT) Received: from localhost.localdomain (antispam.orange.md [127.0.0.1]) by localhost (Postfix) with ESMTP id DC51B27E03E; Fri, 2 Oct 2015 14:53:49 +0300 (EEST) Received: from antispam.orange.md by antispam.orange.md with queue id 249828-1; Fri, 02 Oct 2015 11:53:49 GMT Received: from XCHSRV03.main.orange.md (unknown [192.168.200.63]) by mailfilter.orange.md (Postfix) with ESMTP id B9A0C27E038; Fri, 2 Oct 2015 14:53:49 +0300 (EEST) Received: from XCHSRV01.main.orange.md ([fe80::685f:aef6:93b0:dea7]) by XCHSRV03.main.orange.md ([fe80::6dbc:28dd:213b:8931%14]) with mapi id 14.02.0328.009; Fri, 2 Oct 2015 14:53:29 +0300 From: Veaceslav ROMAN To: Ingemar Johansson S , Eric Dumazet Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8Pq87vsQ9XoBmgDq86x2kNXnZ6oAq81ilTA= Date: Fri, 2 Oct 2015 11:53:28 +0000 Message-ID: <7DBBB686E19D2049ADAACD210B474BB10166E88A4F@XCHSRV01.main.orange.md> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E863AD@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD89C3@ESESSMB205.ericsson.se> In-Reply-To: <81564C0D7D4D2A4B9A86C8C7404A13DA34BD89C3@ESESSMB205.ericsson.se> Accept-Language: en-US, ro-RO Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.107.165] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2015 11:53:40 -0000 VGhhbmsgeW91LiBJJ2xsIGJlIGdyYXRlZnVsIGlmIHlvdSBzaGFyZSB5b3VyIGZ1cnRoZXIgcmVz dWx0cy4NCkFzIGZvciB0aW1lc3RhbXAsIHBvc3NpYmx5IHRoZXkgY2FuIGJlIHVzZWQgZm9yIExU RSwgYnV0IGJlY2F1c2Ugb2YgdGhleSBncmFudWxhcml0eSBvZiAxLCA0IG9yIDEwIG1zIEkgYW0g bm90IHN1cmUgdGhleSB3aWxsIGJlIGVub3VnaCBmb3Igc3ViIG1pbGxpc2Vjb25kIG5leXdvcmtz Lg0KDQpCVFcsIHRoZSBzZXJ2ZXIgdXNlZCBpbiB0ZXN0cyBJIGRlc2NyaWJlZCBiZWxvdyBzZW5k cyB0aW1lc3RhbXBzIGV2ZXJ5IDQgbXMgYW5kIHRoZSBjbGllbnQgc2VuZHMgZXZlcnkgMTAgbXMu ICAgDQoNClZlYWNlc2xhdiBSb21hbg0KVGVjaG5pY2FsIGFuZCBJVCBkaXJlY3Rvcg0KT3Jhbmdl IE1vbGRvdmEgUy5BLg0KRml4OiArMzczMjI1NzU0MDANCk1vYjogKzM3MzY5MTk4NDAwDQpGYXg6 ICszNzMyMjU3NTMwNg0KDQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBJbmdl bWFyIEpvaGFuc3NvbiBTIFttYWlsdG86aW5nZW1hci5zLmpvaGFuc3NvbkBlcmljc3Nvbi5jb21d IA0KU2VudDogVGh1cnNkYXksIDAxIE9jdG9iZXIgMjAxNSAyMDowMg0KVG86IFZlYWNlc2xhdiBS T01BTjsgRXJpYyBEdW1hemV0DQpDYzogRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsOyB0Y3Bt QGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgZW5kMmVuZC1pbnRlcmVzdEBw b3N0ZWwub3JnOyBJbmdlbWFyIEpvaGFuc3NvbiBTDQpTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVd IFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCg0KSGkNCg0KQW4gaW5jcmVhc2VkIHRocmVz aG9sZCB3b3VsZCBwcm9iYWJseSBzb2x2ZSB0aGUgcHJvYmxlbSBhbmQgaXMgcG9zc2libHkgdGhl IHF1aWNrIGZpeC4gDQpJIHdpbGwgYW55d2F5IG5lZWQgdG8gcmVydW4gcGFydHMgb2YgbXkgc2lt dWxhdGlvbnMgYmVjYXVzZSBvZiBteSBvd24gMzJtcyBtaXN0YWtlLCBpdCBpcyBxdWl0ZSBzdHJh aWdodGZvcndhcmQgdG8gYWRkIGEgZmV3IGFsdGVybmF0aXZlIHRocmVzaG9sZCB2YWx1ZXMgaW4g dGhlIHNhbWUgd29yay4NCg0KV29uZGVyIHRob3VnaCBpZiBvbmUgY2FuIGRvIGl0IHNtYXJ0ZXI/ Lg0KRm9yIGluc3RhbmNlLCBhc3N1bWluZyB0aGF0IHRoZSB0aW1lc3RhbXAgb3B0aW9uIGlzIGVu YWJsZWQsIGl0IHNob3VsZCB0aGVyZSBiZSBwb3NzaWJsZSB0byBpbmZlciBpZiBBQ0tzIGJlbG9u ZyB0byB0aGUgc2FtZSBIeVN0YXJ0IHJvdW5kLCBiYXNlZCBvbiBUU3ZhbC4gDQpJdCBvZiBjb3Vy c2UgYWRkcyBzb21lIGV4dHJhIGxpbmVzIG9mIGNvZGUgdG8gdGhlIEh5U3RhcnQgYWxnb3JpdGht IGJ1dCB0aGVuIG9uZSBjYW4gcGVyaGFwcyBrZWVwIHRoZSBjdXJyZW50IHRocmVzaG9sZHMgPy4N Cg0KL0luZ2VtYXINCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBWZWFj ZXNsYXYgUk9NQU4gW21haWx0bzpWZWFjZXNsYXYuUm9tYW5Ab3JhbmdlLm1kXQ0KPiBTZW50OiBk ZW4gMSBva3RvYmVyIDIwMTUgMTc6MTcNCj4gVG86IEVyaWMgRHVtYXpldDsgSW5nZW1hciBKb2hh bnNzb24gUw0KPiBDYzogRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsOyB0Y3BtQGlldGYub3Jn OyBQaWVycyBPJ0hhbmxvbjsgDQo+IFNhbmd0YWUgSGE7IGVuZDJlbmQtaW50ZXJlc3RAcG9zdGVs Lm9yZw0KPiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxv eW1lbnQNCj4gDQo+IFN1cmUgMTAgbXMgd2lsbCBiZSBiZXR0ZXIgYXMgbW9yZSBvZiB0aGUgTFRF IHRlY2hub2xvZ2ljYWwganVtcHMgd2lsbCANCj4gYmUgY292ZXJlZCBhbmQsIGF0IGxlYXN0LCB3 aWxsIGhlbHAgZ2V0IHJpZCBvZiBleGl0IGF0IHRoZSBtaW5pbXVtIA0KPiBleGl0IGN3bmQgKDI5 LCB3aGljaCBhdCAyMCBtcyBSVFQgaXMgfjE2TWJwcykgYW5kIHdpbGwgc2hpZnQgdG8gNDkgDQo+ ICh+MjggTWJwcykgb3IgdG8gODkgKH4gNTAgTWJwcykuDQo+IFdlIGhhdmUgYWxsIG5lY2Vzc2Fy eSB0byByZXBlYXQgdGhvc2UgdGVzdHMuDQo+IA0KPiBWZWFjZXNsYXYgUm9tYW4NCj4gDQo+IA0K PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBFcmljIER1bWF6ZXQgW21haWx0 bzplZHVtYXpldEBnb29nbGUuY29tXQ0KPiBTZW50OiBUaHVyc2RheSwgMDEgT2N0b2JlciAyMDE1 IDE1OjQ0DQo+IFRvOiBJbmdlbWFyIEpvaGFuc3NvbiBTDQo+IENjOiBWZWFjZXNsYXYgUk9NQU47 IEVyaWMgRHVtYXpldDsgTmVhbCBDYXJkd2VsbDsgdGNwbUBpZXRmLm9yZzsgUGllcnMgDQo+IE8n SGFubG9uOyBTYW5ndGFlIEhhOyBlbmQyZW5kLWludGVyZXN0QHBvc3RlbC5vcmcNCj4gU3ViamVj dDogUmU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBwYXRjaCBkZXBsb3ltZW50DQo+IA0KPiBZ ZXMsIHRoaXMgNG1zIHZhbHVlIHByb2JsZW0gaXMgZXZlbiBtb3JlIHZpc2libGUgd2l0aCBUQ1Ag cGFjaW5nIChhcyANCj4gaW1wbGVtZW50ZWQgd2l0aCBsaW51eCBzY2hfZnEgcGFja2V0IHNjaGVk dWxlcikgYmVjYXVzZSBvZiBhcHBhcmVudCANCj4gZGVsYXlzIGNhdXNlZCBieSBzb2pvdXJuIHRp bWUgaW4gcGFja2V0IHNjaGVkdWxlciBpdHNlbGYuDQo+IFRDUCBkb2VzIG5vdCBvZmZzZXQgdGhp cyBzb2pvdXJuIHRpbWUgKGFzIFJUVCBpcyBhbHNvIHVzZWQgZm9yIFJUTyANCj4gZGV0ZXJtaW5h dGlvbiwgYmV0dGVyIGhhdmUgYW4gaW5mbGF0ZWQgdmlldykNCj4gDQo+IElzIHN3aXRjaGluZyB0 byAxMCBtcyB3b3VsZCBiZSBiZXR0ZXIgaW4geW91ciBMVEUgdGVzdHMgPw0KPiANCj4gDQo+IA0K PiBPbiBUaHUsIE9jdCAxLCAyMDE1IGF0IDQ6NTcgQU0sIEluZ2VtYXIgSm9oYW5zc29uIFMgDQo+ IDxpbmdlbWFyLnMuam9oYW5zc29uQGVyaWNzc29uLmNvbT4gd3JvdGU6DQo+ID4gSG1tDQo+ID4g WW91IGFyZSByaWdodCAhLg0KPiA+IFdlbGwuLi4gVGhhdCBvZiBjb3Vyc2Ugc2hvdWxkIGV4cGxh aW4gd2h5IEh5U3RhcnQgcGVyZm9ybXMgYmFkbHkgZm9yIExURS4uLg0KPiBBdGxlYXN0IGl0IGlz IHBhcnQgb2YgdGhlIHJlYXNvbi4NCj4gPg0KPiA+IC9JbmdlbWFyDQo+ID4NCj4gPg0KPiA+DQo+ ID4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+IEZyb206IFZlYWNlc2xhdiBST01B TiBbbWFpbHRvOlZlYWNlc2xhdi5Sb21hbkBvcmFuZ2UubWRdDQo+ID4+IFNlbnQ6IGRlbiAxIG9r dG9iZXIgMjAxNSAxMzo0Ng0KPiA+PiBUbzogSW5nZW1hciBKb2hhbnNzb24gUzsgRXJpYyBEdW1h emV0OyBOZWFsIENhcmR3ZWxsDQo+ID4+IENjOiB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxv bjsgU2FuZ3RhZSBIYTsgRXJpYyBEdW1hemV0OyANCj4gPj4gZW5kMmVuZC0gaW50ZXJlc3RAcG9z dGVsLm9yZw0KPiA+PiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNo IGRlcGxveW1lbnQNCj4gPj4NCj4gPj4gV2VsbCB0aGlzIHNheQ0KPiA+PiAgICAgICBJZiBjdXJy X3J0dCBpcyBoaWdoZXIgdGhhbiBkZWxheV9taW4gKyAoMS84ICogZGVsYXlfbWluIG9yIDQgDQo+ ID4+IG1zLCB3aGljaGV2ZXIgaXMgaGlnaGVyKSAgdGhlbiBleGl0IHNsb3cgc3RhcnQuDQo+ID4+ DQo+ID4+IFdoZW4gYWN0dWFsIG1pbmltdW0gZGVsYXkgaXMgbGVzcyB0aGFuIDMyIG1zIHRoZSBk ZWxheV9taW4gd2lsbCBiZSANCj4gPj4gbGVzcyB0aGFuIDMyPDwzLCB0aGVuIHRoZSBkZWxheV9t aW4+PjMgd2lsbCBiZSBsZXNzIHRoYW4gMzIgYW5kIA0KPiA+PiB3aWxsIGNvbXBhcmUgd2l0aA0K PiA+PiA0PDwzIHdoaWNoIGlzIDMyIGFuZCB0aGVyZWZvciB0aGUgdGhyZXNob2xkIHdpbGwgYmUN Cj4gPj4gKHJlYWxfY3Vycl9ydHQ8PDMpID4NCj4gPj4gKHJlYWxfZGVsYXk8PDMgKyA0PDwzKSBv ciAocmVhbF9jdXJyX3J0dCk+KHJlYWxfZGVsYXkgKzQpDQo+ID4+DQo+ID4+DQo+ID4+IFZlYWNl c2xhdiBSb21hbg0KPiA+Pg0KPiA+PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+PiBG cm9tOiBJbmdlbWFyIEpvaGFuc3NvbiBTIFttYWlsdG86aW5nZW1hci5zLmpvaGFuc3NvbkBlcmlj c3Nvbi5jb21dDQo+ID4+IFNlbnQ6IFRodXJzZGF5LCAwMSBPY3RvYmVyIDIwMTUgMTM6MDANCj4g Pj4gVG86IFZlYWNlc2xhdiBST01BTjsgRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsDQo+ID4+ IENjOiB0Y3BtQGlldGYub3JnOyBQaWVycyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgRXJpYyBEdW1h emV0OyANCj4gPj4gZW5kMmVuZC0gaW50ZXJlc3RAcG9zdGVsLm9yZzsgSW5nZW1hciBKb2hhbnNz b24gUw0KPiA+PiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRl cGxveW1lbnQNCj4gPj4NCj4gPj4gSGkNCj4gPj4NCj4gPj4gWWVzLCB0aGUgdmFyaWFibGUgZGVs YXlfbWluIGlzIGluIFEzIGZvciByZWFzb25zIG9mIHByZWNpc2lvbiBJIA0KPiA+PiBndWVzcyBC dXQgdGhlIGNhbGwgdG8gSFlTVEFSVF9ERUxBWV9USFJFU0ggaXMgd2l0aCBkZWxheV9taW4gDQo+ ID4+IHNoaWZ0ZWQgZG93biB0byBRMA0KPiA+Pg0KPiA+PiBIWVNUQVJUX0RFTEFZX1RIUkVTSChj YS0+ZGVsYXlfbWluID4+IDMpKSAgKHNlZSBodHRwOi8vbHhyLmZyZWUtDQo+ID4+IGVsZWN0cm9u cy5jb20vc291cmNlL25ldC9pcHY0L3RjcF9jdWJpYy5jI0w0MDMgKSBJIG1heSBiZSBvZmYgaGVy ZS4uDQo+ID4+DQo+ID4+IFJlZ2FyZGluZyBwYWNpbmcuIEkgZG9uJ3QgaGF2ZSByZWFsIGRhdGEg dG8gc2hvdyB0aGUgZWZmZWN0cyBvZiANCj4gPj4gcGFjaW5nLCBpbiBzaW11bGF0aW9ucyBpdCBp cyBob3dldmVyIHBvc3NpYmxlIHRvIHNlZSB0aGUgZ2FpbnMgaW4gDQo+ID4+IHRlcm1zIG9mIGxl c3MgY29hbGVzY2luZywgeWVzIHBhY2luZyBkb2VzIG5vdCBzb2x2ZSB0aGUgQUNLIA0KPiA+PiBj b21wcmVzc2lvbiBidXQgaXQgc2VlbSB0byBzb2x2ZSB0aGUgZm9sbG93IHVwIGVmZmVjdCB0aGF0 IEFDSw0KPiBjb21wcmVzc2lvbiBnaXZlcy4NCj4gPj4NCj4gPj4gL0luZ2VtYXINCj4gPj4NCj4g Pj4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+PiA+IEZyb206IFZlYWNlc2xhdiBS T01BTiBbbWFpbHRvOlZlYWNlc2xhdi5Sb21hbkBvcmFuZ2UubWRdDQo+ID4+ID4gU2VudDogZGVu IDEgb2t0b2JlciAyMDE1IDEwOjMyDQo+ID4+ID4gVG86IEluZ2VtYXIgSm9oYW5zc29uIFM7IEVy aWMgRHVtYXpldDsgTmVhbCBDYXJkd2VsbA0KPiA+PiA+IENjOiB0Y3BtQGlldGYub3JnOyBQaWVy cyBPJ0hhbmxvbjsgU2FuZ3RhZSBIYTsgRXJpYyBEdW1hemV0Ow0KPiA+PiA+IGVuZDJlbmQtIGlu dGVyZXN0QHBvc3RlbC5vcmcNCj4gPj4gPiBTdWJqZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBI eVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4gPj4gPg0KPiA+PiA+IFllcywgSFlTVEFSVF9ERUxB WV9NSU4gICg0VTw8MykgLg0KPiA+PiA+IEJ1dCBjdXJyX3J0dCBhbmQgZGVsYXlfbWluIGNvbWUg ZnJvbSBiaWN0Y3BfYWNrZWQgd2hpY2ggaXMgYWxzbyA8PDMgOg0KPiA+PiA+ICAgICBJbiB0Y3Bf Y3ViaWMuYywgZnVuY3Rpb24gIGxpbmUgNDMzIChrZXJuZWwgNC4yKQ0KPiA+PiA+ICAgICAgICAg ICAgIGRlbGF5ID0gKHJ0dF91cyA8PCAzKSAvIFVTRUNfUEVSX01TRUM7IFNvLA0KPiA+PiA+IDRV PDwzIGlzIGVxdWl2YWxlbnQgdG8gNCBtcyAuDQo+ID4+ID4NCj4gPj4gPiBSZWdhcmRpbmcgcGFj aW5nIEkgYW0gbm90IHN1cmUgd2lsbCBoZWxwIHRoZSBwZXJmb3JtYW5jZTogdGhlcmUgDQo+ID4+ ID4gYXJlIGdhcHMgaW4gQUNLIHRyYWlucyBvZiB0aGUgb3JkZXIgb2YgbWlsbGlzZWNvbmRzIGFu ZCB0aGlzIGlzIGEgDQo+ID4+ID4gbG9zdCB0aW1lIGFuZCB0aGVyZSBpcyBubyB3YXkgdG8gY29t cGVuc2F0ZSBpdC4gSWYgaW50cm9kdWNlIEFDSyANCj4gPj4gPiBwYWNpbmcgb3IgZGF0YSBwYWNr ZXRzIHBhY2luZyB0aGlzIHdvdWxkIGFkZCB0byB0aGUgb3ZlcmFsbCBsb3N0IHRpbWUuDQo+ID4+ ID4gUGFjaW5nIGNhbiBoZWxwIG9ubHkgdG8gYWxsZXZpYXRlIGEgcG90ZW50aWFsIHByb2JsZW0g b2Ygc29tZSANCj4gPj4gPiBkb3duc3RyZWFtIHJvdXRlcnMgd2l0aCBpbnN1ZmZpY2llbnQgYnVm ZmVycyB3aGljaCBtYXkgaW50cm9kdWNlIA0KPiA+PiA+IHVuZXhwZWN0ZWQgcGFja2V0cyBkcm9w cyB3aGVuIGxhcmdlIHRyYWluIHNlbnQgYXQgYSBoaWdoIHNwZWVkIGFzIA0KPiA+PiA+IGEgcmVw bHkgdG8gaGlnaGx5IGNvbXByZXNzZWQgQUNLIHRyYWluLCBidXQsIG9uIGFub3RoZXIgaGFuZHMu DQo+ID4+ID4gQnV0IGhvdyBtdWNoIHBhY2luZyBhbmQgaG93IHRvIGFjY29tbW9kYXRlIHRvIGxh cmdlbHkgY2hhbmdpbmcgDQo+ID4+ID4gcmFkaW8gY29uZGl0aW9ucyA/IEFuZCB3aXRoIHBlcm1h bmVudGx5IGNoYW5naW5nIHRlY2hub2xvZ3kgYW5kDQo+IHNwZWVkID8NCj4gPj4gPiBBbnl3YXkg dGhlIHBhY2luZyB3aWxsIGFkZCB0byB0aGUgcHJvYmxlbSBvZiBnYXBzIGFuZCBiZXR0ZXIgc2l6 ZSANCj4gPj4gPiBwcm9wZXJseSByb3V0ZXJzIGJ1ZmZlcnMuDQo+ID4+ID4NCj4gPj4gPiBJZGVh bGx5IEFDSyB0cmFpbiBpcyBzcGxpdCBpbiAyIHBhcnRzLCBidXQgaW4gcHJhY3RpY2UgSSBkbyBz ZWUgbW9yZS4NCj4gPj4gPiBCdXQgYXMgdGhlIHN0cmVhbSBwcm9ncmVzcyB0aGUgbGVzcyBnYXBz LiBUaGUgYmlnZ2VyIHRoZSBkb3dubGluayANCj4gPj4gPiBwcmVzc3VyZSB0aGUgbW9yZSBBQ0sg YW5kIGxlc3MgY2hhbmNlcyB0byBoYXZlIGEgcGF1c2UgaW4gQUNLIA0KPiA+PiA+IHNlbmRpbmcs IHRoZW4gdGhlcmUgd2lsbCBiZSBhIGNvbnRpbnVvdXMgZmxvdyBvZiBCdWZmZXIgU3RhdHVzIA0K PiA+PiA+IFJlcG9ydHMgYW5kIGxlc3Mgb2YgdGhlIFNjaGVkdWxpbmcgUmVxdWVzdCAoU1IsICJI ZXksIEkgaGF2ZSBzb21lIA0KPiA+PiA+IGRhdGEgdG8gc2VuZCIpLiBUaGUgb25seSBpc3N1ZSBJ IGFtIG5vdCBzdXJlIGlzIG9uIHRoZSBwb3RlbnRpYWwgDQo+ID4+ID4gZ2FwcyBpbiBvdGhlciBu ZXR3b3Jrcy4gSSBkaWQgbm90IHB1dCBiZWxsb3cgYWxsIGRldGFpbHMsIGJ1dCB5b3UgDQo+ID4+ ID4gc2hvdWxkIGtub3cgdGhhdCBkZXZpY2UgY2FuIG5vdCBzZW5kIFNSIGluIGFyYml0cmFyeSBt b21lbnRzIG9mIA0KPiA+PiA+IHRpbWUgYnV0IGluIHByZWRlZmluZWQgZm9yIGhpbSBpbnRlcnZh bHMuIEluIG15IG5ldHdvcmsgdGhpcyANCj4gPj4gPiBwZXJpb2QgaXMgNSBtcywgdGhlcmVmb3Jl IHRoZSBSVFQgb2YgdGhlIGZpcnN0IEFDSyBpbiB0aGUgdHJhaW4gDQo+ID4+ID4gd2lsbCBoYXZl IHZhcmlhdGlvbnMgb2YgMi41IG1zLiBCdXQgc3RhbmRhcmRzIGFkbWl0IHRoaXMgcGVyaW9kIA0K PiA+PiA+IHRvIGJlIGFsc28gMTAgbXMgLDIwIG1zLCA0MCBtcywgODAgbXMgYW5kIEkgY2FuIG5v dCBzYXkgd2hhdCBpcyANCj4gPj4gPiB0aGUgcHJhY3RpY2Ugb2Ygb3RoZXJzLiBJIG1heSBvbmx5 IGd1ZXNzIHRoYXQgbW9zdCB3aWxsIHRyeSB0byANCj4gPj4gPiB1c2UNCj4gPj4gdGhlIGJlc3Qg YW5kIGZhc3Rlc3QsIGkuZS4gNSBtcy4NCj4gPj4gPiBBbnl3YXksIHRvIGdldCBhIHJlYWxpc3Rp YyBtb2RlbGluZyBvZiBMVEUgaW50ZXJhY3Rpb24gd2l0aCBUQ1AgDQo+ID4+ID4gdGhlIGRldGFp bGVkIE1BQyBzY2hlZHVsZXIgbW9kZWwgaXMgbmVlZGVkLiBVbmZvcnR1bmF0ZWx5LCBtb3N0IA0K PiA+PiA+IG9mIHRoZSBsaXRlcmF0dXJlIGZvY3VzIG9uIHNjaGVkdWxpbmcgb2YgdGhlIFBoeXNp Y2FsIFJlc291cmNlIA0KPiA+PiA+IEJsb2NrcywgQWRhcHRpdmUgTW9kdWxhdGlvbiBhbmQgQ29k aW5nLCBNSU1PLCBpLmUuLCBmb2N1c2luZyBvbiANCj4gPj4gPiB0aGUgZGlzdHJpYnV0aW9uIG9m IHJlc291cmNlcyBiZXR3ZWVuIHVzZXJzLCB3aGljaCBpcyBnb29kIGFuZCANCj4gPj4gPiBuZWVk ZWQsIGJ1dCBkbyBub3QgYWRkcmVzcyB2ZXJ5IG11Y2ggdGhlIHRpbWluZyBpbnRlcmFjdGlvbiB3 aXRoIA0KPiA+PiA+IFRDUC4gIEF0IGxlYXN0LCBJIG5ldmVyIG1ldCBpbiBMVEUgc2NoZWR1bGlu ZyBsaXRlcmF0dXJlIGEgdGVybSAiVENQIHNlbGYtY2xvY2tpbmciLg0KPiA+PiA+DQo+ID4+ID4g VGhlIDggbXMgIEhZU1RBUlRfREVMQVlfTUlOIGNvdWxkIG9ubHkgYmUgYSBzaW1wbGUgc29sdXRp b24gZm9yIA0KPiA+PiA+IHRoZSBwcm9ibGVtIG9mIGNvdW50aW5nIG9mIHRoZSByb3VuZCcgY3Vy cl9ydHQgZnJvbSB0aGUgc2Vjb25kIA0KPiA+PiA+IEFDSyBvZiB0aGUgdHJhaW4gKGFzIHRoZSBz ZWNvbmQgQUNLIFJUVCB3aWxsIGJlLCB0eXBpY2FsbHksIGR1ZSANCj4gPj4gPiB0byBzY2hlZHVs aW5nLA0KPiA+PiA+IDUtNyBtcyBsb25nZXIgdGhhbiB0aGUgZmlyc3Qgb25lKSwgYnV0IHRoaXMg b25seSBpZiBTUiBwZXJpb2RpY2l0eSBpcyA1IG1zLg0KPiA+PiA+IFRoZSBvdGhlciAyIHByb2Js ZW1zIHdpbGwgcmVtYWluIHVuc29sdmVkLg0KPiA+PiA+DQo+ID4+ID4gVmVhY2VzbGF2IFJvbWFu DQo+ID4+ID4NCj4gPj4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+PiA+IEZyb206 IEluZ2VtYXIgSm9oYW5zc29uIFMNCj4gW21haWx0bzppbmdlbWFyLnMuam9oYW5zc29uQGVyaWNz c29uLmNvbV0NCj4gPj4gPiBTZW50OiBUaHVyc2RheSwgMDEgT2N0b2JlciAyMDE1IDA4OjU5DQo+ ID4+ID4gVG86IFZlYWNlc2xhdiBST01BTjsgRXJpYyBEdW1hemV0OyBOZWFsIENhcmR3ZWxsDQo+ ID4+ID4gQ2M6IHRjcG1AaWV0Zi5vcmc7IFBpZXJzIE8nSGFubG9uOyBTYW5ndGFlIEhhOyBFcmlj IER1bWF6ZXQ7DQo+ID4+ID4gZW5kMmVuZC0gaW50ZXJlc3RAcG9zdGVsLm9yZzsgSW5nZW1hciBK b2hhbnNzb24gUw0KPiA+PiA+IFN1YmplY3Q6IFJFOiBbdGNwbV0gW2UyZV0gVENQIEh5U3RhcnQg cGF0Y2ggZGVwbG95bWVudA0KPiA+PiA+DQo+ID4+ID4gSGkNCj4gPj4gPg0KPiA+PiA+IFRoYW5r cyBmb3IgdGhlIGRldGFpbGVkIHJlcG9ydC4gV2hhdCBzZWVtcyB1bmNsZWFyIHRvIG1lIGlzIHRo ZSANCj4gPj4gPiB2YWx1ZSBvZiBIWVNUQVJUX0RFTEFZX01JTi4gWW91IG1lbnRpb24gOG1zIGJ1 dCB3aGVuIEkgbG9vayBpbnRvIA0KPiA+PiA+IHRoZSA0LjIga2VybmVsIGNvZGUgSSBnZXQgdGhl IGZlZWxpbmcgdGhhdCBpdCBpcyAzMm1zICggI2RlZmluZQ0KPiBIWVNUQVJUX0RFTEFZX01JTg0K PiA+PiA+ICg0VTw8MykgICApLiBJcyB0aGlzIGEgbWlzdGFrZSBmcm9tIG15IHNpZGUgID8NCj4g Pj4gPiBJIGhhdmUgcnVuIExURSBzaW11bGF0aW9ucyBhbmQgdHJpZWQgdG8gd3JpbmcgdGhpcyBI eVN0YXJ0IGlzc3VlIA0KPiA+PiA+IGluc2lkZSBvdXQgYW5kIHVwc2lkZSBkb3duIGJ1dCBzb2Zh ciBpdCBoYXMgYmVlbiB2ZXJ5IGRpZmZpY3VsdCANCj4gPj4gPiB0byBtYWtlIEh5U3RhcnQgZXhp dCBwcmVtYXR1cmVseSBidXQgdGhlbiBvZiBjb3Vyc2UgSSBoYXZlIHJ1biANCj4gPj4gPiB3aXRo IEhZU1RBUlRfREVMQVlfTUlOID0gMzJtcy4NCj4gPj4gPg0KPiA+PiA+IEFzIHJlZ2FyZHMgdG8g dGhlIEFDSy1jb21wcmVzc2lvbiBwcm9ibGVtLCB5ZXMsIEFDSyBjb21wcmVzc2lvbiANCj4gPj4g PiBlYXNpbHkgb2NjdXIgaW4gTFRFLCBldmVuIGluIHNpbXVsYXRpb25zLiBXaGF0IEkgaGF2ZSBz ZWVuIGlzIA0KPiA+PiA+IHRoYXQgcGFja2V0IHBhY2luZyBpcyBhIHZlcnkgZWZmaWNpZW50IHJl bWVkeSwgZm9yIGluc3RhbmNlIHRoZSANCj4gPj4gPiBwYWNrZXQgaW1wbGVtZW50ZWQgaW4gUVVJ QyBzZWVtcyB0byBzb2x2ZSBBQ0sgY29tcHJlc3Npb24gaXNzdWVzIHZlcnkgd2VsbC4NCj4gPj4g Pg0KPiA+PiA+IEl0IGlzIGludGVyZXN0aW5nIHdoYXQgeW91IHNheSB0aGF0IEFDSyB0cmFpbnMg YXJlIHNvIG11Y2ggc3BsaXQgDQo+ID4+ID4gdXAsIEkgY291bGQgdW5kZXJzdGFuZCB0aGF0IHRo ZXkgYXJlIHNwbGl0IHVwIGluIHR3byBwYXJ0cyANCj4gPj4gPiAoaW5pdGlhbCBncmFudCArIGFk ZGl0aW9uYWwgYWZ0ZXIgcmVjZWl2ZWQgQlNSKS4NCj4gPj4gPg0KPiA+PiA+IC9JbmdlbWFyDQo+ ID4+ID4NCj4gPj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+ID4gPiBGcm9t OiBWZWFjZXNsYXYgUk9NQU4gW21haWx0bzpWZWFjZXNsYXYuUm9tYW5Ab3JhbmdlLm1kXQ0KPiA+ PiA+ID4gU2VudDogZGVuIDEgb2t0b2JlciAyMDE1IDAyOjI1DQo+ID4+ID4gPiBUbzogRXJpYyBE dW1hemV0OyBOZWFsIENhcmR3ZWxsDQo+ID4+ID4gPiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMg TydIYW5sb247IFNhbmd0YWUgSGE7IEluZ2VtYXIgDQo+ID4+ID4gPiBKb2hhbnNzb24gUzsgRXJp YyBEdW1hemV0OyBlbmQyZW5kLWludGVyZXN0QHBvc3RlbC5vcmcNCj4gPj4gPiA+IFN1YmplY3Q6 IFJFOiBbdGNwbV0gW2UyZV0gVENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiA+PiA+ID4N Cj4gPj4gPiA+IEZpbmFsbHkgY2FuIHNoYXJlIHNvbWUgcmVzdWx0cyBvZiBIeXN0YXJ0IGludGVy YWN0aW9uIHdpdGggTFRFLg0KPiA+PiA+ID4gIEZldyB3b3JkcyBhYm91dCB0aGUgdGVzdGluZyBj b25maWd1cmF0aW9uOg0KPiA+PiA+ID4gICBEZXZpY2UgU2Ftc3VuZyBHYWxheHkgTm90ZTRMVEUg LCB1cCB0byAxNTAgTWJwcyBjYXBhYmxlDQo+ID4+ID4gPiAgICAgICAgICAgICAgUmFkaW8gTmV0 d29yayBMVEUgMjYwMC8xODAwDQo+ID4+ID4gPiAgICAgICAgICAgICAgRGlzdGFuY2UgYmV0d2Vl biB0aGUgQ29yZSBOZXR3b3JrIChFUEMpIGFuZCBCYXNlIA0KPiA+PiA+ID4gU3RhdGlvbiAtIDEw DQo+ID4+IGttDQo+ID4+ID4gPiAgICAgICAgICAgICAgSFRUUCBTZXJ2ZXIgY29ubmVjdGVkIGRp cmVjdGx5IHRvIHRoZSBDb3JlIE5ldHdvcmsgMSBHYnBzDQo+ID4+ID4gPiAgICAgICAgICAgICAg Q29yZSBuZXR3b3JrIHN3aXRjaGluZyhFUEMpIDEwIEdicHMNCj4gPj4gPiA+ICAgICAgICAgICAg ICBCYWNraGF1bCB0cmFuc3BvcnQgIDEgR2JwcywgZnVsbCBJUCwgTVBMUw0KPiA+PiA+ID4gICAg ICAgICAgICAgIExhc3QgbWlsZSB0cmFuc3BvcnQgZnVsbCBJUCwgTVBMUywgIDMwMCBNYnBzDQo+ ID4+ID4gPg0KPiA+PiA+ID4gICAgICAgICAgICAgU2VydmVyOiBMaW51eCBrNDBzcnYgNC4xLjAt MDQwMTAwLWdlbmVyaWMNCj4gPj4gPiA+ICMyMDE1MDcwMzA5NDAgU01QIEZyaSBKdWwgMw0KPiA+ PiA+ID4gMDk6NDE6NDcgVVRDIDIwMTUgeDg2XzY0IHg4Nl82NCB4ODZfNjQgR05VL0xpbnV4DQo+ ID4+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICBBcGFjaGUvMi40LjEwIChVYnVudHUpLCBi dWlsdDogICBKdWwgMjQgMjAxNSAxNzoyNToxOA0KPiA+PiA+ID4NCj4gPj4gPiA+ICAgICAgICAg ICAgSHlzdGFydF9kZXRlY3Q6IDIgKEhZU1RBUlRfREVMQVkgb25seSksIGFzIHN1Z2dlc3RlZC4N Cj4gPj4gPiA+ICAgICAgICAgICAgbmV0LmlwdjQudGNwX25vX21ldHJpY3Nfc2F2ZSA9IDEgdG8g YXZvaWQgdGNwIA0KPiA+PiA+ID4gbWV0cmljcyBjYWNoaW5nIGludGVyZmVyZW5jZQ0KPiA+PiA+ ID4NCj4gPj4gPiA+ICAgVGVzdHMgdHlwZXM6DQo+ID4+ID4gPiAgICAgICAgICAgMS4gRG93bmxv YWQgMyBNQiBmaWxlLCBpbnRlcmNoYW5naW5nIEh5c3RhcnQgT04NCj4gPj4gPiBhbmQgT0ZGLCBp bg0KPiA+PiA+ID4gY29uZGl0aW9ucyBvZiBMb3cgQmFuZHdpZHRoICgyNS0zNSBNYnBzLCBwb29y IHJhZGlvKSBhbmQgSGlnaCANCj4gPj4gPiA+IEJhbmR3aWR0aCAoMTAwLTEyMCBNYnBzLCBnb29k IHJhZGlvKS4NCj4gPj4gPiA+ICAgICAgICAgICAyLiBEb3dubG9hZCAxMCBNQiBmaWxlLCBpbnRl cmNoYW5naW5nIEh5c3RhcnQNCj4gPj4gPiBPTiBhbmQgT0ZGLCBpbg0KPiA+PiA+ID4gY29uZGl0 aW9ucyBvZiBMb3cgQmFuZHdpZHRoICgyNS0zNSBNYnBzLCBwb29yIHJhZGlvKSBhbmQgSGlnaCAN Cj4gPj4gPiA+IEJhbmR3aWR0aCAoMTAwLTEyMCBNYnBzLCBnb29kIHJhZGlvKS4NCj4gPj4gPiA+ DQo+ID4+ID4gPiAgIDEwMCBEb3dubG9hZCB0ZXN0cyBwZXIgdHlwZS4NCj4gPj4gPiA+ICAgICAg ICAgICAgICBEb3dubG9hZCBkdXJhdGlvbiBhbmQgYmFuZHdpZHRoIG1lYXN1cmVkIGF0IHRoZSAN Cj4gPj4gPiA+IGNsaWVudCBzaWRlIGFzIHNob3duIGJ5IGN1cmwgZnJvbSB0aGUgZmlyc3QgcmVj ZWl2ZWQgZGF0YSANCj4gPj4gPiA+IHBhY2tldCB0aWxsIHRoZSBlbmQgb2Ygc2Vzc2lvbiAodGhp cyBleGNsdWRlIEROUywgMyB3YXkgDQo+ID4+ID4gPiBoYW5kc2hha2UsIGh0dHAgR0VUIGFuZCBp dHMNCj4gPj4gQUNLKS4NCj4gPj4gPiA+ICAgICAgICAgICAgICBIeXN0YXJ0IGV4aXQgY3duZCBl eHRyYWN0ZWQgZnJvbSBuc3RhdCAodGhhbmsgeW91IA0KPiA+PiA+ID4gRXJpYyB0byBwb2ludGlu ZyBvdXQgdG8gaXQpLg0KPiA+PiA+ID4NCj4gPj4gPiA+IE92ZXJhbGwgcmVzdWx0czoNCj4gPj4g PiA+ICAgRG93bmxvYWQgM01CLCBMb3cgQmFuZHdpZHRoczoNCj4gPj4gPiA+ICAgICAgICAgICBI eXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS40MSBzDQo+ID4+ID4gPiAgICAgICAgICAg Tm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS4zNiBzDQo+ID4+ID4gPg0KPiA+PiA+ ID4gICBEb3dubG9hZCAzTUIsIEhpZ2ggQmFuZHdpZHRoczoNCj4gPj4gPiA+ICAgICAgICAgICBI eXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC42NSBzDQo+ID4+ID4gPiAgICAgICAgICAg Tm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC4zNyAgcw0KPiA+PiA+ID4NCj4gPj4g PiA+ICAgRG93bmxvYWQgMTBNQiwgTG93IEJhbmR3aWR0aHM6DQo+ID4+ID4gPiAgICAgICAgICAg SHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDIuNDAgcw0KPiA+PiA+ID4gICAgICAgICAg IE5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDIuMTIgcw0KPiA+PiA+ID4NCj4gPj4g PiA+ICAgRG93bmxvYWQgMTBNQiwgSGlnaCBCYW5kd2lkdGhzOg0KPiA+PiA+ID4gICAgICAgICAg IEh5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAxLjQyIHMNCj4gPj4gPiA+ICAgICAgICAg ICBOb0h5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAwLjg5ICBzDQo+ID4+ID4gPg0KPiA+ PiA+ID4gVGhlc2UgcmVzdWx0cyBzaG93IHRoYXQgSHlzdGFydCBoYXMgbm8gc2lnbmlmaWNhbnQg aW1wYWN0IGluIA0KPiA+PiA+ID4gY29uZGl0aW9ucyBvZiBsb3cgYXZhaWxhYmxlIGJhbmR3aWR0 aCwgYnV0IGF0IGhpZ2ggYXZhaWxhYmxlIA0KPiA+PiA+ID4gdGhlIGRlY3JlYXNlIG9mIHBlcmZv cm1hbmNlIGNhbiByZWFjaCA2MC03MCAlLg0KPiA+PiA+ID4gV2l0aCB0aGUgZGV2ZWxvcG1lbnQg b2YgdGhlIExURS1BIHdoaWNoIHVzZXMgbGFyZ2VyIHNwZWN0cnVtIHRvIA0KPiA+PiA+ID4gcmVh Y2gNCj4gPj4gPiA+IDMwMCBNYnBzIGFuZCBtb3JlIHRoZSBpbXBhY3Qgb2YgSHlzdGFydCB3aWxs IGJlY29tZSBldmVuIG1vcmUNCj4gdmlzaWJsZS4NCj4gPj4gPiA+DQo+ID4+ID4gPiBJIGRvIHNo YXJlIHdpdGggRXJpYyBhbmQgTmVhbCBzb21lIGZpbGVzIGRlc2NyaWJlZCBiZWxvdyBhbmQgDQo+ ID4+ID4gPiBhbHNvIGNhbiBwcm92aWRlIGxpbmtzIHRvIGFueW9uZSBpbnRlcmVzdGVkLg0KPiA+ PiA+ID4NCj4gPj4gPiA+IFN1bW1hcnkgb2YgdGhlc2UgdGVzdHMgcmVzdWx0cyBhcmUgZ2F0aGVy ZWQgaW4gdGhlIGZvbGxvd2luZyANCj4gPj4gPiA+IEV4Y2VsIGZpbGVzIChuYW1lcywgSSBiZWxp ZXZlLCBhcmUgc2VsZiBleHBsYW5hdG9yeSk6DQo+ID4+ID4gPiBUZXN0XzNNX2xvd19CVy54bHN4 LCBUZXN0XzNNX2hpZ2hfQlcueGxzeCwNCj4gVGVzdF8xME1fbG93X0JXLnhsc3gsDQo+ID4+ID4g VGVzdF8xME1faGlnaF9CVy54bHN4Lg0KPiA+PiA+ID4gU2ltaWxhcmx5IHRjcGR1bXAgdHJhY2Vz IGFyZSBjYWxsZWQgVGVzdF8zTV9sb3dfQlcucGNhcCwgDQo+ID4+ID4gPiBUZXN0XzNNX2hpZ2hf QlcuIHBjYXAsIFRlc3RfMTBNX2xvd19CVy4gcGNhcCwNCj4gPj4gVGVzdF8xME1faGlnaF9CVy4N Cj4gPj4gPiA+IHBjYXAgKGFibGUgdG8gY29sbGVjdCBvbmx5IHNlcnZlciBzaWRlIHRyYWNlcywg YnV0IHRoZXkgYXJlIA0KPiA+PiA+ID4gc2VsZg0KPiBzdWZmaWNpZW50KS4NCj4gPj4gPiA+IEZv ciB0aGUgMTBNIGRvd25sb2FkcyB0aGVyZSBhcmUgZmV3IG1pc3NpbmcgdHJhY2VzIGFzIHRoZXJl IA0KPiA+PiA+ID4gdHJhY2UgZmlsZXMgd2VyZSByZWN5Y2xlZC4gSW4gdHJhY2VzIGFyZSBhbHNv IHNzaCBzZXNzaW9ucywgDQo+ID4+ID4gPiBwbGVhc2UsIGlnbm9yZSB0aGVtLCB0aGV5IHdlcmUg dXNlZCB0byBwdXQgaHlzdGFydCBPTiBhbmQgT0ZGIA0KPiA+PiA+ID4gb24gc2VydmVyIGFuZCBl eHRyYWN0IG5zdGF0IHJlc3VsdHMgYmVmb3JlIGFuZCBhZnRlciBlYWNoIGRvd25sb2FkLg0KPiA+ PiA+ID4NCj4gPj4gPiA+IEV4Y2VsIGZpbGVzIGhhdmUgMiB0YWJzOiBjdXJsX3N1bSBhbmQgU3Vt bWFyeS4NCj4gPj4gPiA+IEluIHRoZSBjdXJsX3N1bSB0YWIgdGhlIHN0YXQgaXMgY3VtdWxhdGVk IGluIGNocm9ub2xvZ2ljYWwgDQo+ID4+ID4gPiBvcmRlciBvZg0KPiB0ZXN0cy4NCj4gPj4gPiA+ IEluIHRoZSBTdW1tYXJ5X3RhYiAgdGhlIEh5c3RhcnQgYW5kIE5vSHlzdGFydCB0ZXN0cyBhcmUg c2hvd24gDQo+ID4+ID4gPiBzaWRlIGJ5IHNpZGUgKGFzIHNhaWQsIERvd25sb2FkIDEgd2FzIHdp dGggSHlzdGFydCwgRG93bmxvYWQgMiANCj4gPj4gPiA+IHdpdGhvdXQsIERvd25sb2FkIDMgd2l0 aCBIeXN0YXJ0LCBEb3dubG9hZCA0IHdpdGhvdXQsIGV0Yy4pLiBJdCANCj4gPj4gPiA+IGFsc28g aW5jbHVkZSBncmFwaHMgb2YgVHJhbnNmZXIgdGltZSBhbmQgYSBncmFwaCB3aGljaCBzaG93cyAN Cj4gPj4gPiA+IHRoZSBkaXN0cmlidXRpb24gaW4gJSBvZiBkaWZmZXJlbnQgSHlzdGFydCBleGl0 IHdpbmRvdyBEZWxheUNXTkQuDQo+ID4+ID4gPg0KPiA+PiA+ID4gRmllbGRzOg0KPiA+PiA+ID4g Y29tbWVudHMgIC0gdGVzdCBjYXNlDQo+ID4+ID4gPiBUcmFuc2ZfVGltZSAgICAgICAtIGNhbGN1 bGF0ZWQgZnJvbSBjdXJsIG91dHB1dCB0aGUgZHVyYXRpb24NCj4gPj4gPiBiZXR3ZWVuIGZpcnN0 IGRhdGENCj4gPj4gPiA+IHBhY2tldCBhbmQgZW5kIG9mIHRyYW5zZmVyDQo+ID4+ID4gPiBUcmFu c2Zfc3BlZWQgICAgICAtIGNhbGN1bGF0ZWQgZnJvbSBjdXJsIG91dHB1dCB0aGUgcmF0aW8gb2Yg ZmlsZSBzaXplDQo+ID4+ID4gYW5kIGR1cmF0aW9uDQo+ID4+ID4gPiBiZXR3ZWVuIGZpcnN0IGRh dGEgcGFja2V0IGFuZCBlbmQgb2YgdHJhbnNmZXINCj4gPj4gPiA+IEh5c3RhcnQgICBEX0RlbGF5 IC0gIGh5c3RhcnQgZXhpdCBvY2N1cnJlbmNlIChkaWZmZXJlbmNlIGJldHdlZW4NCj4gPj4gPiA+ IFRjcEV4dFRDUEh5c3RhcnREZWxheURldGVjdCBiZWZvcmUgYW5kIGFmdGVyIGRvd25sb2FkKQ0K PiA+PiA+ID4gRF9EZWxheUN3bmQgICAgICAgICAtIEh5c3RhcnQgZXhpdCBjd25kIGZvciB0aGlz IGRvd25sb2FkDQo+ID4+ID4gPiAoZGlmZmVyZW5jZSBvZiBUY3BFeHRUQ1BIeXN0YXJ0RGVsYXlD d25kIGJlZm9yZSBhbmQgYWZ0ZXIgdGhlDQo+ID4+ID4gPiBkb3dubG9hZCkgRGF0ZV9Pbl9TZXJ2 ZXIgLSB0aW1lIGp1c3QgYmVmb3JlIHRoZSBkb3dubG9hZCBzdGFydCwgDQo+ID4+ID4gPiB0byBo ZWxwIG5hdmlnYXRlIHRoZSBwY2FwDQo+ID4+ID4gPg0KPiA+PiA+ID4gQWRkaXRpb25hbGx5LCBp biB0aGUgZmlsZSBUZXN0XzEwTV9sb3dfQlcueGxzeCBpbiB0aGUgdGFiIA0KPiA+PiA+ID4gY3Vy bF9zdW0gYXJlIGluY2x1ZGVkIGZvciBlYWNoIGRvd25sb2FkIHRjcC5zdHJlYW0gZXh0cmFjdCBm cm9tIA0KPiA+PiA+ID4gcGNhcCBvZiB0aGUgZmlyc3QNCj4gPj4gPiA+IDEwMCB0Y3AuYW5hbHlz aXMuYWNrX3J0dCB3aGljaCBtYXkgaGVscCBpbiB1bmRlcnN0YW5kaW5nIG9mIHRoZSANCj4gPj4g PiA+IGh5c3RhcnQgYmVoYXZpb3IgKGZpZWxkcyBhY2tfcnR0XzEgdG8gYWNrX3J0dF8xMDApLg0K PiA+PiA+ID4gSW4gdGhpcyBmaWxlIHRoZXJlIGFyZSBhbHNvIGZpZWxkcyBVUklfZnJhbWUsIFVS SV90aW1lLA0KPiBVUklfdGltZV9yZWxhdGl2ZSwNCj4gPj4gPiA+ICAgdGNwX3N0cmVhbS4NCj4g Pj4gPiA+IFRoZW4gdGhlcmUgYXJlIGNhbGN1bGF0ZWQgZmllbGRzOiBtaW5SVFQgYXMgYSBtaW5p bXVtIG9mIGFsbCANCj4gPj4gPiA+IDEwMCBBQ0ssDQo+ID4+IHIxXzUsDQo+ID4+ID4gPiByMl84 LCByM184LCAgICAgICByNF84IHdoaWNoIHNpbXVsYXRlIEh5c3RhcnQgY2FsY3VsYXRpb25zIG9m IHRoZQ0KPiA+PiByb3VuZCBjdXJyX3J0dA0KPiA+PiA+ID4gYW5kIHIxIG1pbiwgcjIgbWluLCBy MyBtaW4sIHI0IG1pbiByZXByZXNlbnQgbWluUlRUIG9mIHRoZSANCj4gPj4gPiA+IHdob2xlDQo+ IHJvdW5kLg0KPiA+PiA+ID4gUGxlYXNlLCBjb3JyZWN0IG1lIGlmIEkgYW0gd3JvbmcsIGJ1dCBp biBteSB1bmRlcnN0YW5kaW5nIGZvciANCj4gPj4gPiA+IGVhY2ggQUNLIHRoZSBoeXN0YXJ0X3Vw ZGF0ZSBpcyBjYWxsZWQgYmVmb3JlIHRoZSBoeXN0YXJ0X3Jlc2V0LCANCj4gPj4gPiA+IGFuZCBh cyBhIHJlc3VsdCB0aGUgY29tcHV0YXRpb24gb2YgY3VyX3J0dCBvZiB0aGUgOCBwYWNrZXRzIG9m IA0KPiA+PiA+ID4gdGhlIHJvdW5kIHN0YXJ0cyB3aXRoIHRoZSAyLW5kIHBhY2tldCBvZiB0aGUg cm91bmQsIG5vdCB0aGUgZmlyc3Qgb25lLg0KPiA+PiA+ID4gIEkndmUgdXNlIHIxXzUgYXMgYSBt aW4oYWNrX3J0dF83Li4uYWNrX3J0dF8xMSksIGR1ZSB0byANCj4gPj4gPiA+IGh5c3RhcnRfbG93 X3dpbmRvdz0xNiwgcjJfOCBhcyBhIG1pbihhY2tfcnR0XzEyIC4uLiANCj4gPj4gPiA+IGFja19y dHRfMTkpLA0KPiA+PiA+ID4gcjNfOCBhcyBhDQo+ID4+ID4gPiBtaW4oYWNrX3J0dF8zMiAuLi4g YWNrX3J0dF8zOSkgYW5kICksIHI0XzggYXMgYSBtaW4oYWNrX3J0dF83MiAuLi4NCj4gPj4gPiBh Y2tfcnR0Xzc5KS4NCj4gPj4gPiA+IFRvIHVuZGVyc3RhbmQgdGhlIGltcGFjdCBvZiB0aGlzIHRo ZXJlIGFyZSBhbHNvIHIxIEhPTCwgcjIgSE9MLA0KPiA+PiA+ID4gcjMgSE9MLCByNCBIT0wgKEhl YWQgT2YgTGluZSkgd2hpY2ggY29tcHV0ZSB0aGUgc2FtZSB3aXRoIGZpcnN0IA0KPiA+PiA+ID4g cGFja2V0IGluIHRoZSByb3VuZCBpbmNsdWRlZCBpbiB0aGUgcmVsZXZhbnQgcm91bmQ6ICByMV81 IGFzIGEgDQo+ID4+ID4gPiBtaW4oYWNrX3J0dF82Li4uYWNrX3J0dF8xMCksIHIyXzggYXMgYSBt aW4oYWNrX3J0dF8xMSAuLi4NCj4gPj4gPiA+IGFja19ydHRfMTgpLA0KPiA+PiA+ID4gcjNfOCBh cyBhIG1pbihhY2tfcnR0XzMxIC4uLiBhY2tfcnR0XzM4KSBhbmQgKSwgcjRfOCBhcyBhIA0KPiA+ PiA+ID4gbWluKGFja19ydHRfNzENCj4gLi4uDQo+ID4+ID4gYWNrX3J0dF83OCkuDQo+ID4+ID4g Pg0KPiA+PiA+ID4gSWYgbXkgdW5kZXJzdGFuZGluZyBvZiBIeXN0YXJ0IGlzIGNvcnJlY3QgYW5k LCBhc3N1bWluZyBpbiBlYWNoIA0KPiA+PiA+ID4gdHJhaW4gdGhlIGZpcnN0IHBhY2tldCBoYXMg dGhlIGxvd2VzdCBSVFQgd2hpY2gsIG1vc3QgcHJvYmFibHkgDQo+ID4+ID4gPiBpcyB2YWxpZCBp biBhbGwgdHlwZSBvZiBuZXR3b3JrcywgdGhlbiB0aGUgZXhpdCBmcm9tIGh5c3RhcnQgaW4gDQo+ ID4+ID4gPiB0aGlzIGltcGxlbWVudGF0aW9uIG1heSBoYXBwZW4gYXQgY3duZCAyOSwgNDksIDg5 LCAxNjksIGV0Yy4gDQo+ID4+ID4gPiAoZ2l2ZW4gdGhlIElXIG9mDQo+ID4+ID4gMTApLg0KPiA+ PiA+ID4gVGhlc2UgdGVzdHMgc2hvdyB0aGF0IHRoaXMgaXMgdGhlIGNhc2UsIGhvd2V2ZXIgdGhl cmUgYWxzbyANCj4gPj4gPiA+IGludGVybWVkaWF0ZSB2YWx1ZXMuIEhvd2V2ZXIgdGhlIGN3bmQg b2YgMjkgc2hhbGwgYmUgdGhlIGxvd2VzdCANCj4gPj4gPiA+IHBvc3NpYmxlIHZhbHVlIGFuZCB0 aGlzIGlzIHRoZSBjYXNlIGluIHRoZXNlIHRlc3RzLg0KPiA+PiA+ID4gVW5mb3J0dW5hdGVseSwg aW4gTFRFIHRoZSBmaXJzdCBwYWNrZXQgaW4gdGhlIHRyYWluIGhhcyBjaGFuY2VzIA0KPiA+PiA+ ID4gdG8gYWx3YXlzIGhhdmUgfjYgbXMgbG93ZXIgdGhhbiB0aGUgZm9sbG93aW5nIG9uZXMgYW5k LCBiZWNhdXNlIA0KPiA+PiA+ID4gaXQgaXMgbm90IGNvdW50ZWQgaW4gdGhlIDItbmQgdHJhaW4s IHRoZXJlIGlzIHF1aXRlIGhpZ2ggDQo+ID4+ID4gPiBwZXJjZW50YWdlIG9mIGV4aXQgYXQgMjkg KDE0LTM2JSBpbiB0aGVzZSB0ZXN0cyksIGV2ZW4gdGhvdWdoIA0KPiA+PiA+ID4gbGF0ZXIgb24g Y3ViaWMgaW5jcmVhc2VzIHRoZSBjd25kIHdpdGhvdXQgbG9zc2VzIHVwIHRvIG1hbnkgDQo+ID4+ ID4gPiBodW5kcmVkcy4gRm9yIG1lIHRoaXMgaXMgdG9vIGVhcmx5IGV4aXQgZnJvbQ0KPiA+PiBz bG93IHN0YXJ0Lg0KPiA+PiA+ID4gQXMgaXQgY2FuIGJlIHNlZW4gZnJvbSBjb21wYXJpc29uIG9m IHRoZSBzaW11bGF0ZWQgY2FsY3VsYXRpb24sIA0KPiA+PiA+ID4gaW5jbHVzaW9uIG9mIHRoZSBm aXJzdCBwYWNrZXQgaW4gdGhlIHJvdW5kIGN1cnJfcnR0IGNhbGN1bGF0aW9uIA0KPiA+PiA+ID4g d291bGQgZWxpbWluYXRlIHRoZSBjd25kIDI5IGFuZCBzaGlmdCB0byBjd25kIDQ5IG9yIDg5IG9y IGhpZ2hlci4NCj4gPj4gPiA+DQo+ID4+ID4gPiBCdXQgdGhlcmUgYXJlIGFsc28gaW50ZXJtZWRp YXRlIHZhbHVlcywgZS5nLiA2OS4gTG9va2luZyANCj4gPj4gPiA+IHRocm91Z2ggdHJhY2VzIG9u ZSBtYXkgc2VlIHRoYXQgdGhpcyBpcyBkdWUgdG8gImNvbXByZXNzZWQgQUNLIiANCj4gPj4gPiA+ IHdoaWNoIHJlY2VpdmVzIGEgdGlnaHRseSBwYWNrZWQgdHJhaW4gb2YgbWFueSBBQ0sgc28gdGhh dCBpbiANCj4gPj4gPiA+IHRyYWNlcyB0aGVyZSBhcmUgbm90IGRhdGEgcGFja2V0cyBzZW50IGlu IHR1cm4gYnkgc2VydmVyLiBMb29rcyANCj4gPj4gPiA+IHRvIG1lLCBiZWNhdXNlIGh5c3JhcnRf cmVzZXQgc2V0cyB0aGUgZW5kX3NlcSB0byBzbmRfbmV4dCBhbmQgDQo+ID4+ID4gPiB0aGUgc2Vy dmVyIGRpZCBub3QgbWFuYWdlZCB0byBzZW5kIHlldCBwYWNrZXRzIGluIHJlcGx5IHRvIHRoaXMg DQo+ID4+ID4gPiBoaWdoIHNwZWVkIEFDSyB0cmFpbiB0aGUgc25kX25leHQgcG9pbnRzIHRvIG11 Y2ggbG93ZXIgdmFsdWUgDQo+ID4+ID4gPiB0aGFuIHdvdWxkIG5vcm1hbGx5IGhhcHBlbiBhbmQg dGhpcyBhY3R1YWxseSBkZXN0cm95cyB0aGUgDQo+ID4+ID4gPiBjb3JyZWN0IGlkZW50aWZpY2F0 aW9uIG9mIHRoZSBib3JkZXJzIG9mIHRoZSByb3VuZCwgdGhlIHJvdW5kIA0KPiA+PiA+ID4gZmFp bHMgc29tZXdoZXJlIGluIHRoZSBtaWRkbGUgb2YgdGhlIHRyYWluIG5vdCBhdCBpdHMgYm9yZGVy LCANCj4gPj4gPiA+IHRoaXMgbWFrZXMgdmVyeSBsaWtlbHkgaW5jcmVhc2VkIGRlbGF5IGR1ZSB0 byBxdWV1ZWluZyBhbmQgDQo+ID4+ID4gPiBlYXJsaWVyIGV4aXQgZnJvbSBzbG93X3N0YXJ0IGFu ZCBpbnRlcm1lZGlhdGUgdmFsdWVzIG9mIGV4aXQgDQo+ID4+ID4gPiBjd25kIHdoZXJlDQo+ID4+ ID4gdGhleSBhcmUgbm90IGV4cGVjdGVkLg0KPiA+PiA+ID4NCj4gPj4gPiA+IEFuZCBsYXN0IGJ1 dCBub3QgbGVhc3Q6IHRoZXJlIGFyZSBnYXBzICg0LTcgbXMpIGluIEFDSyB0cmFpbnMuIA0KPiA+ PiA+ID4gT24gdGhlIHBlcmZlY3RseSBzZW50IElXIG9mIDEwIGRhdGEgcGFja2V0cyB0cmFpbiB0 aGUgdHlwaWNhbCANCj4gPj4gPiA+IGZvciBMVEUgd2lsbCBiZSB0byByZWNlaXZlIDEgb3IgMiBB Q0ssIHRoZW4gYSBnYXAgb2YgNC02IG1zIA0KPiA+PiA+ID4gdGhlbiBhbm90aGVyIGNvdXBsZSBv ZiBBQ0ssIHRoZW4gYW5vdGhlciBnYXAsIHRoZW4gNSBBQ0sgDQo+ID4+ID4gPiBjb21wcmVzc2Vk IHRvZ2V0aGVyIGluIGEgZnJhY3Rpb24gb2YgbWljcm9zZWNvbmQuIE9mIGNhdXNlLCB0aGUgDQo+ ID4+ID4gPiBuZXh0IHRyYWluIGZyb20gdGhlIHNlcnZlciB3aWxsIGNvbnRhaW4gdGhlIHNhbWUg Z2Fwcy4gVmVyeSANCj4gPj4gPiA+IHF1aWNrbHksIGUuZy4sIHRoZSBIZWFkIG9mIExpbmUNCj4g Pj4gPiA+IChIT0wpIG9mIHRoZSAzLWQgdHJhaW4gd2lsbCBmb2xsb3cgaW1tZWRpYXRlbHkgdGhl IHRhaWwgb2YgdGhlIA0KPiA+PiA+ID4gMi1uZCB0cmFpbiB3aGljaCB3aWxsIGxlYWQgdG8gcXVl dWVpbmcgaW4gdGhlIGVuZCByb3V0ZXIgDQo+ID4+ID4gPiAoYWN0dWFsbHkgdGhlIHJhZGlvIGJh c2Ugc3RhdGlvbiwgbXVjaCBlYXJsaWVyIHRoYW4gQkRQIA0KPiA+PiA+ID4gcmVhY2hlZCksIGJ1 dCB0aGlzIHdpbGwgbGVhZCB0byBpbmNyZWFzZSBvZiB0aGUgdHJhaW4gUlRUIGFuZCANCj4gPj4g PiA+IHRvbyB5ZWFybHkgZXhpdCBmcm9tIHNsb3cgc3RhcnQuIFdoeSB0b28gZWFybHkgZXhpdCA/ IEJlY2F1c2UsIA0KPiA+PiA+ID4gc2hvdWxkIHNlcnZlciBjb250aW51ZSBpbmNyZWFzaW5nIHRo ZSBzcGVlZCB0aGlzIHdvdWxkIHJlZHVjZSANCj4gPj4gPiA+IHF1aWNrZXIgdGhlIGdhcHMgYm90 aCBpbiBkb3dubGluayBhbmQgdXBsaW5rLCB0aGUgcHJlaWNlIGZvciANCj4gPj4gPiA+IHRoaXMg YmVpbmcgdGhlIGluY3JlYXNlIG9mIHRoZSBSVFQgKEkgZ3Vlc3MgZG91YmxlKSBhZ2FpbnN0IA0K PiA+PiA+ID4gd2hhdCBvbmUgd291bGQgbm9ybWFsbHkgc2VlDQo+ID4+ID4gaW4gdGhlIGVuZC10 by1lbmQgRXRoZXJuZXQgbmV0d29ya3MuDQo+ID4+ID4gPg0KPiA+PiA+ID4NCj4gPj4gPiA+IEkg ZG8gbGlrZSBpZGVhcyBvZiBIeXN0YXJ0LCBidXQgSSAgZG9uJ3Qga25vdyBob3cgY291bGQgaXQg YmUgDQo+ID4+ID4gPiBwb3NzaWJseSByZWNvbmNpbGVkIHdpdGggbXkgTFRFIHByb2JsZW1zLiAg V2l0aCB0aGUgaW5jcmVhc2Ugb2YgDQo+ID4+ID4gPiBMVEUgc3BlZWRzIHRoaXMgZWFybHkgZXhp dCBmcm9tIHNsb3cgc3RhcnQgd2lsbCBiZWNvbWUgZXZlbiANCj4gPj4gPiA+IG1vcmUgb2YgdGhl DQo+ID4+IHByb2JsZW0uDQo+ID4+ID4gPiBGb3IgTFRFLCBub3Qgc3VyZSBob3cgZ29vZCBhbmQg Z2VuZXJhbGx5IGFjY2VwdGFibGUsIHNvbHV0aW9uIA0KPiA+PiA+ID4gd291bGQgYmUsIHBvc3Np Ymx5LCB0byBpbmNyZWFzZSB0aGUgSFlTVEFSVF9ERUxBWV9NSU4gdG8gOCBtcywgDQo+ID4+ID4g PiB3aGljaCwgYXQgbGVhc3QgaW4gbXkgY2FzZSwgd291bGQgZGltaW5pc2ggdGhlIGVhcmxpZXIg ZXhpdCANCj4gPj4gPiA+IHNpZ25pZmljYW50bHksIGlmIEkgYmVsaWV2ZSBpbiB0aGVzZSB0ZXN0 cyByZXN1bHRzLg0KPiA+PiA+ID4gQW5vdGhlciBzb2x1dGlvbiwgd2hpY2ggSSBkb24ndCBsaWtl IHZlcnkgbXVjaCwgYnV0IGFzIEkga25vdyANCj4gPj4gPiA+IHNvbWUgb3BlcmF0b3JzIHVzZSwg d291bGQgYmUgdG8gaW5zZXJ0IGEga2luZCBvZiBUQ1AgDQo+ID4+ID4gPiBhY2NlbGVyYXRvciBi ZXR3ZWVuIHRoZSBJbnRlcm5ldCBhbmQgdGhlIG1vYmlsZSBuZXR3b3JrIHdoaWNoIA0KPiA+PiA+ ID4gd2lsbCBpbnRlcmNlcHQgYWxsIFRDUCBhbmQgd291bGQgc2VuZCBpdCB0byBtb2JpbGUgd2l0 aG91dCANCj4gPj4gPiA+IEh5c3RhcnQgYnV0IHRoaXMgd291bGQga2lsbCBhbnkgZW5kLXRvLWVu ZCBwcmluY2lwbGUgYW5kIHRoaXMgDQo+ID4+ID4gPiB3aWxsIG5vdCBiZSB0aGUgSW50ZXJuZXQN Cj4gYW55bW9yZS4NCj4gPj4gPiA+DQo+ID4+ID4gPg0KPiA+PiA+ID4gSWYgeW91IGhhdmUgdGlt ZSBhbmQgbmV2ZXIgbG9vayBjbG9zZWx5IHRvIExURSB0ZWNobm9sb2d5IEkgZG8gDQo+ID4+ID4g PiBhIHF1aWNrIHN1bW1hcnkgYmVsbG93Lg0KPiA+PiA+ID4NCj4gPj4gPiA+DQo+ID4+ID4gPiBM VEUsIFRDUCBzZWxmLWNsb2NraW5nIGFuZCBBQ0sgY29tcHJlc3Npb24uDQo+ID4+ID4gPg0KPiA+ PiA+ID4gU3BlbnQgc29tZSB0aW1lIHRvIHVuZGVyc3RhbmQgdGhlIGJhc2ljcyBvZiB3aGF0IGNv dWxkIGJlIHRoZSANCj4gPj4gPiA+IHJlYXNvbiBmb3IgYWxsIHRob3NlIHRpbWluZyBlZmZlY3Rz IGR1ZSB0byBMVEUuDQo+ID4+ID4gPiAgIEZpcnN0LCB1bmxpa2Ugd2lyZSBiYXNlZCB0cmFuc3Bv cnQgdGVjaG5vbG9naWVzLCBMVEUgKGJ1dCANCj4gPj4gPiA+IGFsc28NCj4gPj4gPiBVTVRTDQo+ ID4+ID4gPiBhbmQgZnV0dXJlIDVHKSBzZW5kcyBkYXRhLCBib3RoIHVwbGluayBhbmQgZG93bmxp bmsgaW4gDQo+ID4+ID4gPiBwcmVkZWZpbmVkIHRpbWVzbG90cyBjYWxsZWQgVHJhbnNtaXNzaW9u IFRpbWUgSW50ZXJ2YWwgKFRUSSkuIA0KPiA+PiA+ID4gSW4gVU1UUyB0aGlzIGlzIDJtcywgaW4g TFRFIGlzDQo+ID4+ID4gPiAxIG1zIGFuZCBpdCBzYXlzIHRoYXQgaW4gNUcgaXQgd2lsbCBiZSAw LjUgbXMuDQo+ID4+ID4gPiBUaGVzZSBUVEkgaGF2ZSBleGFjdCBib3JkZXJzIGFuZCB0aWdodGx5 IGZvbGxvdyBlYWNoIG90aGVyIGFuZCANCj4gPj4gPiA+IHJlcXVpcmUgZXhhY3Qgc3luY2hyb25p emF0aW9uIGJldHdlZW4gdGhlIG1vYmlsZSBkZXZpY2UgYW5kIA0KPiA+PiA+ID4gUmFkaW8gQmFz ZSBTdGF0aW9uIChjYWxsZWQgZU5vZGVCIGluIExURSkuIFRvIGNvcGUgd2l0aCB2YXJ5aW5nIA0K PiA+PiA+ID4gcmFkaW8gY29uZGl0aW9ucyB0aGUgc2VuZGluZyBkYXRhIGJhbmR3aWR0aCBpcyBt b2RpZmllZCBpbiBzdWNoIA0KPiA+PiA+ID4gYSB3YXkgdGhhdCB0byBrZWVwIGEgY29uc3RhbnQg cHJvYmFiaWxpdHkgb2YgZGF0YSBsb3NzIG9mIDEwJS4gDQo+ID4+ID4gPiBCYWQgcmFkaW8gLSBs ZXNzIGJhbmR3aWR0aCwgZ29vZA0KPiA+PiA+IHJhZGlvIC0gbW9yZSBiYW5kd2lkdGguDQo+ID4+ ID4gPiBUaGUgYmFuZHdpZHRoIGlzIG1vZGlmaWVkIGJ5IHBhY2tpbmcgbGVzcyBvciBtb3JlIGJp dHMgaW4gYSBUVEkgDQo+ID4+ID4gPiBvZiAxIG1zIGluDQo+ID4+ID4gTFRFLg0KPiA+PiA+ID4g QnV0IFRUSSByZW1haW4gdW5jaGFuZ2VkIG9mIGV4YWN0bHkgMSBtcy4gVGhhdCBtZWFucyB0aGF0 LCBhdCANCj4gPj4gPiA+IHRoZSBUQ1AvSVAgbGF5ZXIgdGhlIHByZWNpc2lvbiBvZiB0aGUgdGlt aW5nIGluZm9ybWF0aW9uIGZyb20gDQo+ID4+ID4gPiBkZXZpY2UgdG8gdGhlIG5ldHdvcmsgaXMg MSBtcy4gSW4gdGhlIGNhc2Ugb2YgYSBiYW5kd2lkdGggb2YgMSANCj4gPj4gPiA+IE1icHMgdGhl cmUgd2lsbCBiZSA4MyBJUCBwYWNrZXRzIHBlciBzZWNvbmQgb3Igb25lIHBhY2tldCBldmVyeSAx MiBtcy4NCj4gPj4gPiA+IEluIHRoaXMgY2FzZSwgcG9zc2libHkgdGhlIHJldHVybiBvZiBvbmUg QUNLIGV2ZXJ5IDEyIG1zIChldmVyeQ0KPiA+PiA+ID4gMTIgVFRJKSB3b3VsZCBiZSBtb3JlIG9y IGxlc3Mgc3VmZmljaWVudCB0byBrZWVwIGF0IGEgZ29vZCANCj4gPj4gPiA+IGxldmVsIHRoZSBU Q1Agc2VsZi1jbG9ja2luZyBtZWNoYW5pc20uIEJ1dCwgd2hlbiB0YWxraW5nIGFib3V0IA0KPiA+ PiA+ID4gMTAwIE1icHMgKG9yIDMwMA0KPiA+PiA+ID4gTWJwcykgdGhhdCBtZWFucyBvbmUgcGFj a2V0IChhbmQgb25lIEFDSykgZXZlcnkgMC4xMiBtcyBhbmQgDQo+ID4+ID4gPiB0aGVyZSBpcyBu byB3YXkgdG8gcHJvdmlkZSBpdCB3aXRoIGEgVFRJIG9mIDEgbXMuIFRoaXMgaXMgdGhlIA0KPiA+ PiA+ID4gcmVhc29uIGZvciBBQ0sgY29tcHJlc3Npb24gaW4gTFRFIGFuZCB0aGUgZnV0dXJlIDVH IHdoaWNoIA0KPiA+PiA+ID4gcHJvbWlzZXMgbW9yZSB0aGFuIDFHYnBzIHdpdGggaXRzIDAuNSBt cyBUVEkgd2lsbA0KPiA+PiA+IGJlIGluIGEgYmlnIGNvbmZsaWN0IHdpdGggVENQIHNlbGYtY2xv Y2tpbmcuDQo+ID4+ID4gPiAgIEJ1dCB0aGUgcHJvYmxlbSBkbyBub3Qgc3RvcHMgaGVyZS4gVGhl cmUgaXMgYSBzaWduaWZpY2FudGx5DQo+ID4+ID4gZGlmZmVyZW50DQo+ID4+ID4gPiBtZWNoYW5p c21zIG9mIHRyYW5zbWlzc2lvbiBpbiB0aGUgZG93bmxpbmsgKGZyb20gZU5vZGVCIHRvIA0KPiA+ PiA+ID4gbW9iaWxlDQo+ID4+ID4gPiBkZXZpY2UpIGFuZCBpbiB1cGxpbmsgKGZyb20gbW9iaWxl IGRldmljZSB0byB0aGUgZU5vZGVCKS4gVGhlIA0KPiA+PiA+ID4gZGlmZmVyZW5jZSBjb21lIGZy b20gdGhlIGZhY3QgdGhhdCB0aGUgc2NoZWR1bGVyIGZvciBib3RoIA0KPiA+PiA+ID4gZG93bmxp bmsgYW5kIGluIHVwbGluayBpcyBsb2NhdGVkIGluIHRoZSBlTm9kZUIuIEFzIG9mIG15IA0KPiA+ PiA+ID4gdW5kZXJzdGFuZGluZyB0aGUgcHVycG9zZSBvZiB0aGlzIGlzIHRvIHNpbXBsaWZ5IHRv IHRoZSBtYXhpbXVtIA0KPiA+PiA+ID4gdGhlIGRldmljZSBhbmQgc2F2ZSBpdHMNCj4gPj4gYmF0 dGVyeS4NCj4gPj4gPiA+IEFzIGEgcmVzdWx0IG9mIHRoaXMgZGVzaWduLCB0aGUgZU5vZGVCIGNh biBzZW5kIGluIGVhY2ggVFRJIA0KPiA+PiA+ID4gdG93YXJkIHRoZSBtb2JpbGUgZGV2aWNlIHdo ZW5ldmVyIGRhdGEgZXhpc3QgaW4gaXRzIGJ1ZmZlciANCj4gPj4gPiA+IChlTm9kZUIgaXMgYSBr aW5kIG9mIElQIHdpcmVsZXNzIHJvdXRlcikuIEJ1dCBmb3IgbW9iaWxlIA0KPiA+PiA+ID4gZGV2 aWNlLCBpbiBvcmRlciB0byBzZW5kIHRoZSBkYXRhIGluIHVwbGluaywgYWZ0ZXIgYSBwYXVzZSwg aXQgDQo+ID4+ID4gPiBoYXMgZmlyc3QgdG8gYXNrIHRoZSBuZXR3b3JrIHRvIHNjaGVkdWxlIGl0 LiBBbmQgbW9iaWxlIGhhcyBmb3IgDQo+ID4+ID4gPiB0aGlzIHJlcXVlc3QgKGFmdGVyIGEgcGF1 c2Ugb2YNCj4gPj4gPiA+IHNlbmRpbmcpIGp1c3Qgb25lIGJpdCwgYWN0dWFsbHkgaXMgbm90IGV2 ZW4gYSBkYXRhIGJpdCwgaXMgYSANCj4gPj4gPiA+IHNwZWNpYWwgcmFkaW8gc2lnbmFsIHdoaWNo IHRlbGxzICJIZXksIEkgaGF2ZSBzb21lIGRhdGEgaW4gdGhlIA0KPiA+PiA+ID4gYnVmZmVyIHRv IHNlbmQiLiBMZXRzIGxvb2sgYXQgaW5pdGlhbCB0cmFpbiBvZiBJVyBvZiAxMCBwYWNrZXRzLg0K PiA+PiA+ID4gSW4gZ29vZCByYWRpbyBjb25kaXRpb25zIHRoZSBlTm9kZUIgd2lsbCBmaXQgYWxs IDEwIHBhY2tldHMgaW4gDQo+ID4+ID4gPiAxIFRUSSBvZiAxIG1zIGFuZCBzZW5kIHRoZW0gYWxs IHRvIHRoZSBkZXZpY2UuIERldmljZSB3aWxsIA0KPiA+PiA+ID4gdW5wYWNrIHRoZW0gYW5kIGdl bmVyYXRlIDEwIEFDSyBhbmQgdGhlbiB0ZWxsIHRvIHRoZSBlTm9kZUIgDQo+ID4+ID4gPiAiSGV5 LCBJIGhhdmUgc29tZSBkYXRhIGluIHRoZSBidWZmZXIgdG8gc2VuZCIuIFRoZSBlTm9kZUIgaGFz IA0KPiA+PiA+ID4gbm90IGtub3dsZWRnZSBob3cgbXVjaCBkYXRhIHRoZSBtb2JpbGUgaGFzIHRv IHNlbmQsIHRoZXJlZm9yIA0KPiA+PiA+ID4gd2lsbCBhbGxvY2F0ZSBzb21lIG1pbmltdW0NCj4g Pj4gPiA+IGNhcGFjaXR5OiAiV2VsbCwgSSBnaXZlIHlvdSBhIGdyYW50IHRvIHNlbmQgdXAgdG8g MjAwIEJ5dGVzIGFuZCANCj4gPj4gPiA+IHlvdSBtdXN0IHNlbmQgdGhlbSBleGFjdGx5IGFmdGVy IDQgbXMgeW91IGdldCB0aGlzIG5vdGlmaWNhdGlvbiIuDQo+ID4+ID4gPiBUaGVzZQ0KPiA+PiA+ ID4gNCBtcyBhcmUgYWxsb3dhbmNlIGZvciBtb2JpbGUgZGV2aWNlIHRvIHByZXBhcmUgZGF0YSBm b3IgDQo+ID4+ID4gPiBzZW5kaW5nIHdoaWNoIGlzIGEgdmVyeSBjb21wbGljYXRlZCBwcm9jZXNz LCB0aGVzZSA0IG1zIGlzIGEgDQo+ID4+ID4gPiBtdXN0IGRlbGF5IGJldHdlZW4gdGhlIGdyYW50 DQo+ID4+ID4gYW5kIHRoZSB0cmFuc21pc3Npb24uDQo+ID4+ID4gPiBJZiBtb2JpbGUgaGFzIGlu IGl0cyBidWZmZXIgMTAgQUNLIHRvIHNlbmQgaXQgd2lsbCBzZW5kIG9ubHkgMiANCj4gPj4gPiA+ IEFDSyBhbmQgd2lsbCBjb21wbGVtZW50IHRoZW0gd2l0aCwgc28gY2FsbGVkLCBidWZmZXIgc3Rh dHVzDQo+ID4+ID4gPiByZXBvcnQ6ICJJIGRvIGhhdmUgc3RpbGwgNDgwIGJ5dGVzIHRvIHNlbmQi LiBOb3cgbmV0d29yayBoYXMgDQo+ID4+ID4gPiBtb3JlIGluZm9ybWF0aW9uIGFuZCBhbGxvY2F0 ZSBhcw0KPiA+PiA+ID4gcmVxdWVzdGVkOiAiV2VsbCwgSSBnaXZlIHlvdSBhIGdyYW50IHRvIHNl bmQgdXAgdG8gMjAwIEJ5dGVzIA0KPiA+PiA+ID4gYW5kIHlvdSBtdXN0IHNlbmQgdGhlbSBleGFj dGx5IGFmdGVyIDQgbXMgeW91IGdldCB0aGlzIG5vdGlmaWNhdGlvbiIuDQo+ID4+ID4gPiBCdXQs IGluIHRoZSBtZWFuIHRpbWUgc29tZSBvdGhlciBpbmZvcm1hdGlvbiBvZiBoaWdoZXIgcHJpb3Jp dHkgDQo+ID4+ID4gPiBtYXkgYXBwZWFyIGluIHRoZQ0KPiA+PiA+IGRldmljZSAoZS5nLg0KPiA+ PiA+ID4gc2lnbmFsaW5nKSBhbmQgdGhlIGRldmljZSB3aWxsIHNlbmQgb25seSA2IG91dCBvZiA4 IHJlbWFpbmluZyANCj4gPj4gPiA+IEFDSyB0b2dldGhlciB3aXRoIHNpZ25hbGluZyBhbmQgYW5v dGhlciBCdWZmZXIgU3RhdHVzIFJlcG9ydDogDQo+ID4+ID4gPiAiSGV5LCBJIGRvIGhhdmUgYW5v dGhlciAxMjAgYnl0ZXMgdG8gc2VuZCIuIEFuZCBhZ2FpbiAiV2VsbCwgSSANCj4gPj4gPiA+IGdp dmUgeW91IGEgZ3JhbnQgdG8gc2VuZCB1cCB0byAxMjAgQnl0ZXMgYW5kIHlvdSBtdXN0IHNlbmQg dGhlbSANCj4gPj4gPiA+IGV4YWN0bHkgYWZ0ZXIgNCBtcyB5b3UgZ2V0IHRoaXMgbm90aWZpY2F0 aW9uIi4gQXMgYSByZXN1bHQgdGhlIA0KPiA+PiA+ID4gc2VydmVyIHdpbGwgcmVjZWl2ZSAyIEFD SywgdGhlbiBhZnRlciBhIGdhcCBvZiA2IG1zIGFub3RoZXIgNiANCj4gPj4gPiA+IEFDSywgdGhl biwgYWZ0ZXIgYW5vdGhlciA2IG1zLCB0aGUgcmVtYWluaW5nIDIgQUNLLiBXaXRoIHRoZSANCj4g Pj4gPiA+IHNjaGVkdWxlciBpbiB0aGUgZU5vZGVCIHRoZXNlIGdhcHMgYXJlIGluZXZpdGFibGUu IEFuZCBvbmUgbW9yZSANCj4gPj4gPiA+IHRoaW5rOiBtb2JpbGUgcGFjayB0b2dldGhlciB0aG9z ZSA2IEFDSyBpbiAxIFRUSSBvZiAxIG1zIGFuZCANCj4gPj4gPiA+IHNlbmQgdGhlbSwgdGhlbiB0 aGUgZU5vZGVCIHVucGFjayB0aGVtIGFuZCB0aGVyZSBpcyBubyB3YXkgdG8gDQo+ID4+ID4gPiBy ZWNvdmVyIHRoZSBzcGFjaW5nDQo+IG9mIDAuMTIgbXMgYmV0d2VlbiB0aGVtLg0KPiA+PiA+ID4g ZU5vZGVCIHNpbXBseSBzZW5kIHRoZW0gdG8gdGhlIHNlcnZlciBiYWNrIHRvIGJhY2sgYXQgdGhl IHNwZWVkIA0KPiA+PiA+ID4gb2YgaXRzIGRhdGEgY2FyZCB3aGljaCBpcyB0eXBpY2FsbHkgMSBH YnBzIGFuZCB3ZSBmYWNlIHRoZSBBQ0sgDQo+ID4+ID4gPiBjb21wcmVzc2lvbi4gQWZ0ZXIgdGhl IGxhc3QgMiBBQ0sgd2VyZSBzZW50IHRoZXJlIGlzIG5vdGhpbmcgDQo+ID4+ID4gPiBlbHNlIHRv IHNlbmQsIHNvIHRoZXJlIGlzIG5vIEJ1ZmZlciBTdGF0dXMgUmVwb3J0LCB0aGVyZWZvcmUsIA0K PiA+PiA+ID4gYWZ0ZXIgcGFja2V0cyBvZiB0aGUgbmV4dCB0cmFpbiBhcnJpdmUgYW5kIEFDSyBh cmUgZ2VuZXJhdGVkIA0KPiA+PiA+ID4gZXZlcnl0aGluZyBzdGFydHMgYWdhaW4gd2l0aCBvbmUg Yml0IG9mIGluZm9ybWF0aW9uICJIZXksIEkgDQo+ID4+ID4gPiBoYXZlDQo+ID4+ID4gc29tZSBk YXRhIGluIHRoZSBidWZmZXIgdG8gc2VuZCIuDQo+ID4+ID4gPiBUaGF0J3Mgd2h5IHRoZXJlIGFy ZSBnYXBzIGFuZCBBQ0sgY29tcHJlc3Npb24sIGxvc3MsIG9yIG1vcmUgDQo+ID4+ID4gPiBleGFj dGx5LCBsYWNrIG9mIGFueSB0aW1pbmcgaW5mb3JtYXRpb24gd2hpY2ggd291bGQgYmUgdXNlZnVs IA0KPiA+PiA+ID4gZm9yIHRoZSBUQ1Agc2VsZi0NCj4gPj4gPiBjbG9ja2luZy4NCj4gPj4gPiA+ DQo+ID4+ID4gPiBUaGFuayB5b3UgYW5kIHNvcnJ5IGZvciB0YWtpbmcgeW91ciB0aW1lLg0KPiA+ PiA+ID4NCj4gPj4gPiA+DQo+ID4+ID4gPiBWZWFjZXNsYXYgUm9tYW4NCj4gPj4gPiA+DQo+ID4+ ID4gPg0KPiA+PiA+ID4NCj4gPj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+ ID4gPiBGcm9tOiBWZWFjZXNsYXYgUk9NQU4NCj4gPj4gPiA+IFNlbnQ6IFNhdHVyZGF5LCAwNSBT ZXB0ZW1iZXIgMjAxNSAwMToxMA0KPiA+PiA+ID4gVG86ICdFcmljIER1bWF6ZXQnOyBOZWFsIENh cmR3ZWxsDQo+ID4+ID4gPiBDYzogdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0 YWUgSGE7IEluZ2VtYXIgDQo+ID4+ID4gPiBKb2hhbnNzb24gUzsgRXJpYyBEdW1hemV0OyBlbmQy ZW5kLWludGVyZXN0QHBvc3RlbC5vcmcNCj4gPj4gPiA+IFN1YmplY3Q6IFJFOiBbdGNwbV0gW2Uy ZV0gVENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiA+PiA+ID4NCj4gPj4gPiA+IElmIHdl IGxvb2sgYXQgdGhlIGZpcnN0IHRyYWluOiBhbGwgcGFja2V0cyByZWNlaXZlZCBpbiBsZXNzIHRo YW4gMSBtcy4NCj4gPj4gPiA+IFByb2JhYmx5IHRoaXMgaXMgb25seSBhbiBhcHBlYXJhbmNlIGFz IExURSB0cmFuc21pdHMgaW4sIHNvIA0KPiA+PiA+ID4gY2FsbGVkIHRyYW5zbWlzc2lvbiB0aW1l IGludGVydmFsIChUVEkpIG9mIDEgbXMsIGFuZCB3aGF0IHdlIA0KPiA+PiA+ID4gc2VlIGhlcmUg aXMgdGhhdCBhbGwgMTAgcGFja2V0cyBvZiBpbml0aWFsIHdpbmRvdyBmaXR0ZWQgaW4gMSANCj4g Pj4gPiA+IG1zLCBhbmQsIHdoZW4gZGVjb2RlZCwgd2VyZSBwcmVzZW50ZWQgdG8gdGhlIFRDUC9J UCBsYXllciAoYW5kIA0KPiA+PiA+ID4gcGNhcCkgYWxsDQo+IGF0IG9uY2UuDQo+ID4+ID4gPiBC VFcsIDggcGFja2V0cyBvZiAxMzAyMiBieXRlcyBpbiAxIG1zIG1lYW5zIGluc3RhbnRhbmVvdXMg c3BlZWQgDQo+ID4+ID4gPiBvZg0KPiA+PiA+ID4gMTA0IE1icHMsIGdvb2QgcmFkaW8gY29uZGl0 aW9ucy4gVENQIGdlbmVyYXRlcyAxMCBBQ0sgdHJhaW4gb2YgDQo+ID4+ID4gPiB0aGUgZHVyYXRp b24gb2YNCj4gPj4gPiA+IDEuMiBtcy4gV2lsbCBpdCBiZSBhIEZhc3QgRXRoZXJuZXQgcG9zc2li bHkgd2Ugd291bGQgY29uc2lkZXIgDQo+ID4+ID4gPiB0aGlzIG5vcm1hbCwNCj4gPj4gPiBpc24n dCBpdCA/DQo+ID4+ID4gPiBJIGxvb2tlZCBhdCBob3cgc2VydmVyIHJlcGx5IHRvIHRoZXNlIDEw IEFDS3MuIFRoZXJlIGFyZSAzIGdhcHMgDQo+ID4+ID4gPiBvZiA0LA0KPiA+PiA+ID4gMiBhbmQg NSBtcyBpbiB0aGUgcmVwbHkgdHJhaW4gYW5kIHRoZSB0b3RhbCB0cmFpbiBvZiAxOCAoPywgaXQg DQo+ID4+ID4gPiBzaG91bGQgYmUgMjApIHBhY2tldHMgcmVhY2hlcyBhIGR1cmF0aW9uIG9mIDE0 IG1zLiBBRkFJSyBMVEUgDQo+ID4+ID4gPiBtYXkgaW50cm9kdWNlIGdhcHMgaW4gQUNLIHRyYWlu IGR1ZSB0byB1cGxpbmsgc2NoZWR1bGluZyBtZWNoYW5pc20uDQo+ID4+ID4gPiBNYXkgYmUgdGhl c2UgZ2FwcyB0cmlnZ2VyIEh5c3RhcnQgZWFybHkgZXhpdCA/DQo+ID4+ID4gPg0KPiA+PiA+ID4g VmVhY2VzbGF2IFJvbWFuDQo+ID4+ID4gPiBUZWNobmljYWwgYW5kIElUIGRpcmVjdG9yDQo+ID4+ ID4gPiBPcmFuZ2UgTW9sZG92YSBTLkEuDQo+ID4+ID4gPiBGaXg6ICszNzMyMjU3NTQwMA0KPiA+ PiA+ID4gTW9iOiArMzczNjkxOTg0MDANCj4gPj4gPiA+IEZheDogKzM3MzIyNTc1MzA2DQo+ID4+ ID4gPg0KPiA+PiA+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4gPiA+IEZyb206 IEVyaWMgRHVtYXpldCBbbWFpbHRvOmVyaWMuZHVtYXpldEBnbWFpbC5jb21dDQo+ID4+ID4gPiBT ZW50OiBGcmlkYXksIDA0IFNlcHRlbWJlciAyMDE1IDE5OjQ4DQo+ID4+ID4gPiBUbzogTmVhbCBD YXJkd2VsbA0KPiA+PiA+ID4gQ2M6IFZlYWNlc2xhdiBST01BTjsgdGNwbUBpZXRmLm9yZzsgUGll cnMgTydIYW5sb247IFNhbmd0YWUgSGE7IA0KPiA+PiA+ID4gSW5nZW1hciBKb2hhbnNzb24gUzsg RXJpYyBEdW1hemV0OyBlbmQyZW5kLWludGVyZXN0QHBvc3RlbC5vcmcNCj4gPj4gPiA+IFN1Ympl Y3Q6IFJlOiBbdGNwbV0gW2UyZV0gVENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiA+PiA+ ID4NCj4gPj4gPiA+IE9uIEZyaSwgMjAxNS0wOS0wNCBhdCAxMTozMiAtMDQwMCwgTmVhbCBDYXJk d2VsbCB3cm90ZToNCj4gPj4gPiA+ID4gSGkgVmVhY2VzbGF2LA0KPiA+PiA+ID4gPg0KPiA+PiA+ ID4gPiBJIGFncmVlIHRoYXQgaW4geW91ciBMVEUgdHJhY2VzIGl0IGxvb2tzIGxpa2UgQ1VCSUMg SHlzdGFydCANCj4gPj4gPiA+ID4gaXMgZXhpdGluZyBzbG93IHN0YXJ0IHRvbyBlYXJseS4NCj4g Pj4gPiA+ID4NCj4gPj4gPiA+ID4gU0luY2UgeW91IHNlZW0gdG8gaGF2ZSBhIG5pY2UgTFRFIHRl c3RiZWQsIHdvdWxkIHlvdSBiZSBhYmxlIA0KPiA+PiA+ID4gPiB0byBkbyBzb21lIGV4cGVyaW1l bnRzIHRvIGZpbmQgYSBzZXQgb2YgcGFyYW1ldGVycyBmb3IgDQo+ID4+ID4gPiA+IEh5c3RhcnQg dGhhdCB3b3JrIGJldHRlciBmb3IgeW91ciBMVEUgZW52aXJvbm1lbnQ/IEZvciANCj4gPj4gPiA+ ID4gZXhhbXBsZSwgeW91IG1pZ2h0IHRyeSB0aGUgdHdvIHZhcmlhdGlvbnMgSSBzdWdnZXN0ZWQg ZWFybGllciBpbiB0aGUgdGhyZWFkOg0KPiA+PiA+ID4gPg0KPiA+PiA+ID4gPiAgICAgICAgICAg ICAgICAgICAgICAgICBpZiAoY2EtPmN1cnJfcnR0ID4gY2EtPmRlbGF5X21pbiArDQo+ID4+ID4g PiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICANCj4gPj4gPiA+ID4gSFlTVEFSVF9ERUxB WV9USFJFU0goY2EtPmRlbGF5X21pbg0KPiA+PiA+ID4gPiA+Pg0KPiA+PiA+ID4gPiAyKSkgeyBv cg0KPiA+PiA+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICBpZiAoY2EtPmN1cnJfcnR0ID4g Y2EtPmRlbGF5X21pbiArDQo+ID4+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAN Cj4gPj4gPiA+ID4gSFlTVEFSVF9ERUxBWV9USFJFU0goY2EtPmRlbGF5X21pbg0KPiA+PiA+ID4g PiA+Pg0KPiA+PiA+ID4gPiAxKSkgew0KPiA+PiA+ID4gPg0KPiA+PiA+ID4gPiBEbyBhbnkgb2Yg dGhvc2UgZ2l2ZSBiZXR0ZXIgcmVzdWx0cyBmb3IgeW91ciB0ZXN0cz8NCj4gPj4gPiA+ID4NCj4g Pj4gPiA+ID4gbmVhbA0KPiA+PiA+ID4NCj4gPj4gPiA+IEFsc28sIGh5c3RhcnQgaXMgZm9vbGVk IGJ5IHRvbyBtYW55IEFDSyByZWNlaXZlZCBpbiBzaG9ydCBwZXJpb2QuDQo+ID4+ID4gPg0KPiA+ PiA+ID4gVGhpcyBwcm9ibGVtIHdvdWxkIGJlIHNvbHZlZCBpZiBHUk8gd2FzIHVzZWQgYXQgcmVj ZWl2ZXIsIGFzIA0KPiA+PiA+ID4gbGVzcyBBQ0sgd291bGQgYmUgc2VudC4NCj4gPj4gPiA+DQo+ ID4+ID4gPiBQcmVzdW1hYmx5IHJlY2VpdmVyIGlzIG5vdCBhIGxpbnV4IFRDUCBzdGFjayA/DQo+ ID4+ID4gPg0KPiA+PiA+ID4gTWF5YmUgd2Ugc2hvdWxkIGFkZCBhIGxvZ2ljIGluIGh5c3RhcnRf dXBkYXRlKCkgdG8gdGFrZSBvbmUgQUNLIA0KPiA+PiA+ID4gcGVyDQo+IG1zLg0KPiA+PiA+ID4N Cj4gPj4gPiA+IDA1OjMyOjI0LjEzNDE3MiBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45 NS4xNzguMjA0LjgwOg0KPiA+PiA+ID4gRmxhZ3MgW1NdLCBzZXEgNDI5NDQyMzIxNSwgd2luIDY1 NTM1LCBvcHRpb25zIFttc3MgDQo+ID4+ID4gPiAxNDYwLHNhY2tPSyxUUyB2YWwNCj4gPj4gPiA+ IDExMTIzNTEgZWNyIDAsbm9wLHdzY2FsZSA4XSwgbGVuZ3RoIDANCj4gPj4gPiA+IDA1OjMyOjI0 LjE2MTk4NiBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0Og0KPiA+ PiA+ID4gRmxhZ3MgW1MuXSwgc2VxIDExOTg4Nzc3MjEsIGFjayA0Mjk0NDIzMjE2LCB3aW4gMjg5 NjAsIG9wdGlvbnMgDQo+ID4+ID4gPiBbbXNzIDE0MTYsc2Fja09LLFRTIHZhbA0KPiA+PiA+ID4g MjgwODgwODYzNCBlY3IgMTExMjM1MSxub3Asd3NjYWxlIDddLCBsZW5ndGggMA0KPiA+PiA+ID4g MDU6MzI6MjQuMTYyMTA4IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQu ODA6DQo+ID4+ID4gPiBGbGFncyBbLl0sIGFjayAxLCB3aW4gMzQzLCBvcHRpb25zIFtub3Asbm9w LFRTIHZhbCAxMTEyMzU0IGVjciANCj4gPj4gPiA+IDI4MDg4MDg2MzRdLCBsZW5ndGggMA0KPiA+ PiA+ID4gMDU6MzI6MjQuMTYzMDcyIElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3 OC4yMDQuODA6DQo+ID4+ID4gPiBGbGFncyBbUC5dLCBzZXEgMTo2NTgsIGFjayAxLCB3aW4gMzQz LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbA0KPiA+PiA+ID4gMTExMjM1NCBlY3IgMjgwODgwODYz NF0sIGxlbmd0aCA2NTcNCj4gPj4gPiA+IDA1OjMyOjI0LjIwMzk4NCBJUCAxOTUuOTUuMTc4LjIw NC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0Og0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgNjU4 LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAyODA4ODA4Njc1IA0KPiA+PiA+ID4g ZWNyIDExMTIzNTRdLCBsZW5ndGggMA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwMjc1IElQIDE5NS45 NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbUC5d LCBzZXEgMTozODcsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgDQo+ID4+ ID4gPiB2YWwNCj4gPj4gPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMzg2DQo+ ID4+ID4gPiAwNTozMjoyNC4yMTAzMTUgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMuMTA0 LjE1Ni4zMjkyNDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgc2VxIDM4NzoxNzkxLCBhY2sgNjU4LCB3 aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIA0KPiA+PiA+ID4gdmFsDQo+ID4+ID4gPiAyODA4 ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPj4gPiA+IDA1OjMyOjI0LjIxMDMz MiBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0Og0KPiA+PiA+ID4g RmxhZ3MgWy5dLCBzZXEgMTc5MTozMTk1LCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3As bm9wLFRTIA0KPiA+PiA+ID4gdmFsDQo+ID4+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwg bGVuZ3RoIDE0MDQNCj4gPj4gPiA+IDA1OjMyOjI0LjIxMDM0NyBJUCAxOTUuOTUuMTc4LjIwNC44 MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0Og0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBzZXEgMzE5NTo0 NTk5LCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIFtub3Asbm9wLFRTIA0KPiA+PiA+ID4gdmFs DQo+ID4+ID4gPiAyODA4ODA4Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPj4gPiA+ IDA1OjMyOjI0LjIxMDM1MCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0 LjgwOg0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgMzg3LCB3aW4gMzQ3LCBvcHRpb25zIFtub3As bm9wLFRTIHZhbCAxMTEyMzU5IA0KPiA+PiA+ID4gZWNyIDI4MDg4MDg2NzddLCBsZW5ndGggMA0K PiA+PiA+ID4gMDU6MzI6MjQuMjEwMzYxIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEw NC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbLl0sIHNlcSA0NTk5OjYwMDMsIGFjayA2NTgs IHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgDQo+ID4+ID4gPiB2YWwNCj4gPj4gPiA+IDI4 MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEw Mzc0IElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6DQo+ID4+ID4g PiBGbGFncyBbLl0sIHNlcSA2MDAzOjc0MDcsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25v cCxub3AsVFMgDQo+ID4+ID4gPiB2YWwNCj4gPj4gPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRd LCBsZW5ndGggMTQwNA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwMzg5IElQIDE5NS45NS4xNzguMjA0 LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbLl0sIHNlcSA3NDA3 Ojg4MTEsIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgW25vcCxub3AsVFMgDQo+ID4+ID4gPiB2 YWwNCj4gPj4gPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBsZW5ndGggMTQwNA0KPiA+PiA+ ID4gMDU6MzI6MjQuMjEwNDAyIElQIDE5NS45NS4xNzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYu MzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbLl0sIHNlcSA4ODExOjEwMjE1LCBhY2sgNjU4LCB3aW4g MjM3LCBvcHRpb25zIA0KPiA+PiA+ID4gW25vcCxub3AsVFMgdmFsDQo+ID4+ID4gPiAyODA4ODA4 Njc3IGVjciAxMTEyMzU0XSwgbGVuZ3RoIDE0MDQNCj4gPj4gPiA+IDA1OjMyOjI0LjIxMDQxNiBJ UCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0Og0KPiA+PiA+ID4gRmxh Z3MgWy5dLCBzZXEgMTAyMTU6MTE2MTksIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlvbnMgDQo+ID4+ ID4gPiBbbm9wLG5vcCxUUyB2YWwNCj4gPj4gPiA+IDI4MDg4MDg2NzcgZWNyIDExMTIzNTRdLCBs ZW5ndGggMTQwNA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwNDE4IElQIDk0LjI0My4xMDQuMTU2LjMy OTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6DQo+ID4+ID4gPiBGbGFncyBbLl0sIGFjayAxNzkxLCB3 aW4gMzU4LCBvcHRpb25zIFtub3Asbm9wLFRTIHZhbCAxMTEyMzU5IA0KPiA+PiA+ID4gZWNyIDI4 MDg4MDg2NzddLCBsZW5ndGggMA0KPiA+PiA+ID4gMDU6MzI6MjQuMjEwNDMxIElQIDE5NS45NS4x NzguMjA0LjgwID4gOTQuMjQzLjEwNC4xNTYuMzI5MjQ6DQo+ID4+ID4gPiBGbGFncyBbLl0sIHNl cSAxMTYxOToxMzAyMywgYWNrIDY1OCwgd2luIDIzNywgb3B0aW9ucyANCj4gPj4gPiA+IFtub3As bm9wLFRTIHZhbA0KPiA+PiA+ID4gMjgwODgwODY3NyBlY3IgMTExMjM1NF0sIGxlbmd0aCAxNDA0 DQo+ID4+ID4gPiAwNTozMjoyNC4yMTA0NTUgSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUu OTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDMxOTUsIHdpbiAzNjksIG9w dGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgDQo+ID4+ID4gPiBlY3IgMjgwODgwODY3N10s IGxlbmd0aCAwDQo+ID4+ID4gPiAwNTozMjoyNC4yMTA2ODUgSVAgOTQuMjQzLjEwNC4xNTYuMzI5 MjQgPiAxOTUuOTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDQ1OTksIHdp biAzODAsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgDQo+ID4+ID4gPiBlY3IgMjgw ODgwODY3N10sIGxlbmd0aCAwDQo+ID4+ID4gPiAwNTozMjoyNC4yMTA3MzEgSVAgOTQuMjQzLjEw NC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgYWNr IDYwMDMsIHdpbiAzOTEsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgDQo+ID4+ID4g PiBlY3IgMjgwODgwODY3N10sIGxlbmd0aCAwDQo+ID4+ID4gPiAwNTozMjoyNC4yMTA5MzcgSVAg OTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDoNCj4gPj4gPiA+IEZsYWdz IFsuXSwgYWNrIDc0MDcsIHdpbiA0MDIsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkg DQo+ID4+ID4gPiBlY3IgMjgwODgwODY3N10sIGxlbmd0aCAwDQo+ID4+ID4gPiAwNTozMjoyNC4y MTEwNzggSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44MDoNCj4gPj4g PiA+IEZsYWdzIFsuXSwgYWNrIDg4MTEsIHdpbiA0MTMsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFs IDExMTIzNTkgDQo+ID4+ID4gPiBlY3IgMjgwODgwODY3N10sIGxlbmd0aCAwDQo+ID4+ID4gPiAw NTozMjoyNC4yMTEyMjggSVAgOTQuMjQzLjEwNC4xNTYuMzI5MjQgPiAxOTUuOTUuMTc4LjIwNC44 MDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgYWNrIDEwMjE1LCB3aW4gNDI0LCBvcHRpb25zIFtub3As bm9wLFRTIHZhbCAxMTEyMzU5IA0KPiA+PiA+ID4gZWNyIDI4MDg4MDg2NzddLCBsZW5ndGggMA0K PiA+PiA+ID4gMDU6MzI6MjQuMjExMzc3IElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1 LjE3OC4yMDQuODA6DQo+ID4+ID4gPiBGbGFncyBbLl0sIGFjayAxMTYxOSwgd2luIDQzNSwgb3B0 aW9ucyBbbm9wLG5vcCxUUyB2YWwgMTExMjM1OSANCj4gPj4gPiA+IGVjciAyODA4ODA4Njc3XSwg bGVuZ3RoIDANCj4gPj4gPiA+IDA1OjMyOjI0LjIxMTU0NCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjky NCA+IDE5NS45NS4xNzguMjA0LjgwOg0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgMTMwMjMsIHdp biA0NDYsIG9wdGlvbnMgW25vcCxub3AsVFMgdmFsIDExMTIzNTkgDQo+ID4+ID4gPiBlY3IgMjgw ODgwODY3N10sIGxlbmd0aCAwDQo+ID4+ID4gPiAwNTozMjoyNC4yMzc5OTkgSVAgMTk1Ljk1LjE3 OC4yMDQuODAgPiA5NC4yNDMuMTA0LjE1Ni4zMjkyNDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgc2Vx IDEzMDIzOjE0NDI3LCBhY2sgNjU4LCB3aW4gMjM3LCBvcHRpb25zIA0KPiA+PiA+ID4gW25vcCxu b3AsVFMgdmFsDQo+ID4+ID4gPiAyODA4ODA4NzA5IGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQN Cj4gPj4gPiA+IDA1OjMyOjI0LjIzODA1MyBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4x MDQuMTU2LjMyOTI0Og0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBzZXEgMTQ0Mjc6MTU4MzEsIGFjayA2 NTgsIHdpbiAyMzcsIG9wdGlvbnMgDQo+ID4+ID4gPiBbbm9wLG5vcCxUUyB2YWwNCj4gPj4gPiA+ IDI4MDg4MDg3MDkgZWNyIDExMTIzNTldLCBsZW5ndGggMTQwNA0KPiA+PiA+ID4gMDU6MzI6MjQu MjM4MDkwIElQIDk0LjI0My4xMDQuMTU2LjMyOTI0ID4gMTk1Ljk1LjE3OC4yMDQuODA6DQo+ID4+ ID4gPiBGbGFncyBbLl0sIGFjayAxNDQyNywgd2luIDQ1Nywgb3B0aW9ucyBbbm9wLG5vcCxUUyB2 YWwgMTExMjM2MiANCj4gPj4gPiA+IGVjciAyODA4ODA4NzA5XSwgbGVuZ3RoIDANCj4gPj4gPiA+ IDA1OjMyOjI0LjIzODE2NCBJUCA5NC4yNDMuMTA0LjE1Ni4zMjkyNCA+IDE5NS45NS4xNzguMjA0 LjgwOg0KPiA+PiA+ID4gRmxhZ3MgWy5dLCBhY2sgMTU4MzEsIHdpbiA0NjgsIG9wdGlvbnMgW25v cCxub3AsVFMgdmFsIDExMTIzNjIgDQo+ID4+ID4gPiBlY3IgMjgwODgwODcwOV0sIGxlbmd0aCAw DQo+ID4+ID4gPiAwNTozMjoyNC4yNDQwMjQgSVAgMTk1Ljk1LjE3OC4yMDQuODAgPiA5NC4yNDMu MTA0LjE1Ni4zMjkyNDoNCj4gPj4gPiA+IEZsYWdzIFsuXSwgc2VxIDE1ODMxOjE3MjM1LCBhY2sg NjU4LCB3aW4gMjM3LCBvcHRpb25zIA0KPiA+PiA+ID4gW25vcCxub3AsVFMgdmFsDQo+ID4+ID4g PiAyODA4ODA4NzEzIGVjciAxMTEyMzU5XSwgbGVuZ3RoIDE0MDQNCj4gPj4gPiA+IDA1OjMyOjI0 LjI0NDA2MyBJUCAxOTUuOTUuMTc4LjIwNC44MCA+IDk0LjI0My4xMDQuMTU2LjMyOTI0Og0KPiA+ PiA+ID4gRmxhZ3MgWy5dLCBzZXEgMTcyMzU6MTg2MzksIGFjayA2NTgsIHdpbiAyMzcsIG9wdGlv bnMgDQo+ID4+ID4gPiBbbm9wLG5vcCxUUyB2YWwNCj4gPj4gPiA+IDI4MDg4MDg3MTMgZWNyIDEx MTIzNTldLCBsZW5ndGggMTQwNA0KPiA+PiA+ID4NCj4gPg0K From nobody Fri Oct 2 10:05:59 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 54FBC1B2F68 for ; Fri, 2 Oct 2015 10:05:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.909 X-Spam-Level: X-Spam-Status: No, score=-6.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L05uKyep1OEk for ; Fri, 2 Oct 2015 10:05:53 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 40C9F1B2F67 for ; Fri, 2 Oct 2015 10:05:52 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 1F362BFF21515; Fri, 2 Oct 2015 17:05:48 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t92H5oO7004660 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 2 Oct 2015 19:05:50 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Fri, 2 Oct 2015 19:05:50 +0200 From: "Scharf, Michael (Michael)" To: Kuhn Nicolas , "Lane Wigley (lwigley)" , "tcpm@ietf.org" Thread-Topic: TCP evolution impact on router buffers Thread-Index: AdD3qDJMG7WkdcJDTiy1iIy/H6510wDXEZsQAB6kmpAABLWoIABn/A/A Date: Fri, 2 Oct 2015 17:05:49 +0000 Message-ID: <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <655C07320163294895BBADA28372AF5D484D799E@FR712WXCHMBA15.zeu.alcatel-lucent.com> <46616f47b42841f0bb7a2afb8bdeac98@XCH-ALN-009.cisco.com> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.39] Content-Type: multipart/alternative; boundary="_000_655C07320163294895BBADA28372AF5D484F17F6FR712WXCHMBA15z_" MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] TCP evolution impact on router buffers X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2015 17:05:58 -0000 --_000_655C07320163294895BBADA28372AF5D484F17F6FR712WXCHMBA15z_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Regarding Section 7 in [2], I am surprised that a potential impact of CIR/P= IR is not mentioned. Unfortunately, I have only a very limited understandin= g of MPLS/IP router architectures, but the way bursts hit an AQM could IMHO= depend on that. As [2] is mentioned, one question related to the discussion below. From [2]= : 3.2. Buffer size The size of the buffers should be carefully chosen, and is to be set to the bandwidth-delay product; the bandwidth being the bottleneck capacity and the delay the largest RTT in the considered network. This means of the order of 10GB of buffer size for a 400G router interface,= right? Wouldn't it make sense to relax the statement "... and is to be set= ..." a bit? Michael From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] Sent: Wednesday, September 30, 2015 6:41 PM To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Hi, In [1], the authors evaluate the performance of CUBIC in small buffer regim= e. However, they use the Linux kernel version 2.6.18 (where the IW was 3 an= d not 10). Also, for any tests on CUBIC, small buffers and IW3/10, you may = be interested in looking at the scenario "7. Burst absorption" of the AQM = Characterization Guidelines of the AQM WG [2]. Concerning TCP pacing, there= is an on-going document on a pacing solution in the slow-start that you ma= y want to have a look and comment [3]. I hope this helps, Cheers, Nicolas [1] Jain, S.; Raina, G., "An experimental evaluation of CUBIC TCP in a smal= l buffer regime," in Communications (NCC), 2011 National Conference on , vo= l., no., pp.1-5, 28-30 Jan. 2011 doi: 10.1109/NCC.2011.5734779 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&is= number=3D5734693 [2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08 [3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01 De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley= ) Envoy=E9 : mercredi 30 septembre 2015 15:10 =C0 : Scharf, Michael (Michael); tcpm@ietf.org Objet : Re: [tcpm] TCP evolution impact on router buffers This paper from 2008 shows no drops with a 5 msec buffer on a heavily-conge= sted core link. Low levels of drops start to show up around 2.5 msec. This = was conducted at Level 3 to test the BDP / sqrt # flows proposal from Stanf= ord. Experimental Study of Router Buffer Sizing - Neda Beheshti, Yashar Ganjali,= Monia Ghobadi, Nick McKeown, and Geoff Salmon They also explored TCP pacing as a way to further reduce buffers. I'm tryin= g to assess if CUBIC results in some pacing effect by decoupling the window= growth from acks. - Lane From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Sent: Tuesday, September 29, 2015 6:56 PM To: Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers This question may also be in scope of the AQM WG, and perhaps ICCRG, but si= nce the communities overlap, I avoid cross-posting... Unfortunately, I don't have any recent pointer or own data. But I am aware = of examples for router buffer dimensioning based on tests with a single TCP= connection, using Reno. In those examples, the dimension of the buffers an= d possibly other related parameters (two-color/three-color meters/policiers= , WRED slopes, H-QoS, etc.) are configured to achieve full "line" speed for= a single TCP connection. As to be expected, this typically comes down to t= he old BDP rule of thumb. But it is rather obvious that a single bulk data = Reno connection is not necessarily a realistic workload these days. So, the= test methodology can be one relevant aspect of the dimensioning problem. I believe that a modern TCP stack with a high-speed congestion control such= as CUBIC will typically need less buffer than the outcome of this dimensio= ning method and the publications 10 years ago. As far as I know, most moder= n TCP stacks operate well over a very wide range of buffer sizes, and the b= uffers can be relatively small. To me, the key question is the latency-thro= ughput tradeoff, and there may not be a one-fits-all answer to that one. Personally, I'd also be interested in more recent pointers on router buffer= dimensioning. I could even see some value in documenting dimensioning guid= elines used in practice, if there was a way to generalize them. Michael From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley= ) Sent: Friday, September 25, 2015 5:41 PM To: tcpm@ietf.org Subject: [tcpm] TCP evolution impact on router buffers I'm trying to track down some research or opinions on how the more recent T= CP schemes impact router buffer needs. There are a number of publications f= rom Stanford and Georgia Tech from about 10 years ago, and I'm trying to as= sess how changes to the algorithms (e.g. CUBIC) and parameters (initial con= gestion window) deployed since then may influence those findings in the dir= ection of more or less buffering being needed. I'd appreciate any input and pointers. Thanks. - Lane --_000_655C07320163294895BBADA28372AF5D484F17F6FR712WXCHMBA15z_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Regarding Section 7 in= [2], I am surprised that a potential impact of CIR/PIR is not mentioned. U= nfortunately, I have only a very limited understanding of MPLS/IP router ar= chitectures, but the way bursts hit an AQM could IMHO depend on that.

 

As [2] is mentioned, o= ne question related to the discussion below. From [2]:

 

3.2.  Buffer size=

 

   The size = of the buffers should be carefully chosen, and is to be set

   to the ba= ndwidth-delay product; the bandwidth being the bottleneck=

   capacity = and the delay the largest RTT in the considered network.<= /p>

 

This means of the orde= r of 10GB of buffer size for a 400G router interface, right? Wouldn’t= it make sense to relax the statement “… and is to be set…= ;” a bit?

 

Michael

 

 

From: Kuhn Nic= olas [mailto:Nicolas.Kuhn@cnes.fr]
Sent: Wednesday, September 30, 2015 6:41 PM
To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org<= br> Subject: RE: TCP evolution impact on router buffers

 

Hi,

 =

In [1], the authors eval= uate the performance of CUBIC in small buffer regime. However, they use the= Linux kernel version 2.6.18 (where the IW was 3 and not 10). Also, for any= tests on CUBIC, small buffers and IW3/10, you may be interested in looking at the scenario “7.  Burst abs= orption” of the AQM Characterization Guidelines of the AQM WG [2]. Co= ncerning TCP pacing, there is an on-going document on a pacing solution in = the slow-start that you may want to have a look and comment [3].

 =

I hope this helps, =

Cheers,

 =

Nicolas=

 = ;

[1] Jain, S.; Raina, G.,= "An experimental evaluation of CUBIC TCP in a small buffer regime,&qu= ot; in Communications (NCC), 2011 National Conference on , vol., no., pp.1-= 5, 28-30 Jan. 2011
doi: 10.1109/NCC.2011.5734779
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&= amp;isnumber=3D5734693

[2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08

[3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01

 = ;

De : tcpm [mailto:t= cpm-bounces@ietf.org] De la part de Lane Wigley (lwigley)
Envoy=E9 : mercredi 30 septembre 2015 15:10
=C0 : Scharf, Michael (Michael); tcpm@ietf.org
Objet : Re: [tcpm] TCP evolution impact on router buffers<= /o:p>

 

This paper from 2008 s= hows no drops with a 5 msec buffer on a heavily-congested core link. Low le= vels of drops start to show up around 2.5 msec. This was conducted at Level= 3 to test the BDP / sqrt # flows proposal from Stanford.

 

Experimental Study of Router Buffer Sizing – N= eda Beheshti, Yashar Ganjali, Monia Ghobadi, Nick McKeown, and Geoff Salmon=

 

They also explored TCP pacing as a way to further re= duce buffers. I’m trying to assess if CUBIC results in some pacing ef= fect by decoupling the window growth from acks.

 

- Lane

 

 

From: Scharf, = Michael (Michael) [mai= lto:michael.scharf@alcatel-lucent.com]
Sent: Tuesday, September 29, 2015 6:56 PM
To: Lane Wigley (lwigley); tcpm@iet= f.org
Subject: RE: TCP evolution impact on router buffers

 

This question may also= be in scope of the AQM WG, and perhaps ICCRG, but since the communities ov= erlap, I avoid cross-posting…

 

Unfortunately, I don&#= 8217;t have any recent pointer or own data. But I am aware of examples for = router buffer dimensioning based on tests with a single TCP connection, usi= ng Reno. In those examples, the dimension of the buffers and possibly other related parameters (two-color/three-colo= r meters/policiers, WRED slopes, H-QoS, etc.) are configured to achieve ful= l “line” speed for a single TCP connection. As to be expected, = this typically comes down to the old BDP rule of thumb. But it is rather obvious that a single bulk data Reno connection= is not necessarily a realistic workload these days. So, the test methodolo= gy can be one relevant aspect of the dimensioning problem.

 

I believe that a moder= n TCP stack with a high-speed congestion control such as CUBIC will typical= ly need less buffer than the outcome of this dimensioning method and the pu= blications 10 years ago. As far as I know, most modern TCP stacks operate well over a very wide range of buffer= sizes, and the buffers can be relatively small. To me, the key question is= the latency-throughput tradeoff, and there may not be a one-fits-all answe= r to that one.

 

Personally, I’d = also be interested in more recent pointers on router buffer dimensioning. I= could even see some value in documenting dimensioning guidelines used in p= ractice, if there was a way to generalize them.

 

Michael

 

 

 

From: tcpm [mailto:tcpm-= bounces@ietf.org] On Behalf Of Lane Wigley (lwigley)
Sent: Friday, September 25, 2015 5:41 PM
To: tcpm@ietf.org
Subject: [tcpm] TCP evolution impact on router buffers

 

I’m trying to track down some research or opin= ions on how the more recent TCP schemes impact router buffer needs. There a= re a number of publications from Stanford and Georgia Tech from about 10 ye= ars ago, and I’m trying to assess how changes to the algorithms (e.g. CUBIC) and parameters (initial congestion window) = deployed since then may influence those findings in the direction of more o= r less buffering being needed.

 

I’d appreciate any input and pointers. Thanks.=

 

- Lane

 

 

--_000_655C07320163294895BBADA28372AF5D484F17F6FR712WXCHMBA15z_-- From nobody Sat Oct 3 04:49:12 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A97091B3636 for ; Sat, 3 Oct 2015 04:49:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 3.413 X-Spam-Level: *** X-Spam-Status: No, score=3.413 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, FM_FORGED_GMAIL=0.622, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, RCVD_IN_DNSWL_LOW=-0.7, RELAY_IS_203=0.994, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eaFIvPfFk_dD for ; Sat, 3 Oct 2015 04:49:10 -0700 (PDT) Received: from mail.sfc.wide.ad.jp (shonan.sfc.wide.ad.jp [203.178.142.130]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 59B141B3635 for ; Sat, 3 Oct 2015 04:49:10 -0700 (PDT) Received: from mail-ob0-f171.google.com (mail-ob0-f171.google.com [209.85.214.171]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 0CAF42780C5 for ; Sat, 3 Oct 2015 20:49:09 +0900 (JST) Received: by obcgx8 with SMTP id gx8so99507391obc.3 for ; Sat, 03 Oct 2015 04:49:07 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.182.241.41 with SMTP id wf9mr11720206obc.76.1443872947492; Sat, 03 Oct 2015 04:49:07 -0700 (PDT) Received: by 10.202.200.9 with HTTP; Sat, 3 Oct 2015 04:49:07 -0700 (PDT) Date: Sat, 3 Oct 2015 04:49:07 -0700 Message-ID: From: Yoshifumi Nishida To: "tcpm@ietf.org" Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "tcpm-chairs@tools.ietf.org" Subject: [tcpm] Agenda requests for TCPM at IETF-94 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2015 11:49:11 -0000 Hi, According to the preliminary agenda, we will have a 2.5 hours slot at Thursday (11/5) morning (9:00-11:30) If you are planning to present something in the meeting, please send request to tcpm-chairs with the following information: * Draft name / presentation title * Requested time * Speaker Thanks, -- Yoshi & Pasi & Michael From nobody Mon Oct 5 15:07:19 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0FA2D1A21C4 for ; Mon, 5 Oct 2015 15:07:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.844 X-Spam-Level: ** X-Spam-Status: No, score=2.844 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FREEMAIL_FROM=0.001, FRT_BELOW2=2.154, J_CHICKENPOX_36=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CA1OaT6mc2Sc for ; Mon, 5 Oct 2015 15:07:11 -0700 (PDT) Received: from mailfilter.orange.md (mailfilter.orange.md [94.243.64.204]) by ietfa.amsl.com (Postfix) with ESMTP id 2D3BD1A21C2 for ; Mon, 5 Oct 2015 15:07:10 -0700 (PDT) Received: from localhost.localdomain (antispam.orange.md [127.0.0.1]) by localhost (Postfix) with ESMTP id 1ED913B26F5; Tue, 6 Oct 2015 01:07:26 +0300 (EEST) Received: from antispam.orange.md by antispam.orange.md with queue id 250106-1; Mon, 05 Oct 2015 22:07:25 GMT Received: from XCHSRV04.main.orange.md (unknown [192.168.200.64]) by mailfilter.orange.md (Postfix) with ESMTP id D01043B26F5; Tue, 6 Oct 2015 01:07:25 +0300 (EEST) Received: from XCHSRV01.main.orange.md ([fe80::685f:aef6:93b0:dea7]) by XCHSRV04.main.orange.md ([fe80::bdbd:3bcd:a0d2:9b6%14]) with mapi id 14.02.0328.009; Tue, 6 Oct 2015 01:07:04 +0300 From: Veaceslav ROMAN To: Eric Dumazet , Ingemar Johansson S Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8Pq87vsQ9XoBmgDq7PdWOg== Date: Mon, 5 Oct 2015 22:07:02 +0000 Message-ID: <7DBBB686E19D2049ADAACD210B474BB10166E8BDBA@XCHSRV01.main.orange.md> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se>, In-Reply-To: Accept-Language: en-US, ro-RO Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.77.207] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2015 22:07:18 -0000 We've managed to recompile the kernel 4.1 with tcp_cubic HYSTART_DELAY_MIN = (10U<<3) , 10ms. Well, we had time only for one test cycle, but results are very promising: Download 10MB, High Bandwidths, HYSTART_DELAY_MIN 10 ms: Hystart average download time: 1.16 s NoHystart average download time: 0.93 s Avr. DelayCWND: 159 Min DelayCWND: 49=20 Max DelayCWND 754=20 Of all 50 tests with Hystart, HystartDelay detected in all 50 te= sts.=20 DelayCWND: 49, 69, 73, 75, 77, 81, 83, 84, 85, 87, 88, 89, 133, 137, 139, 1= 45, 150, 153, 160, 162, 165, 301, 307, 321, 454, 640, 707, 754 Cases: 4, 1, 1, 1, 2, 2, 2, 1, 7, 3, 2, = 7, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, = 1, 1, 1, 1, 1, 1 This shows an improvement of download time 20% in comparison with HYSTART_D= ELAY_MIN 4 ms and reduced the gap with NoHystart conditions from -59% (HD= M 4ms) to -25% (HDM 10 ms).=20 Compare with previous round with HYSTART_DELAY_MIN of 4 ms=20 Download 10MB, High Bandwidths, HYSTART_DELAY_MIN 4 ms: Hystart average download time: 1.42 s NoHystart average download time: 0.89 s Avr. DelayCWND: 60 Min DelayCWND: 29 Max DelayCWND 89=20 Of all 50 tests with Hystart, HystartDelay detected in all 50 te= sts.=20 =20 DelayCWND: 29, 45, 49, 63, 69, 81, 83, 85, 86, 87, 88, 89 Cases: 13, 1, 12, 1, 3, 2, 2, 1, 2, 6, 1, 6 I'll be in the trip till the end of the week, will be able to come with mor= e details next week.=20 Best regards Veaceslav Roman ________________________________________ From: Eric Dumazet [edumazet@google.com] Sent: Thursday, October 01, 2015 3:44 PM To: Ingemar Johansson S Cc: Veaceslav ROMAN; Eric Dumazet; Neal Cardwell; tcpm@ietf.org; Piers O'Ha= nlon; Sangtae Ha; end2end-interest@postel.org Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment Yes, this 4ms value problem is even more visible with TCP pacing (as implemented with linux sch_fq packet scheduler) because of apparent delays caused by sojourn time in packet scheduler itsel= f. TCP does not offset this sojourn time (as RTT is also used for RTO determination, better have an inflated view) Is switching to 10 ms would be better in your LTE tests ? On Thu, Oct 1, 2015 at 4:57 AM, Ingemar Johansson S wrote: > Hmm > You are right !. > Well... That of course should explain why HyStart performs badly for LTE.= .. Atleast it is part of the reason. > > /Ingemar > > > >> -----Original Message----- >> From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] >> Sent: den 1 oktober 2015 13:46 >> To: Ingemar Johansson S; Eric Dumazet; Neal Cardwell >> Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Eric Dumazet; end2end- >> interest@postel.org >> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment >> >> Well this say >> If curr_rtt is higher than delay_min + (1/8 * delay_min or 4 ms, >> whichever is higher) then exit slow start. >> >> When actual minimum delay is less than 32 ms the delay_min will be less = than >> 32<<3, then the delay_min>>3 will be less than 32 and will compare with >> 4<<3 which is 32 and therefor the threshold will be (real_curr_rtt<<3) > >> (real_delay<<3 + 4<<3) or (real_curr_rtt)>(real_delay +4) >> >> >> Veaceslav Roman >> >> -----Original Message----- >> From: Ingemar Johansson S [mailto:ingemar.s.johansson@ericsson.com] >> Sent: Thursday, 01 October 2015 13:00 >> To: Veaceslav ROMAN; Eric Dumazet; Neal Cardwell >> Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Eric Dumazet; end2end- >> interest@postel.org; Ingemar Johansson S >> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment >> >> Hi >> >> Yes, the variable delay_min is in Q3 for reasons of precision I guess Bu= t the >> call to HYSTART_DELAY_THRESH is with delay_min shifted down to Q0 >> >> HYSTART_DELAY_THRESH(ca->delay_min >> 3)) (see http://lxr.free- >> electrons.com/source/net/ipv4/tcp_cubic.c#L403 ) I may be off here.. >> >> Regarding pacing. I don't have real data to show the effects of pacing, = in >> simulations it is however possible to see the gains in terms of less coa= lescing, >> yes pacing does not solve the ACK compression but it seem to solve the >> follow up effect that ACK compression gives. >> >> /Ingemar >> >> > -----Original Message----- >> > From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] >> > Sent: den 1 oktober 2015 10:32 >> > To: Ingemar Johansson S; Eric Dumazet; Neal Cardwell >> > Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Eric Dumazet; end2end- >> > interest@postel.org >> > Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment >> > >> > Yes, HYSTART_DELAY_MIN (4U<<3) . >> > But curr_rtt and delay_min come from bictcp_acked which is also <<3 : >> > In tcp_cubic.c, function line 433 (kernel 4.2) >> > delay =3D (rtt_us << 3) / USEC_PER_MSEC; So, >> > 4U<<3 is equivalent to 4 ms . >> > >> > Regarding pacing I am not sure will help the performance: there are >> > gaps in ACK trains of the order of milliseconds and this is a lost >> > time and there is no way to compensate it. If introduce ACK pacing or >> > data packets pacing this would add to the overall lost time. >> > Pacing can help only to alleviate a potential problem of some >> > downstream routers with insufficient buffers which may introduce >> > unexpected packets drops when large train sent at a high speed as a >> > reply to highly compressed ACK train, but, on another hands. >> > But how much pacing and how to accommodate to largely changing radio >> > conditions ? And with permanently changing technology and speed ? >> > Anyway the pacing will add to the problem of gaps and better size >> > properly routers buffers. >> > >> > Ideally ACK train is split in 2 parts, but in practice I do see more. >> > But as the stream progress the less gaps. The bigger the downlink >> > pressure the more ACK and less chances to have a pause in ACK sending, >> > then there will be a continuous flow of Buffer Status Reports and less >> > of the Scheduling Request (SR, "Hey, I have some data to send"). The >> > only issue I am not sure is on the potential gaps in other networks. I >> > did not put bellow all details, but you should know that device can >> > not send SR in arbitrary moments of time but in predefined for him >> > intervals. In my network this period is 5 ms, therefore the RTT of the >> > first ACK in the train will have variations of 2.5 ms. But standards >> > admit this period to be also 10 ms ,20 ms, 40 ms, 80 ms and I can not >> > say what is the practice of others. I may only guess that most will tr= y to use >> the best and fastest, i.e. 5 ms. >> > Anyway, to get a realistic modeling of LTE interaction with TCP the >> > detailed MAC scheduler model is needed. Unfortunately, most of the >> > literature focus on scheduling of the Physical Resource Blocks, >> > Adaptive Modulation and Coding, MIMO, i.e., focusing on the >> > distribution of resources between users, which is good and needed, but >> > do not address very much the timing interaction with TCP. At least, I >> > never met in LTE scheduling literature a term "TCP self-clocking". >> > >> > The 8 ms HYSTART_DELAY_MIN could only be a simple solution for the >> > problem of counting of the round' curr_rtt from the second ACK of the >> > train (as the second ACK RTT will be, typically, due to scheduling, >> > 5-7 ms longer than the first one), but this only if SR periodicity is = 5 ms. >> > The other 2 problems will remain unsolved. >> > >> > Veaceslav Roman >> > >> > -----Original Message----- >> > From: Ingemar Johansson S [mailto:ingemar.s.johansson@ericsson.com] >> > Sent: Thursday, 01 October 2015 08:59 >> > To: Veaceslav ROMAN; Eric Dumazet; Neal Cardwell >> > Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Eric Dumazet; end2end- >> > interest@postel.org; Ingemar Johansson S >> > Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment >> > >> > Hi >> > >> > Thanks for the detailed report. What seems unclear to me is the value >> > of HYSTART_DELAY_MIN. You mention 8ms but when I look into the 4.2 >> > kernel code I get the feeling that it is 32ms ( #define HYSTART_DELAY_= MIN >> > (4U<<3) ). Is this a mistake from my side ? >> > I have run LTE simulations and tried to wring this HyStart issue >> > inside out and upside down but sofar it has been very difficult to >> > make HyStart exit prematurely but then of course I have run with >> > HYSTART_DELAY_MIN =3D 32ms. >> > >> > As regards to the ACK-compression problem, yes, ACK compression easily >> > occur in LTE, even in simulations. What I have seen is that packet >> > pacing is a very efficient remedy, for instance the packet implemented >> > in QUIC seems to solve ACK compression issues very well. >> > >> > It is interesting what you say that ACK trains are so much split up, I >> > could understand that they are split up in two parts (initial grant + >> > additional after received BSR). >> > >> > /Ingemar >> > >> > > -----Original Message----- >> > > From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] >> > > Sent: den 1 oktober 2015 02:25 >> > > To: Eric Dumazet; Neal Cardwell >> > > Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Ingemar Johansson S; >> > > Eric Dumazet; end2end-interest@postel.org >> > > Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment >> > > >> > > Finally can share some results of Hystart interaction with LTE. >> > > Few words about the testing configuration: >> > > Device Samsung Galaxy Note4LTE , up to 150 Mbps capable >> > > Radio Network LTE 2600/1800 >> > > Distance between the Core Network (EPC) and Base Statio= n - 10 >> km >> > > HTTP Server connected directly to the Core Network 1 Gb= ps >> > > Core network switching(EPC) 10 Gbps >> > > Backhaul transport 1 Gbps, full IP, MPLS >> > > Last mile transport full IP, MPLS, 300 Mbps >> > > >> > > Server: Linux k40srv 4.1.0-040100-generic #201507030940 >> > > SMP Fri Jul 3 >> > > 09:41:47 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux >> > > Apache/2.4.10 (Ubuntu), built: Jul 24 2015= 17:25:18 >> > > >> > > Hystart_detect: 2 (HYSTART_DELAY only), as suggested. >> > > net.ipv4.tcp_no_metrics_save =3D 1 to avoid tcp metrics >> > > caching interference >> > > >> > > Tests types: >> > > 1. Download 3 MB file, interchanging Hystart ON >> > and OFF, in >> > > conditions of Low Bandwidth (25-35 Mbps, poor radio) and High >> > > Bandwidth (100-120 Mbps, good radio). >> > > 2. Download 10 MB file, interchanging Hystart >> > ON and OFF, in >> > > conditions of Low Bandwidth (25-35 Mbps, poor radio) and High >> > > Bandwidth (100-120 Mbps, good radio). >> > > >> > > 100 Download tests per type. >> > > Download duration and bandwidth measured at the client >> > > side as shown by curl from the first received data packet till the >> > > end of session (this exclude DNS, 3 way handshake, http GET and its >> ACK). >> > > Hystart exit cwnd extracted from nstat (thank you Eric >> > > to pointing out to it). >> > > >> > > Overall results: >> > > Download 3MB, Low Bandwidths: >> > > Hystart average download time: 1.41 s >> > > NoHystart average download time: 1.36 s >> > > >> > > Download 3MB, High Bandwidths: >> > > Hystart average download time: 0.65 s >> > > NoHystart average download time: 0.37 s >> > > >> > > Download 10MB, Low Bandwidths: >> > > Hystart average download time: 2.40 s >> > > NoHystart average download time: 2.12 s >> > > >> > > Download 10MB, High Bandwidths: >> > > Hystart average download time: 1.42 s >> > > NoHystart average download time: 0.89 s >> > > >> > > These results show that Hystart has no significant impact in >> > > conditions of low available bandwidth, but at high available the >> > > decrease of performance can reach 60-70 %. >> > > With the development of the LTE-A which uses larger spectrum to >> > > reach >> > > 300 Mbps and more the impact of Hystart will become even more visibl= e. >> > > >> > > I do share with Eric and Neal some files described below and also >> > > can provide links to anyone interested. >> > > >> > > Summary of these tests results are gathered in the following Excel >> > > files (names, I believe, are self explanatory): Test_3M_low_BW.xlsx, >> > > Test_3M_high_BW.xlsx, Test_10M_low_BW.xlsx, >> > Test_10M_high_BW.xlsx. >> > > Similarly tcpdump traces are called Test_3M_low_BW.pcap, >> > > Test_3M_high_BW. pcap, Test_10M_low_BW. pcap, >> Test_10M_high_BW. >> > > pcap (able to collect only server side traces, but they are self suf= ficient). >> > > For the 10M downloads there are few missing traces as there trace >> > > files were recycled. In traces are also ssh sessions, please, ignore >> > > them, they were used to put hystart ON and OFF on server and extract >> > > nstat results before and after each download. >> > > >> > > Excel files have 2 tabs: curl_sum and Summary. >> > > In the curl_sum tab the stat is cumulated in chronological order of = tests. >> > > In the Summary_tab the Hystart and NoHystart tests are shown side >> > > by side (as said, Download 1 was with Hystart, Download 2 without, >> > > Download 3 with Hystart, Download 4 without, etc.). It also include >> > > graphs of Transfer time and a graph which shows the distribution in >> > > % of different Hystart exit window DelayCWND. >> > > >> > > Fields: >> > > comments - test case >> > > Transf_Time - calculated from curl output the duration >> > between first data >> > > packet and end of transfer >> > > Transf_speed - calculated from curl output the ratio of file si= ze >> > and duration >> > > between first data packet and end of transfer >> > > Hystart D_Delay - hystart exit occurrence (difference between >> > > TcpExtTCPHystartDelayDetect before and after download) >> > > D_DelayCwnd - Hystart exit cwnd for this download >> > > (difference of TcpExtTCPHystartDelayCwnd before and after the >> > > download) Date_On_Server - time just before the download start, to >> > > help navigate the pcap >> > > >> > > Additionally, in the file Test_10M_low_BW.xlsx in the tab curl_sum >> > > are included for each download tcp.stream extract from pcap of the >> > > first >> > > 100 tcp.analysis.ack_rtt which may help in understanding of the >> > > hystart behavior (fields ack_rtt_1 to ack_rtt_100). >> > > In this file there are also fields URI_frame, URI_time, URI_time_rel= ative, >> > > tcp_stream. >> > > Then there are calculated fields: minRTT as a minimum of all 100 ACK= , >> r1_5, >> > > r2_8, r3_8, r4_8 which simulate Hystart calculations of the >> round curr_rtt >> > > and r1 min, r2 min, r3 min, r4 min represent minRTT of the whole rou= nd. >> > > Please, correct me if I am wrong, but in my understanding for each >> > > ACK the hystart_update is called before the hystart_reset, and as a >> > > result the computation of cur_rtt of the 8 packets of the round >> > > starts with the 2-nd packet of the round, not the first one. >> > > I've use r1_5 as a min(ack_rtt_7...ack_rtt_11), due to >> > > hystart_low_window=3D16, r2_8 as a min(ack_rtt_12 ... ack_rtt_19), >> > > r3_8 as a >> > > min(ack_rtt_32 ... ack_rtt_39) and ), r4_8 as a min(ack_rtt_72 ... >> > ack_rtt_79). >> > > To understand the impact of this there are also r1 HOL, r2 HOL, r3 >> > > HOL, r4 HOL (Head Of Line) which compute the same with first packet >> > > in the round included in the relevant round: r1_5 as a >> > > min(ack_rtt_6...ack_rtt_10), r2_8 as a min(ack_rtt_11 ... >> > > ack_rtt_18), >> > > r3_8 as a min(ack_rtt_31 ... ack_rtt_38) and ), r4_8 as a min(ack_rt= t_71 ... >> > ack_rtt_78). >> > > >> > > If my understanding of Hystart is correct and, assuming in each >> > > train the first packet has the lowest RTT which, most probably is >> > > valid in all type of networks, then the exit from hystart in this >> > > implementation may happen at cwnd 29, 49, 89, 169, etc. (given the >> > > IW of >> > 10). >> > > These tests show that this is the case, however there also >> > > intermediate values. However the cwnd of 29 shall be the lowest >> > > possible value and this is the case in these tests. Unfortunately, >> > > in LTE the first packet in the train has chances to always have ~6 >> > > ms lower than the following ones and, because it is not counted in >> > > the 2-nd train, there is quite high percentage of exit at 29 (14-36% >> > > in these tests), even though later on cubic increases the cwnd >> > > without losses up to many hundreds. For me this is too early exit fr= om >> slow start. >> > > As it can be seen from comparison of the simulated calculation, >> > > inclusion of the first packet in the round curr_rtt calculation >> > > would eliminate the cwnd 29 and shift to cwnd 49 or 89 or higher. >> > > >> > > But there are also intermediate values, e.g. 69. Looking through >> > > traces one may see that this is due to "compressed ACK" which >> > > receives a tightly packed train of many ACK so that in traces there >> > > are not data packets sent in turn by server. Looks to me, because >> > > hysrart_reset sets the end_seq to snd_next and the server did not >> > > managed to send yet packets in reply to this high speed ACK train >> > > the snd_next points to much lower value than would normally happen >> > > and this actually destroys the correct identification of the borders >> > > of the round, the round fails somewhere in the middle of the train >> > > not at its border, this makes very likely increased delay due to >> > > queueing and earlier exit from slow_start and intermediate values of >> > > exit cwnd where >> > they are not expected. >> > > >> > > And last but not least: there are gaps (4-7 ms) in ACK trains. On >> > > the perfectly sent IW of 10 data packets train the typical for LTE >> > > will be to receive 1 or 2 ACK, then a gap of 4-6 ms then another >> > > couple of ACK, then another gap, then 5 ACK compressed together in a >> > > fraction of microsecond. Of cause, the next train from the server >> > > will contain the same gaps. Very quickly, e.g., the Head of Line >> > > (HOL) of the 3-d train will follow immediately the tail of the 2-nd >> > > train which will lead to queueing in the end router (actually the >> > > radio base station, much earlier than BDP reached), but this will >> > > lead to increase of the train RTT and too yearly exit from slow >> > > start. Why too early exit ? Because, should server continue >> > > increasing the speed this would reduce quicker the gaps both in >> > > downlink and uplink, the preice for this being the increase of the >> > > RTT (I guess double) against what one would normally see >> > in the end-to-end Ethernet networks. >> > > >> > > >> > > I do like ideas of Hystart, but I don't know how could it be >> > > possibly reconciled with my LTE problems. With the increase of LTE >> > > speeds this early exit from slow start will become even more of the >> problem. >> > > For LTE, not sure how good and generally acceptable, solution would >> > > be, possibly, to increase the HYSTART_DELAY_MIN to 8 ms, which, at >> > > least in my case, would diminish the earlier exit significantly, if >> > > I believe in these tests results. >> > > Another solution, which I don't like very much, but as I know some >> > > operators use, would be to insert a kind of TCP accelerator between >> > > the Internet and the mobile network which will intercept all TCP and >> > > would send it to mobile without Hystart but this would kill any >> > > end-to-end principle and this will not be the Internet anymore. >> > > >> > > >> > > If you have time and never look closely to LTE technology I do a >> > > quick summary bellow. >> > > >> > > >> > > LTE, TCP self-clocking and ACK compression. >> > > >> > > Spent some time to understand the basics of what could be the reason >> > > for all those timing effects due to LTE. >> > > First, unlike wire based transport technologies, LTE (but also >> > UMTS >> > > and future 5G) sends data, both uplink and downlink in predefined >> > > timeslots called Transmission Time Interval (TTI). In UMTS this is >> > > 2ms, in LTE is >> > > 1 ms and it says that in 5G it will be 0.5 ms. >> > > These TTI have exact borders and tightly follow each other and >> > > require exact synchronization between the mobile device and Radio >> > > Base Station (called eNodeB in LTE). To cope with varying radio >> > > conditions the sending data bandwidth is modified in such a way that >> > > to keep a constant probability of data loss of 10%. Bad radio - less >> > > bandwidth, good >> > radio - more bandwidth. >> > > The bandwidth is modified by packing less or more bits in a TTI of 1 >> > > ms in >> > LTE. >> > > But TTI remain unchanged of exactly 1 ms. That means that, at the >> > > TCP/IP layer the precision of the timing information from device to >> > > the network is 1 ms. In the case of a bandwidth of 1 Mbps there will >> > > be 83 IP packets per second or one packet every 12 ms. In this case, >> > > possibly the return of one ACK every 12 ms (every 12 TTI) would be >> > > more or less sufficient to keep at a good level the TCP >> > > self-clocking mechanism. But, when talking about 100 Mbps (or 300 >> > > Mbps) that means one packet (and one ACK) every 0.12 ms and there is >> > > no way to provide it with a TTI of 1 ms. This is the reason for ACK >> > > compression in LTE and the future 5G which promises more than 1Gbps >> > > with its 0.5 ms TTI will >> > be in a big conflict with TCP self-clocking. >> > > But the problem do not stops here. There is a significantly >> > different >> > > mechanisms of transmission in the downlink (from eNodeB to mobile >> > > device) and in uplink (from mobile device to the eNodeB). The >> > > difference come from the fact that the scheduler for both downlink >> > > and in uplink is located in the eNodeB. As of my understanding the >> > > purpose of this is to simplify to the maximum the device and save it= s >> battery. >> > > As a result of this design, the eNodeB can send in each TTI toward >> > > the mobile device whenever data exist in its buffer (eNodeB is a >> > > kind of IP wireless router). But for mobile device, in order to send >> > > the data in uplink, after a pause, it has first to ask the network >> > > to schedule it. And mobile has for this request (after a pause of >> > > sending) just one bit, actually is not even a data bit, is a special >> > > radio signal which tells "Hey, I have some data in the buffer to >> > > send". Lets look at initial train of IW of 10 packets. In good radio >> > > conditions the eNodeB will fit all 10 packets in 1 TTI of 1 ms and >> > > send them all to the device. Device will unpack them and generate 10 >> > > ACK and then tell to the eNodeB "Hey, I have some data in the buffer >> > > to send". The eNodeB has not knowledge how much data the mobile has >> > > to send, therefor will allocate some minimum >> > > capacity: "Well, I give you a grant to send up to 200 Bytes and you >> > > must send them exactly after 4 ms you get this notification". These >> > > 4 ms are allowance for mobile device to prepare data for sending >> > > which is a very complicated process, these 4 ms is a must delay >> > > between the grant >> > and the transmission. >> > > If mobile has in its buffer 10 ACK to send it will send only 2 ACK >> > > and will complement them with, so called, buffer status report: "I >> > > do have still 480 bytes to send". Now network has more information >> > > and allocate as >> > > requested: "Well, I give you a grant to send up to 200 Bytes and you >> > > must send them exactly after 4 ms you get this notification". But, >> > > in the mean time some other information of higher priority may >> > > appear in the >> > device (e.g. >> > > signaling) and the device will send only 6 out of 8 remaining ACK >> > > together with signaling and another Buffer Status Report: "Hey, I do >> > > have another 120 bytes to send". And again "Well, I give you a grant >> > > to send up to 120 Bytes and you must send them exactly after 4 ms >> > > you get this notification". As a result the server will receive 2 >> > > ACK, then after a gap of 6 ms another 6 ACK, then, after another 6 >> > > ms, the remaining 2 ACK. With the scheduler in the eNodeB these gaps >> > > are inevitable. And one more think: mobile pack together those 6 ACK >> > > in 1 TTI of 1 ms and send them, then the eNodeB unpack them and >> > > there is no way to recover the spacing of 0.12 ms between them. >> > > eNodeB simply send them to the server back to back at the speed of >> > > its data card which is typically 1 Gbps and we face the ACK >> > > compression. After the last 2 ACK were sent there is nothing else to >> > > send, so there is no Buffer Status Report, therefore, after packets >> > > of the next train arrive and ACK are generated everything starts >> > > again with one bit of information "Hey, I have >> > some data in the buffer to send". >> > > That's why there are gaps and ACK compression, loss, or more >> > > exactly, lack of any timing information which would be useful for >> > > the TCP self- >> > clocking. >> > > >> > > Thank you and sorry for taking your time. >> > > >> > > >> > > Veaceslav Roman >> > > >> > > >> > > >> > > -----Original Message----- >> > > From: Veaceslav ROMAN >> > > Sent: Saturday, 05 September 2015 01:10 >> > > To: 'Eric Dumazet'; Neal Cardwell >> > > Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Ingemar Johansson S; >> > > Eric Dumazet; end2end-interest@postel.org >> > > Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment >> > > >> > > If we look at the first train: all packets received in less than 1 m= s. >> > > Probably this is only an appearance as LTE transmits in, so called >> > > transmission time interval (TTI) of 1 ms, and what we see here is >> > > that all 10 packets of initial window fitted in 1 ms, and, when >> > > decoded, were presented to the TCP/IP layer (and pcap) all at once. >> > > BTW, 8 packets of 13022 bytes in 1 ms means instantaneous speed of >> > > 104 Mbps, good radio conditions. TCP generates 10 ACK train of the >> > > duration of >> > > 1.2 ms. Will it be a Fast Ethernet possibly we would consider this >> > > normal, >> > isn't it ? >> > > I looked at how server reply to these 10 ACKs. There are 3 gaps of >> > > 4, >> > > 2 and 5 ms in the reply train and the total train of 18 (?, it >> > > should be 20) packets reaches a duration of 14 ms. AFAIK LTE may >> > > introduce gaps in ACK train due to uplink scheduling mechanism. >> > > May be these gaps trigger Hystart early exit ? >> > > >> > > Veaceslav Roman >> > > Technical and IT director >> > > Orange Moldova S.A. >> > > Fix: +37322575400 >> > > Mob: +37369198400 >> > > Fax: +37322575306 >> > > >> > > -----Original Message----- >> > > From: Eric Dumazet [mailto:eric.dumazet@gmail.com] >> > > Sent: Friday, 04 September 2015 19:48 >> > > To: Neal Cardwell >> > > Cc: Veaceslav ROMAN; tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; >> > > Ingemar Johansson S; Eric Dumazet; end2end-interest@postel.org >> > > Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment >> > > >> > > On Fri, 2015-09-04 at 11:32 -0400, Neal Cardwell wrote: >> > > > Hi Veaceslav, >> > > > >> > > > I agree that in your LTE traces it looks like CUBIC Hystart is >> > > > exiting slow start too early. >> > > > >> > > > SInce you seem to have a nice LTE testbed, would you be able to do >> > > > some experiments to find a set of parameters for Hystart that work >> > > > better for your LTE environment? For example, you might try the >> > > > two variations I suggested earlier in the thread: >> > > > >> > > > if (ca->curr_rtt > ca->delay_min + >> > > > HYSTART_DELAY_THRESH(ca->delay_min >> >> > > > 2)) { or >> > > > if (ca->curr_rtt > ca->delay_min + >> > > > HYSTART_DELAY_THRESH(ca->delay_min >> >> > > > 1)) { >> > > > >> > > > Do any of those give better results for your tests? >> > > > >> > > > neal >> > > >> > > Also, hystart is fooled by too many ACK received in short period. >> > > >> > > This problem would be solved if GRO was used at receiver, as less >> > > ACK would be sent. >> > > >> > > Presumably receiver is not a linux TCP stack ? >> > > >> > > Maybe we should add a logic in hystart_update() to take one ACK per = ms. >> > > >> > > 05:32:24.134172 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [S], seq 4294423215, win 65535, options [mss 1460,sackOK,TS val >> > > 1112351 ecr 0,nop,wscale 8], length 0 >> > > 05:32:24.161986 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [S.], seq 1198877721, ack 4294423216, win 28960, options [mss >> > > 1416,sackOK,TS val >> > > 2808808634 ecr 1112351,nop,wscale 7], length 0 >> > > 05:32:24.162108 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 1, win 343, options [nop,nop,TS val 1112354 ecr >> > > 2808808634], length 0 >> > > 05:32:24.163072 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [P.], seq 1:658, ack 1, win 343, options [nop,nop,TS val 1112354 ecr >> > > 2808808634], length 657 >> > > 05:32:24.203984 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], ack 658, win 237, options [nop,nop,TS val 2808808675 ecr >> > > 1112354], length 0 >> > > 05:32:24.210275 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [P.], seq 1:387, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 386 >> > > 05:32:24.210315 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 387:1791, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 1404 >> > > 05:32:24.210332 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 1791:3195, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 1404 >> > > 05:32:24.210347 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 3195:4599, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 1404 >> > > 05:32:24.210350 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 387, win 347, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.210361 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 4599:6003, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 1404 >> > > 05:32:24.210374 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 6003:7407, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 1404 >> > > 05:32:24.210389 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 7407:8811, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 1404 >> > > 05:32:24.210402 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 8811:10215, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 1404 >> > > 05:32:24.210416 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 10215:11619, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 1404 >> > > 05:32:24.210418 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 1791, win 358, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.210431 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 11619:13023, ack 658, win 237, options [nop,nop,TS val >> > > 2808808677 ecr 1112354], length 1404 >> > > 05:32:24.210455 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 3195, win 369, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.210685 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 4599, win 380, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.210731 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 6003, win 391, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.210937 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 7407, win 402, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.211078 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 8811, win 413, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.211228 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 10215, win 424, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.211377 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 11619, win 435, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.211544 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 13023, win 446, options [nop,nop,TS val 1112359 ecr >> > > 2808808677], length 0 >> > > 05:32:24.237999 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 13023:14427, ack 658, win 237, options [nop,nop,TS val >> > > 2808808709 ecr 1112359], length 1404 >> > > 05:32:24.238053 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 14427:15831, ack 658, win 237, options [nop,nop,TS val >> > > 2808808709 ecr 1112359], length 1404 >> > > 05:32:24.238090 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 14427, win 457, options [nop,nop,TS val 1112362 ecr >> > > 2808808709], length 0 >> > > 05:32:24.238164 IP 94.243.104.156.32924 > 195.95.178.204.80: Flags >> > > [.], ack 15831, win 468, options [nop,nop,TS val 1112362 ecr >> > > 2808808709], length 0 >> > > 05:32:24.244024 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 15831:17235, ack 658, win 237, options [nop,nop,TS val >> > > 2808808713 ecr 1112359], length 1404 >> > > 05:32:24.244063 IP 195.95.178.204.80 > 94.243.104.156.32924: Flags >> > > [.], seq 17235:18639, ack 658, win 237, options [nop,nop,TS val >> > > 2808808713 ecr 1112359], length 1404 >> > > >= From nobody Mon Oct 5 15:54:33 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F10141A6FFA for ; Mon, 5 Oct 2015 15:54:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.609 X-Spam-Level: X-Spam-Status: No, score=-2.609 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id loJ-FSr-Q0VM for ; Mon, 5 Oct 2015 15:54:30 -0700 (PDT) Received: from mailfilter.orange.md (mailfilter.orange.md [94.243.64.204]) by ietfa.amsl.com (Postfix) with ESMTP id 65B651A6FF9 for ; Mon, 5 Oct 2015 15:54:29 -0700 (PDT) Received: from localhost.localdomain (antispam.orange.md [127.0.0.1]) by localhost (Postfix) with ESMTP id A7ECF27E047; Tue, 6 Oct 2015 01:54:49 +0300 (EEST) Received: from antispam.orange.md by antispam.orange.md with queue id 250110-1; Mon, 05 Oct 2015 22:54:49 GMT Received: from XCHSRV03.main.orange.md (unknown [192.168.200.63]) by mailfilter.orange.md (Postfix) with ESMTP id 8E15A27E039; Tue, 6 Oct 2015 01:54:49 +0300 (EEST) Received: from XCHSRV01.main.orange.md ([fe80::685f:aef6:93b0:dea7]) by XCHSRV03.main.orange.md ([fe80::6dbc:28dd:213b:8931%14]) with mapi id 14.02.0328.009; Tue, 6 Oct 2015 01:54:28 +0300 From: Veaceslav ROMAN To: "Scharf, Michael (Michael)" , "Kuhn Nicolas" , "Lane Wigley (lwigley)" , "tcpm@ietf.org" Thread-Topic: TCP evolution impact on router buffers Thread-Index: AdD3qDJMG7WkdcJDTiy1iIy/H6510wDXEZsQAB6kmpAABLWoIABn/A/AAKN/QXA= Date: Mon, 5 Oct 2015 22:54:26 +0000 Message-ID: <7DBBB686E19D2049ADAACD210B474BB10166E8BDE1@XCHSRV01.main.orange.md> References: <655C07320163294895BBADA28372AF5D484D799E@FR712WXCHMBA15.zeu.alcatel-lucent.com> <46616f47b42841f0bb7a2afb8bdeac98@XCH-ALN-009.cisco.com> , <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: en-US, ro-RO Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.77.207] Content-Type: multipart/alternative; boundary="_000_7DBBB686E19D2049ADAACD210B474BB10166E8BDE1XCHSRV01maino_" MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] TCP evolution impact on router buffers X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2015 22:54:33 -0000 --_000_7DBBB686E19D2049ADAACD210B474BB10166E8BDE1XCHSRV01maino_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Can be 400G router interface considered as a bottleneck capacity if all dow= nstream receivers are at a maximum of 1G ? Best regards Veaceslav Roman ________________________________ From: tcpm [tcpm-bounces@ietf.org] on behalf of Scharf, Michael (Michael) [= michael.scharf@alcatel-lucent.com] Sent: Friday, October 02, 2015 8:05 PM To: Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org Subject: Re: [tcpm] TCP evolution impact on router buffers Regarding Section 7 in [2], I am surprised that a potential impact of CIR/P= IR is not mentioned. Unfortunately, I have only a very limited understandin= g of MPLS/IP router architectures, but the way bursts hit an AQM could IMHO= depend on that. As [2] is mentioned, one question related to the discussion below. From [2]= : 3.2. Buffer size The size of the buffers should be carefully chosen, and is to be set to the bandwidth-delay product; the bandwidth being the bottleneck capacity and the delay the largest RTT in the considered network. This means of the order of 10GB of buffer size for a 400G router interface,= right? Wouldn=92t it make sense to relax the statement =93=85 and is to be= set=85=94 a bit? Michael From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] Sent: Wednesday, September 30, 2015 6:41 PM To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Hi, In [1], the authors evaluate the performance of CUBIC in small buffer regim= e. However, they use the Linux kernel version 2.6.18 (where the IW was 3 an= d not 10). Also, for any tests on CUBIC, small buffers and IW3/10, you may = be interested in looking at the scenario =937. Burst absorption=94 of the = AQM Characterization Guidelines of the AQM WG [2]. Concerning TCP pacing, t= here is an on-going document on a pacing solution in the slow-start that yo= u may want to have a look and comment [3]. I hope this helps, Cheers, Nicolas [1] Jain, S.; Raina, G., "An experimental evaluation of CUBIC TCP in a smal= l buffer regime," in Communications (NCC), 2011 National Conference on , vo= l., no., pp.1-5, 28-30 Jan. 2011 doi: 10.1109/NCC.2011.5734779 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&is= number=3D5734693 [2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08 [3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01 De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley= ) Envoy=E9 : mercredi 30 septembre 2015 15:10 =C0 : Scharf, Michael (Michael); tcpm@ietf.org Objet : Re: [tcpm] TCP evolution impact on router buffers This paper from 2008 shows no drops with a 5 msec buffer on a heavily-conge= sted core link. Low levels of drops start to show up around 2.5 msec. This = was conducted at Level 3 to test the BDP / sqrt # flows proposal from Stanf= ord. Experimental Study of Router Buffer Sizing =96 Neda Beheshti, Yashar Ganjal= i, Monia Ghobadi, Nick McKeown, and Geoff Salmon They also explored TCP pacing as a way to further reduce buffers. I=92m try= ing to assess if CUBIC results in some pacing effect by decoupling the wind= ow growth from acks. - Lane From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Sent: Tuesday, September 29, 2015 6:56 PM To: Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers This question may also be in scope of the AQM WG, and perhaps ICCRG, but si= nce the communities overlap, I avoid cross-posting=85 Unfortunately, I don=92t have any recent pointer or own data. But I am awar= e of examples for router buffer dimensioning based on tests with a single T= CP connection, using Reno. In those examples, the dimension of the buffers = and possibly other related parameters (two-color/three-color meters/policie= rs, WRED slopes, H-QoS, etc.) are configured to achieve full =93line=94 spe= ed for a single TCP connection. As to be expected, this typically comes dow= n to the old BDP rule of thumb. But it is rather obvious that a single bulk= data Reno connection is not necessarily a realistic workload these days. S= o, the test methodology can be one relevant aspect of the dimensioning prob= lem. I believe that a modern TCP stack with a high-speed congestion control such= as CUBIC will typically need less buffer than the outcome of this dimensio= ning method and the publications 10 years ago. As far as I know, most moder= n TCP stacks operate well over a very wide range of buffer sizes, and the b= uffers can be relatively small. To me, the key question is the latency-thro= ughput tradeoff, and there may not be a one-fits-all answer to that one. Personally, I=92d also be interested in more recent pointers on router buff= er dimensioning. I could even see some value in documenting dimensioning gu= idelines used in practice, if there was a way to generalize them. Michael From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley= ) Sent: Friday, September 25, 2015 5:41 PM To: tcpm@ietf.org Subject: [tcpm] TCP evolution impact on router buffers I=92m trying to track down some research or opinions on how the more recent= TCP schemes impact router buffer needs. There are a number of publications= from Stanford and Georgia Tech from about 10 years ago, and I=92m trying t= o assess how changes to the algorithms (e.g. CUBIC) and parameters (initial= congestion window) deployed since then may influence those findings in the= direction of more or less buffering being needed. I=92d appreciate any input and pointers. Thanks. - Lane --_000_7DBBB686E19D2049ADAACD210B474BB10166E8BDE1XCHSRV01maino_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

Can be 400G router interface considered as a bottleneck capacity if= all downstream receivers are at a maximum of 1G ?

 

Best regards

 

Veaceslav Roman

 

 

From: tcpm [tcpm-bounces@ietf.org] on beha= lf of Scharf, Michael (Michael) [michael.scharf@alcatel-lucent.com]
Sent: Friday, October 02, 2015 8:05 PM
To: Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org
Subject: Re: [tcpm] TCP evolution impact on router buffers

Regarding Section 7 i= n [2], I am surprised that a potential impact of CIR/PIR is not mentioned. = Unfortunately, I have only a very limited understanding of MPLS/IP router a= rchitectures, but the way bursts hit an AQM could IMHO depend on that.

 

As [2] is mentioned, = one question related to the discussion below. From [2]:

 

3.2.  Buffer siz= e

 

   The size= of the buffers should be carefully chosen, and is to be set

   to the b= andwidth-delay product; the bandwidth being the bottleneck

   capacity= and the delay the largest RTT in the considered network.

 

This means of the ord= er of 10GB of buffer size for a 400G router interface, right? Wouldn=92t it= make sense to relax the statement =93=85 and is to be set=85=94 a bit?

 

Michael

 

 

From: Kuhn N= icolas [mailto:Nicolas.Kuhn@cnes.fr]
Sent: Wednesday, September 30, 2015 6:41 PM
To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org<= br> Subject: RE: TCP evolution impact on router buffers

 

Hi,

 

In [1], the authors eva= luate the performance of CUBIC in small buffer regime. However, they use th= e Linux kernel version 2.6.18 (where the IW was 3 and not 10). Also, for an= y tests on CUBIC, small buffers and IW3/10, you may be interested in looking at the scenario =937.  Burst= absorption=94 of the AQM Characterization Guidelines of the AQM WG [2]. Co= ncerning TCP pacing, there is an on-going document on a pacing solution in = the slow-start that you may want to have a look and comment [3].

 

I hope this helps,

Cheers,

 

Nicolas

&n= bsp;

[1] Jain, S.; Raina, G.= , "An experimental evaluation of CUBIC TCP in a small buffer regime,&q= uot; in Communications (NCC), 2011 National Conference on , vol., no., pp.1-= 5, 28-30 Jan. 2011
doi: 10.1109/NCC.2011.5734779
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&a= mp;arnumber=3D5734779&isnumber=3D5734693

[2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08

[3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01

&n= bsp;

De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley)
Envoy=E9 : mercredi 30 septembre 2015 15:10
=C0 : Scharf, Michael (Michael); tcpm@ietf.org
Objet : Re: [tcpm] TCP evolution impact on router buffers

 

This paper from 2008 = shows no drops with a 5 msec buffer on a heavily-congested core link. Low l= evels of drops start to show up around 2.5 msec. This was conducted at Leve= l 3 to test the BDP / sqrt # flows proposal from Stanford.

 

Experimental Study of Router Buffer Sizing =96 Neda = Beheshti, Yashar Ganjali, Monia Ghobadi, Nick McKeown, and Geoff Salmon

 

They also explored TCP pacing as a way to further re= duce buffers. I=92m trying to assess if CUBIC results in some pacing effect= by decoupling the window growth from acks.

 

- Lane

 

 

From: Scharf= , Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com]
Sent: Tuesday, September 29, 2015 6:56 PM
To: Lane Wigley (lwigley); tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

 

This question may als= o be in scope of the AQM WG, and perhaps ICCRG, but since the communities o= verlap, I avoid cross-posting=85

 

Unfortunately, I don= =92t have any recent pointer or own data. But I am aware of examples for ro= uter buffer dimensioning based on tests with a single TCP connection, using= Reno. In those examples, the dimension of the buffers and possibly other related parameters (two-color/three-colo= r meters/policiers, WRED slopes, H-QoS, etc.) are configured to achieve ful= l =93line=94 speed for a single TCP connection. As to be expected, this typ= ically comes down to the old BDP rule of thumb. But it is rather obvious that a single bulk data Reno connection= is not necessarily a realistic workload these days. So, the test methodolo= gy can be one relevant aspect of the dimensioning problem.

 

I believe that a mode= rn TCP stack with a high-speed congestion control such as CUBIC will typica= lly need less buffer than the outcome of this dimensioning method and the p= ublications 10 years ago. As far as I know, most modern TCP stacks operate well over a very wide range of buff= er sizes, and the buffers can be relatively small. To me, the key question = is the latency-throughput tradeoff, and there may not be a one-fits-all ans= wer to that one.

 

Personally, I=92d als= o be interested in more recent pointers on router buffer dimensioning. I co= uld even see some value in documenting dimensioning guidelines used in prac= tice, if there was a way to generalize them.

 

Michael

 

 

 

From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley)
Sent: Friday, September 25, 2015 5:41 PM
To: tcpm@ietf.org=
Subject: [tcpm] TCP evolution impact on router buffers

 

I=92m trying to track down some research or opinions= on how the more recent TCP schemes impact router buffer needs. There are a= number of publications from Stanford and Georgia Tech from about 10 years = ago, and I=92m trying to assess how changes to the algorithms (e.g. CUBIC) and parameters (initial congestion window) = deployed since then may influence those findings in the direction of more o= r less buffering being needed.

 

I=92d appreciate any input and pointers. Thanks.

 

- Lane

 

 

--_000_7DBBB686E19D2049ADAACD210B474BB10166E8BDE1XCHSRV01maino_-- From nobody Mon Oct 5 16:11:07 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A5F061B2AEA for ; Mon, 5 Oct 2015 16:11:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.909 X-Spam-Level: X-Spam-Status: No, score=-6.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kH2szh0vz_yp for ; Mon, 5 Oct 2015 16:11:00 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D1A5B1B2B2C for ; Mon, 5 Oct 2015 16:10:59 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (unknown [135.239.2.42]) by Websense Email Security Gateway with ESMTPS id 309656ED85313; Mon, 5 Oct 2015 23:10:52 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id t95NAulM000543 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 6 Oct 2015 01:10:56 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Tue, 6 Oct 2015 01:10:56 +0200 From: "Scharf, Michael (Michael)" To: Veaceslav ROMAN , Kuhn Nicolas , "Lane Wigley (lwigley)" , "tcpm@ietf.org" Thread-Topic: TCP evolution impact on router buffers Thread-Index: AdD3qDJMG7WkdcJDTiy1iIy/H6510wDXEZsQAB6kmpAABLWoIABn/A/AAKN/QXAAAEdhcA== Date: Mon, 5 Oct 2015 23:10:55 +0000 Message-ID: <655C07320163294895BBADA28372AF5D484F4C66@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <655C07320163294895BBADA28372AF5D484D799E@FR712WXCHMBA15.zeu.alcatel-lucent.com> <46616f47b42841f0bb7a2afb8bdeac98@XCH-ALN-009.cisco.com> , <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> <7DBBB686E19D2049ADAACD210B474BB10166E8BDE1@XCHSRV01.main.orange.md> In-Reply-To: <7DBBB686E19D2049ADAACD210B474BB10166E8BDE1@XCHSRV01.main.orange.md> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.39] Content-Type: multipart/alternative; boundary="_000_655C07320163294895BBADA28372AF5D484F4C66FR712WXCHMBA15z_" MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] TCP evolution impact on router buffers X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2015 23:11:05 -0000 --_000_655C07320163294895BBADA28372AF5D484F4C66FR712WXCHMBA15z_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable What prevents congestion in the case you mention? E.g., what happens if the= re significantly more than 400 customers/customer ports with 1G each? I don't know whether such a fast interface really benefits from AQM, and wh= ether interfaces of that order of magnitude are in scope of document [2], o= r not. But just having a very fast link does not imply that congestion can = never occur. Otherwise TCPM's job would be much simpler ;) Michael From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] Sent: Tuesday, October 06, 2015 12:54 AM To: Scharf, Michael (Michael); Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ie= tf.org Subject: RE: TCP evolution impact on router buffers Can be 400G router interface considered as a bottleneck capacity if all dow= nstream receivers are at a maximum of 1G ? Best regards Veaceslav Roman ________________________________ From: tcpm [tcpm-bounces@ietf.org] on behalf of Scharf, Michael (Michael) [= michael.scharf@alcatel-lucent.com] Sent: Friday, October 02, 2015 8:05 PM To: Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org Subject: Re: [tcpm] TCP evolution impact on router buffers Regarding Section 7 in [2], I am surprised that a potential impact of CIR/P= IR is not mentioned. Unfortunately, I have only a very limited understandin= g of MPLS/IP router architectures, but the way bursts hit an AQM could IMHO= depend on that. As [2] is mentioned, one question related to the discussion below. From [2]= : 3.2. Buffer size The size of the buffers should be carefully chosen, and is to be set to the bandwidth-delay product; the bandwidth being the bottleneck capacity and the delay the largest RTT in the considered network. This means of the order of 10GB of buffer size for a 400G router interface,= right? Wouldn't it make sense to relax the statement "... and is to be set= ..." a bit? Michael From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] Sent: Wednesday, September 30, 2015 6:41 PM To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Hi, In [1], the authors evaluate the performance of CUBIC in small buffer regim= e. However, they use the Linux kernel version 2.6.18 (where the IW was 3 an= d not 10). Also, for any tests on CUBIC, small buffers and IW3/10, you may = be interested in looking at the scenario "7. Burst absorption" of the AQM = Characterization Guidelines of the AQM WG [2]. Concerning TCP pacing, there= is an on-going document on a pacing solution in the slow-start that you ma= y want to have a look and comment [3]. I hope this helps, Cheers, Nicolas [1] Jain, S.; Raina, G., "An experimental evaluation of CUBIC TCP in a smal= l buffer regime," in Communications (NCC), 2011 National Conference on , vo= l., no., pp.1-5, 28-30 Jan. 2011 doi: 10.1109/NCC.2011.5734779 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&is= number=3D5734693 [2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08 [3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01 De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley= ) Envoy=E9 : mercredi 30 septembre 2015 15:10 =C0 : Scharf, Michael (Michael); tcpm@ietf.org Objet : Re: [tcpm] TCP evolution impact on router buffers This paper from 2008 shows no drops with a 5 msec buffer on a heavily-conge= sted core link. Low levels of drops start to show up around 2.5 msec. This = was conducted at Level 3 to test the BDP / sqrt # flows proposal from Stanf= ord. Experimental Study of Router Buffer Sizing - Neda Beheshti, Yashar Ganjali,= Monia Ghobadi, Nick McKeown, and Geoff Salmon They also explored TCP pacing as a way to further reduce buffers. I'm tryin= g to assess if CUBIC results in some pacing effect by decoupling the window= growth from acks. - Lane From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Sent: Tuesday, September 29, 2015 6:56 PM To: Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers This question may also be in scope of the AQM WG, and perhaps ICCRG, but si= nce the communities overlap, I avoid cross-posting... Unfortunately, I don't have any recent pointer or own data. But I am aware = of examples for router buffer dimensioning based on tests with a single TCP= connection, using Reno. In those examples, the dimension of the buffers an= d possibly other related parameters (two-color/three-color meters/policiers= , WRED slopes, H-QoS, etc.) are configured to achieve full "line" speed for= a single TCP connection. As to be expected, this typically comes down to t= he old BDP rule of thumb. But it is rather obvious that a single bulk data = Reno connection is not necessarily a realistic workload these days. So, the= test methodology can be one relevant aspect of the dimensioning problem. I believe that a modern TCP stack with a high-speed congestion control such= as CUBIC will typically need less buffer than the outcome of this dimensio= ning method and the publications 10 years ago. As far as I know, most moder= n TCP stacks operate well over a very wide range of buffer sizes, and the b= uffers can be relatively small. To me, the key question is the latency-thro= ughput tradeoff, and there may not be a one-fits-all answer to that one. Personally, I'd also be interested in more recent pointers on router buffer= dimensioning. I could even see some value in documenting dimensioning guid= elines used in practice, if there was a way to generalize them. Michael From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley= ) Sent: Friday, September 25, 2015 5:41 PM To: tcpm@ietf.org Subject: [tcpm] TCP evolution impact on router buffers I'm trying to track down some research or opinions on how the more recent T= CP schemes impact router buffer needs. There are a number of publications f= rom Stanford and Georgia Tech from about 10 years ago, and I'm trying to as= sess how changes to the algorithms (e.g. CUBIC) and parameters (initial con= gestion window) deployed since then may influence those findings in the dir= ection of more or less buffering being needed. I'd appreciate any input and pointers. Thanks. - Lane --_000_655C07320163294895BBADA28372AF5D484F4C66FR712WXCHMBA15z_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

What pr= events congestion in the case you mention? E.g., what happens if there sign= ificantly more than 400 customers/customer ports with 1G each?

&n= bsp;

I don&#= 8217;t know whether such a fast interface really benefits from AQM, and whe= ther interfaces of that order of magnitude are in scope of document [2], or= not. But just having a very fast link does not imply that congestion can never occur. Otherwise TCPM’s job woul= d be much simpler ;)

&n= bsp;

Michael=

&n= bsp;

&n= bsp;

From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md]
Sent: Tuesday, October 06, 2015 12:54 AM
To: Scharf, Michael (Michael); Kuhn Nicolas; Lane Wigley (lwigley); = tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

 

Can be 400G router interface= considered as a bottleneck capacity if all downstream receivers are at a m= aximum of 1G ?

 

Best regards

 

Veaceslav Roman=

 

 

From: tcpm [tcpm-bounces@ietf.org] on behalf of Scharf, Michael (Michael) [michael.scharf@alcatel-lucent.com]=
Sent: Friday, October 02, 2015 8:05 PM
To: Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org
Subject: Re: [tcpm] TCP evolution impact on router buffers

Regardi= ng Section 7 in [2], I am surprised that a potential impact of CIR/PIR is n= ot mentioned. Unfortunately, I have only a very limited understanding of MP= LS/IP router architectures, but the way bursts hit an AQM could IMHO depend on that.

 

As [2] = is mentioned, one question related to the discussion below. From [2]:

 

3.2.&nb= sp; Buffer size

 

 &= nbsp; The size of the buffers should be carefully chosen, and is to be set<= /span>

 &= nbsp; to the bandwidth-delay product; the bandwidth being the bottleneck

 &= nbsp; capacity and the delay the largest RTT in the considered network.

 

This me= ans of the order of 10GB of buffer size for a 400G router interface, right?= Wouldn’t it make sense to relax the statement “… and is = to be set…” a bit?

 

Michael=

 

 

From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr]
Sent: Wednesday, September 30, 2015 6:41 PM
To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

 

Hi, =

 

In [1], t= he authors evaluate the performance of CUBIC in small buffer regime. Howeve= r, they use the Linux kernel version 2.6.18 (where the IW was 3 and not 10)= . Also, for any tests on CUBIC, small buffers and IW3/10, you may be interested in looking at the scenario ̶= 0;7.  Burst absorption” of the AQM Characterization Guidelines o= f the AQM WG [2]. Concerning TCP pacing, there is an on-going document on a= pacing solution in the slow-start that you may want to have a look and comment [3].

 

I hope th= is helps,

Cheers,

 

Nicolas

 

[1] Jain,= S.; Raina, G., "An experimental evaluation of CUBIC TCP in a small bu= ffer regime," in Communications (NCC), 2011 National Conference on , vol., no., pp.1-= 5, 28-30 Jan. 2011
doi: 10.1109/NCC.2011.5734779
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&a= rnumber=3D5734779&isnumber=3D5734693

[2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08

[3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01

 

De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley)
Envoy=E9 : mercredi 30 septembre 2015 15:10
=C0 : Scharf, Michael (Michael); tcpm@ietf.org
Objet : Re: [tcpm] TCP evolution impact on router buffers

 

This pa= per from 2008 shows no drops with a 5 msec buffer on a heavily-congested co= re link. Low levels of drops start to show up around 2.5 msec. This was con= ducted at Level 3 to test the BDP / sqrt # flows proposal from Stanford.

 

Experimen= tal Study of Router Buffer Sizing – Neda Beheshti, Yashar Ganjali, Mo= nia Ghobadi, Nick McKeown, and Geoff Salmon

 

They also= explored TCP pacing as a way to further reduce buffers. I’m trying t= o assess if CUBIC results in some pacing effect by decoupling the window gr= owth from acks.

 

- Lane

 

 

From: Scharf, Michael (Michael) [mailto= :michael.scharf@alcatel-lucent.com]
Sent: Tuesday, September 29, 2015 6:56 PM
To: Lane Wigley (lwigley); tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

 

This qu= estion may also be in scope of the AQM WG, and perhaps ICCRG, but since the= communities overlap, I avoid cross-posting…

 

Unfortu= nately, I don’t have any recent pointer or own data. But I am aware o= f examples for router buffer dimensioning based on tests with a single TCP = connection, using Reno. In those examples, the dimension of the buffers and possibly other related parameters (two-color/= three-color meters/policiers, WRED slopes, H-QoS, etc.) are configured to a= chieve full “line” speed for a single TCP connection. As to be = expected, this typically comes down to the old BDP rule of thumb. But it is rather obvious that a single bulk data Reno c= onnection is not necessarily a realistic workload these days. So, the test = methodology can be one relevant aspect of the dimensioning problem.<= span lang=3D"EN-US" style=3D"color:black">

 

I belie= ve that a modern TCP stack with a high-speed congestion control such as CUB= IC will typically need less buffer than the outcome of this dimensioning me= thod and the publications 10 years ago. As far as I know, most modern TCP stacks operate well over a very wide ran= ge of buffer sizes, and the buffers can be relatively small. To me, the key= question is the latency-throughput tradeoff, and there may not be a one-fi= ts-all answer to that one.

 

Persona= lly, I’d also be interested in more recent pointers on router buffer = dimensioning. I could even see some value in documenting dimensioning guide= lines used in practice, if there was a way to generalize them.

 

Michael=

 

 

 

From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley)
Sent: Friday, September 25, 2015 5:41 PM
To: tcpm@ietf.org=
Subject: [tcpm] TCP evolution impact on router buffers

 

I’m= trying to track down some research or opinions on how the more recent TCP = schemes impact router buffer needs. There are a number of publications from= Stanford and Georgia Tech from about 10 years ago, and I’m trying to assess how changes to the algorithms (e.g. CU= BIC) and parameters (initial congestion window) deployed since then may inf= luence those findings in the direction of more or less buffering being need= ed.

 

I’d= appreciate any input and pointers. Thanks.

 

- Lane

 

 

--_000_655C07320163294895BBADA28372AF5D484F4C66FR712WXCHMBA15z_-- From nobody Mon Oct 5 18:06:01 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AC0A51B301B for ; Mon, 5 Oct 2015 18:05:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n4KTMFPFbqhW for ; Mon, 5 Oct 2015 18:05:58 -0700 (PDT) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BC0601B301A for ; Mon, 5 Oct 2015 18:05:58 -0700 (PDT) Received: by obbda8 with SMTP id da8so142890915obb.1 for ; Mon, 05 Oct 2015 18:05:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=tzEtajBI+4XYehlU/bqdf471scC/1KXD/U0ah5BxO5U=; b=YvZdMrsW5ly5/30PGSo18H6JowdBF1hYeY6vFX9iphTe9I6Ri9aRZfbg1L9/A5F6u6 U1peiloHJWyn7EGBA5GDFp9zyoOn2dSugaKgMS4NpF+lF0/oeBCUehHQBXF+dqxzTLyC 8mUsCrIeX8gDkL2F39aAwkZyZJ+i9POMPjaHlQyXBst12hw3S7D33OTlOo+PEGpKPN+X /8Aun5TCQopVkojZRDKkWkH69Vhc9rf6k79fRBHtIg+bXsVXg90dKui3oJkIJWKyF6+u FeazwekiXgtE2OLzP2WddRpl2LcB4NPOG7pkLFbBsvqUEPOZ21LQOnsAcWYTfzhQvYVj VDew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=tzEtajBI+4XYehlU/bqdf471scC/1KXD/U0ah5BxO5U=; b=X5q3kDqAVwBYqSs2v+lYNTtAhZeiMTocSM9wsS+LuGux+oJ6nRiM6zSPEH++DRA0Zf ewOxjOzLPyU2npG+2CeFPABXJJpsoVkzNxeFHJhMQMLqPwtmz5niwounWw9txs+5pV+q /t7KT6hEdIy0dCQqWEuJoyOYk/L7yQZaz1c2UyX7clDprd4JxJljpidbXCg84JkUHM3U Ybo5CFqEA/OjfOopH9UK2r+Dk9qPaDHMGNdcNZfnOdjsjrTTHxC6cuW0jfbjOgUyEh1W CjguG5cwbXI0s5gr6l9KwgFu4MpeduXluxg7MbKPOlB8G2KIJGZHbhgLd7uXHZiqvcgN qboQ== X-Gm-Message-State: ALoCoQnZKCeaRyFVtCcHxKLAgqM+35AgtX3NfqpsuvCCfGIufoFJEP3ZXeZDM7B0H6yvSGJgukSb MIME-Version: 1.0 X-Received: by 10.60.51.5 with SMTP id g5mr20212883oeo.35.1444093557044; Mon, 05 Oct 2015 18:05:57 -0700 (PDT) Received: by 10.202.213.136 with HTTP; Mon, 5 Oct 2015 18:05:56 -0700 (PDT) In-Reply-To: <7DBBB686E19D2049ADAACD210B474BB10166E8BDBA@XCHSRV01.main.orange.md> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E8BDBA@XCHSRV01.main.orange.md> Date: Mon, 5 Oct 2015 21:05:56 -0400 Message-ID: From: Neal Cardwell To: Veaceslav ROMAN Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , Ingemar Johansson S , Eric Dumazet , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Oct 2015 01:05:59 -0000 On Mon, Oct 5, 2015 at 6:07 PM, Veaceslav ROMAN wrote: > We've managed to recompile the kernel 4.1 with tcp_cubic HYSTART_DELAY_MIN (10U<<3) , 10ms. > Well, we had time only for one test cycle, but results are very promising: > > Download 10MB, High Bandwidths, HYSTART_DELAY_MIN 10 ms: > Hystart average download time: 1.16 s > NoHystart average download time: 0.93 s ... > Download 10MB, High Bandwidths, HYSTART_DELAY_MIN 4 ms: > Hystart average download time: 1.42 s > NoHystart average download time: 0.89 s Thanks! That is very useful and interesting data. Would you be able to try a few other values, like 15ms and 20ms? neal From nobody Mon Oct 5 23:18:07 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 601B61A6F56 for ; Mon, 5 Oct 2015 23:18:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.609 X-Spam-Level: X-Spam-Status: No, score=-2.609 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Os74v_ZTiswQ for ; Mon, 5 Oct 2015 23:18:02 -0700 (PDT) Received: from mailfilter.orange.md (mailfilter.orange.md [94.243.64.204]) by ietfa.amsl.com (Postfix) with ESMTP id 8FCC81A1A48 for ; Mon, 5 Oct 2015 23:18:01 -0700 (PDT) Received: from localhost.localdomain (antispam.orange.md [127.0.0.1]) by localhost (Postfix) with ESMTP id 0D61827E020; Tue, 6 Oct 2015 09:18:21 +0300 (EEST) Received: from antispam.orange.md by antispam.orange.md with queue id 250138-1; Tue, 06 Oct 2015 06:18:20 GMT Received: from XCHSRV03.main.orange.md (unknown [192.168.200.63]) by mailfilter.orange.md (Postfix) with ESMTP id CC7503B2640; Tue, 6 Oct 2015 09:18:20 +0300 (EEST) Received: from XCHSRV01.main.orange.md ([fe80::685f:aef6:93b0:dea7]) by XCHSRV03.main.orange.md ([fe80::6dbc:28dd:213b:8931%14]) with mapi id 14.02.0328.009; Tue, 6 Oct 2015 09:17:59 +0300 From: Veaceslav ROMAN To: "Scharf, Michael (Michael)" , "Kuhn Nicolas" , "Lane Wigley (lwigley)" , "tcpm@ietf.org" Thread-Topic: TCP evolution impact on router buffers Thread-Index: AdD3qDJMG7WkdcJDTiy1iIy/H6510wDXEZsQAB6kmpAABLWoIABn/A/AAKN/QXAAAEdhcAAPdKKg Date: Tue, 6 Oct 2015 06:17:58 +0000 Message-ID: <1q60nkv7j85juo1c62vkyl83.1444112016101@email.android.com> References: <655C07320163294895BBADA28372AF5D484D799E@FR712WXCHMBA15.zeu.alcatel-lucent.com> <46616f47b42841f0bb7a2afb8bdeac98@XCH-ALN-009.cisco.com> , <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> <7DBBB686E19D2049ADAACD210B474BB10166E8BDE1@XCHSRV01.main.orange.md>, <655C07320163294895BBADA28372AF5D484F4C66@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: <655C07320163294895BBADA28372AF5D484F4C66@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: en-US, ro-RO Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_1q60nkv7j85juo1c62vkyl831444112016101emailandroidcom_" MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] TCP evolution impact on router buffers X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Oct 2015 06:18:06 -0000 --_000_1q60nkv7j85juo1c62vkyl831444112016101emailandroidcom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Sure, congestion may happen even at 400. The question is what B one should = use in BDP estimation for buffer sizing and if BDP approach is the right on= e in conditions when individual stream is physically limited to much lower = spead? Sent from my Samsung device -------- Original message -------- From: "Scharf, Michael (Michael)" Date: 2015/10/06 02:10 (GMT+02:00) To: Veaceslav ROMAN , Kuhn Nicolas , "Lane Wigley (lwigley)" , tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers What prevents congestion in the case you mention? E.g., what happens if the= re significantly more than 400 customers/customer ports with 1G each? I don=92t know whether such a fast interface really benefits from AQM, and = whether interfaces of that order of magnitude are in scope of document [2],= or not. But just having a very fast link does not imply that congestion ca= n never occur. Otherwise TCPM=92s job would be much simpler ;) Michael From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] Sent: Tuesday, October 06, 2015 12:54 AM To: Scharf, Michael (Michael); Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ie= tf.org Subject: RE: TCP evolution impact on router buffers Can be 400G router interface considered as a bottleneck capacity if all dow= nstream receivers are at a maximum of 1G ? Best regards Veaceslav Roman ________________________________ From: tcpm [tcpm-bounces@ietf.org] on behalf of Scharf, Michael (Michael) [= michael.scharf@alcatel-lucent.com] Sent: Friday, October 02, 2015 8:05 PM To: Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org Subject: Re: [tcpm] TCP evolution impact on router buffers Regarding Section 7 in [2], I am surprised that a potential impact of CIR/P= IR is not mentioned. Unfortunately, I have only a very limited understandin= g of MPLS/IP router architectures, but the way bursts hit an AQM could IMHO= depend on that. As [2] is mentioned, one question related to the discussion below. From [2]= : 3.2. Buffer size The size of the buffers should be carefully chosen, and is to be set to the bandwidth-delay product; the bandwidth being the bottleneck capacity and the delay the largest RTT in the considered network. This means of the order of 10GB of buffer size for a 400G router interface,= right? Wouldn=92t it make sense to relax the statement =93=85 and is to be= set=85=94 a bit? Michael From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] Sent: Wednesday, September 30, 2015 6:41 PM To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Hi, In [1], the authors evaluate the performance of CUBIC in small buffer regim= e. However, they use the Linux kernel version 2.6.18 (where the IW was 3 an= d not 10). Also, for any tests on CUBIC, small buffers and IW3/10, you may = be interested in looking at the scenario =937. Burst absorption=94 of the = AQM Characterization Guidelines of the AQM WG [2]. Concerning TCP pacing, t= here is an on-going document on a pacing solution in the slow-start that yo= u may want to have a look and comment [3]. I hope this helps, Cheers, Nicolas [1] Jain, S.; Raina, G., "An experimental evaluation of CUBIC TCP in a smal= l buffer regime," in Communications (NCC), 2011 National Conference on , vo= l., no., pp.1-5, 28-30 Jan. 2011 doi: 10.1109/NCC.2011.5734779 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&is= number=3D5734693 [2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08 [3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01 De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley= ) Envoy=E9 : mercredi 30 septembre 2015 15:10 =C0 : Scharf, Michael (Michael); tcpm@ietf.org Objet : Re: [tcpm] TCP evolution impact on router buffers This paper from 2008 shows no drops with a 5 msec buffer on a heavily-conge= sted core link. Low levels of drops start to show up around 2.5 msec. This = was conducted at Level 3 to test the BDP / sqrt # flows proposal from Stanf= ord. Experimental Study of Router Buffer Sizing =96 Neda Beheshti, Yashar Ganjal= i, Monia Ghobadi, Nick McKeown, and Geoff Salmon They also explored TCP pacing as a way to further reduce buffers. I=92m try= ing to assess if CUBIC results in some pacing effect by decoupling the wind= ow growth from acks. - Lane From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Sent: Tuesday, September 29, 2015 6:56 PM To: Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers This question may also be in scope of the AQM WG, and perhaps ICCRG, but si= nce the communities overlap, I avoid cross-posting=85 Unfortunately, I don=92t have any recent pointer or own data. But I am awar= e of examples for router buffer dimensioning based on tests with a single T= CP connection, using Reno. In those examples, the dimension of the buffers = and possibly other related parameters (two-color/three-color meters/policie= rs, WRED slopes, H-QoS, etc.) are configured to achieve full =93line=94 spe= ed for a single TCP connection. As to be expected, this typically comes dow= n to the old BDP rule of thumb. But it is rather obvious that a single bulk= data Reno connection is not necessarily a realistic workload these days. S= o, the test methodology can be one relevant aspect of the dimensioning prob= lem. I believe that a modern TCP stack with a high-speed congestion control such= as CUBIC will typically need less buffer than the outcome of this dimensio= ning method and the publications 10 years ago. As far as I know, most moder= n TCP stacks operate well over a very wide range of buffer sizes, and the b= uffers can be relatively small. To me, the key question is the latency-thro= ughput tradeoff, and there may not be a one-fits-all answer to that one. Personally, I=92d also be interested in more recent pointers on router buff= er dimensioning. I could even see some value in documenting dimensioning gu= idelines used in practice, if there was a way to generalize them. Michael From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley= ) Sent: Friday, September 25, 2015 5:41 PM To: tcpm@ietf.org Subject: [tcpm] TCP evolution impact on router buffers I=92m trying to track down some research or opinions on how the more recent= TCP schemes impact router buffer needs. There are a number of publications= from Stanford and Georgia Tech from about 10 years ago, and I=92m trying t= o assess how changes to the algorithms (e.g. CUBIC) and parameters (initial= congestion window) deployed since then may influence those findings in the= direction of more or less buffering being needed. I=92d appreciate any input and pointers. Thanks. - Lane --_000_1q60nkv7j85juo1c62vkyl831444112016101emailandroidcom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Sure, congestion may happen even at 400. The question is what B one sh= ould use in BDP estimation for buffer sizing and if BDP approach is the rig= ht one in conditions when individual stream is physically limited to much l= ower spead? 



Sent from my Samsung device


-------- Original message --------
From: "Scharf, Michael (Michael)" <michael.scharf@alcatel-luce= nt.com>
Date: 2015/10/06 02:10 (GMT+02:00)
To: Veaceslav ROMAN <Veaceslav.Roman@orange.md>, Kuhn Nicolas <Nic= olas.Kuhn@cnes.fr>, "Lane Wigley (lwigley)" <lwigley@cisco.= com>, tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

What pr= events congestion in the case you mention? E.g., what happens if there sign= ificantly more than 400 customers/customer ports with 1G each?

 <= /span>

I don= =92t know whether such a fast interface really benefits from AQM, and wheth= er interfaces of that order of magnitude are in scope of document [2], or n= ot. But just having a very fast link does not imply that congestion can never occur. Otherwise TCPM=92s job would be= much simpler ;)

 <= /span>

Michael=

 <= /span>

 <= /span>

From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md]
Sent: Tuesday, October 06, 2015 12:54 AM
To: Scharf, Michael (Michael); Kuhn Nicolas; Lane Wigley (lwigley); = tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

 

Can be 400G router interfa= ce considered as a bottleneck capacity if all downstream receivers are at a= maximum of 1G ?

 

Best regards<= /span>

 =

Veaceslav Rom= an

 

 

From: tcpm [tcpm-bounces@ietf.org] on behalf of Scharf, Michael (Michael) [michael.sc= harf@alcatel-lucent.com]
Sent: Friday, October 02, 2015 8:05 PM
To: Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org
Subject: Re: [tcpm] TCP evolution impact on router buffers

Regardi= ng Section 7 in [2], I am surprised that a potential impact of CIR/PIR is n= ot mentioned. Unfortunately, I have only a very limited understanding of MP= LS/IP router architectures, but the way bursts hit an AQM could IMHO depend on that.

 

As [2] = is mentioned, one question related to the discussion below. From [2]:

 

3.2.&nb= sp; Buffer size

 

 &= nbsp; The size of the buffers should be carefully chosen, and is to be set<= /span>

 &= nbsp; to the bandwidth-delay product; the bandwidth being the bottleneck

 &= nbsp; capacity and the delay the largest RTT in the considered network.

 

This me= ans of the order of 10GB of buffer size for a 400G router interface, right?= Wouldn=92t it make sense to relax the statement =93=85 and is to be set=85= =94 a bit?

 

Michael=

 

 

From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr]
Sent: Wednesday, September 30, 2015 6:41 PM
To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

 

Hi,

 

In [1], t= he authors evaluate the performance of CUBIC in small buffer regime. Howeve= r, they use the Linux kernel version 2.6.18 (where the IW was 3 and not 10)= . Also, for any tests on CUBIC, small buffers and IW3/10, you may be interested in looking at the scenario =937.=   Burst absorption=94 of the AQM Characterization Guidelines of the AQ= M WG [2]. Concerning TCP pacing, there is an on-going document on a pacing = solution in the slow-start that you may want to have a look and comment [3].

 

I hope th= is helps,

Cheers,

 

Nicolas

 

[1] Jain,= S.; Raina, G., "An experimental evaluation of CUBIC TCP in a small bu= ffer regime," in Communications (NCC), 2011 National Conference on , vol., no., pp.1-= 5, 28-30 Jan. 2011
doi: 10.1109/NCC.2011.5734779
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&= arnumber=3D5734779&isnumber=3D5734693

[2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08

[3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01

 

De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley)
Envoy=E9 : mercredi 30 septembre 2015 15:10
=C0 : Scharf, Michael (Michael); tcpm@ietf.org
Objet : Re: [tcpm] TCP evolution impact on router buffers

 

This pa= per from 2008 shows no drops with a 5 msec buffer on a heavily-congested co= re link. Low levels of drops start to show up around 2.5 msec. This was con= ducted at Level 3 to test the BDP / sqrt # flows proposal from Stanford.

 

Experimen= tal Study of Router Buffer Sizing =96 Neda Beheshti, Yashar Ganjali, Monia = Ghobadi, Nick McKeown, and Geoff Salmon

 

They also= explored TCP pacing as a way to further reduce buffers. I=92m trying to as= sess if CUBIC results in some pacing effect by decoupling the window growth= from acks.

 

- Lane

 

 

From: Scharf, Michael (Michael)= [ma= ilto:michael.scharf@alcatel-lucent.com]
Sent: Tuesday, September 29, 2015 6:56 PM
To: Lane Wigley (lwigley); tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

 

This qu= estion may also be in scope of the AQM WG, and perhaps ICCRG, but since the= communities overlap, I avoid cross-posting=85

 

Unfortu= nately, I don=92t have any recent pointer or own data. But I am aware of ex= amples for router buffer dimensioning based on tests with a single TCP conn= ection, using Reno. In those examples, the dimension of the buffers and possibly other related parameters (two-color/= three-color meters/policiers, WRED slopes, H-QoS, etc.) are configured to a= chieve full =93line=94 speed for a single TCP connection. As to be expected= , this typically comes down to the old BDP rule of thumb. But it is rather obvious that a single bulk data Reno c= onnection is not necessarily a realistic workload these days. So, the test = methodology can be one relevant aspect of the dimensioning problem.<= span lang=3D"EN-US" style=3D"color:black">

 

I belie= ve that a modern TCP stack with a high-speed congestion control such as CUB= IC will typically need less buffer than the outcome of this dimensioning me= thod and the publications 10 years ago. As far as I know, most modern TCP stacks operate well over a very wide ran= ge of buffer sizes, and the buffers can be relatively small. To me, the key= question is the latency-throughput tradeoff, and there may not be a one-fi= ts-all answer to that one.

 

Persona= lly, I=92d also be interested in more recent pointers on router buffer dime= nsioning. I could even see some value in documenting dimensioning guideline= s used in practice, if there was a way to generalize them.<= /p>

 

Michael=

 

 

 

From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley)
Sent: Friday, September 25, 2015 5:41 PM
To: tcpm@ietf.org=
Subject: [tcpm] TCP evolution impact on router buffers

 

I=92m try= ing to track down some research or opinions on how the more recent TCP sche= mes impact router buffer needs. There are a number of publications from Sta= nford and Georgia Tech from about 10 years ago, and I=92m trying to assess how changes to the algorithms (e.g. CUBIC)= and parameters (initial congestion window) deployed since then may influen= ce those findings in the direction of more or less buffering being needed.<= /span>

 

I=92d app= reciate any input and pointers. Thanks.

 

- Lane

 

 

--_000_1q60nkv7j85juo1c62vkyl831444112016101emailandroidcom_-- From nobody Tue Oct 6 06:57:33 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E5D131ACC88 for ; Tue, 6 Oct 2015 06:57:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -12.711 X-Spam-Level: X-Spam-Status: No, score=-12.711 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, J_CHICKENPOX_31=0.6, J_CHICKENPOX_41=0.6, J_CHICKENPOX_61=0.6, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MXjhy1bBr7k5 for ; Tue, 6 Oct 2015 06:57:29 -0700 (PDT) Received: from alln-iport-1.cisco.com (alln-iport-1.cisco.com [173.37.142.88]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1AB9B1ACC83 for ; Tue, 6 Oct 2015 06:57:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=9649; q=dns/txt; s=iport; t=1444139849; x=1445349449; h=from:to:subject:date:message-id: content-transfer-encoding:mime-version; bh=MyKD01YLRoCMzPQfvYqIsh7Lmw273BI8NHIRz26FdTk=; b=Rf0YhLuli8Y4HPP/GdCI6BNGxaeYjmG7lVtb/7pDNxApI6lS2RZCtTT4 rFCJ53xYz9LVdK8ADyZlEq29yE5xMsrdAeMYw4ypfeyPirxOUrO4K84x6 LQZEuEh/1U9D+xVhiVTcdUGMUvUFM9owtiDvvRPpv4y/Zo4ovzAav/YC6 8=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0D8AQDR0hNW/5NdJa1egydUbga+HgENgVoXCoV5AoEyOBQBAQEBAQEBfwuEJAEBAQMBAQEBNy4GEA0BCBEDAQEBHwkuCxQJCQEEEwgTiAsIDZsYo3EBAQEBAQEBAwEBAQEBAQEBARmGc4R+hCoKBwEmOIQoBYV/CYEwjkwBhReCb4UJgV5IkQeIRwEfAQFChAJxAYZ1CRcjgQYBAQE X-IronPort-AV: E=Sophos;i="5.17,644,1437436800"; d="scan'208";a="195082253" Received: from rcdn-core-11.cisco.com ([173.37.93.147]) by alln-iport-1.cisco.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 06 Oct 2015 13:57:28 +0000 Received: from XCH-RCD-007.cisco.com (xch-rcd-007.cisco.com [173.37.102.17]) by rcdn-core-11.cisco.com (8.14.5/8.14.5) with ESMTP id t96DvQwk030382 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL) for ; Tue, 6 Oct 2015 13:57:27 GMT Received: from xch-aln-009.cisco.com (173.36.7.19) by XCH-RCD-007.cisco.com (173.37.102.17) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Tue, 6 Oct 2015 08:57:26 -0500 Received: from xch-aln-009.cisco.com ([173.36.7.19]) by XCH-ALN-009.cisco.com ([173.36.7.19]) with mapi id 15.00.1104.000; Tue, 6 Oct 2015 08:57:26 -0500 From: "Lane Wigley (lwigley)" To: "tcpm@ietf.org" Thread-Topic: TCP evolution impact on router buffers Thread-Index: AdEAOmhRaNCRCT+GQW2PzfJxLmBguQ== Date: Tue, 6 Oct 2015 13:57:25 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.116.71.38] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] TCP evolution impact on router buffers X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Oct 2015 13:57:32 -0000 I think you want to use "B" of 400, but if this is an Internet link carryin= g TCP, this is a case where the Stanford work suggesting BDP / sqrt (# flow= s) [1] should apply as the TCP flows don't get highly synchronized when the= y are many of them. I'd want a bit of margin for error [2], but the trend s= ince that work seems to be toward more smoothing of individual TCP sessions= and thus less margin needed. The key item from the first paper is that in 2008, they took 3 highly-conge= sted links at Level 3 and changed the 190 msec BDP buffers to 10, 5, 2.5, a= nd 1 msec. They didn't see any drops at 5 msec and saw 0.02% at 2.5 msec. W= ith 10000 flows, this was inline with their predictions. After some back an= d forth with other researchers (including Georgia Tech), they suggested BDP= / 10 as a conservative sizing. [1] Behishti [www-afs.secure-endpoints.com/afs/cs.pitt.edu/usr/.../Buff-Siz= ing-Exp.pdf] [2] Ganuali [sysweb.cs.toronto.edu/publication_files/.../Buffers-Update---C= CR-06.pdf] - Lane > Date: Tue, 6 Oct 2015 06:17:58 +0000 > From: Veaceslav ROMAN > To: "Scharf, Michael (Michael)" , > "Kuhn Nicolas" , "Lane Wigley (lwigley)" > , "tcpm@ietf.org" > Subject: Re: [tcpm] TCP evolution impact on router buffers > Message-ID: > <1q60nkv7j85juo1c62vkyl83.1444112016101@email.android.com> > Content-Type: text/plain; charset=3D"windows-1252" >=20 > Sure, congestion may happen even at 400. The question is what B one > should use in BDP estimation for buffer sizing and if BDP approach is the > right one in conditions when individual stream is physically limited to m= uch > lower spead? >=20 >=20 >=20 > Sent from my Samsung device >=20 >=20 > -------- Original message -------- > From: "Scharf, Michael (Michael)" > Date: 2015/10/06 02:10 (GMT+02:00) > To: Veaceslav ROMAN , Kuhn Nicolas > , "Lane Wigley (lwigley)" , > tcpm@ietf.org > Subject: RE: TCP evolution impact on router buffers >=20 > What prevents congestion in the case you mention? E.g., what happens if > there significantly more than 400 customers/customer ports with 1G each? >=20 > I don?t know whether such a fast interface really benefits from AQM, and > whether interfaces of that order of magnitude are in scope of document [2= ], > or not. But just having a very fast link does not imply that congestion c= an > never occur. Otherwise TCPM?s job would be much simpler ;) >=20 > Michael >=20 >=20 > From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] > Sent: Tuesday, October 06, 2015 12:54 AM > To: Scharf, Michael (Michael); Kuhn Nicolas; Lane Wigley (lwigley); > tcpm@ietf.org > Subject: RE: TCP evolution impact on router buffers >=20 >=20 > Can be 400G router interface considered as a bottleneck capacity if all > downstream receivers are at a maximum of 1G ? >=20 >=20 > Best regards >=20 > Veaceslav Roman >=20 >=20 > ________________________________ > From: tcpm [tcpm-bounces@ietf.org] on behalf of Scharf, Michael (Michael) > [michael.scharf@alcatel-lucent.com] > Sent: Friday, October 02, 2015 8:05 PM > To: Kuhn Nicolas; Lane Wigley (lwigley); > tcpm@ietf.org > Subject: Re: [tcpm] TCP evolution impact on router buffers Regarding Sect= ion > 7 in [2], I am surprised that a potential impact of CIR/PIR is not mentio= ned. > Unfortunately, I have only a very limited understanding of MPLS/IP router > architectures, but the way bursts hit an AQM could IMHO depend on that. >=20 > As [2] is mentioned, one question related to the discussion below. From [= 2]: >=20 > 3.2. Buffer size >=20 > The size of the buffers should be carefully chosen, and is to be set > to the bandwidth-delay product; the bandwidth being the bottleneck > capacity and the delay the largest RTT in the considered network. >=20 > This means of the order of 10GB of buffer size for a 400G router interfac= e, > right? Wouldn?t it make sense to relax the statement ?? and is to be set?= ? a > bit? >=20 > Michael >=20 >=20 > From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] > Sent: Wednesday, September 30, 2015 6:41 PM > To: Lane Wigley (lwigley); Scharf, Michael (Michael); > tcpm@ietf.org > Subject: RE: TCP evolution impact on router buffers >=20 > Hi, >=20 > In [1], the authors evaluate the performance of CUBIC in small buffer reg= ime. > However, they use the Linux kernel version 2.6.18 (where the IW was 3 and > not 10). Also, for any tests on CUBIC, small buffers and IW3/10, you may = be > interested in looking at the scenario ?7. Burst absorption? of the AQM > Characterization Guidelines of the AQM WG [2]. Concerning TCP pacing, > there is an on-going document on a pacing solution in the slow-start that > you may want to have a look and comment [3]. >=20 > I hope this helps, > Cheers, >=20 > Nicolas >=20 > [1] Jain, S.; Raina, G., "An experimental evaluation of CUBIC TCP in a sm= all > buffer regime," in Communications (NCC), 2011 National Conference on , > vol., no., pp.1-5, 28-30 Jan. 2011 > doi: 10.1109/NCC.2011.5734779 > URL: > http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&isnum > ber=3D5734693 > [2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08 > [3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01 >=20 > De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigl= ey) > Envoy? : mercredi 30 septembre 2015 15:10 ? : Scharf, Michael (Michael); > tcpm@ietf.org > Objet : Re: [tcpm] TCP evolution impact on router buffers >=20 > This paper from 2008 shows no drops with a 5 msec buffer on a heavily- > congested core link. Low levels of drops start to show up around 2.5 msec= . > This was conducted at Level 3 to test the BDP / sqrt # flows proposal fro= m > Stanford. >=20 > Experimental Study of Router Buffer Sizing ? Neda Beheshti, Yashar Ganjal= i, > Monia Ghobadi, Nick McKeown, and Geoff Salmon >=20 > They also explored TCP pacing as a way to further reduce buffers. I?m try= ing > to assess if CUBIC results in some pacing effect by decoupling the window > growth from acks. >=20 > - Lane >=20 >=20 > From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com= ] > Sent: Tuesday, September 29, 2015 6:56 PM > To: Lane Wigley (lwigley); tcpm@ietf.org > Subject: RE: TCP evolution impact on router buffers >=20 > This question may also be in scope of the AQM WG, and perhaps ICCRG, but > since the communities overlap, I avoid cross-posting? >=20 > Unfortunately, I don?t have any recent pointer or own data. But I am awar= e > of examples for router buffer dimensioning based on tests with a single T= CP > connection, using Reno. In those examples, the dimension of the buffers a= nd > possibly other related parameters (two-color/three-color meters/policiers= , > WRED slopes, H-QoS, etc.) are configured to achieve full ?line? speed for= a > single TCP connection. As to be expected, this typically comes down to th= e > old BDP rule of thumb. But it is rather obvious that a single bulk data R= eno > connection is not necessarily a realistic workload these days. So, the te= st > methodology can be one relevant aspect of the dimensioning problem. >=20 > I believe that a modern TCP stack with a high-speed congestion control su= ch > as CUBIC will typically need less buffer than the outcome of this > dimensioning method and the publications 10 years ago. As far as I know, > most modern TCP stacks operate well over a very wide range of buffer size= s, > and the buffers can be relatively small. To me, the key question is the > latency-throughput tradeoff, and there may not be a one-fits-all answer t= o > that one. >=20 > Personally, I?d also be interested in more recent pointers on router buff= er > dimensioning. I could even see some value in documenting dimensioning > guidelines used in practice, if there was a way to generalize them. >=20 > Michael >=20 >=20 >=20 > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley > (lwigley) > Sent: Friday, September 25, 2015 5:41 PM > To: tcpm@ietf.org > Subject: [tcpm] TCP evolution impact on router buffers >=20 > I?m trying to track down some research or opinions on how the more recent > TCP schemes impact router buffer needs. There are a number of publication= s > from Stanford and Georgia Tech from about 10 years ago, and I?m trying to > assess how changes to the algorithms (e.g. CUBIC) and parameters (initial > congestion window) deployed since then may influence those findings in th= e > direction of more or less buffering being needed. >=20 > I?d appreciate any input and pointers. Thanks. >=20 > - Lane >=20 >=20 > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > a599/attachment.html> >=20 > ------------------------------ >=20 > Subject: Digest Footer >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm >=20 >=20 > ------------------------------ >=20 > End of tcpm Digest, Vol 138, Issue 12 > ************************************* From nobody Wed Oct 7 03:16:42 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AD7871B2D62 for ; Wed, 7 Oct 2015 03:16:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.26 X-Spam-Level: X-Spam-Status: No, score=-2.26 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_FR=0.35, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w0Fs0htZQqPE for ; Wed, 7 Oct 2015 03:16:34 -0700 (PDT) Received: from TW-EDGE-P04.cnesnet.ad.cnes.fr (edge4.cnes.fr [132.149.49.4]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 023A61B2D60 for ; Wed, 7 Oct 2015 03:16:33 -0700 (PDT) From: Kuhn Nicolas To: "Scharf, Michael (Michael)" , "Lane Wigley (lwigley)" , "tcpm@ietf.org" Thread-Topic: TCP evolution impact on router buffers Thread-Index: AdD3qDJMG7WkdcJDTiy1iIy/H6510wDXEZsQAB6kmpAABLWoIABn/A/AAOxXFcA= Date: Wed, 7 Oct 2015 10:16:30 +0000 Message-ID: References: <655C07320163294895BBADA28372AF5D484D799E@FR712WXCHMBA15.zeu.alcatel-lucent.com> <46616f47b42841f0bb7a2afb8bdeac98@XCH-ALN-009.cisco.com> <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: en-US Content-Language: fr-FR X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: SMEX-11.0.0.4179-8.000.1202-21864.005 x-tm-as-result: No--37.930500-8.000000-31 x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No Content-Type: multipart/alternative; boundary="_000_F3B0A07CFD358240926B78A680E166FF835AB9TWMBXP03cnesnetad_" MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] TCP evolution impact on router buffers X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Oct 2015 10:16:40 -0000 --_000_F3B0A07CFD358240926B78A680E166FF835AB9TWMBXP03cnesnetad_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, 1- Regarding the CIR/PIR IMHO this is out of the scope of the characterization guidelines. However, = if you think that we should introduce more information on the MPLS/IP route= r architectures please let us know. The only problem being that we want to = be quite generic and do not want to focus on specific architectures. 2- Regarding buffer sizes Since buffer sizing is not the only problem of evaluating AQMs, we have loo= ked for RFC clearly stipulating how the "adequate" buffer size can be found= . As said in RFC7597: " The optimum buffer size is a function of operational requirements and should generally be sized to be sufficient to buffer the largest normal traffic burst that is expected. This size depends on the amount and burstiness of traffic arriving at the queue and the rate at which traffic leaves the queue. " In the characterization draft, the recommendation on the buffer size is not= a "MUST". If you think that we should more clearly say that the size of the buffer MA= Y be set to the BDP, let us know. In its current state, it has been voluntary that the draft does not impose = any buffer size. Cheers, Nicolas De : Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Envoy=E9 : vendredi 2 octobre 2015 19:06 =C0 : Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org Objet : RE: TCP evolution impact on router buffers Regarding Section 7 in [2], I am surprised that a potential impact of CIR/P= IR is not mentioned. Unfortunately, I have only a very limited understandin= g of MPLS/IP router architectures, but the way bursts hit an AQM could IMHO= depend on that. As [2] is mentioned, one question related to the discussion below. From [2]= : 3.2. Buffer size The size of the buffers should be carefully chosen, and is to be set to the bandwidth-delay product; the bandwidth being the bottleneck capacity and the delay the largest RTT in the considered network. This means of the order of 10GB of buffer size for a 400G router interface,= right? Wouldn't it make sense to relax the statement "... and is to be set= ..." a bit? Michael From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] Sent: Wednesday, September 30, 2015 6:41 PM To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Hi, In [1], the authors evaluate the performance of CUBIC in small buffer regim= e. However, they use the Linux kernel version 2.6.18 (where the IW was 3 an= d not 10). Also, for any tests on CUBIC, small buffers and IW3/10, you may = be interested in looking at the scenario "7. Burst absorption" of the AQM = Characterization Guidelines of the AQM WG [2]. Concerning TCP pacing, there= is an on-going document on a pacing solution in the slow-start that you ma= y want to have a look and comment [3]. I hope this helps, Cheers, Nicolas [1] Jain, S.; Raina, G., "An experimental evaluation of CUBIC TCP in a smal= l buffer regime," in Communications (NCC), 2011 National Conference on , vo= l., no., pp.1-5, 28-30 Jan. 2011 doi: 10.1109/NCC.2011.5734779 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&is= number=3D5734693 [2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08 [3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01 De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley= ) Envoy=E9 : mercredi 30 septembre 2015 15:10 =C0 : Scharf, Michael (Michael); tcpm@ietf.org Objet : Re: [tcpm] TCP evolution impact on router buffers This paper from 2008 shows no drops with a 5 msec buffer on a heavily-conge= sted core link. Low levels of drops start to show up around 2.5 msec. This = was conducted at Level 3 to test the BDP / sqrt # flows proposal from Stanf= ord. Experimental Study of Router Buffer Sizing - Neda Beheshti, Yashar Ganjali,= Monia Ghobadi, Nick McKeown, and Geoff Salmon They also explored TCP pacing as a way to further reduce buffers. I'm tryin= g to assess if CUBIC results in some pacing effect by decoupling the window= growth from acks. - Lane From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Sent: Tuesday, September 29, 2015 6:56 PM To: Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers This question may also be in scope of the AQM WG, and perhaps ICCRG, but si= nce the communities overlap, I avoid cross-posting... Unfortunately, I don't have any recent pointer or own data. But I am aware = of examples for router buffer dimensioning based on tests with a single TCP= connection, using Reno. In those examples, the dimension of the buffers an= d possibly other related parameters (two-color/three-color meters/policiers= , WRED slopes, H-QoS, etc.) are configured to achieve full "line" speed for= a single TCP connection. As to be expected, this typically comes down to t= he old BDP rule of thumb. But it is rather obvious that a single bulk data = Reno connection is not necessarily a realistic workload these days. So, the= test methodology can be one relevant aspect of the dimensioning problem. I believe that a modern TCP stack with a high-speed congestion control such= as CUBIC will typically need less buffer than the outcome of this dimensio= ning method and the publications 10 years ago. As far as I know, most moder= n TCP stacks operate well over a very wide range of buffer sizes, and the b= uffers can be relatively small. To me, the key question is the latency-thro= ughput tradeoff, and there may not be a one-fits-all answer to that one. Personally, I'd also be interested in more recent pointers on router buffer= dimensioning. I could even see some value in documenting dimensioning guid= elines used in practice, if there was a way to generalize them. Michael From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley= ) Sent: Friday, September 25, 2015 5:41 PM To: tcpm@ietf.org Subject: [tcpm] TCP evolution impact on router buffers I'm trying to track down some research or opinions on how the more recent T= CP schemes impact router buffer needs. There are a number of publications f= rom Stanford and Georgia Tech from about 10 years ago, and I'm trying to as= sess how changes to the algorithms (e.g. CUBIC) and parameters (initial con= gestion window) deployed since then may influence those findings in the dir= ection of more or less buffering being needed. I'd appreciate any input and pointers. Thanks. - Lane --_000_F3B0A07CFD358240926B78A680E166FF835AB9TWMBXP03cnesnetad_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi, =

 

1-&= nbsp;     Regarding the = CIR/PIR

 

IMHO th= is is out of the scope of the characterization guidelines. However, if you = think that we should introduce more information on the MPLS/IP router archi= tectures please let us know. The only problem being that we want to be quite generic and do not want to focus on= specific architectures.

&n= bsp;

= 2-      Regarding buffer sizes

&n= bsp;

Since b= uffer sizing is not the only problem of evaluating AQMs, we have looked for= RFC clearly stipulating how the “adequate” buffer size can be = found. As said in RFC7597:

“=

The opt= imum buffer size is a function of

 &= nbsp; operational requirements and should generally be sized to be

 &= nbsp; sufficient to buffer the largest normal traffic burst that is

 &= nbsp; expected.  This size depends on the amount and burstiness of tra= ffic

 &= nbsp; arriving at the queue and the rate at which traffic leaves the queue.=

“=

&n= bsp;

In the = characterization draft, the recommendation on the buffer size is not a R= 20;MUST”.

If you = think that we should more clearly say that the size of the buffer MAY be se= t to the BDP, let us know.

In its = current state, it has been voluntary that the draft does not impose any buf= fer size.

&n= bsp;

Cheers,=

&n= bsp;

Nicolas

 

De : Scha= rf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com]
Envoy=E9 : vendredi 2 octobre 2015 19:06
=C0 : Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org
Objet : RE: TCP evolution impact on router buffers

 

Regardi= ng Section 7 in [2], I am surprised that a potential impact of CIR/PIR is n= ot mentioned. Unfortunately, I have only a very limited understanding of MP= LS/IP router architectures, but the way bursts hit an AQM could IMHO depend on that.

&n= bsp;

As [2] = is mentioned, one question related to the discussion below. From [2]:<= /o:p>

&n= bsp;

3.2.&nb= sp; Buffer size

&n= bsp;

 &= nbsp; The size of the buffers should be carefully chosen, and is to be set<= o:p>

 &= nbsp; to the bandwidth-delay product; the bandwidth being the bottleneck

 &= nbsp; capacity and the delay the largest RTT in the considered network.

&n= bsp;

This me= ans of the order of 10GB of buffer size for a 400G router interface, right?= Wouldn’t it make sense to relax the statement “… and is = to be set…” a bit?

&n= bsp;

Michael=

&n= bsp;

&n= bsp;

From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr]
Sent: Wednesday, September 30, 2015 6:41 PM
To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

 

Hi, =

&nbs= p;

In [1], t= he authors evaluate the performance of CUBIC in small buffer regime. Howeve= r, they use the Linux kernel version 2.6.18 (where the IW was 3 and not 10)= . Also, for any tests on CUBIC, small buffers and IW3/10, you may be interested in looking at the scenario ̶= 0;7.  Burst absorption” of the AQM Characterization Guidelines o= f the AQM WG [2]. Concerning TCP pacing, there is an on-going document on a= pacing solution in the slow-start that you may want to have a look and comment [3].

&nbs= p;

I hope th= is helps,

Cheers,

&nbs= p;

Nicolas

 

[1] Jain,= S.; Raina, G., "An experimental evaluation of CUBIC TCP in a small bu= ffer regime," in Communications (NCC), 2011 National Conference on , vol., no., pp.1-= 5, 28-30 Jan. 2011
doi: 10.1109/NCC.2011.5734779
URL: http://ie= eexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&isnumber= =3D5734693

[2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08

[3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01

 

De : tcpm= [mailto:tcpm-bounces@ietf.org= ] De la part de Lane Wigley (lwigley)
Envoy=E9 : mercredi 30 septembre 2015 15:10
=C0 : Scharf, Michael (Michael); tcpm@ietf.org
Objet : Re: [tcpm] TCP evolution impact on router buffers<= /o:p>

 

This pa= per from 2008 shows no drops with a 5 msec buffer on a heavily-congested co= re link. Low levels of drops start to show up around 2.5 msec. This was con= ducted at Level 3 to test the BDP / sqrt # flows proposal from Stanford.

&n= bsp;

Experimental Study of Router Bu= ffer Sizing – Neda Beheshti, Yashar Ganjali, Monia Ghobadi, Nick McKe= own, and Geoff Salmon

 

They also explored TCP pacing a= s a way to further reduce buffers. I’m trying to assess if CUBIC resu= lts in some pacing effect by decoupling the window growth from acks.

 

- Lane

&n= bsp;

&n= bsp;

From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com]
Sent: Tuesday, September 29, 2015 6:56 PM
To: Lane Wigley (lwigley); tcpm@iet= f.org
Subject: RE: TCP evolution impact on router buffers

 

This qu= estion may also be in scope of the AQM WG, and perhaps ICCRG, but since the= communities overlap, I avoid cross-posting…

&n= bsp;

Unfortu= nately, I don’t have any recent pointer or own data. But I am aware o= f examples for router buffer dimensioning based on tests with a single TCP = connection, using Reno. In those examples, the dimension of the buffers and possibly other related parameters (two-color/= three-color meters/policiers, WRED slopes, H-QoS, etc.) are configured to a= chieve full “line” speed for a single TCP connection. As to be = expected, this typically comes down to the old BDP rule of thumb. But it is rather obvious that a single bulk data Reno c= onnection is not necessarily a realistic workload these days. So, the test = methodology can be one relevant aspect of the dimensioning problem.

&n= bsp;

I belie= ve that a modern TCP stack with a high-speed congestion control such as CUB= IC will typically need less buffer than the outcome of this dimensioning me= thod and the publications 10 years ago. As far as I know, most modern TCP stacks operate well over a very wide ran= ge of buffer sizes, and the buffers can be relatively small. To me, the key= question is the latency-throughput tradeoff, and there may not be a one-fi= ts-all answer to that one.

&n= bsp;

Persona= lly, I’d also be interested in more recent pointers on router buffer = dimensioning. I could even see some value in documenting dimensioning guide= lines used in practice, if there was a way to generalize them.

&n= bsp;

Michael=

&n= bsp;

&n= bsp;

&n= bsp;

From: tcpm [mailto:tcpm-= bounces@ietf.org] On Behalf Of Lane Wigley (lwigley)
Sent: Friday, September 25, 2015 5:41 PM
To: tcpm@ietf.org
Subject: [tcpm] TCP evolution impact on router buffers

 

I’m trying to track down = some research or opinions on how the more recent TCP schemes impact router = buffer needs. There are a number of publications from Stanford and Georgia = Tech from about 10 years ago, and I’m trying to assess how changes to the algorithms (e.g. CUBIC) and parameters (initi= al congestion window) deployed since then may influence those findings in t= he direction of more or less buffering being needed.

 

I’d appreciate any input = and pointers. Thanks.

 

- Lane

 

 

--_000_F3B0A07CFD358240926B78A680E166FF835AB9TWMBXP03cnesnetad_-- From nobody Wed Oct 7 03:58:12 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DE3BB1B2DDC for ; Wed, 7 Oct 2015 03:58:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.61 X-Spam-Level: X-Spam-Status: No, score=-2.61 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FGCjfTgtNAwJ for ; Wed, 7 Oct 2015 03:58:08 -0700 (PDT) Received: from mailfilter.orange.md (mailfilter.orange.md [94.243.64.204]) by ietfa.amsl.com (Postfix) with ESMTP id 653C91B2DC4 for ; Wed, 7 Oct 2015 03:58:08 -0700 (PDT) Received: from localhost.localdomain (antispam.orange.md [127.0.0.1]) by localhost (Postfix) with ESMTP id 271FB3B27A6; Wed, 7 Oct 2015 13:58:28 +0300 (EEST) Received: from antispam.orange.md by antispam.orange.md with queue id 250342-1; Wed, 07 Oct 2015 10:58:28 GMT Received: from XCHSRV03.main.orange.md (unknown [192.168.200.63]) by mailfilter.orange.md (Postfix) with ESMTP id 0A0A93B274C; Wed, 7 Oct 2015 13:58:28 +0300 (EEST) Received: from XCHSRV01.main.orange.md ([fe80::685f:aef6:93b0:dea7]) by XCHSRV03.main.orange.md ([fe80::6dbc:28dd:213b:8931%14]) with mapi id 14.02.0328.009; Wed, 7 Oct 2015 13:58:06 +0300 From: Veaceslav ROMAN To: Neal Cardwell Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8Pq87vsQ9XoBmgDq7PdWOtXZ4gsAq7Fatsw= Date: Wed, 7 Oct 2015 10:58:04 +0000 Message-ID: <7DBBB686E19D2049ADAACD210B474BB10167E5E9AA@XCHSRV01.main.orange.md> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E8BDBA@XCHSRV01.main.orange.md>, In-Reply-To: Accept-Language: en-US, ro-RO Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.77.207] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , Ingemar Johansson S , Eric Dumazet , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Oct 2015 10:58:11 -0000 Yes, we'll test it too, but next week. It will be very much helpful if someone can rewrite tcp_cubic.c to make HY= START_DELAY_MIN externally settable, otherwise, according to our IT it take= s almost a whole working day to recompile the kernel. Best regards Veaceslav Roman ________________________________________ From: Neal Cardwell [ncardwell@google.com] Sent: Tuesday, October 06, 2015 4:05 AM To: Veaceslav ROMAN Cc: Eric Dumazet; Ingemar Johansson S; Eric Dumazet; tcpm@ietf.org; Piers O= 'Hanlon; Sangtae Ha; end2end-interest@postel.org Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment On Mon, Oct 5, 2015 at 6:07 PM, Veaceslav ROMAN wrote: > We've managed to recompile the kernel 4.1 with tcp_cubic HYSTART_DELAY_MI= N (10U<<3) , 10ms. > Well, we had time only for one test cycle, but results are very promising= : > > Download 10MB, High Bandwidths, HYSTART_DELAY_MIN 10 ms: > Hystart average download time: 1.16 s > NoHystart average download time: 0.93 s ... > Download 10MB, High Bandwidths, HYSTART_DELAY_MIN 4 ms: > Hystart average download time: 1.42 s > NoHystart average download time: 0.89 s Thanks! That is very useful and interesting data. Would you be able to try a few other values, like 15ms and 20ms? neal= From nobody Wed Oct 7 04:35:35 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 61A161B2E3C for ; Wed, 7 Oct 2015 04:35:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.909 X-Spam-Level: X-Spam-Status: No, score=-6.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rhEMsISeZV9k for ; Wed, 7 Oct 2015 04:35:28 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 159541B2E3D for ; Wed, 7 Oct 2015 04:35:28 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 0158017E8DA57; Wed, 7 Oct 2015 11:35:23 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t97BZPhf014353 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 7 Oct 2015 13:35:26 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Wed, 7 Oct 2015 13:35:25 +0200 From: "Scharf, Michael (Michael)" To: Kuhn Nicolas , "Lane Wigley (lwigley)" , "tcpm@ietf.org" Thread-Topic: TCP evolution impact on router buffers Thread-Index: AdD3qDJMG7WkdcJDTiy1iIy/H6510wDXEZsQAB6kmpAABLWoIABn/A/AAOxXFcAAAu7VEA== Date: Wed, 7 Oct 2015 11:35:25 +0000 Message-ID: <655C07320163294895BBADA28372AF5D484F7BD9@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <655C07320163294895BBADA28372AF5D484D799E@FR712WXCHMBA15.zeu.alcatel-lucent.com> <46616f47b42841f0bb7a2afb8bdeac98@XCH-ALN-009.cisco.com> <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: multipart/alternative; boundary="_000_655C07320163294895BBADA28372AF5D484F7BD9FR712WXCHMBA15z_" MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] TCP evolution impact on router buffers X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Oct 2015 11:35:34 -0000 --_000_655C07320163294895BBADA28372AF5D484F7BD9FR712WXCHMBA15z_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Regarding 1: Sure, it doesn't make any sense to add architecture-specific g= uidelines. I just wonder if the text could mention a bit more explicitly th= at a number of potential router components and algorithms (e.g., DiffServ) = are omitted (or abstracted) to keep the guidelines generic and simple. Regarding 2: Yes, given the references mentioned by Lane, I'd suggest to so= mehow replace the wording "... and is to be set to the bandwidth-delay prod= uct", e.g., by a MAY. But I leave this to the authors - this is not really in TCPM scope, and I d= on't have cycles to follow the AQM WG. Michael From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] Sent: Wednesday, October 07, 2015 12:17 PM To: Scharf, Michael (Michael); Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Hi, 1- Regarding the CIR/PIR IMHO this is out of the scope of the characterization guidelines. However, = if you think that we should introduce more information on the MPLS/IP route= r architectures please let us know. The only problem being that we want to = be quite generic and do not want to focus on specific architectures. 2- Regarding buffer sizes Since buffer sizing is not the only problem of evaluating AQMs, we have loo= ked for RFC clearly stipulating how the "adequate" buffer size can be found= . As said in RFC7597: " The optimum buffer size is a function of operational requirements and should generally be sized to be sufficient to buffer the largest normal traffic burst that is expected. This size depends on the amount and burstiness of traffic arriving at the queue and the rate at which traffic leaves the queue. " In the characterization draft, the recommendation on the buffer size is not= a "MUST". If you think that we should more clearly say that the size of the buffer MA= Y be set to the BDP, let us know. In its current state, it has been voluntary that the draft does not impose = any buffer size. Cheers, Nicolas De : Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Envoy=E9 : vendredi 2 octobre 2015 19:06 =C0 : Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org Objet : RE: TCP evolution impact on router buffers Regarding Section 7 in [2], I am surprised that a potential impact of CIR/P= IR is not mentioned. Unfortunately, I have only a very limited understandin= g of MPLS/IP router architectures, but the way bursts hit an AQM could IMHO= depend on that. As [2] is mentioned, one question related to the discussion below. From [2]= : 3.2. Buffer size The size of the buffers should be carefully chosen, and is to be set to the bandwidth-delay product; the bandwidth being the bottleneck capacity and the delay the largest RTT in the considered network. This means of the order of 10GB of buffer size for a 400G router interface,= right? Wouldn't it make sense to relax the statement "... and is to be set= ..." a bit? Michael From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] Sent: Wednesday, September 30, 2015 6:41 PM To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Hi, In [1], the authors evaluate the performance of CUBIC in small buffer regim= e. However, they use the Linux kernel version 2.6.18 (where the IW was 3 an= d not 10). Also, for any tests on CUBIC, small buffers and IW3/10, you may = be interested in looking at the scenario "7. Burst absorption" of the AQM = Characterization Guidelines of the AQM WG [2]. Concerning TCP pacing, there= is an on-going document on a pacing solution in the slow-start that you ma= y want to have a look and comment [3]. I hope this helps, Cheers, Nicolas [1] Jain, S.; Raina, G., "An experimental evaluation of CUBIC TCP in a smal= l buffer regime," in Communications (NCC), 2011 National Conference on , vo= l., no., pp.1-5, 28-30 Jan. 2011 doi: 10.1109/NCC.2011.5734779 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&is= number=3D5734693 [2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08 [3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01 De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley= ) Envoy=E9 : mercredi 30 septembre 2015 15:10 =C0 : Scharf, Michael (Michael); tcpm@ietf.org Objet : Re: [tcpm] TCP evolution impact on router buffers This paper from 2008 shows no drops with a 5 msec buffer on a heavily-conge= sted core link. Low levels of drops start to show up around 2.5 msec. This = was conducted at Level 3 to test the BDP / sqrt # flows proposal from Stanf= ord. Experimental Study of Router Buffer Sizing - Neda Beheshti, Yashar Ganjali,= Monia Ghobadi, Nick McKeown, and Geoff Salmon They also explored TCP pacing as a way to further reduce buffers. I'm tryin= g to assess if CUBIC results in some pacing effect by decoupling the window= growth from acks. - Lane From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Sent: Tuesday, September 29, 2015 6:56 PM To: Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers This question may also be in scope of the AQM WG, and perhaps ICCRG, but si= nce the communities overlap, I avoid cross-posting... Unfortunately, I don't have any recent pointer or own data. But I am aware = of examples for router buffer dimensioning based on tests with a single TCP= connection, using Reno. In those examples, the dimension of the buffers an= d possibly other related parameters (two-color/three-color meters/policiers= , WRED slopes, H-QoS, etc.) are configured to achieve full "line" speed for= a single TCP connection. As to be expected, this typically comes down to t= he old BDP rule of thumb. But it is rather obvious that a single bulk data = Reno connection is not necessarily a realistic workload these days. So, the= test methodology can be one relevant aspect of the dimensioning problem. I believe that a modern TCP stack with a high-speed congestion control such= as CUBIC will typically need less buffer than the outcome of this dimensio= ning method and the publications 10 years ago. As far as I know, most moder= n TCP stacks operate well over a very wide range of buffer sizes, and the b= uffers can be relatively small. To me, the key question is the latency-thro= ughput tradeoff, and there may not be a one-fits-all answer to that one. Personally, I'd also be interested in more recent pointers on router buffer= dimensioning. I could even see some value in documenting dimensioning guid= elines used in practice, if there was a way to generalize them. Michael From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley= ) Sent: Friday, September 25, 2015 5:41 PM To: tcpm@ietf.org Subject: [tcpm] TCP evolution impact on router buffers I'm trying to track down some research or opinions on how the more recent T= CP schemes impact router buffer needs. There are a number of publications f= rom Stanford and Georgia Tech from about 10 years ago, and I'm trying to as= sess how changes to the algorithms (e.g. CUBIC) and parameters (initial con= gestion window) deployed since then may influence those findings in the dir= ection of more or less buffering being needed. I'd appreciate any input and pointers. Thanks. - Lane --_000_655C07320163294895BBADA28372AF5D484F7BD9FR712WXCHMBA15z_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Regarding 1: Sure, it = doesn’t make any sense to add architecture-specific guidelines. I jus= t wonder if the text could mention a bit more explicitly that a number of p= otential router components and algorithms (e.g., DiffServ) are omitted (or abstracted) to keep the guidelines generi= c and simple.

 

Regarding 2: Yes, give= n the references mentioned by Lane, I’d suggest to somehow replace th= e wording “… and is to be set to the bandwidth-dela= y product”, e.g., by a MAY.

 

But I leave this to th= e authors – this is not really in TCPM scope, and I don’t have = cycles to follow the AQM WG.

 

Michael

 

 

From: Kuhn Nic= olas [mailto:Nicolas.Kuhn@cnes.fr]
Sent: Wednesday, October 07, 2015 12:17 PM
To: Scharf, Michael (Michael); Lane Wigley (lwigley); tcpm@ietf.org<= br> Subject: RE: TCP evolution impact on router buffers

 

Hi, <= /o:p>

 = ;

1-      Re= garding the CIR/PIR

 = ;

IMHO this is out of th= e scope of the characterization guidelines. However, if you think that we s= hould introduce more information on the MPLS/IP router architectures please= let us know. The only problem being that we want to be quite generic and do not want to focus on specific arch= itectures.

 

2-&= nbsp;     Regarding buff= er sizes

 

Since buffer sizing is= not the only problem of evaluating AQMs, we have looked for RFC clearly st= ipulating how the “adequate” buffer size can be found. As said = in RFC7597:

“

The optimum buffer siz= e is a function of

   operation= al requirements and should generally be sized to be

   sufficien= t to buffer the largest normal traffic burst that is

   expected.=   This size depends on the amount and burstiness of traffic=

   arriving = at the queue and the rate at which traffic leaves the queue.

“

 

In the characterizatio= n draft, the recommendation on the buffer size is not a “MUST”.

If you think that we s= hould more clearly say that the size of the buffer MAY be set to the BDP, l= et us know.

In its current state, = it has been voluntary that the draft does not impose any buffer size.

 

Cheers,

 

Nicolas

 = ;

De : Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com]
Envoy=E9 : vendredi 2 octobre 2015 19:06
=C0 : Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org
Objet : RE: TCP evolution impact on router buffers

 

Regarding Section 7 in= [2], I am surprised that a potential impact of CIR/PIR is not mentioned. U= nfortunately, I have only a very limited understanding of MPLS/IP router ar= chitectures, but the way bursts hit an AQM could IMHO depend on that.

 

As [2] is mentioned, o= ne question related to the discussion below. From [2]:

 

3.2.  Buffer size=

 

   The size = of the buffers should be carefully chosen, and is to be set

   to the ba= ndwidth-delay product; the bandwidth being the bottleneck=

   capacity = and the delay the largest RTT in the considered network.<= /p>

 

This means of the orde= r of 10GB of buffer size for a 400G router interface, right? Wouldn’t= it make sense to relax the statement “… and is to be set…= ;” a bit?

 

Michael

 

 

 

Hi,

 =

In [1], the authors eval= uate the performance of CUBIC in small buffer regime. However, they use the= Linux kernel version 2.6.18 (where the IW was 3 and not 10). Also, for any= tests on CUBIC, small buffers and IW3/10, you may be interested in looking at the scenario “7.  Burst abs= orption” of the AQM Characterization Guidelines of the AQM WG [2]. Co= ncerning TCP pacing, there is an on-going document on a pacing solution in = the slow-start that you may want to have a look and comment [3].

 =

I hope this helps, =

Cheers,

 =

Nicolas=

 = ;

[1] Jain, S.; Raina, G.,= "An experimental evaluation of CUBIC TCP in a small buffer regime,&qu= ot; in Communications (NCC), 2011 National Conference on , vol., no., pp.1-= 5, 28-30 Jan. 2011
doi: 10.1109/NCC.2011.5734779
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&= amp;isnumber=3D5734693

[2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08

[3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01

 = ;

De : tcpm [mailto:t= cpm-bounces@ietf.org] De la part de Lane Wigley (lwigley)
Envoy=E9 : mercredi 30 septembre 2015 15:10
=C0 : Scharf, Michael (Michael); tcpm@ietf.org
Objet : Re: [tcpm] TCP evolution impact on router buffers<= /o:p>

 

This paper from 2008 s= hows no drops with a 5 msec buffer on a heavily-congested core link. Low le= vels of drops start to show up around 2.5 msec. This was conducted at Level= 3 to test the BDP / sqrt # flows proposal from Stanford.

 

Experimental Study of Router Buffer Sizing – N= eda Beheshti, Yashar Ganjali, Monia Ghobadi, Nick McKeown, and Geoff Salmon=

 

They also explored TCP pacing as a way to further re= duce buffers. I’m trying to assess if CUBIC results in some pacing ef= fect by decoupling the window growth from acks.

 

- Lane

 

 

From: Scharf, = Michael (Michael) [mai= lto:michael.scharf@alcatel-lucent.com]
Sent: Tuesday, September 29, 2015 6:56 PM
To: Lane Wigley (lwigley); tcpm@iet= f.org
Subject: RE: TCP evolution impact on router buffers

 

This question may also= be in scope of the AQM WG, and perhaps ICCRG, but since the communities ov= erlap, I avoid cross-posting…

 

Unfortunately, I don&#= 8217;t have any recent pointer or own data. But I am aware of examples for = router buffer dimensioning based on tests with a single TCP connection, usi= ng Reno. In those examples, the dimension of the buffers and possibly other related parameters (two-color/three-colo= r meters/policiers, WRED slopes, H-QoS, etc.) are configured to achieve ful= l “line” speed for a single TCP connection. As to be expected, = this typically comes down to the old BDP rule of thumb. But it is rather obvious that a single bulk data Reno connection= is not necessarily a realistic workload these days. So, the test methodolo= gy can be one relevant aspect of the dimensioning problem.

 

I believe that a moder= n TCP stack with a high-speed congestion control such as CUBIC will typical= ly need less buffer than the outcome of this dimensioning method and the pu= blications 10 years ago. As far as I know, most modern TCP stacks operate well over a very wide range of buffer= sizes, and the buffers can be relatively small. To me, the key question is= the latency-throughput tradeoff, and there may not be a one-fits-all answe= r to that one.

 

Personally, I’d = also be interested in more recent pointers on router buffer dimensioning. I= could even see some value in documenting dimensioning guidelines used in p= ractice, if there was a way to generalize them.

 

Michael

 

 

 

From: tcpm [mailto:tcpm-= bounces@ietf.org] On Behalf Of Lane Wigley (lwigley)
Sent: Friday, September 25, 2015 5:41 PM
To: tcpm@ietf.org
Subject: [tcpm] TCP evolution impact on router buffers

 

I’m trying to track down some research or opin= ions on how the more recent TCP schemes impact router buffer needs. There a= re a number of publications from Stanford and Georgia Tech from about 10 ye= ars ago, and I’m trying to assess how changes to the algorithms (e.g. CUBIC) and parameters (initial congestion window) = deployed since then may influence those findings in the direction of more o= r less buffering being needed.

 

I’d appreciate any input and pointers. Thanks.=

 

- Lane

 

 

--_000_655C07320163294895BBADA28372AF5D484F7BD9FR712WXCHMBA15z_-- From nobody Wed Oct 7 09:21:46 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 28C4A1ACDC7 for ; Wed, 7 Oct 2015 09:21:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.002 X-Spam-Level: X-Spam-Status: No, score=-2.002 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TA-En7_Rpr1R for ; Wed, 7 Oct 2015 09:21:30 -0700 (PDT) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0116.outbound.protection.outlook.com [65.55.169.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5D79E1ACD13 for ; Wed, 7 Oct 2015 09:21:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=3C/fbG1NlJwJgBYPYAbs1Vg/ZdBrlPTWXRFewqlExDo=; b=lLlcqEoZsx+jpIueQvApMGDz7NeWUwXR7VhUpV6AFP6u8cfxC09dcT4zZYXGk9/s/linzlXF12BzjbUhKGFOifqbHz+9zmPKBwjH/1AB9T+tTv9BbvFl8zQe9RH5hPMzGPNIQyJFzBNImRR1ss/z5ibtONyduqN5yoXCIoN9p5E= Received: from BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) by BN1PR03MB007.namprd03.prod.outlook.com (10.255.224.37) with Microsoft SMTP Server (TLS) id 15.1.293.16; Wed, 7 Oct 2015 16:21:27 +0000 Received: from BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.173]) by BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.173]) with mapi id 15.01.0293.007; Wed, 7 Oct 2015 16:21:27 +0000 From: Praveen Balasubramanian To: "tcpm@ietf.org" Thread-Topic: [tcpm] I-D Action: draft-ietf-tcpm-dctcp-00.txt Thread-Index: AQHQ9QyoK6O2j7Jje0aEPi9maSJ3Ap5gTMTQ Date: Wed, 7 Oct 2015 16:21:26 +0000 Message-ID: References: <20150922075939.25385.97529.idtracker@ietfa.amsl.com> In-Reply-To: <20150922075939.25385.97529.idtracker@ietfa.amsl.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com; x-originating-ip: [2001:4898:80e8:5::584] x-microsoft-exchange-diagnostics: 1; BN1PR03MB007; 5:d/No8wy5V9MgSERV6A5pOtGAiO5NbZYvB26Z96Zz2LzbmqoYFQsf+fUSCb0i9XzZCgENJLlWwS7XogTpMDBuHEtesvuamfawTLBsFG+Cze8/o133kXH5VBkkSuonDDl7gOFlmW1a4HVM9DuRs78ngQ==; 24:Y484bIZevbZm8WlsScAeGB9jCZhdA+1C+0KnCS8hM/pr8pvWiHCJXJ5cirCw4zsv35RBSuq+3DeHiir4yxWgYJmJHR/B42EnztY14yUXXi4=; 20:O+9YGDiUNQSrtj4Ar6dxSPgD39CK2j/hc3BO5N1ce91dAFveGNVrcQoR+LU9+dwqpGT6V6EGNZeQEbCXJEmLdQ== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN1PR03MB007; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425024)(601004)(2401047)(5005006)(520078)(8121501046)(3002001)(61426024)(61427024); SRVR:BN1PR03MB007; BCL:0; PCL:0; RULEID:; SRVR:BN1PR03MB007; x-forefront-prvs: 0722981D2A x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(377454003)(377424004)(53754006)(199003)(13464003)(189002)(97736004)(5007970100001)(81156007)(5008740100001)(86612001)(230783001)(5004730100002)(5003600100002)(19580395003)(46102003)(106116001)(19580405001)(106356001)(92566002)(105586002)(10090500001)(99286002)(86362001)(15975445007)(2351001)(102836002)(122556002)(40100003)(10290500002)(110136002)(5005710100001)(2501003)(10400500002)(107886002)(5001960100002)(50986999)(87936001)(101416001)(64706001)(450100001)(76176999)(5002640100001)(2900100001)(2950100001)(8990500004)(33656002)(76576001)(11100500001)(54356999)(74316001)(189998001)(3826002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR03MB007; H:BN1PR03MB008.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Oct 2015 16:21:26.7643 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR03MB007 Archived-At: Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-dctcp-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Oct 2015 16:21:32 -0000 Hi all, As discussed in the Prague meeting, this new Internet-Draft addresses the f= eedback received so far. Please review and provide feedback so we can addre= ss it in time for the next meeting.=20 Thanks -----Original Message----- From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of internet-drafts@ietf= .org Sent: Tuesday, September 22, 2015 1:00 AM To: i-d-announce@ietf.org Cc: tcpm@ietf.org Subject: [tcpm] I-D Action: draft-ietf-tcpm-dctcp-00.txt A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the TCP Maintenance and Minor Extensions Work= ing Group of the IETF. Title : Datacenter TCP (DCTCP): TCP Congestion Control fo= r Datacenters Authors : Stephen Bensley Lars Eggert Dave Thaler Praveen Balasubramanian Glenn Judd Filename : draft-ietf-tcpm-dctcp-00.txt Pages : 14 Date : 2015-09-22 Abstract: This memo describes Datacenter TCP (DCTCP), an improvement to TCP congestion control for datacenter traffic. DCTCP uses improved Explicit Congestion Notification (ECN) processing to estimate the fraction of bytes that encounter congestion, rather than simply detecting that some congestion has occurred. DCTCP then scales the TCP congestion window based on this estimate. This method achieves high burst tolerance, low latency, and high throughput with shallow- buffered switches. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-tcpm-dctcp/ There's also a htmlized version available at: https://tools.ietf.org/html/draft-ietf-tcpm-dctcp-00 Please note that it may take a couple of minutes from the time of submissio= n until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ _______________________________________________ tcpm mailing list tcpm@ietf.org https://www.ietf.org/mailman/listinfo/tcpm From nobody Wed Oct 7 10:58:48 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2D0491ACD92 for ; Wed, 7 Oct 2015 10:58:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 89pn39ap98kY for ; Wed, 7 Oct 2015 10:58:45 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 317101ACD90 for ; Wed, 7 Oct 2015 10:58:45 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id D7F416DC2717F for ; Wed, 7 Oct 2015 17:58:39 +0000 (GMT) Received: from FR711WXCHHUB02.zeu.alcatel-lucent.com (fr711wxchhub02.zeu.alcatel-lucent.com [135.239.2.112]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t97Hwgig017727 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Wed, 7 Oct 2015 19:58:42 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR711WXCHHUB02.zeu.alcatel-lucent.com ([135.239.2.112]) with mapi id 14.03.0195.001; Wed, 7 Oct 2015 19:58:42 +0200 From: "Scharf, Michael (Michael)" To: "tcpm@ietf.org" Thread-Topic: [aqm] TCP ACK Suppression Thread-Index: AQHRAAfbmdqr5Xk9lE2VWzCubgsRu55eiGYAgAD9UYCAAAl1AIAAK3+AgABsWwCAAATHAIAABJaAgAAioOA= Date: Wed, 7 Oct 2015 17:58:42 +0000 Message-ID: <655C07320163294895BBADA28372AF5D484F882F@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5614D4B8.5060403@kit.edu> <5615581C.5010907@rogers.com> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Oct 2015 17:58:47 -0000 VGhpcyB0aHJlYWQgaXMgbWF5IGJlIHJlbGV2YW50IHRvIFRDUE0NCg0KTWljaGFlbA0KDQoNCi0t LS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBhcW0gW21haWx0bzphcW0tYm91bmNlc0Bp ZXRmLm9yZ10gT24gQmVoYWxmIE9mIEpvbmF0aGFuIE1vcnRvbg0KU2VudDogV2VkbmVzZGF5LCBP Y3RvYmVyIDA3LCAyMDE1IDc6NTMgUE0NClRvOiBkYXZlY2JAc3BhbWNvcC5uZXQNCkNjOiBhcW1A aWV0Zi5vcmcNClN1YmplY3Q6IFJlOiBbYXFtXSBUQ1AgQUNLIFN1cHByZXNzaW9uDQoNCg0KPiBP biA3IE9jdCwgMjAxNSwgYXQgMjA6MzYsIERhdmlkIENvbGxpZXItQnJvd24gPGRhdmVjLWJAcm9n ZXJzLmNvbT4gd3JvdGU6DQo+IA0KPiBPbiAwNy8xMC8xNSAwMToxOSBQTSwgRGF2aWQgTGFuZyB3 cm90ZToNCj4+IE9uIFdlZCwgNyBPY3QgMjAxNSwgTWlrYWVsIEFicmFoYW1zc29uIHdyb3RlOg0K Pj4gDQo+Pj4+IE9oLCBJIGhvcGUgdGhhdCB0aGlzIGlzIGFuIGV4Y2VwdGlvbi4gU3VjaCBraW5k IG9mIG9wdGltaXphdGlvbnMgbWF5IGNhdXNlIGEgbG90IG9mIHRyb3VibGUgc2luY2UgYSBsaW5r IGxheWVyIGRldmljZSBpcyBpbnRlcmZlcmluZyB3aXRoIHRyYW5zcG9ydCBsYXllciBzZW1hbnRp Y3MuIFdlIGFsbCBrbm93IHRoYXQgZXhhY3RseSB0aGVzZSBraW5kcyBvZiBpbnRlcmZlcmVuY2Ug ZXZlbnR1YWxseSBlbmQgdXAgaW4gcHJvYmxlbXMgd2l0aCBlbmQtdG8tZW5kIHRyYW5zcGFyZW5j eSBhbmQgZGVwbG95bWVudCBvZiBuZXcgcHJvdG9jb2wgb3B0aW9ucy4gQXQgbGVhc3QgaXQgaW50 ZXJmZXJlcyB3aXRoIHRoZSBBQ0sgY2xvY2tpbmcgZXhwZWN0YXRpb24gb2Ygc29tZSBjb25nZXN0 aW9uIGNvbnRyb2wgYWxnb3JpdGhtcy4uLg0KPj4+IA0KPj4+IFBlcnNvbmFsbHksIEkgdGhpbmsg eW91J3JlIGdvaW5nIHRvIHNlZSBtb3JlIGFuZCBtb3JlIG9mIHRoaXMuIFRoZXJlIGFyZSBtdWxp dHBsZSBzaGFyZWQgYWNjZXNzIG1lZGl1bSB3aGVyZSB5b3UncmUgYWxsb3dlZCB0byBzZW5kIG9u bHkgcGFydCBvZiB0aGUgdGltZSwgYW5kIGl0J3Mgc29tZW9uZSBlbHNlIHdobyB0ZWxscyB5b3Ug d2hlbiB5b3UgbWF5IHNlbmQuDQo+PiANCj4+IGl0IGRvZXNuJ3QgZXZlbiByZXF1aXJlIHRoYXQg c29tZW9uZSBlbHNlIHRlbGxzIHlvdSB3aGVuIHlvdSBtYXkgc2VuZC4gSXQgY2FuIGp1c3QgYmUg d2FpdGluZyBmb3IgYW4gYXZhaWxhYmxlIHRyYW5zbWl0IHRpbWVzbG90IChXaWZpIGZvciBleGFt cGxlKQ0KPj4gDQo+PiBjb2xsYXBzaW5nIG11bHRpcGxlIEFDS3MgdGhhdCBhcmUgZ29pbmcgdG8g YmUgc2VudCBhdCBvbmNlIGlzIGFsbW9zdCBhbHdheXMgZ29pbmcgdG8gYmUgYSB3aW4uDQo+IA0K PiBJIHF1aXRlIGFncmVlLCBidXQgaWYgdGhlcmUgaXMgYSBjb25nZXN0aW9uIGNvbnRyb2wgaW1w bGVtZW50YXRpb24gImluIHRoZSB3aWxkIiB0aGF0IGFzc3VtZXMgaXQgd2lsbCBnZXQgYSBzdHJl YW0gb2YgYWNrcywgdGhhdCBvbmUncyBnb2luZyB0byBuZWVkIHNvbWUgd29yayAoOi0pKQ0KPiAN Cj4gQW55b25lIGtub3cgaWYgdGhhdCdzIHRoZSBjYXNlPyBUaGUgY29tbWVudCBhYm92ZSBzdWdn ZXN0IGl0IG1heSBiZS4uLg0KDQpJdOKAmXMgbm90IOKAnGluIHRoZSB3aWxk4oCdLCBidXQgdGhp cyBzb3J0IG9mIHRoaW5nIGlzIGEgaGVhZGFjaGUgZm9yIEVMUi4gIEl0IGVzc2VudGlhbGx5IG1l YW5zIHRoYXQgaXQgd29u4oCZdCBiZSBhYmxlIHRvIHVzZSBBY2NFQ04gZm9yIGl0cyBmZWVkYmFj ayAoc2luY2UgQWNjRUNOIGRvZXNu4oCZdCBhbGxvdyByZWNvbnN0cnVjdGluZyBhIGNvbXBsZXgg MzItcGFja2V0IGhpc3Rvcnkgb2YgRUNOIGNvZGVwb2ludHMgZnJvbSBhIHNpbmdsZSBBQ0spLCBi dXQgbXVzdCBpbnRyb2R1Y2Ugc29tZSBvdGhlciBtZWNoYW5pc20gdG8gZmVlZCBiYWNrIHRoZSBy ZXF1aXJlZCBkYXRhIHRvIHRoZSBzZW5kZXIuDQoNCknigJltIGxlYW5pbmcgdG93YXJkcyBtb3Zp bmcgdGhlIEVMUi1hd2FyZSBjb25nZXN0aW9uIGNvbnRyb2wgYWxnb3JpdGhtIHRvIHRoZSByZWNl aXZlciwgYW5kIGhhdmluZyB0aGUgZmVlZGJhY2sgc2ltcGx5IGJlIHRoZSByZXF1ZXN0ZWQgY3du ZDsgaXTigJlzIHRoZSBzaW1wbGVzdCBhbmQgbW9zdCBjb21wYWN0IHNvbHV0aW9uIEkgY2FuIHRo aW5rIG9mLCBhbmQgd291bGQgd29yayBmb3IgYW55IGNvbmdlc3Rpb24gY29udHJvbCBzY2hlbWUs IG5vdCBqdXN0IEVMUi4gIFRoZSBzZW5kZXIgbWF5IG9mIGNvdXJzZSBtYWludGFpbiBpdHMgb3du IGlkZWEgb2YgY3duZCBpbiB0aGUgY29udmVudGlvbmFsIHdheSwgYW5kIGNvbnNlcnZhdGl2ZWx5 IHRha2UgdGhlIG1pbmltdW0gb2YgdGhlIHR3byBhcyBlZmZlY3RpdmU7IHRoaXMgd291bGQgcHJv dGVjdCBhZ2FpbnN0IG5haXZlbHkgZ3JlZWR5IHJlY2VpdmVycy4NCg0KSSBub3RlIHRoYXQgQ1VC SUMgZG9lc27igJl0IGNhcmUgaG93IG1hbnkgYWNrcyBpdCBnZXRzLCBzaW5jZSBpdHMgc2NhbGlu ZyBjdXJ2ZSBkZXBlbmRzIG9ubHkgb24gdGltZSBzbyBsb25nIGFzIHRoZSBkYXRhIGZsb3cgaXMg bm90IGFwcGxpY2F0aW9uLWxpbWl0ZWQuICBUaGF04oCZcyBnb29kIG5ld3MgZm9yIExpbnV4LCBn aXZlbiB0aGUgbGFjayBvZiBBQkMuICBEbyB0aGUgQlNEcyBhbmQgV2luZG93cyB1c2UgQUJDPw0K DQogLSBKb25hdGhhbiBNb3J0b24NCg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX18NCmFxbSBtYWlsaW5nIGxpc3QNCmFxbUBpZXRmLm9yZw0KaHR0cHM6Ly93 d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9hcW0NCg== From nobody Wed Oct 7 12:03:36 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A20041A8A28 for ; Wed, 7 Oct 2015 12:03:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.002 X-Spam-Level: X-Spam-Status: No, score=-2.002 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VHgGN_UDHAqE for ; Wed, 7 Oct 2015 12:03:31 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bon0790.outbound.protection.outlook.com [IPv6:2a01:111:f400:fc10::1:790]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 22EEB1A1BFF for ; Wed, 7 Oct 2015 12:03:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=OvyVpFBPOHu/MjDIMzGMqWCfCPfdCsLZSPPqwpxsIOM=; b=LRT58+NJyxFE7eWxdEyo4yyNi+f/AeoQfnyq+wbDg8Z4htAFyMRSxAafnzgLtzXFiRUkZikXQ6SMn8MddRmnq6/5Ve/lRHnZOY+wpn2jdkzsMxZWsn5ozu4IfUqBXDJwdeEM5H1HjlbJPGIK83IQzj2KsHQ1PjKu1CHSz7apqJk= Received: from BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) by BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) with Microsoft SMTP Server (TLS) id 15.1.293.16; Wed, 7 Oct 2015 19:03:13 +0000 Received: from BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.173]) by BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.173]) with mapi id 15.01.0293.007; Wed, 7 Oct 2015 19:03:13 +0000 From: Praveen Balasubramanian To: "Scharf, Michael (Michael)" , "tcpm@ietf.org" Thread-Topic: [aqm] TCP ACK Suppression Thread-Index: AQHRAAfbmdqr5Xk9lE2VWzCubgsRu55eiGYAgAD9UYCAAAl1AIAAK3+AgABsWwCAAATHAIAABJaAgAAioOCAAAglYA== Date: Wed, 7 Oct 2015 19:03:13 +0000 Message-ID: References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5614D4B8.5060403@kit.edu> <5615581C.5010907@rogers.com> <655C07320163294895BBADA28372AF5D484F882F@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: <655C07320163294895BBADA28372AF5D484F882F@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com; x-originating-ip: [2001:4898:80e8:5::584] x-microsoft-exchange-diagnostics: 1; BN1PR03MB008; 5:q/apC7TYzRplTuwFDkLd85Nypqb9GLQ1Z+kSeos9JiZvKkEa78t3Otmp21PCSD5RBqzDP75pWeGNYaC6iVv2iAq+CZvdcRgd+3mK7sVjXDvo/Z+S4NjkgFsqXacVmWljnL2TEuyquk2TVrbo9Aiudw==; 24:D1l8Nvl4bnsomRMFObH+XwwEuz8JhQbbXMX/HATAt4m7M69mgy+5zxb/yOuu0Fsd7m+i8M9NbdzsAsorzUqmwRWN6lY8j1ZWhjgTliF2Ot8=; 20:mFSJYo5A2AKLDCD5bB6saYGzhC31PLQGeyDyGjj2lj5Va3UD4lmzSQ60d64gkljXbnuZRo8SR3oBnrwcSr1N+A== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN1PR03MB008; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425024)(601004)(2401047)(8121501046)(5005006)(520078)(3002001)(61426024)(61427024); SRVR:BN1PR03MB008; BCL:0; PCL:0; RULEID:; SRVR:BN1PR03MB008; x-forefront-prvs: 0722981D2A x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(24454002)(479174004)(13464003)(377454003)(189002)(199003)(5005710100001)(8990500004)(10290500002)(93886004)(74316001)(122556002)(40100003)(64706001)(54356999)(76176999)(10400500002)(50986999)(101416001)(92566002)(5003600100002)(19580395003)(5007970100001)(107886002)(5001960100002)(5004730100002)(105586002)(86612001)(33656002)(97736004)(5001770100001)(81156007)(2900100001)(2950100001)(106116001)(76576001)(106356001)(10090500001)(99286002)(102836002)(15975445007)(19580405001)(46102003)(87936001)(5008740100001)(561944003)(86362001)(2501003)(189998001)(11100500001)(5002640100001)(3826002)(217873001); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR03MB008; H:BN1PR03MB008.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Oct 2015 19:03:13.0222 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR03MB008 Archived-At: Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Oct 2015 19:03:34 -0000 VGhhbmtzIE1pY2hhZWwsIGl0IGlzIGluZGVlZCB2ZXJ5IHJlbGV2YW50LiANCg0KV2luZG93cyBk b2VzIHBlcmZvcm0gc3RyZXRjaCBBQ0tzIGFzIGFuIG9wdGltaXphdGlvbi4gSG93ZXZlciwgQUNL IGNvYWxlc2NpbmcgaXMgbGltaXRlZCBpbiB0aGUgd29yc3QgY2FzZSBzaW5jZSBwYWNrZXRzIGFy ZSBwcm9jZXNzZWQgYnkgYmF0Y2hpbmcgKHdoaWNoIGlzIGxpbWl0ZWQgdG8gYSBzaXplIE4pLiBU aGF0IGlzLCB3aXRoIGEgc2luZ2xlIGNvbm5lY3Rpb24gYW5kIHZlcnkgc21hbGwgaW50ZXIgcGFj a2V0IHNwYWNpbmcsIFdpbmRvd3Mgd2lsbCBnZW5lcmF0ZSBvbmUgQUNLIGZvciBOIHNlZ21lbnRz LiBJbiBwcmFjdGljZSBob3dldmVyIHRoZXJlIGFyZSBtdWx0aXBsZSBmbG93cyBhbmQgcGFja2V0 IHNwYWNpbmcgc3VjaCB0aGF0IEFDS3MgY292ZXIgYSBudW1iZXIgbXVjaCBsZXNzIHRoYW4gTi4g VGhlIHJlY2VpdmVyIEFDS2luZyBiZWhhdmlvciBpcyBoZW5jZSBub3QgY29tcGxldGVseSBkZXRl cm1pbmlzdGljIGFuZCBkZXBlbmRzIG9uIHBhY2tldCBhcnJpdmFsIHJhdGUsIGludGVycnVwdCBh bmQgRFBDICh+IGRlZmVycmVkIHdvcmsgZnJvbSBpbnRlcnJ1cHRzKSByYXRlcyBhbmQgbnVtYmVy IG9mIGNvbm5lY3Rpb25zIGV0Yy4gV2UgYXJlIGxvb2tpbmcgaW50byB0aGUgaW1wYWN0IG9mIHRo aXMgYmVoYXZpb3Igb24gc2VuZGVyIHdpbmRvdyBncm93dGggZXNwZWNpYWxseSBkdXJpbmcgc2xv dyBzdGFydC4gQSBwb3NzaWJsZSBtaXRpZ2F0aW9uIGlzIHRvIHJlZHVjZSB0aGUgc2l6ZSBOIG9y IGludHJvZHVjZSBhIHBlciBjb25uZWN0aW9uIGxpbWl0IE0gdG8gZW5zdXJlIGEgc2luZ2xlIGRl bGF5ZWQgQUNLIGFja25vd2xlZGdlcyBubyBtb3JlIHRoYW4gTSBzZWdtZW50cy4gSSBhbHJlYWR5 IHByb3ZpZGVkIGZlZWRiYWNrIG9uIHRoZSBBY2NFQ04gcHJvcG9zYWwgYW5kIGl0IGxvb2tzIGxp a2UgdGhlIGxpbWl0IE0gd291bGQgbmVlZCB0byBiZSA2IHdoZW4gcHJvY2Vzc2luZyBDRSBtYXJr ZWQgcGFja2V0cyB3aGljaCBkb2VzbuKAmXQgc2VlbSBvbmVyb3VzLiANCg0KVGhlIEFDS2luZyBi ZWhhdmlvciBkaXJlY3RseSB0aWVzIGluIHRvIEFCQyBvbiB0aGUgc2VuZGVyLiBXaW5kb3dzIGRv ZXMgc3VwcG9ydCBBQkMgZHVyaW5nIGJvdGggc2xvdyBzdGFydCBhbmQgY29uZ2VzdGlvbiBhdm9p ZGFuY2UgYW5kIGJhc2VzIHRoZSBtYWduaXR1ZGUgb2YgdGhlIGN3bmQgaW5jcmVhc2Ugb24gdGhl IG51bWJlciBvZiBuZXdseSBhY2tub3dsZWRnZWQgYnl0ZXMgKHBlciBSRkMgMzQ2NSkgdXAgdG8g YSBmaXhlZCBmYWN0b3IgWCBvZiB0aGUgTVNTLiBBbGwgdGhlIGNvbmdlc3Rpb24gYWxnb3JpdGht cyBzdXBwb3J0ZWQgaW4gV2luZG93cyBsaWtlIE5ld1Jlbm8sIENUQ1AgYW5kIERDVENQIGZvbGxv dyB0aGlzIGJlaGF2aW9yLCBhbmQgQ1RDUCB0YWtlcyBhZHZhbnRhZ2UgdGhlIGR3bmQgKGRlbGF5 IHdpbmRvdykgZ3Jvd3RoIGFzIHdlbGwuIEl0IG1pZ2h0IGJlIHVzZWZ1bCB0byBoYXZlIFggPSBN Lg0KDQpBbnkgdGhvdWdodHMgb24gc3RyZXRjaCBBQ0tzIGFzIGFuIG9wdGltaXphdGlvbiB3aGVu IHVzZWQgaW4gY29uanVuY3Rpb24gd2l0aCBBQkM/IA0KDQpUaGFua3MNCg0KLS0tLS1PcmlnaW5h bCBNZXNzYWdlLS0tLS0NCkZyb206IHRjcG0gW21haWx0bzp0Y3BtLWJvdW5jZXNAaWV0Zi5vcmdd IE9uIEJlaGFsZiBPZiBTY2hhcmYsIE1pY2hhZWwgKE1pY2hhZWwpDQpTZW50OiBXZWRuZXNkYXks IE9jdG9iZXIgNywgMjAxNSAxMDo1OSBBTQ0KVG86IHRjcG1AaWV0Zi5vcmcNClN1YmplY3Q6IFJl OiBbdGNwbV0gW2FxbV0gVENQIEFDSyBTdXBwcmVzc2lvbg0KDQpUaGlzIHRocmVhZCBpcyBtYXkg YmUgcmVsZXZhbnQgdG8gVENQTQ0KDQpNaWNoYWVsDQoNCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdl LS0tLS0NCkZyb206IGFxbSBbbWFpbHRvOmFxbS1ib3VuY2VzQGlldGYub3JnXSBPbiBCZWhhbGYg T2YgSm9uYXRoYW4gTW9ydG9uDQpTZW50OiBXZWRuZXNkYXksIE9jdG9iZXIgMDcsIDIwMTUgNzo1 MyBQTQ0KVG86IGRhdmVjYkBzcGFtY29wLm5ldA0KQ2M6IGFxbUBpZXRmLm9yZw0KU3ViamVjdDog UmU6IFthcW1dIFRDUCBBQ0sgU3VwcHJlc3Npb24NCg0KDQo+IE9uIDcgT2N0LCAyMDE1LCBhdCAy MDozNiwgRGF2aWQgQ29sbGllci1Ccm93biA8ZGF2ZWMtYkByb2dlcnMuY29tPiB3cm90ZToNCj4g DQo+IE9uIDA3LzEwLzE1IDAxOjE5IFBNLCBEYXZpZCBMYW5nIHdyb3RlOg0KPj4gT24gV2VkLCA3 IE9jdCAyMDE1LCBNaWthZWwgQWJyYWhhbXNzb24gd3JvdGU6DQo+PiANCj4+Pj4gT2gsIEkgaG9w ZSB0aGF0IHRoaXMgaXMgYW4gZXhjZXB0aW9uLiBTdWNoIGtpbmQgb2Ygb3B0aW1pemF0aW9ucyBt YXkgY2F1c2UgYSBsb3Qgb2YgdHJvdWJsZSBzaW5jZSBhIGxpbmsgbGF5ZXIgZGV2aWNlIGlzIGlu dGVyZmVyaW5nIHdpdGggdHJhbnNwb3J0IGxheWVyIHNlbWFudGljcy4gV2UgYWxsIGtub3cgdGhh dCBleGFjdGx5IHRoZXNlIGtpbmRzIG9mIGludGVyZmVyZW5jZSBldmVudHVhbGx5IGVuZCB1cCBp biBwcm9ibGVtcyB3aXRoIGVuZC10by1lbmQgdHJhbnNwYXJlbmN5IGFuZCBkZXBsb3ltZW50IG9m IG5ldyBwcm90b2NvbCBvcHRpb25zLiBBdCBsZWFzdCBpdCBpbnRlcmZlcmVzIHdpdGggdGhlIEFD SyBjbG9ja2luZyBleHBlY3RhdGlvbiBvZiBzb21lIGNvbmdlc3Rpb24gY29udHJvbCBhbGdvcml0 aG1zLi4uDQo+Pj4gDQo+Pj4gUGVyc29uYWxseSwgSSB0aGluayB5b3UncmUgZ29pbmcgdG8gc2Vl IG1vcmUgYW5kIG1vcmUgb2YgdGhpcy4gVGhlcmUgYXJlIG11bGl0cGxlIHNoYXJlZCBhY2Nlc3Mg bWVkaXVtIHdoZXJlIHlvdSdyZSBhbGxvd2VkIHRvIHNlbmQgb25seSBwYXJ0IG9mIHRoZSB0aW1l LCBhbmQgaXQncyBzb21lb25lIGVsc2Ugd2hvIHRlbGxzIHlvdSB3aGVuIHlvdSBtYXkgc2VuZC4N Cj4+IA0KPj4gaXQgZG9lc24ndCBldmVuIHJlcXVpcmUgdGhhdCBzb21lb25lIGVsc2UgdGVsbHMg eW91IHdoZW4geW91IG1heSBzZW5kLiBJdCBjYW4ganVzdCBiZSB3YWl0aW5nIGZvciBhbiBhdmFp bGFibGUgdHJhbnNtaXQgdGltZXNsb3QgKFdpZmkgZm9yIGV4YW1wbGUpDQo+PiANCj4+IGNvbGxh cHNpbmcgbXVsdGlwbGUgQUNLcyB0aGF0IGFyZSBnb2luZyB0byBiZSBzZW50IGF0IG9uY2UgaXMg YWxtb3N0IGFsd2F5cyBnb2luZyB0byBiZSBhIHdpbi4NCj4gDQo+IEkgcXVpdGUgYWdyZWUsIGJ1 dCBpZiB0aGVyZSBpcyBhIGNvbmdlc3Rpb24gY29udHJvbCBpbXBsZW1lbnRhdGlvbiAiaW4gdGhl IHdpbGQiIHRoYXQgYXNzdW1lcyBpdCB3aWxsIGdldCBhIHN0cmVhbSBvZiBhY2tzLCB0aGF0IG9u ZSdzIGdvaW5nIHRvIG5lZWQgc29tZSB3b3JrICg6LSkpDQo+IA0KPiBBbnlvbmUga25vdyBpZiB0 aGF0J3MgdGhlIGNhc2U/IFRoZSBjb21tZW50IGFib3ZlIHN1Z2dlc3QgaXQgbWF5IGJlLi4uDQoN Ckl04oCZcyBub3Qg4oCcaW4gdGhlIHdpbGTigJ0sIGJ1dCB0aGlzIHNvcnQgb2YgdGhpbmcgaXMg YSBoZWFkYWNoZSBmb3IgRUxSLiAgSXQgZXNzZW50aWFsbHkgbWVhbnMgdGhhdCBpdCB3b27igJl0 IGJlIGFibGUgdG8gdXNlIEFjY0VDTiBmb3IgaXRzIGZlZWRiYWNrIChzaW5jZSBBY2NFQ04gZG9l c27igJl0IGFsbG93IHJlY29uc3RydWN0aW5nIGEgY29tcGxleCAzMi1wYWNrZXQgaGlzdG9yeSBv ZiBFQ04gY29kZXBvaW50cyBmcm9tIGEgc2luZ2xlIEFDSyksIGJ1dCBtdXN0IGludHJvZHVjZSBz b21lIG90aGVyIG1lY2hhbmlzbSB0byBmZWVkIGJhY2sgdGhlIHJlcXVpcmVkIGRhdGEgdG8gdGhl IHNlbmRlci4NCg0KSeKAmW0gbGVhbmluZyB0b3dhcmRzIG1vdmluZyB0aGUgRUxSLWF3YXJlIGNv bmdlc3Rpb24gY29udHJvbCBhbGdvcml0aG0gdG8gdGhlIHJlY2VpdmVyLCBhbmQgaGF2aW5nIHRo ZSBmZWVkYmFjayBzaW1wbHkgYmUgdGhlIHJlcXVlc3RlZCBjd25kOyBpdOKAmXMgdGhlIHNpbXBs ZXN0IGFuZCBtb3N0IGNvbXBhY3Qgc29sdXRpb24gSSBjYW4gdGhpbmsgb2YsIGFuZCB3b3VsZCB3 b3JrIGZvciBhbnkgY29uZ2VzdGlvbiBjb250cm9sIHNjaGVtZSwgbm90IGp1c3QgRUxSLiAgVGhl IHNlbmRlciBtYXkgb2YgY291cnNlIG1haW50YWluIGl0cyBvd24gaWRlYSBvZiBjd25kIGluIHRo ZSBjb252ZW50aW9uYWwgd2F5LCBhbmQgY29uc2VydmF0aXZlbHkgdGFrZSB0aGUgbWluaW11bSBv ZiB0aGUgdHdvIGFzIGVmZmVjdGl2ZTsgdGhpcyB3b3VsZCBwcm90ZWN0IGFnYWluc3QgbmFpdmVs eSBncmVlZHkgcmVjZWl2ZXJzLg0KDQpJIG5vdGUgdGhhdCBDVUJJQyBkb2VzbuKAmXQgY2FyZSBo b3cgbWFueSBhY2tzIGl0IGdldHMsIHNpbmNlIGl0cyBzY2FsaW5nIGN1cnZlIGRlcGVuZHMgb25s eSBvbiB0aW1lIHNvIGxvbmcgYXMgdGhlIGRhdGEgZmxvdyBpcyBub3QgYXBwbGljYXRpb24tbGlt aXRlZC4gIFRoYXTigJlzIGdvb2QgbmV3cyBmb3IgTGludXgsIGdpdmVuIHRoZSBsYWNrIG9mIEFC Qy4gIERvIHRoZSBCU0RzIGFuZCBXaW5kb3dzIHVzZSBBQkM/DQoNCiAtIEpvbmF0aGFuIE1vcnRv bg0KDQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KYXFt IG1haWxpbmcgbGlzdA0KYXFtQGlldGYub3JnDQpodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFu L2xpc3RpbmZvL2FxbQ0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18NCnRjcG0gbWFpbGluZyBsaXN0DQp0Y3BtQGlldGYub3JnDQpodHRwczovL3d3dy5pZXRm Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG0NCg== From nobody Wed Oct 7 12:05:31 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5993A1ACEA9 for ; Wed, 7 Oct 2015 12:05:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.49 X-Spam-Level: X-Spam-Status: No, score=0.49 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_21=0.6, J_CHICKENPOX_31=0.6, J_CHICKENPOX_41=0.6, J_CHICKENPOX_51=0.6, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P2_f5Hj-Z2PI for ; Wed, 7 Oct 2015 12:05:29 -0700 (PDT) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 543631ACE98 for ; Wed, 7 Oct 2015 12:05:29 -0700 (PDT) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id 0596DD4C13; Wed, 7 Oct 2015 12:05:29 -0700 (PDT) Date: Wed, 7 Oct 2015 12:05:28 -0700 From: hiren panchasara To: "Scharf, Michael (Michael)" Message-ID: <20151007190528.GB42742@strugglingcoder.info> References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5614D4B8.5060403@kit.edu> <5615581C.5010907@rogers.com> <655C07320163294895BBADA28372AF5D484F882F@FR712WXCHMBA15.zeu.alcatel-lucent.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="cvVnyQ+4j833TQvp" Content-Disposition: inline In-Reply-To: <655C07320163294895BBADA28372AF5D484F882F@FR712WXCHMBA15.zeu.alcatel-lucent.com> User-Agent: Mutt/1.5.23 (2014-03-12) Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Oct 2015 19:05:30 -0000 --cvVnyQ+4j833TQvp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 10/07/15 at 05:58P, Scharf, Michael (Michael) wrote: > This thread is may be relevant to TCPM >=20 > Michael >=20 >=20 > -----Original Message----- > From: aqm [mailto:aqm-bounces@ietf.org] On Behalf Of Jonathan Morton > Sent: Wednesday, October 07, 2015 7:53 PM > To: davecb@spamcop.net > Cc: aqm@ietf.org > Subject: Re: [aqm] TCP ACK Suppression >=20 >=20 > > On 7 Oct, 2015, at 20:36, David Collier-Brown wrot= e: > >=20 > > On 07/10/15 01:19 PM, David Lang wrote: > >> On Wed, 7 Oct 2015, Mikael Abrahamsson wrote: > >>=20 > >>>> Oh, I hope that this is an exception. Such kind of optimizations may= cause a lot of trouble since a link layer device is interfering with trans= port layer semantics. We all know that exactly these kinds of interference = eventually end up in problems with end-to-end transparency and deployment o= f new protocol options. At least it interferes with the ACK clocking expect= ation of some congestion control algorithms... > >>>=20 > >>> Personally, I think you're going to see more and more of this. There = are mulitple shared access medium where you're allowed to send only part of= the time, and it's someone else who tells you when you may send. > >>=20 > >> it doesn't even require that someone else tells you when you may send.= It can just be waiting for an available transmit timeslot (Wifi for exampl= e) > >>=20 > >> collapsing multiple ACKs that are going to be sent at once is almost a= lways going to be a win. > >=20 > > I quite agree, but if there is a congestion control implementation "in = the wild" that assumes it will get a stream of acks, that one's going to ne= ed some work (:-)) > >=20 > > Anyone know if that's the case? The comment above suggest it may be... >=20 > It?s not ?in the wild?, but this sort of thing is a headache for ELR. It= essentially means that it won?t be able to use AccECN for its feedback (si= nce AccECN doesn?t allow reconstructing a complex 32-packet history of ECN = codepoints from a single ACK), but must introduce some other mechanism to f= eed back the required data to the sender. >=20 > I?m leaning towards moving the ELR-aware congestion control algorithm to = the receiver, and having the feedback simply be the requested cwnd; it?s th= e simplest and most compact solution I can think of, and would work for any= congestion control scheme, not just ELR. The sender may of course maintai= n its own idea of cwnd in the conventional way, and conservatively take the= minimum of the two as effective; this would protect against naively greedy= receivers. >=20 > I note that CUBIC doesn?t care how many acks it gets, since its scaling c= urve depends only on time so long as the data flow is not application-limit= ed. That?s good news for Linux, given the lack of ABC. Do the BSDs and Wi= ndows use ABC? > FreeBSD uses ABC and is turned on by default. Cheers, Hiren --cvVnyQ+4j833TQvp Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJWFWz0XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lvKYH/2x/PKgYrX3Ny8VDqjzrPKkg RSfhpQ+MeD/1teUmQApRHJL471tBmp1GFqQIWpsdy5RDRpVKA93kEeW0cCul45ig 7PXlR+08VPzrLUnv47NYb83APTaKnvIiDUntlU1x4Jg9QM8XbG/bnqoBuvJztEBa 2fGNdrL+8OXe9i6CiS0wr4Huat6Q59jLacLLu083+aQNH38PRh0lN6XIigP0twZB 81IW0VJEmOCoTiCp5LI96x/gVvGRe7Mi3iJC6rZLCee5hC225OyqP+8xlfqOCfJC NqdWji8+W+8MiRzFsd5AA4bUBPe3BL5EpP3ScJMIun5esE06o0hB/giRZzfntng= =j7lB -----END PGP SIGNATURE----- --cvVnyQ+4j833TQvp-- From nobody Thu Oct 8 02:45:03 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CA1D51A904C; Thu, 8 Oct 2015 02:45:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.85 X-Spam-Level: X-Spam-Status: No, score=-3.85 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HGsWUBKfjb2c; Thu, 8 Oct 2015 02:44:58 -0700 (PDT) Received: from iramx2.ira.uni-karlsruhe.de (iramx2.ira.uni-karlsruhe.de [141.3.10.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7D0701A904B; Thu, 8 Oct 2015 02:44:57 -0700 (PDT) Received: from i72vorta.tm.uni-karlsruhe.de ([141.3.71.26] helo=i72vorta.tm.kit.edu) by iramx2.ira.uni-karlsruhe.de with esmtp port 25 iface 141.3.10.81 id 1Zk7ks-00020B-8n; Thu, 08 Oct 2015 11:44:54 +0200 Received: from [IPv6:::1] (ip6-localhost [IPv6:::1]) by i72vorta.tm.kit.edu (Postfix) with ESMTPS id 2D021B006E1; Thu, 8 Oct 2015 11:44:54 +0200 (CEST) To: dpreed@reed.com, aqm@ietf.org References: <1444247538.3556484@apps.rackspace.com> From: "Bless, Roland (TM)" X-Enigmail-Draft-Status: N1110 Organization: Institute of Telematics, Karlsruhe Institute of Technology Message-ID: <56163B16.5020504@kit.edu> Date: Thu, 8 Oct 2015 11:44:54 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <1444247538.3556484@apps.rackspace.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-ATIS-AV: ClamAV (iramx2.ira.uni-karlsruhe.de) X-ATIS-Timestamp: iramx2.ira.uni-karlsruhe.de 1444297494. Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] [aqm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Oct 2015 09:45:01 -0000 Hi David, Am 07.10.2015 um 21:52 schrieb dpreed@reed.com: > Nonetheless I am troubled by the very fact of the discussion taking > place, for two reasons: > > 1) TCP ACKs are TCP's business only. It's not a gateway or router's job > to get involved in or to understand end-to-end protocols, *even if* the > router thinks it knows exactly what the endpoints' goals are. And the > router cannot know that for every protocol, not even the many higher > level protocols on top of TCP, which use TCP quite differently. The > idea that routers can be omniscient, merely by looking at packets and > taking the designers' personal prejudices into account, seems > ridiculous. TCP endpoints on both ends of a connection can reduce the > number of ACKs they send if they want. If ACKs are filling up buffers in > intermediate routers, just drop them or mark them to notify that they > are contributing to congestion. The endpoints have to slow down > something, and they can slow down ACKs by mutual agreement. +1 > 2) The hypothetical that there will be a sufficiently long sequence of > ACKs for a single end-to-end flow buffered in a single router queue may > seem plausible, *until it becomes clear that in the big picture, having > so many packets in a queue means that the network is extremely congested > by that point*. In other words, in order for this "optimization" to > apply, you would have to operate the network at an unacceptable > operating point! It's like saying that when a highway has slowed to a > crawl, we can load all the cars going to a particular destination onto > single "car carrier" to save gas. Far better to insure that queues are > not built up! The purpose of queueing is to absorb bursts that can't be > anticipated, not to build up congestion in order to have enough data to > perform a dubious optimization that can best be done at the source of > traffic in cooperation with the destination. +1 Maybe an incentive for some people to think about alternative link layer access schemes that will be better suited for such kind of Internet traffic. As already pointed out the specific optimization will be useless for newer transport protocols as well as for tunneled or encrypted traffic or advanced TCP features, including ECN feedback. > It's said that in committees the amount of time spent on trivialities > far exceeds the time spent on important issues. That seems to be true > as I watch the discussion on this list. I think the "discussion" seems to be necessary since Mikael's original question was: >> Now, this kind of mechanism, how should it be treated when it comes >> to AQM? This mechanism is basically done at de-queue, when a number of >> packets are emptied from the queue at one time, which is then allowed to >> fill up again until the next transmit opportunity arises. I really hate it if the IETF tries to work around "broken" (short-sighted, cross-layer optimized, and so on) behavior of middleboxes or other devices. So I don't think that the AQM WG should take into account this particular optimization of specific link layer boxes. Otherwise, we'll make the situation only worse for transport protocol evolution. We've got enough examples for ossification due to non-farseeing implementations of middlebox stuff. It's quite perverted if we start to design mechanisms according to such kind of "special/broken" behavior. Regards, Roland From nobody Thu Oct 8 07:41:14 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 931831A1A38 for ; Thu, 8 Oct 2015 07:41:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.51 X-Spam-Level: X-Spam-Status: No, score=-14.51 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H37LW3tq1eHN for ; Thu, 8 Oct 2015 07:41:06 -0700 (PDT) Received: from alln-iport-3.cisco.com (alln-iport-3.cisco.com [173.37.142.90]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 428761A1A3D for ; Thu, 8 Oct 2015 07:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=70998; q=dns/txt; s=iport; t=1444315266; x=1445524866; h=from:to:subject:date:message-id:references:in-reply-to: mime-version; bh=mjIWmzQWO2KdwPeJANlJ1Zer/rQdyiX138V48MEKSPk=; b=U+bRODvr2Ee5w6W7jMS9jdKDhr6xbuEYQ2gRh6ZO6pL9ICrUKFbAn/1b bHcPs9N61I8qMaWh21LeIlgWfFIDyaoZuhB4alYXJ/QVldQCEK2GI9oOE 8uCpVqN6/9z0POcr+gSdhvZC6lB0ZwHCI1cy/weuZ24qGW9luEr3exofA Y=; X-Files: image001.jpg : 25518 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0CzAwBdfxZW/5ldJa1eglpNVG4GrECRAg6BWiN4gXiCCn8CgUg4FAEBAQEBAQGBCoQmAQEBBAUgCAE6IQIBCBEBAwEBBgEBAQoOAQYHAgUQAQwCDBQDBggCBAERAQYCBg2IEw3DJAEBAQEBAQEBAQEBAQEBAQEBAQEBAReGc4R+hDQ+FwoBhC4FjUOIRwGELQFpKIJHhQqBX0iDcgeVWQcBEQ4BQ4IRHYFUcQGGIkOBBgEBAQ X-IronPort-AV: E=Sophos;i="5.17,655,1437436800"; d="jpg'145?scan'145,208,217,145";a="195319162" Received: from rcdn-core-2.cisco.com ([173.37.93.153]) by alln-iport-3.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Oct 2015 14:41:04 +0000 Received: from XCH-ALN-009.cisco.com (xch-aln-009.cisco.com [173.36.7.19]) by rcdn-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id t98Ef4AC022143 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 8 Oct 2015 14:41:04 GMT Received: from xch-aln-009.cisco.com (173.36.7.19) by XCH-ALN-009.cisco.com (173.36.7.19) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Thu, 8 Oct 2015 09:41:03 -0500 Received: from xch-aln-009.cisco.com ([173.36.7.19]) by XCH-ALN-009.cisco.com ([173.36.7.19]) with mapi id 15.00.1104.000; Thu, 8 Oct 2015 09:41:03 -0500 From: "Lane Wigley (lwigley)" To: "Scharf, Michael (Michael)" , "Kuhn Nicolas" , "tcpm@ietf.org" Thread-Topic: TCP evolution impact on router buffers Thread-Index: AdD3qDJMG7WkdcJDTiy1iIy/H6510wDXEZsQAB6kmpAABLWoIABn/A/AAOxXFcAAAu7VEAA5/SuQ Date: Thu, 8 Oct 2015 14:41:03 +0000 Message-ID: <068cb32c994d4438880ec3907a6bbcd5@XCH-ALN-009.cisco.com> References: <655C07320163294895BBADA28372AF5D484D799E@FR712WXCHMBA15.zeu.alcatel-lucent.com> <46616f47b42841f0bb7a2afb8bdeac98@XCH-ALN-009.cisco.com> <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> <655C07320163294895BBADA28372AF5D484F7BD9@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: <655C07320163294895BBADA28372AF5D484F7BD9@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.116.71.37] Content-Type: multipart/related; boundary="_004_068cb32c994d4438880ec3907a6bbcd5XCHALN009ciscocom_"; type="multipart/alternative" MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] TCP evolution impact on router buffers X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Oct 2015 14:41:12 -0000 --_004_068cb32c994d4438880ec3907a6bbcd5XCHALN009ciscocom_ Content-Type: multipart/alternative; boundary="_000_068cb32c994d4438880ec3907a6bbcd5XCHALN009ciscocom_" --_000_068cb32c994d4438880ec3907a6bbcd5XCHALN009ciscocom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Just to wrap this thread up, I found a paper that addressed my original que= stion. It looked at the impact of mixing different TCP implementations and = one of the calculations was the standard deviation of the congestion window= with various algorithms. All of the newer ones have lower standard deviati= on than Reno. [cid:image001.jpg@01D101B5.D389B220] http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=3D6742797 - Lane From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Sent: Wednesday, October 07, 2015 7:35 AM To: Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Regarding 1: Sure, it doesn't make any sense to add architecture-specific g= uidelines. I just wonder if the text could mention a bit more explicitly th= at a number of potential router components and algorithms (e.g., DiffServ) = are omitted (or abstracted) to keep the guidelines generic and simple. Regarding 2: Yes, given the references mentioned by Lane, I'd suggest to so= mehow replace the wording "... and is to be set to the bandwidth-delay prod= uct", e.g., by a MAY. But I leave this to the authors - this is not really in TCPM scope, and I d= on't have cycles to follow the AQM WG. Michael From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] Sent: Wednesday, October 07, 2015 12:17 PM To: Scharf, Michael (Michael); Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Hi, 1- Regarding the CIR/PIR IMHO this is out of the scope of the characterization guidelines. However, = if you think that we should introduce more information on the MPLS/IP route= r architectures please let us know. The only problem being that we want to = be quite generic and do not want to focus on specific architectures. 2- Regarding buffer sizes Since buffer sizing is not the only problem of evaluating AQMs, we have loo= ked for RFC clearly stipulating how the "adequate" buffer size can be found= . As said in RFC7597: " The optimum buffer size is a function of operational requirements and should generally be sized to be sufficient to buffer the largest normal traffic burst that is expected. This size depends on the amount and burstiness of traffic arriving at the queue and the rate at which traffic leaves the queue. " In the characterization draft, the recommendation on the buffer size is not= a "MUST". If you think that we should more clearly say that the size of the buffer MA= Y be set to the BDP, let us know. In its current state, it has been voluntary that the draft does not impose = any buffer size. Cheers, Nicolas De : Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Envoy=E9 : vendredi 2 octobre 2015 19:06 =C0 : Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org Objet : RE: TCP evolution impact on router buffers Regarding Section 7 in [2], I am surprised that a potential impact of CIR/P= IR is not mentioned. Unfortunately, I have only a very limited understandin= g of MPLS/IP router architectures, but the way bursts hit an AQM could IMHO= depend on that. As [2] is mentioned, one question related to the discussion below. From [2]= : 3.2. Buffer size The size of the buffers should be carefully chosen, and is to be set to the bandwidth-delay product; the bandwidth being the bottleneck capacity and the delay the largest RTT in the considered network. This means of the order of 10GB of buffer size for a 400G router interface,= right? Wouldn't it make sense to relax the statement "... and is to be set= ..." a bit? Michael From: Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] Sent: Wednesday, September 30, 2015 6:41 PM To: Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers Hi, In [1], the authors evaluate the performance of CUBIC in small buffer regim= e. However, they use the Linux kernel version 2.6.18 (where the IW was 3 an= d not 10). Also, for any tests on CUBIC, small buffers and IW3/10, you may = be interested in looking at the scenario "7. Burst absorption" of the AQM = Characterization Guidelines of the AQM WG [2]. Concerning TCP pacing, there= is an on-going document on a pacing solution in the slow-start that you ma= y want to have a look and comment [3]. I hope this helps, Cheers, Nicolas [1] Jain, S.; Raina, G., "An experimental evaluation of CUBIC TCP in a smal= l buffer regime," in Communications (NCC), 2011 National Conference on , vo= l., no., pp.1-5, 28-30 Jan. 2011 doi: 10.1109/NCC.2011.5734779 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&is= number=3D5734693 [2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08 [3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01 De : tcpm [mailto:tcpm-bounces@ietf.org] De la part de Lane Wigley (lwigley= ) Envoy=E9 : mercredi 30 septembre 2015 15:10 =C0 : Scharf, Michael (Michael); tcpm@ietf.org Objet : Re: [tcpm] TCP evolution impact on router buffers This paper from 2008 shows no drops with a 5 msec buffer on a heavily-conge= sted core link. Low levels of drops start to show up around 2.5 msec. This = was conducted at Level 3 to test the BDP / sqrt # flows proposal from Stanf= ord. Experimental Study of Router Buffer Sizing - Neda Beheshti, Yashar Ganjali,= Monia Ghobadi, Nick McKeown, and Geoff Salmon They also explored TCP pacing as a way to further reduce buffers. I'm tryin= g to assess if CUBIC results in some pacing effect by decoupling the window= growth from acks. - Lane From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] Sent: Tuesday, September 29, 2015 6:56 PM To: Lane Wigley (lwigley); tcpm@ietf.org Subject: RE: TCP evolution impact on router buffers This question may also be in scope of the AQM WG, and perhaps ICCRG, but si= nce the communities overlap, I avoid cross-posting... Unfortunately, I don't have any recent pointer or own data. But I am aware = of examples for router buffer dimensioning based on tests with a single TCP= connection, using Reno. In those examples, the dimension of the buffers an= d possibly other related parameters (two-color/three-color meters/policiers= , WRED slopes, H-QoS, etc.) are configured to achieve full "line" speed for= a single TCP connection. As to be expected, this typically comes down to t= he old BDP rule of thumb. But it is rather obvious that a single bulk data = Reno connection is not necessarily a realistic workload these days. So, the= test methodology can be one relevant aspect of the dimensioning problem. I believe that a modern TCP stack with a high-speed congestion control such= as CUBIC will typically need less buffer than the outcome of this dimensio= ning method and the publications 10 years ago. As far as I know, most moder= n TCP stacks operate well over a very wide range of buffer sizes, and the b= uffers can be relatively small. To me, the key question is the latency-thro= ughput tradeoff, and there may not be a one-fits-all answer to that one. Personally, I'd also be interested in more recent pointers on router buffer= dimensioning. I could even see some value in documenting dimensioning guid= elines used in practice, if there was a way to generalize them. Michael From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Lane Wigley (lwigley= ) Sent: Friday, September 25, 2015 5:41 PM To: tcpm@ietf.org Subject: [tcpm] TCP evolution impact on router buffers I'm trying to track down some research or opinions on how the more recent T= CP schemes impact router buffer needs. There are a number of publications f= rom Stanford and Georgia Tech from about 10 years ago, and I'm trying to as= sess how changes to the algorithms (e.g. CUBIC) and parameters (initial con= gestion window) deployed since then may influence those findings in the dir= ection of more or less buffering being needed. I'd appreciate any input and pointers. Thanks. - Lane --_000_068cb32c994d4438880ec3907a6bbcd5XCHALN009ciscocom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Just to wrap this thre= ad up, I found a paper that addressed my original question. It looked at th= e impact of mixing different TCP implementations and one of the calculation= s was the standard deviation of the congestion window with various algorithms. All of the newer ones have lowe= r standard deviation than Reno.

 

 

http://ieeexplore.ieee= .org/stamp/stamp.jsp?arnumber=3D6742797

 

- Lane

 

From: Scharf, = Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com]
Sent: Wednesday, October 07, 2015 7:35 AM
To: Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org
Subject: RE: TCP evolution impact on router buffers

 

Regarding 1: Sure, it = doesn’t make any sense to add architecture-specific guidelines. I jus= t wonder if the text could mention a bit more explicitly that a number of p= otential router components and algorithms (e.g., DiffServ) are omitted (or abstracted) to keep the guidelines generi= c and simple.

 

Regarding 2: Yes, give= n the references mentioned by Lane, I’d suggest to somehow replace th= e wording “… and is to be set to the bandwidth-delay product= 221;, e.g., by a MAY.

 

But I leave this to th= e authors – this is not really in TCPM scope, and I don’t have = cycles to follow the AQM WG.

 

Michael

 

 

 

Hi, <= /o:p>

 = ;

1-     Re= garding the CIR/PIR

 = ;

IMHO this is out of th= e scope of the characterization guidelines. However, if you think that we s= hould introduce more information on the MPLS/IP router architectures please= let us know. The only problem being that we want to be quite generic and do not want to focus on specific arch= itectures.

 

2-&n= bsp;    Regarding buff= er sizes

 

Since buffer sizing is= not the only problem of evaluating AQMs, we have looked for RFC clearly st= ipulating how the “adequate” buffer size can be found. As said = in RFC7597:

“

The optimum buffer siz= e is a function of

   operation= al requirements and should generally be sized to be

   sufficien= t to buffer the largest normal traffic burst that is

   expected.=   This size depends on the amount and burstiness of traffic=

   arriving = at the queue and the rate at which traffic leaves the queue.

“

 

In the characterizatio= n draft, the recommendation on the buffer size is not a “MUST”.

If you think that we s= hould more clearly say that the size of the buffer MAY be set to the BDP, l= et us know.

In its current state, = it has been voluntary that the draft does not impose any buffer size.

 

Cheers,

 

Nicolas

 = ;

De : Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com]
Envoy=E9 : vendredi 2 octobre 2015 19:06
=C0 : Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org
Objet : RE: TCP evolution impact on router buffers

 

Regarding Section 7 in= [2], I am surprised that a potential impact of CIR/PIR is not mentioned. U= nfortunately, I have only a very limited understanding of MPLS/IP router ar= chitectures, but the way bursts hit an AQM could IMHO depend on that.

 

As [2] is mentioned, o= ne question related to the discussion below. From [2]:

 

3.2.  Buffer size=

 

   The size = of the buffers should be carefully chosen, and is to be set

   to the ba= ndwidth-delay product; the bandwidth being the bottleneck=

   capacity = and the delay the largest RTT in the considered network.<= /p>

 

This means of the orde= r of 10GB of buffer size for a 400G router interface, right? Wouldn’t= it make sense to relax the statement “… and is to be set…= ;” a bit?

 

Michael

 

 

 

Hi,

 =

In [1], the authors eval= uate the performance of CUBIC in small buffer regime. However, they use the= Linux kernel version 2.6.18 (where the IW was 3 and not 10). Also, for any= tests on CUBIC, small buffers and IW3/10, you may be interested in looking at the scenario “7.  Burst abs= orption” of the AQM Characterization Guidelines of the AQM WG [2]. Co= ncerning TCP pacing, there is an on-going document on a pacing solution in = the slow-start that you may want to have a look and comment [3].

 =

I hope this helps, =

Cheers,

 =

Nicolas=

 = ;

[1] Jain, S.; Raina, G.,= "An experimental evaluation of CUBIC TCP in a small buffer regime,&qu= ot; in Communications (NCC), 2011 National Conference on , vol., no., pp.1-= 5, 28-30 Jan. 2011
doi: 10.1109/NCC.2011.5734779
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=3D&arnumber=3D5734779&= amp;isnumber=3D5734693

[2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08

[3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01

 = ;

De : tcpm [mailto:t= cpm-bounces@ietf.org] De la part de Lane Wigley (lwigley)
Envoy=E9 : mercredi 30 septembre 2015 15:10
=C0 : Scharf, Michael (Michael); tcpm@ietf.org
Objet : Re: [tcpm] TCP evolution impact on router buffers<= /o:p>

 

This paper from 2008 s= hows no drops with a 5 msec buffer on a heavily-congested core link. Low le= vels of drops start to show up around 2.5 msec. This was conducted at Level= 3 to test the BDP / sqrt # flows proposal from Stanford.

 

Experimental Study of Router Buffer Sizing – N= eda Beheshti, Yashar Ganjali, Monia Ghobadi, Nick McKeown, and Geoff Salmon=

 

They also explored TCP pacing as a way to further re= duce buffers. I’m trying to assess if CUBIC results in some pacing ef= fect by decoupling the window growth from acks.

 

- Lane

 

 

From: Scharf, = Michael (Michael) [mai= lto:michael.scharf@alcatel-lucent.com]
Sent: Tuesday, September 29, 2015 6:56 PM
To: Lane Wigley (lwigley); tcpm@iet= f.org
Subject: RE: TCP evolution impact on router buffers

 

This question may also= be in scope of the AQM WG, and perhaps ICCRG, but since the communities ov= erlap, I avoid cross-posting…

 

Unfortunately, I don&#= 8217;t have any recent pointer or own data. But I am aware of examples for = router buffer dimensioning based on tests with a single TCP connection, usi= ng Reno. In those examples, the dimension of the buffers and possibly other related parameters (two-color/three-colo= r meters/policiers, WRED slopes, H-QoS, etc.) are configured to achieve ful= l “line” speed for a single TCP connection. As to be expected, = this typically comes down to the old BDP rule of thumb. But it is rather obvious that a single bulk data Reno connection= is not necessarily a realistic workload these days. So, the test methodolo= gy can be one relevant aspect of the dimensioning problem.

 

I believe that a moder= n TCP stack with a high-speed congestion control such as CUBIC will typical= ly need less buffer than the outcome of this dimensioning method and the pu= blications 10 years ago. As far as I know, most modern TCP stacks operate well over a very wide range of buffer= sizes, and the buffers can be relatively small. To me, the key question is= the latency-throughput tradeoff, and there may not be a one-fits-all answe= r to that one.

 

Personally, I’d = also be interested in more recent pointers on router buffer dimensioning. I= could even see some value in documenting dimensioning guidelines used in p= ractice, if there was a way to generalize them.

 

Michael

 

 

 

From: tcpm [mailto:tcpm-= bounces@ietf.org] On Behalf Of Lane Wigley (lwigley)
Sent: Friday, September 25, 2015 5:41 PM
To: tcpm@ietf.org
Subject: [tcpm] TCP evolution impact on router buffers

 

I’m trying to track down some research or opin= ions on how the more recent TCP schemes impact router buffer needs. There a= re a number of publications from Stanford and Georgia Tech from about 10 ye= ars ago, and I’m trying to assess how changes to the algorithms (e.g. CUBIC) and parameters (initial congestion window) = deployed since then may influence those findings in the direction of more o= r less buffering being needed.

 

I’d appreciate any input and pointers. Thanks.=

 

- Lane

 

 

--_000_068cb32c994d4438880ec3907a6bbcd5XCHALN009ciscocom_-- --_004_068cb32c994d4438880ec3907a6bbcd5XCHALN009ciscocom_ Content-Type: image/jpeg; name="image001.jpg" Content-Description: image001.jpg Content-Disposition: inline; filename="image001.jpg"; size=25518; creation-date="Thu, 08 Oct 2015 14:41:03 GMT"; modification-date="Thu, 08 Oct 2015 14:41:03 GMT" Content-ID: Content-Transfer-Encoding: base64 /9j/4AAQSkZJRgABAQEAeAB4AAD/2wBDAAoHBwkHBgoJCAkLCwoMDxkQDw4ODx4WFxIZJCAmJSMg IyIoLTkwKCo2KyIjMkQyNjs9QEBAJjBGS0U+Sjk/QD3/2wBDAQsLCw8NDx0QEB09KSMpPT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT3/wAARCAGhAqgDASIA AhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQA AAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3 ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWm p6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEA AwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSEx BhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElK U1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3 uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD2aiii gAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKA CiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoo ooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiii gAooooAKKKKACiiigAorktQ8Z3um6IdVn0NltfMCKDdDewLbQcY4z161rWuq3z6nFa3umfZ0ljZ0 mScSLkY+UjAIODQD0Neisy31+zufEN1oyMftVtEsrehB7D6cZ+opfEGrPoei3OorbfaFt0MjoJNh 2jrjg0dLh1saVFc1L4tmstGh1fUNLeLT5ER2kimEjRK2MFlwOORnGam8TeJ/+Ee0uC/jtPtkE0iR grKFOX+6eR0osC1N+imRM7RKZUCORyobcB+NYF74mubbxTFocOmrLLNA1xHIbgKpUHBz8pINAHRU Vl2+o351GO2vNNEMciMwmScSAEY+UjAI6/pWmxIUlQC2OATjNAC0Vz+ieJLjWrzU7cacIW0+YwOW nBDvjPGB096Tw/4luNfhvnj00Qm0ne3w9wDvdevQcD3oA6GisLwr4jfxPp8l59iNrEsrRANKHJKn B6DpW7QAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUU AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQA UUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQBx/xQ48Ey/wDXeH/0MV015dpYabJd SjKwxlsDqeOg9z0rO8U+Hm8TaX9gN41rEXV2KxhiSpyOvTmpb/SLq/t7OJr8J5EqyyYhBE205AIz wM80dBvocVrF2dIbSNeFpfJc2kp+3vJbMivHKfnyTxwSMfSur8bOsngPV3QhkazcgjuMVp6vpw1b SLmwdwi3EZjZim7AIx0rI/4RSdvBx8Pyaq7xmLyPPaEb/L6Y64zjvSeqaBbpmBq1tqsnw6tjK8E2 mJZxvdQRKY5pIgoJCuSQOBzxzUnju7tr74faZc2QItpbm1aIHqF3DArak8KXVzoqaRdaxKbERLC6 wwLG7oBjBbJ6gc4FP1/wjHrOi2ulW9ybK0tnRkVIwx+T7o5PSrvrfzJjokvI6HtXDaz9pb4saX9h aAS/2ZJgzAlcb/at680fVr22aB9c8pH4cw2qqxHcAknGaguvCs8viWDWbfUvIkt7c20UXkBlCHnn nJNSt7/1sPoXdNXV11S5bU3t2gMSCHyAQoOW3ZyevStasdNIvpNTt7m91UzxW+WW3SARqzEYDE5J OMmtdgSpCnBxwcZxQBx/gj/kO+LP+wmf/QRSfDr/AI9dc/7C0/8AMVp6J4an0W81O4XUfObUJjO4 aAAK+McYPT2pNA8NT6BDfJFqRm+1ztcEvABsduuMHp7UL9P8gev3/wCZmfCz/kUpP+v24/8AQzXZ 1h+FfDjeGNPksxetdRNK0oLRhSCxyeh6VuU2HVhRRRSAKKKKACiiigAooooAKKKKACiiigAooooA KKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAo ormfiLfT6f4F1Ka1ZkkKBN69VDEAn8jSbsNK7NJfEVjNI62rTXQjYq728TSIpHUbgMflU+mavZaz DJLp84mjjkMbkAjDDqOfrTdCtILDQrG2tVVYY4EChfp1/Gs7xLeL4X8PapqVjEouJCHwR8pkOEDH 9PyqnoxLU6CisKfRXj0R2ju7gaikO4XXmHcXAzkjpjPbGMVj3c1z4t8E6PeW9w9jf3Dxsk0ZI2Pz ngdQSOnpSA7WiuOttXHiDTo7bUomt9VsbuKO7gVyuDuHzDB5Rh0q3Jcy6x4zuNJaR47GwtkkkSNi pmd84BI52gDp3JoA6aisOCOwsfEkVtBeypM1u2LJnZlIyPnAPT0/Gudsn0yPxJ4jTVDO6w3CGJQZ XCKYwTjbkDnNAHfUVzOqJdReD4G0NLiZA0cpiEh814S25kVjznBxS+HpNN1YXVxpF5OsbxCGW2kZ g9u/P8Lcqef0oA6WiuKu9GtLbxjolggmNvJbTmRTO58wqFwTzyev51J4ssE0/RdKghln2rqcKA+a 27Y0nKk5yRg45oWtvP8AzsB2NFc3qd5NceLdP0KKR4rY273U5RsM6ghVTPUDJyaj1mZ/Dmq6PLaO 4tby6FpPAzFlO4Hawz0II/EGgDqKK4bVp7/w94mvNatWln02MRpfWgJbapGfNQeo7gdq39L+zXOs 3d7auZI5oInRxISpB3cgZxzxQBtUVheMdOm1Dw3dfZXkS6gXzoTG5Ull528HkHGPxrAutYhfW/Dm uW5l+yXUaxXI8w7I/M4jJGcZDAihagzvKKy9Pto5r6/vfmxM/kqC5xtXgkDoMnP5CsPwhEsvhjU/ NaSTN1cpl5GJ2hiAAc5GKAOworhvDN/daV5Ph/XZnniu4fM0+7cnMoIyYmP94dvUUviCIQfDi3MT yq6tBhxK27l1B+bOeQTTsB3FFcr4mgbw1pDavpMskRs2V5YC5aOZCwDAg5wcHgipLrUJdT8YW2kJ I8Voln9smCna0uW2queuOpPrS3A6aiuY1Kd/D/iHR1tnf7HqErWssDMWAbaWV1z0PBB9azhDHY6r 4i0a6lnEN1B9stn81tygjayqc5GGwRj1oA7iiuM0G4bV/Cthpt2HW8jn+z3SiRgytEcscg55AH/f VdkAAuB0AxTaAWiuK0i4m0XxpcW000j2GqySeQHcsIpo+qjPQFece1LLLLefEfS5Wml+yvDcCOIO Qh2FRvI6E5J/DFJa2DudpRXDymwXx/q0WpNM0ItoHjQGVlDHduIC9M4FdNoK2o0wSWJl+zSu0iCQ sSozjvyBx0oB7mlRXK22rXcPi+P7TITp2qxslqD0R48/+hrk/hU3jDTru/isf7Mumtb9Jt0MgY7S QpO1h0KnGDR5gdJRXIW2rxeJLbTnnie3vra9WG7tt5UxOFbIODypwCPWuvp2AKK5fxXqd5YSQ3to 7fZdNkSW9RR/rEb5SP8AgIO78q1tbSO78P3fzHY8DMroxBHHBBFS9rh1saVFcPpuoXVrZ3fhrXpX ku47VpLS6yV+1RBeuf769/zqXxLAqaF4eRWlUNe20bbZWBZW+8Cc5OaqwdDs6K5TxAH8LJZ6lp0s q263EcNzas5ZHR225APRgSORXV0gCiuDt7lH1GbTNdlubHWHuWa2uWkYRXCbsqEOduMcFeta3i/S 4RomsakzzG4W0YxESsojKqcEAHrnnNJuyuNK7sdNRWJ4f0m3t7SyvojKsj2qiQGRmD5AOSCeue/v VCz1a7h8XqLqQnT9ViP2MHojx9v+BL834VVtbEp3VzqqK5rxhp95fCwOlXTWuoRyl4X3HaxVSdjD oVOMVXtdXh8SwaZLLFJb3tve+TdWxcqYnCMSpAPIyAR60lqM62iuSvLOK7+I6QT+Y0LaaZDH5rBd 4kxuwDjOKnt7i40fxnFpPnyT2N9bPNCsrbmhdCNwDHkqQRwelC1B6HTUVyutatdWGvWV8shGlRTC zuV7bnxh/wDgLbR+Jq543yPBerOjOjx2zujIxUqQOCCKOlxpXdjeorhLG6juNS0iPw1JO8sTqdS+ Z/LWMpzvDfxE9Mc1f122jufHWiQS+YYpoLgyIJGVW2hcZAPbJp2EtTrKK5ieWbw/4r0y2hnkk0/U 98RgkYt5MiruDKTyAQCCK375FksJ1bODG3Q47Unorgt7FiiuM8LaOL/wPY3EVzcwX8sG4XSzMW3Z OCckgj2NUNX1iXWPhvZalclorj7TEkrRMy8iUI+Mc4IB496dtbAtrnoVFc9pq6NJrEa6cJ1nijaQ 58xVKnjBDcHr+lVUX+w/H+13kNprEP7re5KxzJyVAJwNy8/UUgOrorI0m2juHvr47it3IQoLnGxf lBAzxkgnj1rP+HwJ8LRyM8ju00uWdyxOJGA5J9BQB09FcrJq11beLrWaWQ/2VfbrONeySryG/wCB fMPwFQ+OdLit/Dmrakkk/wBrYKyyCZx5fKjCgHA/+vQB2FFY8ei2tpBcSweYqzQbXj81iuRzkZPB +lczp+sT6N8J7O+gZnvJlSNZJWL/ADu+0Mc9cZz+FAI76isDU9Ja00Gea1up1v4ITIty0hJZlGfm B4IOORisnUnvPFHh3QtQ0y4ay1GYiaNgx27gjMUYd1JGOaAO1orkrPVofEttp8k8T299bXgiurbe VMUgVsg4PIOAR611tABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAVV1LTrfVtNuLG8TfBcIU dfY1aooA57SLDXNFtI7DzLS/toRsimlZo5Ao6BgAQ2OmRirt9o51jRbqw1SUOLpSreWu0R+m36Hn mtSih6gtDFks9Xm0g6e0tuJGj8prwE5xjBYJj72PfGaH0d7HTtLsdKiiMFnIhPmyFTtUdsA5Jrao p3DyMLV/DMeoazYarBJ5F3ayL5hHSaMHOxvXB5FOu9Fnj8QDWdNkjWd4hBcQy52zIDlTkfdYc889 a26KQGLDpd1P4ki1e68qHy7VrcQxsXzuYMSWwPT0qpp2navpus6xdra2k0d/OsiD7SylQqBefkPp XS0UAZAi1n7JA4a0juFmLPCGPl+VyNoOM5xg5x1pljorx+JbrWZliheaBYPKiOd2CTuY4GT2HHAr aooAwr7S7248X6bqUSwfZbWKWN90hDkvjoMY4x60eKdKvNXtrKKyEOYLyK4YyuV4Rs4GAeTW7RQt LAY+paNJc6rZataukd7aq0ZVuVljbqpPbkZBptzpE+rapY3WoeXHDYuZYoI2LbpMYDMcDgAnAx1N bVFAGdaW1yuqahJcRQ/Z59gjw5YkAYO4Yx+pqp4f8Np4eur8W0pNncOrwwk/6jrlR/s5OR9a3KKA E61y0PgiCHw1qmkrMcXkskkb/wDPHJygH+6ea6qigCtZWv2DTobaP5vKjCgk/eIHUn3rG0DSNQ0v Q761nS3aaaaaWPZKSvzkkAnbxjNdFRQ9QWhhXXh5dY8Lw6bqSrHPHGuySFiTFIo4dTgHg1W1jQL+ 78HW+k27wSXMflb5JHKqdhBJ4B64/WumooAwdT0u+8QQR2l+ILWy3q88cUhkeUKchc4AAJAz3qa/ 0Z21u11exdEuoYmgdH+7LGTnGR0IIyDWxRQBiyaPNqWtWV/qHloljuaCCNi37xhgszYHQdBjvS6v 4ej1XWdKv2co1i7FgP8AlopH3T7bgD+FbNFAGPYeH47DxHqOqo5/0xU/d9lYDDN9ThfyrXpaKAOY v/Dl3q+lzQTvHaXS3v2q1nhcsYzuznkDnGR+NTT6JdDxTpV7bJALKxt5ISGkIc7scgYxxj1710NF C0/r5Ac4mnara+LNQ1SG3tZYLqGKJVa4KsNmeT8hHOatSR6zLpl2vl2kVxK+2KNZTtjQ9SW25LdT 09K2aKAOV1zwdFNpkX9iwxwahbSJLbyPK21GU/jwRkdO9a93FfTT6dIsMH7p984Mp4+Uj5fl55Pf FadFAGDf+GI7jxLY61bSGGeJsXCj7s6YIXPuCeDW62Qp2jJxwCcUtFHSwGBH4agvrSdtYtYpLu5L GXZKxUg8AdugwOnaotL0vV7Pwa2l3P2aa6jjaGF/NbaydF3HbnIGB07V0lFAGFr/AIcXxFoaW05+ z3sS7oJ42yYZMYyDxkHofUVFr+jX1/p+lQWYgZ7O5hnfzJCoITsMA9a6KincOljCvNJvNcntP7S8 mCzt5VnMETFzK6/d3MQPlB5xjmtwjIIpaKQHM6loeo65okem6mbQsJEY3SElgFYEFVxw2BjOfWtP xDYz6l4evrG08vzriFolMjEKMjGSQDWnRQ9VYadncxzaajH4Zhs4VgF2sSQsTIdoAADEHbnpnHFZ 2ueD45dPgbRIYoNQtZUlt5JJW2oVPI78EZHSupoovrcS0VjNuob6a606VIYNsTFpsynjKkfL8vPX viql74Yim8T2etW0hhmjOLlB92ddpC59wTwfSt2ijzA56603Ul8YLq1tDbSwLZ/ZtjzlGJ37s/dI qWLSLs6rNrF00Mt6sBhtYVJEcSk5OWxkknGTjtW5RQBzNz4Ptr/w/Nb3VujX08ZMjiVseaedwP15 6U+80vV9Q8CS6XdG2bUpbbyHkEjeWTjG7O3PvjFdHRQCdtTmZ9Av459O1PTjBDqUEawXMbOfLuYh 1UnGcjqDipdV03Up/Eumanaw2zx2cUqOjzFSxcDp8p6YroaKAMVNIuLzXINU1J4wbVGW2t4skIW4 Zix6nHHQYrUu1ke0lWJVaRkIUMcDOPWpqKHqrB1ucxo2l65p3hi30hRZwyxReX9qWRpMdeQu0c89 zTdW8LSjwnaaNo4ixbyxPunkK52OGJ4B5JB/Oupop31uBlq+rTXcG+1tLeEN+9cTmRyvoo2Dqcd6 Z4m0L+39LW3jmNvcRSpNDMvWNgeo/DI/GteikBEkQtrVYoEGI0CopOBwOOawNC0rVtG8KSWIW1N6 GkaNhKdnzMSCTtzxn0rpKKAOV1jwZb3WgGDT4Ui1BArwStK2I5VOQ3fv7dzVvxFpuo614RlsFW2W 9nRVcmQ+WpBBJB25PT0rfooArOszacyBE84x7du/5c49cf0rA0/wq7+A4vD+qFFdI9nmQMWwwbcr DIHIOK6iigNjEu7LVr/SH06aS3jMsflS3SEklTwxCY4JHucVI2mSWf8AZNvp0MX2SyOCHkIIUIVG ODk8+1a9FAGFe+GY5vE1nrVtIYZ4ztuFH3Z1wQufcE8Gt2iigAooooAKKKKACiiigAooooAKKKKA CiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoo ooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiii gAooooAKKKKACiiigCne6tp+nOiX1/a2zOMqJplQsPbJqv8A8JPoX/Qa03/wKj/xq3c6fZ3rK13a W87KMKZYwxH0yKh/sLSv+gZZf+A6f4UARf8ACT6F/wBBrTf/AAKj/wAaP+En0L/oNab/AOBUf+NS /wBhaV/0DLL/AMB0/wAKP7C0r/oGWX/gOn+FAEX/AAk+hf8AQa03/wACo/8AGj/hJ9C/6DWm/wDg VH/jUv8AYWlf9Ayy/wDAdP8ACj+wtK/6Bll/4Dp/hQBF/wAJPoX/AEGtN/8AAqP/ABo/4SfQv+g1 pv8A4FR/41L/AGFpX/QMsv8AwHT/AAo/sLSv+gZZf+A6f4UARf8ACT6F/wBBrTf/AAKj/wAaP+En 0L/oNab/AOBUf+NS/wBhaV/0DLL/AMB0/wAKP7C0r/oGWX/gOn+FAEX/AAk+hf8AQa03/wACo/8A Gj/hJ9C/6DWm/wDgVH/jUv8AYWlf9Ayy/wDAdP8ACj+wtK/6Bll/4Dp/hQBF/wAJPoX/AEGtN/8A AqP/ABo/4SfQv+g1pv8A4FR/41L/AGFpX/QMsv8AwHT/AAo/sLSv+gZZf+A6f4UARf8ACT6F/wBB rTf/AAKj/wAaP+En0L/oNab/AOBUf+NS/wBhaV/0DLL/AMB0/wAKP7C0r/oGWX/gOn+FAEX/AAk+ hf8AQa03/wACo/8AGj/hJ9C/6DWm/wDgVH/jUv8AYWlf9Ayy/wDAdP8ACj+wtK/6Bll/4Dp/hQBF /wAJPoX/AEGtN/8AAqP/ABo/4SfQv+g1pv8A4FR/41L/AGFpX/QMsv8AwHT/AAo/sLSv+gZZf+A6 f4UARf8ACT6F/wBBrTf/AAKj/wAaP+En0L/oNab/AOBUf+NS/wBhaV/0DLL/AMB0/wAKP7C0r/oG WX/gOn+FAEX/AAk+hf8AQa03/wACo/8AGj/hJ9C/6DWm/wDgVH/jUv8AYWlf9Ayy/wDAdP8ACj+w tK/6Bll/4Dp/hQBF/wAJPoX/AEGtN/8AAqP/ABo/4SfQv+g1pv8A4FR/41L/AGFpX/QMsv8AwHT/ AAo/sLSv+gZZf+A6f4UARf8ACT6F/wBBrTf/AAKj/wAaP+En0L/oNab/AOBUf+NS/wBhaV/0DLL/ AMB0/wAKP7C0r/oGWX/gOn+FAEX/AAk+hf8AQa03/wACo/8AGj/hJ9C/6DWm/wDgVH/jUv8AYWlf 9Ayy/wDAdP8ACj+wtK/6Bll/4Dp/hQBF/wAJPoX/AEGtN/8AAqP/ABo/4SfQv+g1pv8A4FR/41L/ AGFpX/QMsv8AwHT/AAo/sLSv+gZZf+A6f4UARf8ACT6F/wBBrTf/AAKj/wAaP+En0L/oNab/AOBU f+NS/wBhaV/0DLL/AMB0/wAKP7C0r/oGWX/gOn+FAEX/AAk+hf8AQa03/wACo/8AGj/hJ9C/6DWm /wDgVH/jUv8AYWlf9Ayy/wDAdP8ACj+wtK/6Bll/4Dp/hQBF/wAJPoX/AEGtN/8AAqP/ABo/4SfQ v+g1pv8A4FR/41L/AGFpX/QMsv8AwHT/AAo/sLSv+gZZf+A6f4UARf8ACT6F/wBBrTf/AAKj/wAa P+En0L/oNab/AOBUf+NS/wBhaV/0DLL/AMB0/wAKP7C0r/oGWX/gOn+FAEX/AAk+hf8AQa03/wAC o/8AGj/hJ9C/6DWm/wDgVH/jUv8AYWlf9Ayy/wDAdP8ACj+wtK/6Bll/4Dp/hQBF/wAJPoX/AEGt N/8AAqP/ABo/4SfQv+g1pv8A4FR/41L/AGFpX/QMsv8AwHT/AAo/sLSv+gZZf+A6f4UARf8ACT6F /wBBrTf/AAKj/wAaP+En0L/oNab/AOBUf+NS/wBhaV/0DLL/AMB0/wAKP7C0r/oGWX/gOn+FAEX/ AAk+hf8AQa03/wACo/8AGj/hJ9C/6DWm/wDgVH/jUv8AYWlf9Ayy/wDAdP8ACj+wtK/6Bll/4Dp/ hQBF/wAJPoX/AEGtN/8AAqP/ABo/4SfQv+g1pv8A4FR/41L/AGFpX/QMsv8AwHT/AAo/sLSv+gZZ f+A6f4UARf8ACT6F/wBBrTf/AAKj/wAaP+En0L/oNab/AOBUf+NS/wBhaV/0DLL/AMB0/wAKP7C0 r/oGWX/gOn+FAEX/AAk+hf8AQa03/wACo/8AGj/hJ9C/6DWm/wDgVH/jUv8AYWlf9Ayy/wDAdP8A Cj+wtK/6Bll/4Dp/hQBF/wAJPoX/AEGtN/8AAqP/ABo/4SfQv+g1pv8A4FR/41L/AGFpX/QMsv8A wHT/AAo/sLSv+gZZf+A6f4UARf8ACT6F/wBBrTf/AAKj/wAaP+En0L/oNab/AOBUf+NS/wBhaV/0 DLL/AMB0/wAKP7C0r/oGWX/gOn+FAEX/AAk+hf8AQa03/wACo/8AGj/hJ9C/6DWm/wDgVH/jUv8A YWlf9Ayy/wDAdP8ACj+wtK/6Bll/4Dp/hQBF/wAJPoX/AEGtN/8AAqP/ABo/4SfQv+g1pv8A4FR/ 41L/AGFpX/QMsv8AwHT/AAo/sLSv+gZZf+A6f4UARf8ACT6F/wBBrTf/AAKj/wAaP+En0L/oNab/ AOBUf+NS/wBhaV/0DLL/AMB0/wAKP7C0r/oGWX/gOn+FAEX/AAk+hf8AQa03/wACo/8AGj/hJ9C/ 6DWm/wDgVH/jUv8AYWlf9Ayy/wDAdP8ACj+wtK/6Bll/4Dp/hQBF/wAJPoX/AEGtN/8AAqP/ABo/ 4SfQv+g1pv8A4FR/41L/AGFpX/QMsv8AwHT/AAo/sLSv+gZZf+A6f4UAW7e4hu4EntpY5oXGVkjY MrD2I61JTIYY7eJYoY0jjUYVEUAAewFPoAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiig AooooAKKoavpSaxbR28skiRCVZHEblS4HO3IIODXHx6RaH4oTacVl+xjTBKIfPfaH343deuKFqB3 9FcVqslzocul+GtJuZEk1K5kbz3Yu8EI+ZsFs5POAT0rbufDNvJYSxW1xeW9yyYW6S4fzA2OCSTz R0uHWxtUVwHiOyaPXvB1vcvIzzO0N1tlYCXbGOuDzzzVzzJdI+IthpmmTyvZ3NtJJc2zOXWHH3XG clcnjHenYDs6KZNKIYXkYMQgJIUZJ+g71zf/AAnunHS01NbTUG08sVkuBB8sWDjLDOcZ9AcUgOno rFvfFNlZ38VkI7me4mga4hWGLd5qj+6ehNSQeIIrryo7e2uXuZIBO1uyhHiU9N+TgHrxnsaANaiu X1rxitv4XvdS0yEyzWsnkyxSfK0L5A+YH0yOnWtZtYWBLVLiCVLu5JWK3BVnbAyTwcYxznNAGlRW baa7a3N3c2kge2urVBJLFNgEIejAg4K+4NQN4ntY3s2lhuEtr2QRW9wVGx2P3eM5AOOCRigDZorN k1mP7RcQ2sE129rgTeSB8hIzt5Iycdh6iue8XatDqPhC21HS7qUI93CoaNyh5kAZWH5gg0AdnRRR QAUUUUAFFc94207+0vDssMcrwzu8aRzRuVZCzqM5B96j8Da1JqXh3y79tt9p7ta3YY8hk43H6jBo QHS0V57prTX/AMUUuLqWZoZ7A3EEBchEUPtQ7c4yRz+NdVqHiW206CW6khnks4H2T3EagrGc4ORn JAPUgHFHQOpsUVny6zbpew2cO6e5mhM6ImOUGBuyeOpFU5vF2nQaJFqreebWSXysrHko27bhh254 oA3KKyrXxBb3GstpjwXNvceV50fnJgSpnBK89j2ODWrQAUVVGo2zao2nq+blIhMyAHhScA5+opbW /trya5it5Q720nlSgA/K2AcfkRQBZooooAKKKKACiiigAooooAKKK4jXdWgtPG6WviJ5YdKmt1+x vvZIjLn5t5HfpjNAHb0Vzaxy+GNG1e9tpJb+AA3MCSTbiFCDK7j/AAjGRU1h4j/4kFhe6nC8Mt35 UcajafNdwMYweB9cUAb1FZaa/bk6gjxzLNYANPFty20jIIweQQD+VU4vGVhLHp0whu1tNQZUhuWj wgdvuqecjP0xQB0FFULjUovtb2MUUlzMqB5Ujx8ingZJI64OB14rl/Bmq22meE768u3kWBNRnRQ+ S3Mm1Vwec8gUIDt6KzY9aha+uLKSKaO6giE5jIBLISRlcE55BGKp6d4wsdU+zyW0F59mnSRvtLQ4 ij2ZyGOeDxQBu0tcP411BL3RdOuIoLhI3vrfyZ/uhlLjsDkAj1FdxR0AKKoXGpRG7ksYopLmZEDS pHj5FPTJJHJweOtc34Ev4rbw1O8hlO/Up44kbJcnecLz7D8MUAdnRWIPE8Ob+FrS5F7Yx+bJa4Xe yHoynOGHB71JB4hhutDs9Ut7eaSO7KCOMFd/zHAzzjjvzQBr0VhveWB8aRW/n3n282jYhBYQ7Awy xHQtk9RT5vFFjFBNc4la1guPs0s6gbUfcF6ZzgEgZxQBs0Vz0uv3a+NBo8dizwLaicyK65OW25wT 0HPvT9I1DTX1XWzb3V0zwujXP2gkRxHb/AD0GBk0eYG9RWLJ4otILSG9minj0+ZlCXTKNnzHCkjO QD6kdxT7nxHbW2r/ANmeTcyXZhM6KkeQ65xwelAGvRWfo2s2+uWTXFssqbJGikjlXa8bqcFSKvsw VSzEAAZJPagBvmJ5nl7134ztzz+VPrjrCfS9Q8USWl7bRTTlzeafd7MGVO4DdTtP4EYrsaACiiig AooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK4aK8t/wDhcc375MHTBEDn jfvztz647V3NJQtHcOljk/GOm3aato/iCwge5fTZGE8MYy7xMMEqO5HXFbEPiXTLmLdb3Blf/nkq N5mfQrjIP1rVoo8gOB8bzWtx4o8JxXe1V+0SNNGzfcBQY3EdOa7Ox0yx08MbK2hi3/eZF5b6nqat UtHQGI33T9K888P3tvJ8M59PUiW7mFzFHbgfM7M7gYHpyOegr0GbzPJfylVpMfKGOAT7msbwjpF3 oWiJYXpgdo3dhJExIO5i3QjjGaLXuh3tZmBbxJpnjjw5YzTKZLfSnhc5/i+Xj8cHFaFtv0bx7qk1 7lLPUoYmhnb7gZBtKE9jzkZ611VLTv1Jseb69plxJonjDUIoZDDeSwtCgU5dY9oZwPQ4P4Cta/nz 4n0PX4N8umeTJbSyKpxHuwVcj0yME12VFJaf18hnEa9pNx4g1u+udMPyLpEtqsoPyySOchQe+AOT 71f8O+J7bUNPtLP7NcLqMKLHNavCymJgMEkkYA9DmuoooX9f18wZyXh120LVtcs9TPlCe8a7t5X+ 7KjgcA+oIxjrXPapZvpngedrgGH7brK3UULcMqGUEcduBk+ma9OooWlv6/rYBFZXUMpDKRkEdDS0 UUAFFFFAGJ4rvbey0qM3MyR7rmHG49cSKT+lYF/pF5D43L6YN2na9Btu3U8IU6sPdl4/Gu5paEBw 5urVPi9EiSxhV0wwgA8Bt+dv1x2qho39j2q3ejeIrCRtQW4l2q0TuLtGYsrLjg8HFejUtAHJ3mqi z8RWGlzxvYWjWe5GiQlncEDygwHAA54rlZJ44vhs9s+6OaLVQGjdSGX9/nv7c16tRQv6++4eX9bH I31xE3xP0gCRTmwm6H1YY/PFddRTW3FDtIDY4J9aOgdTktA1O1l8ReI72R/3n2hbaJcfM6Rrj5R3 +Yt0qTwbO8WjSySQSNczyy3VwoHzI7OQEI9cAfTFbHh3SW0TRILOSQSypuaSQDG52YsT+ZrRddyM uSMjGR1FGwGbod3BqC3dzDAYm+0NHI27cJGT5cg+nGPwrUrP0fSE0ezS3jleRI1CLu4wBk/mSSSe 9aFABRRRQAUUUUAFFFFABXPXuqaXe3d/o+uQxCKPaQZ0/dyKVzwx4yOfeuhpKAPPtB0y6t/Dfiiz sxPJpREi6asmdxBQ7guedueBVhfES2ngvQXt9whLQW9zcGEt9m+Xk4I65GM9s13VFH/A/AP+D+Jw NpcW8HiHxR88wSewieJ5g37wBHBOT7kVQluIl+G3hMNIoK3drkemDzn6d69Nop3/AE/ADj7C/XRP GmtR6oWjj1BoprScqSkgCbSgI7jHT3rI03VZ9I8GalPHbS7v7VkLmSBj5Uby/wCs2kc4HNej0UgO Gsru1i8fPcCeZoJ9KAS4mDYkIck4JGAAPoKr6Dby6r8Hp7KwYNdPBOgUHB3FmOPxH869Boo6WDrc 8813WrfV/BumpbJN9oiurbzrcQtvhKsNwYY4xivQVYOoZehGRkYp1FO4rHH6ffLofjPW4dU3RLfv HNaTMpKyAJtKA+ox0965m3SSPRLe9msrma1s9auJbqEROG8pywDgd8ZzxXq1FIq5zugHQ7u7a80S 0Vt0eyS78tlyM8JluT/SqfhrTLmw1e80ySJhp9hO1xaOehEoyFH+7l/zFddRQI5K8niHxU05fMXI 06VSM9CXXA+vFYGt6sdX8M6wLhZ47u3u8GzjiYLEiyr87YHzZAzk/hXplFH9fjcDjjeRRfEa2u3L i3u9LEcD7Gw7CTOOnXBzVFYDrGp+ObG0mUT3McaRkHqfLI/nxXf0UW0sNOzv6fgcTeTnWPhu2lrC w1KW2W1Noy4dJBheR2AxnPTFSRbLX4kafbvMGeLRzExJ6sHH68ZrsqKd9b/1/WpNtLf1/WhyfgSa OX+3vLcNnVZm49DjB+laHi+Ce50FoYQ5jeaMXAQZYw7xvx+HX2zW5RS7fL8B9zm54I9V8VaTPZhW ttNjld5U+7ll2qgP5kjtgVds9QhvdemiNq6XFvAreYWzhXJwpHZvlzj0rXrOs9HSzv7m6WaRjcSm ZlP97aF69wAOB2oA0aKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAMvVtft 9Hljjntr+YyAsDbWjzAY9SoOKof8JvY/9A/Wv/BZN/8AE10dFAHOf8JvY/8AQP1r/wAFk3/xNH/C b2P/AED9a/8ABZN/8TXR0UAc5/wm9j/0D9a/8Fk3/wATR/wm9j/0D9a/8Fk3/wATXR0UAc5/wm9j /wBA/Wv/AAWTf/E0f8JvY/8AQP1r/wAFk3/xNdHRQBzn/Cb2P/QP1r/wWTf/ABNH/Cb2P/QP1r/w WTf/ABNdHRQBzn/Cb2P/AED9a/8ABZN/8TR/wm9j/wBA/Wv/AAWTf/E10dFAHOf8JvY/9A/Wv/BZ N/8AE0f8JvY/9A/Wv/BZN/8AE10dFAHOf8JvY/8AQP1r/wAFk3/xNH/Cb2P/AED9a/8ABZN/8TXR 0UAc5/wm9j/0D9a/8Fk3/wATR/wm9j/0D9a/8Fk3/wATXR0UAc5/wm9j/wBA/Wv/AAWTf/E0f8Jv Y/8AQP1r/wAFk3/xNdHRQBzn/Cb2P/QP1r/wWTf/ABNH/Cb2P/QP1r/wWTf/ABNdHRQBzn/Cb2P/ AED9a/8ABZN/8TR/wm9j/wBA/Wv/AAWTf/E10dFAHOf8JvY/9A/Wv/BZN/8AE0f8JvY/9A/Wv/BZ N/8AE10dFAHOf8JvY/8AQP1r/wAFk3/xNH/Cb2P/AED9a/8ABZN/8TXR0UAc5/wm9j/0D9a/8Fk3 /wATR/wm9j/0D9a/8Fk3/wATXR0UAc5/wm9j/wBA/Wv/AAWTf/E0f8JvY/8AQP1r/wAFk3/xNdHR QBzn/Cb2P/QP1r/wWTf/ABNH/Cb2P/QP1r/wWTf/ABNdHRQBzn/Cb2P/AED9a/8ABZN/8TR/wm9j /wBA/Wv/AAWTf/E10dFAHOf8JvY/9A/Wv/BZN/8AE0f8JvY/9A/Wv/BZN/8AE10dFAHOf8JvY/8A QP1r/wAFk3/xNH/Cb2P/AED9a/8ABZN/8TXR0UAc5/wm9j/0D9a/8Fk3/wATR/wm9j/0D9a/8Fk3 /wATXR0UAc5/wm9j/wBA/Wv/AAWTf/E0f8JvY/8AQP1r/wAFk3/xNdHRQBzn/Cb2P/QP1r/wWTf/ ABNH/Cb2P/QP1r/wWTf/ABNdHRQBzn/Cb2P/AED9a/8ABZN/8TR/wm9j/wBA/Wv/AAWTf/E10dFA HOf8JvY/9A/Wv/BZN/8AE0f8JvY/9A/Wv/BZN/8AE10dFAHOf8JvY/8AQP1r/wAFk3/xNH/Cb2P/ AED9a/8ABZN/8TXR0UAc5/wm9j/0D9a/8Fk3/wATR/wm9j/0D9a/8Fk3/wATXR0UAc5/wm9j/wBA /Wv/AAWTf/E0f8JvY/8AQP1r/wAFk3/xNdHRQBzn/Cb2P/QP1r/wWTf/ABNH/Cb2P/QP1r/wWTf/ ABNdHRQBzn/Cb2P/AED9a/8ABZN/8TR/wm9j/wBA/Wv/AAWTf/E10dFAHOf8JvY/9A/Wv/BZN/8A E0f8JvY/9A/Wv/BZN/8AE10dFAHOf8JvY/8AQP1r/wAFk3/xNH/Cb2P/AED9a/8ABZN/8TXR0UAc 5/wm9j/0D9a/8Fk3/wATR/wm9j/0D9a/8Fk3/wATXR0UAc5/wm9j/wBA/Wv/AAWTf/E0f8JvY/8A QP1r/wAFk3/xNdHRQBBZXaX9nFcxpKiSruCyxlHH1U8ip6KKACiiigAooooAKKKKACiiigAooooA KKKKACiiigAooooAKKKKAGu6ou52Cj1JxTUuIpG2pKjN6BgTVPXYY59Bv0mjWRPs7/KwyPumvOdO l0NvhfaW8awyay1r+5jtRm583naRt+Yc45NK+47HqjyJGAXdVB9Tio/tlvnH2iLP++K43xXb3LfC K4Grqsl/HZKZCQCRJxk59adBc+E28PWcE8enSyTRRQmNI1LlmAX065NVbVoS2TOz81N+zeu7rtzz SvIkYy7qo9WOK4vxj4ak1C60r+yZRbanaRySW8+BligXCsfQ0Jrdr4v8FXyX1qiX1sPLu7WReYpA eo9j2NINjtVYOoZSCD0INNSWOQkI6sV6gHOK5HXrl7nxXo/hi3Jt7OWJ7m5EXybkXonHQE9cVe8R 6Fbx6Dcz6ZGtle2sTTQTQAIwZRnBx1Bxgg0Xsrgld2OjorK8Maude8NWGpOoV7iEM4HQN0P6g1q0 2rOwk7hRRRSGFFFFABRRRQAUUUUAFFFFABXK6nqmp2UKkSxnVJLkJDYKysske/GemR8uTnPGK6mu a1bTdX1Xw9Jpc3ltdyMP9NBCKmGyGVQc5A/WjqB01FV5JlsrVTKzSEYQYHzO3Qfiaz/+Ensl0a81 KQSJFZyNFMuMsHU4IGODyR0oA2KKoNq0Savb6e0U4kuIWmR9nyYXGVJ9eelX6ACiiigAooooAKKK KACiiigAooooAKKKKACiiigArF8V317pmg3N5YyRI0Sg5dNx5YDjt3rarF8V2d1qWg3FjZweZJOA AxcKFwwPOfpQCNlTlQfalqK3d3hUyRNE391iCf0qWgEFFFZlrr9nd6NPqi71tITJlmHUISCR7cHF AGnRUNpcpeWcNzGGVJkWRQ4wQCMjIqRHWRA8bBlPQqcg0AOooooAKKKKACiiigAooooAKKKKACii igAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAoa5I I9DvTtdi0DqFjQsxJUgAAc1w+meHrq78C6PdafHLZ+INJj/d+bGYyxyd0bZAyrCvR6KBnEeI9Vl1 /wCGl+F069hvp4fLNo1u+8SZGQOOR71owazp8fh22iura9laKCPdCtlKW3KBwPl65FdNRRcRzn9p PLrOjPc2txHI8EhkCwuViLBSqs2MA8fpWd4x8LXMl4utaEdl6wEN5Eo4uYcjPH94dq7SigOljmPE Oj3S+ItL8Q6fEZ5bMNDPApw0kTddue4POO9Sa7qNxqmjXFjo9rcPeXSGHdLE0aQ7hgsxYDoCema6 OijpYL63M/QtJj0LQ7PTYmLLbRBNx/iPc/nWhRRQ3cFoFFFFABRRRQAUUUUAFFFFABRRRQBFcW8d 1bSwTAmOVSjBWKnBGDyOR+Fcdqnh3w7pdzZW7W19JJdzrCAt/P8AJn+I/P04/Gu1OcHHXtmuL8RP c2r6ULhLYztqMU0jCc9Bnr8vCjpmjqg6M6i306LTtLFpYocRK3lCV2chuSMsxJ6muPSxnHg7StMa 0uTLHcxS6ivlnIAfdIf9rLemeK7tG3xq3HIB4OR+dOo2DoZymS7umuoozsiiKQiQFd7HknB5A4A/ OsZtf1BdG1e/ZoBHYSyRwsE/15XjBGePm+Xiuqqj/Y1h/Z4sfs6/ZQ/meXk4Lbt2T/wLmgCgdVvI NW062uPJzdRPJPGP+WCqBzu7/MQOaWTWZZ9Q1KG3ZIYNNQGaZ13bmK7toHoFxk+9aU2m2txeJdSw hplXYGyeVznB9RkZ5pk+kWVy8zTQBvPXbMuSFkGMfMOh49aGCK+lX95qGg2dzIkMV1cIrOoJxGG5 6dc4xxVXTNR1K/lldZbXyIL1rcllIMqKOSOfvbsj04rQOjWJto4DCdkTiRPnbKsOAc5zxT7XSbOy I+zw7AGLBdxKqx6kDoCcnmnpcOhcooopAFFFFABRRRQAUUUUAFZur6BYa35Rv1mbyc7fLuJIsZxn O0jPTvWlVTU7I6hZNbieSHLKxaPGSAQcc9jjB+tAGKngXQJBlEumHqNQnP8A7PTv+EC0L/nlef8A gfP/APF1Fp2I/G04dfsBezAjsxjE+G5k44yM4x19a1NM1xNQ1C+sZLeS2ubQqTHIRl0YfK4x26j2 xQBh6n4T0LT4Y9lreTXE0gjhi/tGdd7H33cADJJ9BUR8G2wcIdNUMeg/ty5yf0rb0z/ia6lJqrcw R5hsx6rn55P+BEYHsPesrxRp7Wujag371/PkMrXzbc2S5HK4+YhcZGKALswXwv4MvnSHyTDHI6oL h5/mPT5n55JHFY81vOfhxa6RaWlyZWiiiugYiDGpIMp9/wCLpmul1PS49f0eC3N0TAzxSs6gHzVU hsfQ4FatGwX2Me81OTTtEluzCiQxKzDfkBUUcZHXJwAB71e007tNtm+zrbFo1YwrjEZIyV49KNR0 +DVLNra5DGNmVvlODlSCP1AqaCBbePYmTySSTkknqTQBJRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA BRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAF FFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUU UUAFFFFABRRRQAUUUUAFVNR06LUoY45iwMUiyoVOMMvIPuPardFAGZDoif2ump3czXF3FEYYiVCr GpOTgDucDmqev+HZdUv7S8srr7JOgMFw4HMkDfeX656HtzW/RQAyGGO3hSGFQkcahVUdAB0FZEvh uN7G5sI7mWOyumdpYxyx3nLAMeRnNbVFAbEdvbx2ttFBAgSKJQiKOwAwBUlFFABRRRQAUUUUAFFF FABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUU AFFFFABRRRQAUUUUAQXl0tlayTvHI6xjcVjXc2PpWLb+NdNuNMXUhFepp7Dd9pe3bYBnGT3A98Vs 3/8AyD7n/rk38jXnvh671SP4UW0Vnpgnje3kQyiUEqpLZbZ1bHPGeaXcOqO51LXLTTNIOpymSWzV PMMkC7wF/vcdqfa6vbXWkjUiWgtSnmb5xswuM5Oegrk7kWSfBidNMuTc2qaeypKwwWwOcjsc54qn 4hczaD4K0+Un7HeTwLOOzAKCFPsT/Kqa1a9PxDon6/gdeviWya3Nysd2bQLu+0fZ32Y9emce+KvW F/b6pYQ3lnJ5lvMu+N8Y3D1qwAAMADHpUVpaQ2NslvbpsiTO1R25zSAmooooAKKKKACiiigAoooo AKKKKACiiigAooooAgu7uCxt2nuZBHGuBk9yeAAO5J7VFBqkE9yLZlkhnZS6xyrtLKOpHrWH4vkc at4Zh/5ZyakC49dqMR+tJ4skaHXvC7x5DHUDGSP7pjbIoQM6mqEmsW0atIRKYF+9OEJjHrz6e/Sr rlQjFyAoHJJ6CvO/7Q1rRdJkiMUl/wCG5MrHexx/v4YT1JT+IAdG9OcUAdpq/iDTdBt4bjU7lYIJ n2LIQSucZ5I6dOtO0vXLDWhI2mzfaI4zgyqp2E+gJ4P4VVlt47230ptPW3ntVXK+acqUKYBx37Ul vrDW+qzaPNbxi5jtvtEAh4WVM4wAehB/nT6h0Rt0Vz1jrs/2rWlujE9vpqqTKowN+ws6e4Axz70Q apq1xouiXoit0lu5IzcxsDwj5Py89Rx+Rpf194HQ0UUUAFFFFABRRRQAUUUUAFFFFABVO51KG3uB b4kluCu/yol3MF6ZPoKuVy/hyRpvGPilpOqSwRr/ALojyP1JoA6G0vIL6Dzbd96hip4wVYcEEdiK Lu8gsbczXMgRAQvqSTwAB3J9K53wxI3/AAlfimHny1uonA7ZaMZ/kKXxXI3/AAkHheH/AJZvfMzD sSsbEUdg7m5b6nBPc/ZiJIZyu8RyrtLL0yPWo7vWraz8wsk8iRf614oy6x8Z5x/TNY3iiR4fFHhZ o8hmu5IzjupjOR+g/KtBYoPDGnX081zcXCSzvMsch3Hcx4jQD1PQe9HQDUtrmG8to7i2lWWGVQyO hyGB7ioU1K1l1OXT0lDXUUYlkQfwqTgZ/KsTwtZXfh/wla2lwg+2yM7JCDkRl2LBfooPP0qPSrYW vj7UEyWY2ELO56uxdsk0+thdDbudbsLS7htZbhfPmlEKooJO4jODjpx61frnPFn/AB/eHv8AsJp/ 6C1aWv6n/Y2g3t+FDNBEWRT3b+Efnil0uPrY0aKwtU1e803SdNOyJ7+8mhgKkHbub75x7ANWxcTf Z4Gk8uSXb/BGMsfoKAJaKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiii gAooooAKKKKACiiigAooooAKKKKACiiigAooooArahHPLYTR2yxmV0Kr5jFV5GOSAa5rQ9E8QaL4 Ug0aI6b5kUbILkyO2MknOzaM9fWuuooA5N/B8tl8P5PDmmSxu8kTRtNOSo3MclsAH16VYn8LHVfC drpWpssVxarH5U9sxJjdBw4yB+VdJRQBkWqa+kQiuZNOkYDH2hQ4Le5Tpn/gVacEbRQqkkjSuBy7 AAk/hUlFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQBm63pA1aCDZIIri1nW4gkIyA69iPQg kH61FJpU2oatZXuoeUq2O5oooyWy7DG4kgdBnA9616KAM37JfXF3qEd7NC+nTxhII0UiRMgh9x75 7VFbWep2tglgHtZI44/LW4bIbaBgZTGCce9a9FAGL/wj0Vo+nT2KI02nwtBEJWIBUgZ5HQ8dcGnW ejSf27LrN80ZumhFvFHHkrEmcnk8kk98CtiigDmZPDFydN1fTYriIQalLLKZmBLpv6rt6H656Vq2 lncf6N9r8lVtlwiREkFsY3cgY46D361o0UAczDbNJ4n1J5nnNlbW6h0JO2WRiXJx32jAGKzFjuk8 JadNJJcLd3dyhWVic2sbPux7AKMc+uK7mihAc9f3MtxrljFc+ZBpjW7SsG482TICo30BJx3/AAqD w/eGPS9TmuGu1HnTShSrM0EYJCqAec4XOPeuoooA44Jb32s3il7pLL7JGsQjdg0sjEkuvqwGBntm uvUbVA54Hc5NOooAKKKKACiiigArI/sqaz1y51KxMbfa0RZ4pCVBZchWBAPY4Ix6Vr0UAY1no1xp 9pqUtvNF/ad87TGVlOxXxhRjrtAApL3RrnUdO083M8X9p2UiTrMiHYZAMNx12nJFbVFAGQ2lTX2s 2d/f+Uosg/kxRksN7DBYkgduAMdzWbe2XiuTWJLq2/sRoUOLdZzKWjX14GNx9fwrqaKAMi10drvT 4Rr8dtc3iszsY87FJPRc84xisux8KPaeKJNTNrYrC6IiRrI5aMqSdwO3BJz04rq6KOtw6WMLXNM1 DU7zT3gFqsdldLcDfI2XwCMYC8dfeqPjP7TfWel6UnlR3N9dpuGSyBY/nbsDj5R+ddXUElnby3cN 1JErTwhhG5HKhuuPrihAZGo6RqGoalp16z2yGyZ2WDJKlmUru3Y6jPAx+NQ67NcWc+l20V5KryyL ucngRoN0jH+8SMAD3rpKa8aSY3orbTkZGcGgARg6KwzgjPIwadRRQAUUUUAFFFFABRRRQAUUUUAF FFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUU UUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRR QAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA BRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQBBe3sGn WU13dyCOCFC7uewFZOm6nqus2aX1va29rbSjdClwWMjr2JA4XPpzVP4lW89z4C1NLcFmVFdlUZJU MCw/IVt6Lcw3miWNxbMrQyQIyFemMChdQfQwNT8T6tYeGdW1GSwt4LnTpCphdy6yKApDBhjrn0rf 02e6u9HguJjCLiaJZPkU7BkZ7nNZHxCXzfAWtrGNzi3JIHXsf5VraGwfQNPZSCpto8Ef7op9H8gf T5mWurauvhe91GVbH7Rb+ayoqvsIQkcnOcnFU4fEusjwhF4hktrKaAwC4kt4yyOE6nDEkEgewqzN z4C1RhyGhumB9QS9c/aaPeXvwssXj1FzCliJHtJFAjlULkozDDAHHY1PRv0/UaW3zOi8ReJZ9P8A Bx1/S0hlQRJMI5wRuVsY6Hg81Pf+Iv7H0W0ub5BPeXRSOG3txgyyN0UZP5k1zviPU4tY+DEt9BAL eOW2QrEOiYYDA9uKXxOTba74KvZ+LOOUxux6I7IApNVbVrzJvp8jpZrjX4bJ7gWljNKqbhapIwJP pvIwT+FRSazqJg0a4t7CJ4rwr9qDS7XhyB90fxYOc/St6mKI2CsoUgfdI7fSkMfRRRQAUUUUAFFF FABRRRQAUUUUAZ+saqulQQkJ5k9zMsEEecbnb1PYAZJ+lQtqs1jqlnZagIj9t3LDLFkDeoztIPqM 4PtWZ4wRv7X8MS/8s01IBj6EowH60eLVaTXvCyR8uNQL4H90Rtk0L9Qf6HTSyJDE8kh2oilmJ7Ad awU8RXU3h867Fbxmx2GYREnzWiH8WegOOcfrW5cQJc20sEn3JUKN9CMGuUuLa70T4f3enHyjDa2c kaXYYEFApx8vXdjj0zSbsmxpXaRqaxrl1a2VjcaPYf2kbpsiJZAhKbS2QTxnpxVvSLvUb2N5dQ09 bFTjy4jKJH9y2OB9OaxNESKLTPDWl3ktxHeraeYFjU8gIAwZu33h6VPHeXVv4wn0WKZ3glsftMbS EuYX3bep5IPXn0qmrOyFe6TOlorjbG/azi8Uaokkn2S0JhiQsTueNDvf6ljj8KtQabOdD8OxXF1c G8ikjldhIcudpLhvUcn9KX/A/EDqKKpHV7EXDQfaFMyjJjAJYjOOB359KaNc04xCUXSGMttLgHap zjDH+E545xQBfoqG4u4bRVM8gXedqjqWPoAOSfpTba/tbxJWt50cRMUkwfuMOoPoaALFFZ0uv6bB v866VNieY25SMJ/e6fd9+laCsGUMpyCMgigBaKKKACiiigArKbVJbrWLjTrAR77VEaeSUEhS2dqg DGTgZ/KtWuW8Nq0XjLxSsn3mmgkUH+6Y8D+RoA2NJ1Yal9qidBHc2cxhnQHIBwCCD6EEGl1fVV0x LdVTzLi6mWCBCcAse5PoACT9KxvDCsfFniqX/lmbqJAf9oRjP8xR4rVh4h8LS/8ALNb5lY9gTGwF Hb5B3+Zqf2rNZ6ta2GoCLN4G8iWLIBZRkqQfbkH2NIuqy31/dW2nLGVtCEmmkBI34zsUDqQCMnPe szxSjS+KPCqx/eF3I5x/dEZyf1qp4JsJZrLV1uZ50kGp3AxG5Qqd3U469uvHSj+vyD+vzOp0u8lv 9OhuZ7aW1kcHdFKMMuDjkfhn8at1xH/CSXlz8Nru8kbF7uks45FGN77/AC1YfmDV/XvOtLHQtHt5 5FluLmKJnDHcY4xuc5+i/rTA6iiql1dqtuvkXNukkuPKaU5Vv1Gat0gCiiigAooooAKKKKACiiig AooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAC iiigBCAwIYAg8EHvWPb+F7Oxdv7Plu7OJmLNBBMRHk9cKc4/DFbNFAFeKwtobZ4FiBjkzvDfNvz1 3E9fxrNtvC1pZw/Z7a4vYrMAgWy3B8tR6DuB7A1tUUAUb7SoL3Sn07dJBbvH5ZEJCnbjGOnpWZH4 Msk0uPTGvNQk09EEYt2nwpX+6SACR+NdDRQBkar4asdW0UaTJ5sFiFCeVbsEG0dB06cVYfRrSfSP 7NvEN5bbQpE+GJA6c8c+9X6KAMm38PQW8flfa7+S3AwIZLhioHpnrj6mtREWJFSNQqKMBQMACnUU AFFFFABRRRQAUUUUAFFFFABRRRQBXvbG31G2a3uo98ZIPXBBByCD2IPeooNKhiu1upHlnuEUokkz ZKKeoHYZx161dooAo2+kW1tqd5foZTNeBVlDSErhRgYHQVWg8M2NvAlsDM1nG25LVnzGpznpjJ55 wSa16KAK91ZpdBSXkikTO2SNsMuev/6jUVlpdvYyyzJvkuJseZNK252A6DPYD0HFXaKAMqXw5YSr dRssn2e7YvPbh8Rux6kjrz7GrVpp0doVPmSysi7EaVslV9B/nNW6KAMaDSpotfv9RMcXMCQ2qjsB lmz6ZY/pWcPDl1H4WsNO2xO4uEnvVDY807t7AH/ex+ArqqKAMSbTbw69Z6k4S48m2eHyw20I7EEu M+wx61FpGnajp+nX6SRW0lxNJLN8zEpK7MSM8ZC42iugooA5+HT9QbWbm+nt4Mz20cEalsrEASWy O4JPb0rfHApaKACiiigAooooAKpXGlwz3gu1aSG52eWZYmwWXOcHsau0UAUBo1oumT2CIyQ3AbzW VyHYt95i3XJ9aDotm2kw6c6M8ECqIyzkupX7rbuuRjrV+igClBpcMN2Lp2knuFQoskrZKqeoHYZq ObRIJLiWeKWe2efHneQ+0S4GMn3x3GDWjRQByniHR47qTQtDtrZo7ATmaXysgIsakjn1LEVrTeH7 W5nguJ5Lh7mDISYyfMARgr6YI9q1aKAMDXtLluJ9OS3tt9skoecKQCQgzGvP8O7k/St1N2xd+N2O cdM06igAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo AKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigA ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACi iigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKK KACiiigAooooAKKKKACiiigAooooAKKKKAMzxHrUfh7QLzU5E3iBMqmfvN0A/Miqej6ZeXemw3er 39y95OgkZYZTHHFkZ2qo649Tml8a6LL4g8JX1hbY+0OgaIE8FlIIH44pvh/xLZXmlQLczJaXsSBJ 7ac7HjcDB4Pb3FC6g+hkeJbfVNI8Ea7LNql3JPAxltLlZNjquFwDtwDjkdK6bRYt2gWYeSZzJAjO 7ysWJKjJ3ZzWd4pifxF4L1a209XkeWFliO3AkI5+X16Yp2geItMfw5Zu15FG0UCLJE7YkRgACCvX OaOj+Q30+ZT+zvD4H1OQXl80yLcOkzXLl1KltuDntgVkRzyQ/DS31hNYvE1T7IsoZrhpPNk67SjE g5PGAK3b3Fr4B1Frn9yZLed9shwRu3EA+/Nc1ZxeHj8MLG4uHtUvYrINFJEwE6y7eNuPm3ZpdH8v 1BdPn+hs+Mby+Hw2fUjJcWWoJBHIfJkZCjkrkcH3PWpNY1i50qx0TS9NkJ1DVGWNJp2MnlrtBdzn qcdBWV4kurxvguz624S/ktk8wSEBidw6j1xjNWfEltLt8NeIbGNrqLTWBnSH5mMTKAWAHXFV1fqS tl6HQzaBIbF0t9V1CO7KYW4M247vXafl/DFQ2Wnaoupafcz6hckR2nl3UBwYnfH3h33Z5+lXYvEO lXFsJ4b+CRCOAr5b6beufbFX4pPNiV9rJuGdrDBH1FIZm6BY3lhazx3l7cXe6Zmje4xvCntx2znH tWrRRQAUUUUAFFFFABRRRQAUUUUAYfiXU5bI6bZ27mOXULtbfzB1RcFmI98DA+tRahdvoWuaRFHL K9tfytbyJI5fDbSysCeR0IPbmo/GFs3naLqIUmKwvlkmwM7UIKlvoMgmma7GuseI/D8Voyyi1uGu 5mQ5CKEIGT7kjH40L9Qf6HRXlx9lsp7jaW8qNn2jvgZrl7W5vLzwINd+1yrfvbG7XDny143BNvTb jj1966GLVbK6v7vT45Q9xaqpnj2n5QwyPY5FcrdfYX8G3FvpVxM9vPG8dtYEYfcxIC4+8Fz2Pb2p O9nYa3VzWi1WfWP7GMVvciyv7czyzQnAQ7QQrN1AOT054qOxvLmx8bTaKZ5Li0ksxdR+Y25oTu2l dx5IPUZo/tKx8EeGdNsr+6gjmWFYYxLIFDMByc+g9ah8P6lpNxe3b6ZfQapqs8ZluJY24AHCoP7q 5OAPqap2u7eZK21ILG1v5brNz4tkW3mlLC1CIJB8x+QSdcfrXUXmo2unR77uYRoBkk5OAOpPoPc1 zep3kU/gG5iuIraO/nt2H2S2OSJj0Cjrndjn1pviC1uP+EW023vf9ddPbWt9Nn7qZy+T2BPGfeku w/M6l723juIIHmQSzgmJM8uAMnH0ptlqFrqKytZzpKIZGikK/wALr1Fc8txbTePvNeWNYLLT9sJJ +UszfNg9DgKo49auNdjRdKvNSW2yJXkunH3cKB1PHUhR+Jo8w8jeorFfX5Un06NrFwdQB8pN/wA6 YXcd46Afias2GrC7lv4pUEbWMgSRw2UPyhuDgdAeaANGisE+J91nZ3sVoz2t7OkEB34dtxwH24+7 gE9c4q9q2pvplpcXPkb4reIyyMzbQQOw4OTxQBoUVR0+8ubtmM9k0EZRHRy4O7cMlcdiO/bmr1AB RRRQAVz9rePrPiXVLRpZEtdOEcYSNyhd2G4kkc8DAA+tdBXL6VGNJ8Za6LlhGl95VxC7nAfC7WAP qCBx70dQ6Fvw9qctxfatp1w5kfT7gIsjdWRlDLn3GSPwpfEepy2s+l2Fu5jk1G6EJkHVUClmx74G PxqhoJSzvfEuu3B8uynnDJIQfmjjTBYe2c/lSeIWjvn8Oa7bMZLO3uhK7gHiORCu7HoMjNHb5B3+ Zbvbx9E8Q6TbpLK9tqLPCySOX2uF3KwJ57EEfSotBuJvEkOoXc888IS7lt4Eicp5aodueOpJ55zT dajXV/FWgJasJVs5ZLqZ0OQi7Cq5PqSf0NPim0y3m1SS1vZbRBO32mErjfJgZZARnnjkdTR0An8M a9/afhoX166K8DSR3Djhcxkgt+QzWo+p2cdhHevcIttIFKSMcA7sbfzyK4KW0fw58K3tLkG3m1G4 2FD1jE0nQ+4XNbPiKe1lu9B0yORDapciaVgflCxKWVSemSQOPan/AF/mHp5/8A66isfVNWgjtIUu obgJcNGhCHaylzhRkHJOeuO2a2KQBRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA BRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAVFJbwyuryQxu69GZQSKloo AKi+zQmbzfJj83+/tG786looAZLDHOm2WNJF9GUEVFHYWkUgeO1gRx0ZYwD+dWKKAI5beGcATRRy AdA6g4pY40hQJGioo6KowKfRQBEtvCkplWGNZD1cKMn8aloooAKKKKACiiigAooooAKKKKACiiig BKbHDHCCIo0QHkhRin0UANCKGLBQGbqQOTTVt4UlaVIo1kb7zhQCfqakooAimtoLggzQxyEdN6A4 /OiK1gtyTDDHGT1KIBn8qlooAj8iLzvO8pPNxjftG7H1p7KGUqwBB4IPelooAj+zw7VXyo9qcqNo wv0qvqmmx6tp8lnOzrFIV3bDgkAg4+hxirlFAFJ9Mik1eHUGZzJDC0SJ/CNxBJ+vAFQ2mhW9qLpd 8kkdw8jsrHu/3vr6DPQVp0UAYn/CMx+TYxC7nCWB/wBHAx8o2leeOTg9al1DQE1KOWGW7uhbyxLG 0If5flOQR7+vritaigCC3tzDuZ5WldsAk8DA6YA4FT0UUAFFFFABTJIo5l2yorr1wwyKfRQAhUFd pA24xjHGKAoC7QAFAxjHFLRQAyOKOFSsUaoDzhRikNvC0wlaKMyjgOVG4fjUlFAGRq2kzalrGkzF k+yWcjzSIerNt2px7ZJrTFvCECCKMIDkLtGM+uKkooAzdU0k6jdWU4nMZtHZ1UrkEldu76jJx9a0 EXairknAxk9TTqKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK KKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooqnNq9hbT+TNe26SjqjSDI+o7UAXKKoT6 5pltEss+oWqRu+xWMq4LYzge+Kr/APCV6Flh/a1llfvDzl+X6+lAGvRUKXlvJaLdJPE1sy7xKGGw r656Yqumt6ZJnZf2xx/01FAF6is668QaVZXX2a61C2inxny3kAbH0q1a3ttfR+ZaXEU6A4LRuGAP 4UAT0UUUAFFFFABRRRQAUUUUAFFFFABRRRQAUU13WNC7sFVRksTgAVFb3ttd7vs88cpXqFbOKAJ6 KKqLqlg67kvbYj1Eq/40AW6Kha7t0g85poxEej7hg/jRb3cF2he2mjlUHBKNnB96AJqKKKACiiig AooooAKKKKACiiigAooooAKKKguLy3tApuJo4t3Tc2M0AT0U2ORJY1eN1dGGQynINOoAKKjhuIbl WaCVJFVipKMCAR1H1qOfULS1kCT3MUbnszAGgCxRSAggEcg0tABRRRQAUUUUAFFFFABRRRQAUUUU AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQA UUUUAc9471efQ/B1/eWjbbgKEjb+6zEDP4Zq54d0i20rQ7a3hQEtGGlkPLSsRksx7kmpde0eHX9E u9NuCVjuIyu4dVPY/gazdFuNZ02xhsNT057iWBRGt1bOpSUDgEgkFTj60LqD6Fyx8Nadp1y8tvAi AzPOiBQFR2UBiB74/U1heGVVvHvi0MoILwZBH+xXV2r3Lxs91GkbE5WNDkqPc9zXM+HLS/t/GWv3 dzp9xDbXzRmGRihB2rg5AbIoW/yB7FK4t4rzx/YeHSv/ABK9Oszd/Zz92Ry2FyO4XsK6vWNEs9b0 5rO7hRk4KHaMxsOQR6YNZOu6Fef8JDZeINHEb3dvGYZreRtoniPYN2YHkZ4rTj1C/uUCx6ZLbyE4 Zrh12r7/ACkk/pR0B7nM6/dQWPxT0Ka6lWONbGcF26VPpEDXfxEvNU06F4tMNosUshQotxLnIIB6 4H8VP1Ozv5PiPpWox2E8lnbW0kMkylMbm6YBbOK6+n0Xz/UH29AooopAFFFFABRRRQAUUUUAFFFF ABRRRQBzPi+4b7ZoFif9Td6gqyr2ZVUtg+2QKTxNO1j4i8Nzw/K8121s5H8SMhOD+IBq94j0mXUU sbi1Cm5sLpbiNWOA4GQy57ZBNQ3dhPrWtaXcSQPb22nu0/7wjc8hXaoABPAyTn6UL9Qf6G6wLKQD gkYyO1cje+DPDcb29kmkWT3lyD+9eIFgB95z78/mRXR21zeyapdwz2QitIwnkTiQEykj5vl7YrPS xvW8dy30kY+xLYCGJ9w++Xywx9AKOoEr6Fa2o01rVpreDTN3l28A+VwVxgjHPrxWVpDiTxbrerFj DEII4TasMSnbk+Yyds9B6itm/XUY9Wtbm3lZ7CONxPbIil5GONpBOOlZ1ppl5d+NTrkkTWlsln9m WJyN8pLbssBkADtzQtw6F2y14T3uo288HlGxRJJGDbgAylsH0YAcj3FRxeI5J9K0m/i0+VotQkRW G8ZhVs4Y+vb86y5NG1KPS/ENjHDuutSmleO43AJsYAKCeoIHHStqyt5JYrGJrZ7eC0Vcq5GSwXAA weg9fpQv8v8Agg/8zWornYrm9n8S3to15ttLe3V5iqjKuxJUKewCjms+PVtSfw1ZX32om7vLlY7V No2shfgv6/ICSRigDsqKxLnVJbrXLTTrR/Lgkt2upZwOSoIUKufc5J9PrUOhavJdadfXFxeRmNZp fs8kmAFhQ7QzEe4PNAHQ0Vy7XV3cavfWsGqtFBb2kcjTFFbbK5JB+m0dPcV0y52DJyccnHWgB1FF FABRRRQAVzOhXBvfGfiJ5eTatDbxZ/hXbuOPqT+grpqwYbCfSfEeoX8MDT22oLGzrHjckiDHQkZB GPyo6h0K3he4ZfEHiSwHEFvdI8a9l3oCwH4gn8a6Ke3iuo/LnjWRMg7WGRkViaZp15pkes6l5Cy3 99KZkt94GAq7UQt0zxyfetWG6uP7NhnubR1uGRTJBGwcox6jPAOKP+AHUxPAiqmlX6qAqjUrkAAc D94aZ4RybfW/7T2/aft8wuN/9z+DP+ztxipvCMF3p9vdW95ZTQtNeTzqxKldrNkZIPXmppTdvdXJ n0aKWdXP2WVdu0pgbd7HkHOeADQ/0/yH/mZHg/X1g+H89/cF3i09p0Xd1KIx2j8sCt29177Bo9ne TWzebdPFGsCsMh5COM+2T+Vc3relSaN4NsdCt8XN1fXaLJg7fNJbzJDz2wDWtq9vfajqWkTx2L/Y 7OZpXjZlDl9hCcZxgE+tP+v8xf8AB/4B0FxcRWsDTTuEjXqx6CpawtW1G+sH06FGhae4lSPaRw3e Rj6BVBP1rcVgyhlIIIyCO9IBaKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK ACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKy9Wu9Yt5YxpWmW94hB3tLdeUVPYAbT mqH9p+K/+hesP/Bl/wDa6AOjornP7T8V/wDQvWH/AIMv/tdH9p+K/wDoXrD/AMGX/wBroA6Oiuc/ tPxX/wBC9Yf+DL/7XR/afiv/AKF6w/8ABl/9roA6Oiuc/tPxX/0L1h/4Mv8A7XR/afiv/oXrD/wZ f/a6AOjornP7T8V/9C9Yf+DL/wC10f2n4r/6F6w/8GX/ANroA6Oiuc/tPxX/ANC9Yf8Agy/+10f2 n4r/AOhesP8AwZf/AGugDo6K5z+0/Ff/AEL1h/4Mv/tdH9p+K/8AoXrD/wAGX/2ugDo6K5z+0/Ff /QvWH/gy/wDtdH9p+K/+hesP/Bl/9roA6Oiuc/tPxX/0L1h/4Mv/ALXR/afiv/oXrD/wZf8A2ugD o6K5z+0/Ff8A0L1h/wCDL/7XR/afiv8A6F6w/wDBl/8Aa6AOjornP7T8V/8AQvWH/gy/+10f2n4r /wChesP/AAZf/a6AOjornP7T8V/9C9Yf+DL/AO10f2n4r/6F6w/8GX/2ugDo6K5z+0/Ff/QvWH/g y/8AtdH9p+K/+hesP/Bl/wDa6AOjornP7T8V/wDQvWH/AIMv/tdH9p+K/wDoXrD/AMGX/wBroA6O iuc/tPxX/wBC9Yf+DL/7XR/afiv/AKF6w/8ABl/9roA3xbwgykRJmX/WHaPn4xz68Ux7K2kgSB7e Iwx42IUG1cdMDtWH/afiv/oXrD/wZf8A2uj+0/Ff/QvWH/gy/wDtdAG7LaQT7POhjfZ93coOPpSJ ZWqJIiW8SpL99Qgw31HesP8AtPxX/wBC9Yf+DL/7XR/afiv/AKF6w/8ABl/9roA2k06zjZWS1hVk +6Qg4qzXOf2n4r/6F6w/8GX/ANro/tPxX/0L1h/4Mv8A7XQB0dFc5/afiv8A6F6w/wDBl/8Aa6P7 T8V/9C9Yf+DL/wC10AdHRXOf2n4r/wChesP/AAZf/a6P7T8V/wDQvWH/AIMv/tdAHR0Vzn9p+K/+ hesP/Bl/9ro/tPxX/wBC9Yf+DL/7XQB0dFc5/afiv/oXrD/wZf8A2uj+0/Ff/QvWH/gy/wDtdAHR 0Vzn9p+K/wDoXrD/AMGX/wBro/tPxX/0L1h/4Mv/ALXQBrXOlwXepWd7LvMtnv8AKGflywwSR64/ nV2uc/tPxX/0L1h/4Mv/ALXR/afiv/oXrD/wZf8A2ugDaubC2vJInuIVd4s7GOQRnqPofSrAAAAA wB2rnf7T8V/9C9Yf+DL/AO10f2n4r/6F6w/8GX/2ugDo6K5z+0/Ff/QvWH/gy/8AtdH9p+K/+hes P/Bl/wDa6AOjornP7T8V/wDQvWH/AIMv/tdH9p+K/wDoXrD/AMGX/wBroA6Oiuc/tPxX/wBC9Yf+ DL/7XR/afiv/AKF6w/8ABl/9roA6Oiuc/tPxX/0L1h/4Mv8A7XR/afiv/oXrD/wZf/a6AOjornP7 T8V/9C9Yf+DL/wC10f2n4r/6F6w/8GX/ANroA6Oiuc/tPxX/ANC9Yf8Agy/+10f2n4r/AOhesP8A wZf/AGugDo6K5z+0/Ff/AEL1h/4Mv/tdH9p+K/8AoXrD/wAGX/2ugDo6K5z+0/Ff/QvWH/gy/wDt dH9p+K/+hesP/Bl/9roA6Oiuc/tPxX/0L1h/4Mv/ALXR/afiv/oXrD/wZf8A2ugDo6KgspLmWzie 9gSC4ZcyRJJvCn0DYGfyqegAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo AKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigA ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACi iigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKK KACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo AKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigA ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACi iigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKK KACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo AKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA//Z --_004_068cb32c994d4438880ec3907a6bbcd5XCHALN009ciscocom_-- From nobody Thu Oct 8 11:22:58 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3B88C1A1A6E; Thu, 8 Oct 2015 11:22:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -114.511 X-Spam-Level: X-Spam-Status: No, score=-114.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p6zpYTJ-nKR3; Thu, 8 Oct 2015 11:22:55 -0700 (PDT) Received: from rcdn-iport-4.cisco.com (rcdn-iport-4.cisco.com [173.37.86.75]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 522AD1A1A67; Thu, 8 Oct 2015 11:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4175; q=dns/txt; s=iport; t=1444328575; x=1445538175; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=S4i8Z7/5vfVXBUXzCEGLcxVPw2ZQ+DAA9COlNfzBOzo=; b=hVNv6s52sk6sKM4UZdYWuTLvRRnIceWmDOEYJ8HKkBqaBv6I/dT5iuC7 EUiuc5DazvR0cEcSPFUiMWKW9EhgDdNsjMT9RplJ0y+F9J8Yza8vK76TM cBQY1smsZYhgOi1ovkWLUEFCLbKq8Vf+W5Y9NoohS+lhnBSp6EsuisyHW 8=; X-Files: signature.asc : 833 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0BVAwBpsxZW/5ldJa1egydUbga9Qw6BWhcKgnKCCn8CgVI4FAEBAQEBAQGBCoQmAQEBAwEBAQEgSwsFCwIBCBgqAgInCyUCBA4FDogYCA2vEZQlAQEBAQEBAQEBAQEBAQEBAQEBAQEBEwSJA4JuhQ0HgmkxgRQFlgoBgkyBYYhqBYFTh16SQwEfAUOEAnGGZoEGAQEB X-IronPort-AV: E=Sophos;i="5.17,655,1437436800"; d="asc'?scan'208";a="35684619" Received: from rcdn-core-2.cisco.com ([173.37.93.153]) by rcdn-iport-4.cisco.com with ESMTP; 08 Oct 2015 18:22:54 +0000 Received: from XCH-ALN-011.cisco.com (xch-aln-011.cisco.com [173.36.7.21]) by rcdn-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id t98IMsfk007858 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 8 Oct 2015 18:22:54 GMT Received: from xch-aln-013.cisco.com (173.36.7.23) by XCH-ALN-011.cisco.com (173.36.7.21) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Thu, 8 Oct 2015 13:22:53 -0500 Received: from xch-aln-013.cisco.com ([173.36.7.23]) by XCH-ALN-013.cisco.com ([173.36.7.23]) with mapi id 15.00.1104.000; Thu, 8 Oct 2015 13:22:52 -0500 From: "Fred Baker (fred)" To: David Lang Thread-Topic: [aqm] ACK Suppression Thread-Index: AQHRAfZYPlH6DUEonU27XILiYKhTSQ== Date: Thu, 8 Oct 2015 18:22:52 +0000 Message-ID: References: <1444247538.3556484@apps.rackspace.com> <7A2801D5E40DD64A85E38DF22117852C8812629D@wdc1exchmbxp05.hq.corp.viasat.com> <06C46DC6-854F-4CDE-8C7B-745F3362FA9E@gmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.19.64.119] Content-Type: multipart/signed; boundary="Apple-Mail=_85CCEFC2-2C47-45FE-ADC6-A6B7A02DE58B"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" , "dpreed@reed.com" , Jonathan Morton , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Oct 2015 18:22:57 -0000 --Apple-Mail=_85CCEFC2-2C47-45FE-ADC6-A6B7A02DE58B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 I'm not sure why this discussion is happening on aqm@ instead of = tcpm@... I have added cpm to the cc line, and would recommend that = anyone responding to this thread do the same and remove aqm@. On Oct 7, 2015, at 2:13 PM, David Lang wrote: > So things that reduce the flow of acks can result in very real = benefits to users. Dumb question of the month. What would it take to see wide deployment of = RFC 5690? That would result in the data/ack ratio being reduced, on = average, to whatever amount had been negotiated. Summary for those that haven't read it - TCP implementations today = generally ack every other packet, with caveats for isolated or final = data packets. This proposal allows consenting adults to change that = ratio, acking every third or fourth packet, or every tenth. If ack reductions are so very valuable, what's the chance of doing that = on an end to end basis instead of in the network? > On Oct 7, 2015, at 2:13 PM, David Lang wrote: > On Wed, 7 Oct 2015, Jonathan Morton wrote: >>> On 7 Oct, 2015, at 23:40, Agarwal, Anil = wrote: >>>> Since the cable modem link will lead to clumped ACKs the difference = between sending 100 ACKs vs. 1 ACK is probably not that big... (except = w.r.t. reliability). >>> The difference may not be big in the spacing of new packets that a = sender will send, unless the sender implements some sort of pacing or if = the return link is very thin. >>=20 >>> But with ABC, there will be a difference in the amount of cwnd = increase at the sender. >>=20 >> There is also a potential difference for detecting packet loss in the = forward direction. It=E2=80=99s entirely possible that thinning would = cause a DupAck condition to be recognised only after three MAC grants in = the reverse direction have elapsed, rather than one. Receivers are = REQUIRED to send an ack for every received packet under these = conditions, but that would be subverted by the modem. AckCC would not = induce this effect, because the receiver would still produce the extra = acks as required. >>=20 >> Packet loss causes head-of-line blocking at the application level, = which is perceived as latency and jerkiness by the end-user, until the = lost packet is retransmitted and actually arrives. Hence the addition = of two MAC grant delays (60ms?) may make the difference between an = imperceptible problem and a noticeable one. >=20 > and excessive ack traffic causes congestion and results in packet loss = on real-world hightly asymmetric links. >=20 > So things that reduce the flow of acks can result in very real = benefits to users. >=20 > David Lang_______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm --Apple-Mail=_85CCEFC2-2C47-45FE-ADC6-A6B7A02DE58B Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQIVAwUBVha0lEayAOS/EQ8MAQLpgw//eKZ+Qon5Rtgqhne7q7xkTZF/MKVOHKK4 dwAX6P8SwgsTxZpIljWy3Ran5tEPThZpVa48A3Dvt+ofPupY5c0igr6KXOB4loCP sow6u0o4z5HShLiB2cmpi8mDxkRiKUGFXOq2iK01gNnTQdQv39b0YFRETgnnef7V F4are5R61lIbcUsxQ9CGqNF0cFTZMjCi2j2yFLK2+naoKAKChofbx/CYQMfE5h0I Obb1mMHmcQGLHpf4rQxJE7Z9UH9WM2YldVnYQXDthonUPfqLnN4II7NIn35+uA3g 1ekExzLfPA68/ba1kOUC6ISkl7lKdpbczKVViQgJFOCle8jlBcIpgHj4kZQiBsFN 6xizCaT+kwRTwUFpTeothfC+MccR8CxOzK/8YjA2BkG4ylhuMkNGQPe+GrOuXCPy g9562tQj/JD5lO74dBFeCrclOdiLRKN5bdELzBIMml8FGO7S96RTRvYTvhR83kY/ R9JAhPQCKVK09Mv4HAoh7pXE+7GyXUuOfTYzlk337PhqFHk6ixGrE1sZ4uWapqWQ avT1I1KMq97UThocXfcXHUPntcYSW+v+MGgDURplxmsJFtOPjkq6PZqzzTdomTdm GAby3pBTqVdZiHQRX1Sh5nH228XuaSmb/ZXA4yQIUBeg9ulTqKMAj8GG7YFlUxpL AFcI38AFy9Y= =N9/3 -----END PGP SIGNATURE----- --Apple-Mail=_85CCEFC2-2C47-45FE-ADC6-A6B7A02DE58B-- From nobody Thu Oct 8 12:22:22 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E39E91A1ADD; Thu, 8 Oct 2015 12:22:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -114.511 X-Spam-Level: X-Spam-Status: No, score=-114.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cR4JtvwfEQPV; Thu, 8 Oct 2015 12:22:19 -0700 (PDT) Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E27021A1AD3; Thu, 8 Oct 2015 12:22:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2995; q=dns/txt; s=iport; t=1444332138; x=1445541738; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=vzfX7iB1PThizf0qDhjSqPLXMyjwvJqI6xyz3Ssnxgs=; b=UykiXFLe2DYgLUFX0Mx8WPiwfpzh4NKh8SP8oeXhCWH+ssKWw0XQBUFm HHJKz8w9hItHeDteuUU8ycJ/EFWvToGWcHBi1uupS/TYLyvMl9kHz3fzR XIB2cLj/wxQkT17blMumQ7w4j02V3NWwvaxPbYL937dfmmwMrMyCZpwPJ g=; X-Files: signature.asc : 833 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0BWAwAkwRZW/4kNJK1egydUbgEFvUMOgVojgnCCCn8CgVQ4FAEBAQEBAQGBCoQmAQEBAwF5BQsCAQgYLjIXAQ0CBA4FDogYCA3DKwEBAQEBAQEBAQEBAQEBAQEBAQEBAReJA4JuhQ0HgxqBFAWWCgGCTIFhiGqBWJFaiEcBHwFDghEdgVRxAYZlgQYBAQE X-IronPort-AV: E=Sophos;i="5.17,655,1437436800"; d="asc'?scan'208";a="33848608" Received: from alln-core-4.cisco.com ([173.36.13.137]) by rcdn-iport-9.cisco.com with ESMTP; 08 Oct 2015 19:22:17 +0000 Received: from XCH-ALN-015.cisco.com (xch-aln-015.cisco.com [173.36.7.25]) by alln-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id t98JMHlE026642 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 8 Oct 2015 19:22:17 GMT Received: from xch-aln-013.cisco.com (173.36.7.23) by XCH-ALN-015.cisco.com (173.36.7.25) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Thu, 8 Oct 2015 14:22:16 -0500 Received: from xch-aln-013.cisco.com ([173.36.7.23]) by XCH-ALN-013.cisco.com ([173.36.7.23]) with mapi id 15.00.1104.000; Thu, 8 Oct 2015 14:22:16 -0500 From: "Fred Baker (fred)" To: David Lang Thread-Topic: [aqm] ACK Suppression Thread-Index: AQHRAf6kOyffdZB/5EOtc50V23snxw== Date: Thu, 8 Oct 2015 19:22:16 +0000 Message-ID: <41060A6E-F64D-418A-825A-6BB1FDA35D02@cisco.com> References: <1444247538.3556484@apps.rackspace.com> <7A2801D5E40DD64A85E38DF22117852C8812629D@wdc1exchmbxp05.hq.corp.viasat.com> <06C46DC6-854F-4CDE-8C7B-745F3362FA9E@gmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.19.64.119] Content-Type: multipart/signed; boundary="Apple-Mail=_C24D76C8-6C6A-4D5D-8736-0CF8236E2609"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" , "dpreed@reed.com" , Jonathan Morton , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Oct 2015 19:22:21 -0000 --Apple-Mail=_C24D76C8-6C6A-4D5D-8736-0CF8236E2609 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Oct 8, 2015, at 11:34 AM, David Lang wrote: >=20 >> If ack reductions are so very valuable, what's the chance of doing = that on an end to end basis instead of in the network? >=20 > a little less than the chance of shutting off IPv4 :-) Hey, we can dream, can't we? :-) > requiring changes to all the software on each end to enable this is = wishful thinking. The major servers could get the update pretty promtly, = but updating the client side?? not for a long time. Yes, of course. That said, consider trends like = http://www.zdnet.com/article/latest-os-share-data-shows-windows-still-domi= nating-in-pcs/ = http://www.netmarketshare.com/operating-system-market-share.aspx?qprid=3D1= 1&qpcustomb=3D0 https://www.pinterest.com/pin/137148751123435826/ What that basically says is that corporations tend to pick a version and = install every update after testing it. The folks still on XP are very = likely people who installed a (potentially pirated) release in 2003 and = have probably never updated it. If we can get this into the next MacOSX = 10.11 and Windows 10 update cycle, roughly 50% of computers will have it = within six months, maybe three. If we can get it into common Linux = distributions, we can probably cover the servers as well. 100% = deployment - that might take until the XP computers actually all fry. = Getting it significantly deployed - I don't see the cause of the = pessimism. You're correct that one has to update set-top-boxes and smart TVs. They = get updates too, or at least mine do. It doesn't require buying a new = TV, it requires getting the updated software. --Apple-Mail=_C24D76C8-6C6A-4D5D-8736-0CF8236E2609 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQIVAwUBVhbCg0ayAOS/EQ8MAQKzxg//b/rv4LQEADEK9HlOuXAeAD3JSuFlEnIy MJqQ97jU6DIYDlavglc406CqDeHyRWw1/FKlYnygayWND06Nl9A945wteog5n0mE itEfd/qEsNrWe+o5pO8C8s+S0qARrN2bAgx90TeUV3k/Vdb0OPwPK04grDddvLvl s8qpDY4Zs6IXOcgMNAau67B53LBga1vLTwq5Ah+7txlwT8etf9Yx3a4me80CqnKt FrY+jPWDgEp02LcHc0KkgmC1ZIzsOrWs0MihyMeROV3XCfw3sHJo8pn9LgAGHutq fh+QmSj3iXvLnTHybmOeu1RmMKagtMY85Hv2NB/NXMxrjkr+OLdxiNoQODnDVCGi cBtFc5QJtCwHQBS2n0kH5CRyMqFx1AlVT02J0cWR044pVZQfhYDSKYuY2W43wNhg hjpp4Dhx9JKFK0CB98dg7z+pnRbI+FVeseX7d/+3JpGpkkGGqhlvGJu7rhwpEqNC OoZeHVoDlaugeyu8LyBqIHU06RLYzzuwMl/qv5q9geONBlQoAMZgpNSUgapNUPcn qGdf8QMKwTEHPCdin7lJjJWL2H5Lutq0R5XmaqF6+lkv9pGoreRcU9FHaNHr9hIu u1tGFZ69vVZTHrjON3cC8NnSZpRxz0v/hPwbd3hn5ROtVMI4HaSvu4OSHArFzGYb c2v7JTKkssA= =+sBr -----END PGP SIGNATURE----- --Apple-Mail=_C24D76C8-6C6A-4D5D-8736-0CF8236E2609-- From nobody Thu Oct 8 14:28:04 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F3CF31ACEAB; Thu, 8 Oct 2015 14:28:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xx1Mx-QV9g6f; Thu, 8 Oct 2015 14:28:00 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B4E101ACEBC; Thu, 8 Oct 2015 14:28:00 -0700 (PDT) Received: from [10.123.86.139] (usc-secure-wireless-206-139.usc.edu [68.181.206.139]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t98LRUCC024657 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 8 Oct 2015 14:27:31 -0700 (PDT) To: "Francini, Andrea (Andrea)" , Steve Bauer References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <1BFAC0A1D7955144A2444E902CB628F865B0C249@US70TWXCHMBA12.zam.alcatel-lucent.com> From: Joe Touch X-Enigmail-Draft-Status: N1110 Message-ID: <5616DFBF.9000204@isi.edu> Date: Thu, 8 Oct 2015 14:27:27 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <1BFAC0A1D7955144A2444E902CB628F865B0C249@US70TWXCHMBA12.zam.alcatel-lucent.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: Greg White , "tcpm@ietf.org" , "aqm@ietf.org" , touch@isi.edu Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Oct 2015 21:28:02 -0000 CC'ing TCPM on my responses to this thread... On 10/7/2015 8:50 AM, Francini, Andrea (Andrea) wrote: > How about the effect of ACK suppression on the growth of the > congestion window, for TCP sources where the trigger for window growth > is the arrival of an ACK, not the number of bytes it acknowledges? > > The Appropriate Byte Counting (ABC) option > (https://tools.ietf.org/html/rfc3465) was addressing a similar issue > with delayed ACKs, but it looks like it is no longer available in Linux > (http://www.spinics.net/lists/netdev/msg225164.html). Even with ABC, killing off some of the ACKs would result in sender bursts because you're interfering with ACK clocking -- unless ABC were coupled with some sort of rate pacing. Note also that when you kill off intermediate ACKs you're increasing the round-trip delay. The sender can't start sending what you ACK until the ACK arrives; if you kill off earlier ACKs, then the later ACK will (by definition) arrive later. This is bad on many levels. Joe From nobody Thu Oct 8 20:03:47 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F6251B2D93; Thu, 8 Oct 2015 20:03:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.211 X-Spam-Level: X-Spam-Status: No, score=-4.211 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rG0mxe1r9Fdj; Thu, 8 Oct 2015 20:03:42 -0700 (PDT) Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 5A2AB1B2D8D; Thu, 8 Oct 2015 20:03:42 -0700 (PDT) Received: from erg.abdn.ac.uk (galactica.erg.abdn.ac.uk [139.133.210.32]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPA id CCAB21B0030F; Fri, 9 Oct 2015 04:10:48 +0100 (BST) Received: from 212.159.18.54 (SquirrelMail authenticated user gorry) by erg.abdn.ac.uk with HTTP; Fri, 9 Oct 2015 04:03:41 +0100 Message-ID: In-Reply-To: References: <1444247538.3556484@apps.rackspace.com> <7A2801D5E40DD64A85E38DF22117852C8812629D@wdc1exchmbxp05.hq.corp.viasat.com> <06C46DC6-854F-4CDE-8C7B-745F3362FA9E@gmail.com> Date: Fri, 9 Oct 2015 04:03:41 +0100 From: gorry@erg.abdn.ac.uk To: "Fred Baker (fred)" User-Agent: SquirrelMail/1.4.23 [SVN] MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Archived-At: Cc: David Lang , Jonathan Morton , "dpreed@reed.com" , "tcpm@ietf.org" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 03:03:44 -0000 RFC3449 pointed to some of the reasons & problems with manipulating TCP ACKs, and could provide a useful background if people needed to get up to speed on the history... Gorry > I'm not sure why this discussion is happening on aqm@ instead of tcpm@... > I have added cpm to the cc line, and would recommend that anyone > responding to this thread do the same and remove aqm@. > > On Oct 7, 2015, at 2:13 PM, David Lang wrote: >> So things that reduce the flow of acks can result in very real benefits >> to users. > > Dumb question of the month. What would it take to see wide deployment of > RFC 5690? That would result in the data/ack ratio being reduced, on > average, to whatever amount had been negotiated. > > Summary for those that haven't read it - TCP implementations today > generally ack every other packet, with caveats for isolated or final data > packets. This proposal allows consenting adults to change that ratio, > acking every third or fourth packet, or every tenth. > > If ack reductions are so very valuable, what's the chance of doing that on > an end to end basis instead of in the network? > >> On Oct 7, 2015, at 2:13 PM, David Lang wrote: >> On Wed, 7 Oct 2015, Jonathan Morton wrote: >>>> On 7 Oct, 2015, at 23:40, Agarwal, Anil >>>> wrote: >>>>> Since the cable modem link will lead to clumped ACKs the difference >>>>> between sending 100 ACKs vs. 1 ACK is probably not that big... >>>>> (except w.r.t. reliability). >>>> The difference may not be big in the spacing of new packets that a >>>> sender will send, unless the sender implements some sort of pacing or >>>> if the return link is very thin. >>> >>>> But with ABC, there will be a difference in the amount of cwnd >>>> increase at the sender. >>> >>> There is also a potential difference for detecting packet loss in the >>> forward direction. It’s entirely possible that thinning would cause >>> a DupAck condition to be recognised only after three MAC grants in the >>> reverse direction have elapsed, rather than one. Receivers are >>> REQUIRED to send an ack for every received packet under these >>> conditions, but that would be subverted by the modem. AckCC would not >>> induce this effect, because the receiver would still produce the extra >>> acks as required. >>> >>> Packet loss causes head-of-line blocking at the application level, >>> which is perceived as latency and jerkiness by the end-user, until the >>> lost packet is retransmitted and actually arrives. Hence the addition >>> of two MAC grant delays (60ms?) may make the difference between an >>> imperceptible problem and a noticeable one. >> >> and excessive ack traffic causes congestion and results in packet loss >> on real-world hightly asymmetric links. >> >> So things that reduce the flow of acks can result in very real benefits >> to users. >> >> David Lang_______________________________________________ >> aqm mailing list >> aqm@ietf.org >> https://www.ietf.org/mailman/listinfo/aqm > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Thu Oct 8 23:43:52 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0EAD71B35ED; Thu, 8 Oct 2015 23:43:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.961 X-Spam-Level: X-Spam-Status: No, score=-3.961 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5w_OrAbyIkXs; Thu, 8 Oct 2015 23:43:48 -0700 (PDT) Received: from uplift.swm.pp.se (swm.pp.se [212.247.200.143]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 87C851B35E5; Thu, 8 Oct 2015 23:43:47 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id DF227A1; Fri, 9 Oct 2015 08:43:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1444373024; bh=fWNw1TX9/WKqOXCvxKZrRFHpmIithvRiM7Z+uCrRmns=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=zPNiQ9VHdGUuE1jAYYsJhrb0ewVNGLDikxHzQBq3q0jxo4P3ujRKRsYDtpE4zFtTY fP54kjRCaRaP1E/phFWstajkUX037bwx2IFQ5nhAg4ANFGnOb8ndfhqpzVWngkaTnE TNJWhtznNwT1aa/cXFkPsdkTyQu/3NdA24V88lE8= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id DA4A19F; Fri, 9 Oct 2015 08:43:44 +0200 (CEST) Date: Fri, 9 Oct 2015 08:43:44 +0200 (CEST) From: Mikael Abrahamsson To: David Lang In-Reply-To: Message-ID: References: <1444247538.3556484@apps.rackspace.com> <7A2801D5E40DD64A85E38DF22117852C8812629D@wdc1exchmbxp05.hq.corp.viasat.com> <06C46DC6-854F-4CDE-8C7B-745F3362FA9E@gmail.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: Cc: "tcpm@ietf.org" , "dpreed@reed.com" , "Fred Baker \(fred\)" , Jonathan Morton , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 06:43:50 -0000 On Thu, 8 Oct 2015, David Lang wrote: > Ok, so it turns out that what we are talking about here is talked about > explicitly in section 5.2.1 of RFC3449. It warns of increased sender > burst sizes, but I really question if that's a valid concern in the > cases that we are talking about here. The choice in these cases isn't to > send more acks spaced out, but rather to send a lot of acks back to back > in one transmit slot vs one stretch ack per transmit slot. Exactly. And in my specific case, we're talking 20kPPS of data down, and 10kPPS of ACKs, over a medium that allows the modem to send a bunch of packets only once every few milliseconds. So there are two alternatives: You get approx 10-60 ACKs as a "train of ACKs" within 0.3 milliseconds, every 1-4 milliseconds. You get 1-3 ACKs within 0.1ms, every 2-3 milliseconds. These ACKs will acknowledge 16-32 packets at a time. By the nature of the beast, I'd say that this kind of medium implicitly will be able to handle the "bursts" we're talking about here, if you ACK 24-48kilobyte of data per ACK (when a middle box does ACK Suppression to remove intermediate ACKs), it's implicit that the medium designers know that they can handle a wirespeed burst of data that is the result of the ACKing of these fairly large number of bytes. But again, I question the wisdom in ACKing every 2 packets when the congestion window is in the hundreds of kilobytes and you're sending tens of thousands of packets per second. I suggested in another part of the thread that perhaps the host itself should do ACK suppression and don't send ACKs more closely together than once per millisecond or 1/10th RTT, whatever is lower. Perhaps the RTT value needs to be even lower, 1/25 of RTT or something. In my specific test, using once per millisecond or 1/25 of RTT would cut down the number of ACKs to around 2kpps instead of the 10kpps ACKs I see now, but it wouldn't cut it down to the around 500-1000 ACKs per second that the DOCSIS modem is cutting it down to... I think that as bandwidths go up, there are changes to TCP that can be done to make it work better, and reducing the frequency of ACKs sent depending on RTT and different window sizes might be a way forward. I fully believe there are cases where ACKing every 2 packets is exactly the right way to go, but when you're sending 1 gigabit/s over 100ms RTT and the medium itself clumps the packets together because that's what it does, then these ACKs don't carry the same information anymore and some of them could be optimized away (preferrably by the host stack) to spare upstream bandwidth and transmit opportunities. -- Mikael Abrahamsson email: swmike@swm.pp.se From nobody Fri Oct 9 01:58:13 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6CA021A8A7E for ; Fri, 9 Oct 2015 01:58:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1iHc_ZllHfax for ; Fri, 9 Oct 2015 01:58:09 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DE5D51A8A4E for ; Fri, 9 Oct 2015 01:58:08 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (unknown [135.239.2.42]) by Websense Email Security Gateway with ESMTPS id 2D521BBC76FF0; Fri, 9 Oct 2015 08:58:05 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id t998w6PE009559 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 9 Oct 2015 10:58:06 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Fri, 9 Oct 2015 10:58:06 +0200 From: "Scharf, Michael (Michael)" To: Praveen Balasubramanian , "tcpm@ietf.org" Thread-Topic: [aqm] TCP ACK Suppression Thread-Index: AQHRAAfbmdqr5Xk9lE2VWzCubgsRu55eiGYAgAD9UYCAAAl1AIAAK3+AgABsWwCAAATHAIAABJaAgAAioOCAAAglYIAChWRA Date: Fri, 9 Oct 2015 08:58:05 +0000 Message-ID: <655C07320163294895BBADA28372AF5D484FC968@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5614D4B8.5060403@kit.edu> <5615581C.5010907@rogers.com> <655C07320163294895BBADA28372AF5D484F882F@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 08:58:11 -0000 V2h5IGRvbid0IHlvdSB3cml0ZSB1cCB5b3VyIGZpbmRpbmdzIGluIGEgc2hvcnQgSS1EPyBCYXNl ZCBvbiB0aGUgbGVuZ3RoIG9mIHRoaXMgdGhyZWFkLCBpdCBzZWVtcyB0aGF0IHRoZXJlIGlzIHNv bWUgY29tbXVuaXR5IGludGVyZXN0IGluIGZ1cnRoZXIgZGlzY3Vzc2luZyBwcm9zIGFuZCBjb25z IG9mIHN0cmV0Y2ggQUNLcy4NCg0KTWljaGFlbA0KDQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0t LS0tDQpGcm9tOiBQcmF2ZWVuIEJhbGFzdWJyYW1hbmlhbiBbbWFpbHRvOnByYXZiQG1pY3Jvc29m dC5jb21dIA0KU2VudDogV2VkbmVzZGF5LCBPY3RvYmVyIDA3LCAyMDE1IDk6MDMgUE0NClRvOiBT Y2hhcmYsIE1pY2hhZWwgKE1pY2hhZWwpOyB0Y3BtQGlldGYub3JnDQpTdWJqZWN0OiBSRTogW2Fx bV0gVENQIEFDSyBTdXBwcmVzc2lvbg0KDQpUaGFua3MgTWljaGFlbCwgaXQgaXMgaW5kZWVkIHZl cnkgcmVsZXZhbnQuIA0KDQpXaW5kb3dzIGRvZXMgcGVyZm9ybSBzdHJldGNoIEFDS3MgYXMgYW4g b3B0aW1pemF0aW9uLiBIb3dldmVyLCBBQ0sgY29hbGVzY2luZyBpcyBsaW1pdGVkIGluIHRoZSB3 b3JzdCBjYXNlIHNpbmNlIHBhY2tldHMgYXJlIHByb2Nlc3NlZCBieSBiYXRjaGluZyAod2hpY2gg aXMgbGltaXRlZCB0byBhIHNpemUgTikuIFRoYXQgaXMsIHdpdGggYSBzaW5nbGUgY29ubmVjdGlv biBhbmQgdmVyeSBzbWFsbCBpbnRlciBwYWNrZXQgc3BhY2luZywgV2luZG93cyB3aWxsIGdlbmVy YXRlIG9uZSBBQ0sgZm9yIE4gc2VnbWVudHMuIEluIHByYWN0aWNlIGhvd2V2ZXIgdGhlcmUgYXJl IG11bHRpcGxlIGZsb3dzIGFuZCBwYWNrZXQgc3BhY2luZyBzdWNoIHRoYXQgQUNLcyBjb3ZlciBh IG51bWJlciBtdWNoIGxlc3MgdGhhbiBOLiBUaGUgcmVjZWl2ZXIgQUNLaW5nIGJlaGF2aW9yIGlz IGhlbmNlIG5vdCBjb21wbGV0ZWx5IGRldGVybWluaXN0aWMgYW5kIGRlcGVuZHMgb24gcGFja2V0 IGFycml2YWwgcmF0ZSwgaW50ZXJydXB0IGFuZCBEUEMgKH4gZGVmZXJyZWQgd29yayBmcm9tIGlu dGVycnVwdHMpIHJhdGVzIGFuZCBudW1iZXIgb2YgY29ubmVjdGlvbnMgZXRjLiBXZSBhcmUgbG9v a2luZyBpbnRvIHRoZSBpbXBhY3Qgb2YgdGhpcyBiZWhhdmlvciBvbiBzZW5kZXIgd2luZG93IGdy b3d0aCBlc3BlY2lhbGx5IGR1cmluZyBzbG93IHN0YXJ0LiBBIHBvc3NpYmxlIG1pdGlnYXRpb24g aXMgdG8gcmVkdWNlIHRoZSBzaXplIE4gb3IgaW50cm9kdWNlIGEgcGVyIGNvbm5lY3Rpb24gbGlt aXQgTSB0byBlbnN1cmUgYSBzaW5nbGUgZGVsYXllZCBBQ0sgYWNrbm93bGVkZ2VzIG5vIG1vcmUg dGhhbiBNIHNlZ21lbnRzLiBJIGFscmVhZHkgcHJvdmlkZWQgZmVlZGJhY2sgb24gdGhlIEFjY0VD TiBwcm9wb3NhbCBhbmQgaXQgbG9va3MgbGlrZSB0aGUgbGltaXQgTSB3b3VsZCBuZWVkIHRvIGJl IDYgd2hlbiBwcm9jZXNzaW5nIENFIG1hcmtlZCBwYWNrZXRzIHdoaWNoIGRvZXNu4oCZdCBzZWVt IG9uZXJvdXMuIA0KDQpUaGUgQUNLaW5nIGJlaGF2aW9yIGRpcmVjdGx5IHRpZXMgaW4gdG8gQUJD IG9uIHRoZSBzZW5kZXIuIFdpbmRvd3MgZG9lcyBzdXBwb3J0IEFCQyBkdXJpbmcgYm90aCBzbG93 IHN0YXJ0IGFuZCBjb25nZXN0aW9uIGF2b2lkYW5jZSBhbmQgYmFzZXMgdGhlIG1hZ25pdHVkZSBv ZiB0aGUgY3duZCBpbmNyZWFzZSBvbiB0aGUgbnVtYmVyIG9mIG5ld2x5IGFja25vd2xlZGdlZCBi eXRlcyAocGVyIFJGQyAzNDY1KSB1cCB0byBhIGZpeGVkIGZhY3RvciBYIG9mIHRoZSBNU1MuIEFs bCB0aGUgY29uZ2VzdGlvbiBhbGdvcml0aG1zIHN1cHBvcnRlZCBpbiBXaW5kb3dzIGxpa2UgTmV3 UmVubywgQ1RDUCBhbmQgRENUQ1AgZm9sbG93IHRoaXMgYmVoYXZpb3IsIGFuZCBDVENQIHRha2Vz IGFkdmFudGFnZSB0aGUgZHduZCAoZGVsYXkgd2luZG93KSBncm93dGggYXMgd2VsbC4gSXQgbWln aHQgYmUgdXNlZnVsIHRvIGhhdmUgWCA9IE0uDQoNCkFueSB0aG91Z2h0cyBvbiBzdHJldGNoIEFD S3MgYXMgYW4gb3B0aW1pemF0aW9uIHdoZW4gdXNlZCBpbiBjb25qdW5jdGlvbiB3aXRoIEFCQz8g DQoNClRoYW5rcw0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogdGNwbSBbbWFp bHRvOnRjcG0tYm91bmNlc0BpZXRmLm9yZ10gT24gQmVoYWxmIE9mIFNjaGFyZiwgTWljaGFlbCAo TWljaGFlbCkNClNlbnQ6IFdlZG5lc2RheSwgT2N0b2JlciA3LCAyMDE1IDEwOjU5IEFNDQpUbzog dGNwbUBpZXRmLm9yZw0KU3ViamVjdDogUmU6IFt0Y3BtXSBbYXFtXSBUQ1AgQUNLIFN1cHByZXNz aW9uDQoNClRoaXMgdGhyZWFkIGlzIG1heSBiZSByZWxldmFudCB0byBUQ1BNDQoNCk1pY2hhZWwN Cg0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogYXFtIFttYWlsdG86YXFtLWJv dW5jZXNAaWV0Zi5vcmddIE9uIEJlaGFsZiBPZiBKb25hdGhhbiBNb3J0b24NClNlbnQ6IFdlZG5l c2RheSwgT2N0b2JlciAwNywgMjAxNSA3OjUzIFBNDQpUbzogZGF2ZWNiQHNwYW1jb3AubmV0DQpD YzogYXFtQGlldGYub3JnDQpTdWJqZWN0OiBSZTogW2FxbV0gVENQIEFDSyBTdXBwcmVzc2lvbg0K DQoNCj4gT24gNyBPY3QsIDIwMTUsIGF0IDIwOjM2LCBEYXZpZCBDb2xsaWVyLUJyb3duIDxkYXZl Yy1iQHJvZ2Vycy5jb20+IHdyb3RlOg0KPiANCj4gT24gMDcvMTAvMTUgMDE6MTkgUE0sIERhdmlk IExhbmcgd3JvdGU6DQo+PiBPbiBXZWQsIDcgT2N0IDIwMTUsIE1pa2FlbCBBYnJhaGFtc3NvbiB3 cm90ZToNCj4+IA0KPj4+PiBPaCwgSSBob3BlIHRoYXQgdGhpcyBpcyBhbiBleGNlcHRpb24uIFN1 Y2gga2luZCBvZiBvcHRpbWl6YXRpb25zIG1heSBjYXVzZSBhIGxvdCBvZiB0cm91YmxlIHNpbmNl IGEgbGluayBsYXllciBkZXZpY2UgaXMgaW50ZXJmZXJpbmcgd2l0aCB0cmFuc3BvcnQgbGF5ZXIg c2VtYW50aWNzLiBXZSBhbGwga25vdyB0aGF0IGV4YWN0bHkgdGhlc2Uga2luZHMgb2YgaW50ZXJm ZXJlbmNlIGV2ZW50dWFsbHkgZW5kIHVwIGluIHByb2JsZW1zIHdpdGggZW5kLXRvLWVuZCB0cmFu c3BhcmVuY3kgYW5kIGRlcGxveW1lbnQgb2YgbmV3IHByb3RvY29sIG9wdGlvbnMuIEF0IGxlYXN0 IGl0IGludGVyZmVyZXMgd2l0aCB0aGUgQUNLIGNsb2NraW5nIGV4cGVjdGF0aW9uIG9mIHNvbWUg Y29uZ2VzdGlvbiBjb250cm9sIGFsZ29yaXRobXMuLi4NCj4+PiANCj4+PiBQZXJzb25hbGx5LCBJ IHRoaW5rIHlvdSdyZSBnb2luZyB0byBzZWUgbW9yZSBhbmQgbW9yZSBvZiB0aGlzLiBUaGVyZSBh cmUgbXVsaXRwbGUgc2hhcmVkIGFjY2VzcyBtZWRpdW0gd2hlcmUgeW91J3JlIGFsbG93ZWQgdG8g c2VuZCBvbmx5IHBhcnQgb2YgdGhlIHRpbWUsIGFuZCBpdCdzIHNvbWVvbmUgZWxzZSB3aG8gdGVs bHMgeW91IHdoZW4geW91IG1heSBzZW5kLg0KPj4gDQo+PiBpdCBkb2Vzbid0IGV2ZW4gcmVxdWly ZSB0aGF0IHNvbWVvbmUgZWxzZSB0ZWxscyB5b3Ugd2hlbiB5b3UgbWF5IHNlbmQuIEl0IGNhbiBq dXN0IGJlIHdhaXRpbmcgZm9yIGFuIGF2YWlsYWJsZSB0cmFuc21pdCB0aW1lc2xvdCAoV2lmaSBm b3IgZXhhbXBsZSkNCj4+IA0KPj4gY29sbGFwc2luZyBtdWx0aXBsZSBBQ0tzIHRoYXQgYXJlIGdv aW5nIHRvIGJlIHNlbnQgYXQgb25jZSBpcyBhbG1vc3QgYWx3YXlzIGdvaW5nIHRvIGJlIGEgd2lu Lg0KPiANCj4gSSBxdWl0ZSBhZ3JlZSwgYnV0IGlmIHRoZXJlIGlzIGEgY29uZ2VzdGlvbiBjb250 cm9sIGltcGxlbWVudGF0aW9uICJpbiB0aGUgd2lsZCIgdGhhdCBhc3N1bWVzIGl0IHdpbGwgZ2V0 IGEgc3RyZWFtIG9mIGFja3MsIHRoYXQgb25lJ3MgZ29pbmcgdG8gbmVlZCBzb21lIHdvcmsgKDot KSkNCj4gDQo+IEFueW9uZSBrbm93IGlmIHRoYXQncyB0aGUgY2FzZT8gVGhlIGNvbW1lbnQgYWJv dmUgc3VnZ2VzdCBpdCBtYXkgYmUuLi4NCg0KSXTigJlzIG5vdCDigJxpbiB0aGUgd2lsZOKAnSwg YnV0IHRoaXMgc29ydCBvZiB0aGluZyBpcyBhIGhlYWRhY2hlIGZvciBFTFIuICBJdCBlc3NlbnRp YWxseSBtZWFucyB0aGF0IGl0IHdvbuKAmXQgYmUgYWJsZSB0byB1c2UgQWNjRUNOIGZvciBpdHMg ZmVlZGJhY2sgKHNpbmNlIEFjY0VDTiBkb2VzbuKAmXQgYWxsb3cgcmVjb25zdHJ1Y3RpbmcgYSBj b21wbGV4IDMyLXBhY2tldCBoaXN0b3J5IG9mIEVDTiBjb2RlcG9pbnRzIGZyb20gYSBzaW5nbGUg QUNLKSwgYnV0IG11c3QgaW50cm9kdWNlIHNvbWUgb3RoZXIgbWVjaGFuaXNtIHRvIGZlZWQgYmFj ayB0aGUgcmVxdWlyZWQgZGF0YSB0byB0aGUgc2VuZGVyLg0KDQpJ4oCZbSBsZWFuaW5nIHRvd2Fy ZHMgbW92aW5nIHRoZSBFTFItYXdhcmUgY29uZ2VzdGlvbiBjb250cm9sIGFsZ29yaXRobSB0byB0 aGUgcmVjZWl2ZXIsIGFuZCBoYXZpbmcgdGhlIGZlZWRiYWNrIHNpbXBseSBiZSB0aGUgcmVxdWVz dGVkIGN3bmQ7IGl04oCZcyB0aGUgc2ltcGxlc3QgYW5kIG1vc3QgY29tcGFjdCBzb2x1dGlvbiBJ IGNhbiB0aGluayBvZiwgYW5kIHdvdWxkIHdvcmsgZm9yIGFueSBjb25nZXN0aW9uIGNvbnRyb2wg c2NoZW1lLCBub3QganVzdCBFTFIuICBUaGUgc2VuZGVyIG1heSBvZiBjb3Vyc2UgbWFpbnRhaW4g aXRzIG93biBpZGVhIG9mIGN3bmQgaW4gdGhlIGNvbnZlbnRpb25hbCB3YXksIGFuZCBjb25zZXJ2 YXRpdmVseSB0YWtlIHRoZSBtaW5pbXVtIG9mIHRoZSB0d28gYXMgZWZmZWN0aXZlOyB0aGlzIHdv dWxkIHByb3RlY3QgYWdhaW5zdCBuYWl2ZWx5IGdyZWVkeSByZWNlaXZlcnMuDQoNCkkgbm90ZSB0 aGF0IENVQklDIGRvZXNu4oCZdCBjYXJlIGhvdyBtYW55IGFja3MgaXQgZ2V0cywgc2luY2UgaXRz IHNjYWxpbmcgY3VydmUgZGVwZW5kcyBvbmx5IG9uIHRpbWUgc28gbG9uZyBhcyB0aGUgZGF0YSBm bG93IGlzIG5vdCBhcHBsaWNhdGlvbi1saW1pdGVkLiAgVGhhdOKAmXMgZ29vZCBuZXdzIGZvciBM aW51eCwgZ2l2ZW4gdGhlIGxhY2sgb2YgQUJDLiAgRG8gdGhlIEJTRHMgYW5kIFdpbmRvd3MgdXNl IEFCQz8NCg0KIC0gSm9uYXRoYW4gTW9ydG9uDQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fDQphcW0gbWFpbGluZyBsaXN0DQphcW1AaWV0Zi5vcmcNCmh0 dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vYXFtDQpfX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KdGNwbSBtYWlsaW5nIGxpc3QNCnRjcG1A aWV0Zi5vcmcNCmh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vdGNwbQ0K From nobody Fri Oct 9 02:20:43 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E6AB31B2FEA for ; Fri, 9 Oct 2015 02:20:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.91 X-Spam-Level: X-Spam-Status: No, score=-3.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id etODjqXk8QPJ for ; Fri, 9 Oct 2015 02:20:38 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 856B31B2FF2 for ; Fri, 9 Oct 2015 02:20:38 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 26DC0D9307 for ; Fri, 9 Oct 2015 11:20:37 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 1v05fGA3QYhy for ; Fri, 9 Oct 2015 11:20:36 +0200 (MEST) Received: from [82.130.103.143] (nb-10510.ethz.ch [82.130.103.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: mirjak) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id D866DD9302 for ; Fri, 9 Oct 2015 11:20:36 +0200 (MEST) To: tcpm@ietf.org References: <655C07320163294895BBADA28372AF5D484D799E@FR712WXCHMBA15.zeu.alcatel-lucent.com> <46616f47b42841f0bb7a2afb8bdeac98@XCH-ALN-009.cisco.com> <655C07320163294895BBADA28372AF5D484F17F6@FR712WXCHMBA15.zeu.alcatel-lucent.com> <655C07320163294895BBADA28372AF5D484F7BD9@FR712WXCHMBA15.zeu.alcatel-lucent.com> <068cb32c994d4438880ec3907a6bbcd5@XCH-ALN-009.cisco.com> From: =?UTF-8?Q?Mirja_K=c3=bchlewind?= Message-ID: <561786BF.300@tik.ee.ethz.ch> Date: Fri, 9 Oct 2015 11:19:59 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <068cb32c994d4438880ec3907a6bbcd5@XCH-ALN-009.cisco.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [tcpm] TCP evolution impact on router buffers X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 09:20:42 -0000 Hi all, one more comment. How much buffer is need mostly depends on the decrease factor of your congestion control which is 0.5 for Reno (and many others) and 0.7 for Cubic. That means Cubic reduces less on congestion and therefore needs less buffer to keep the link full during the time period where the window was reduced. This is easy to calculate if there is only one flow. Of course if you have different flows that are not synchronized and might use different schemes, it's more complicated. And then also the increase behavior influences how much you'll underutilize the link if the buffer is 'too small'. However, the actual point I wanted to make it that most schemes use fixed decrease values which (at least theoretically) induce a dependency on the buffer size. There is e.g. H-TCP (and also a scheme I've proposed and presented a while ago at an ICCRG meeting called TCP SIAD) that calculate the decrease factor dynamically based on RTT measurements. These schemes can basically work well with any buffer size, however, of course they are slightly more complicated and as there was not really a big need for them up to now, they are not really deployed. Mirja On 08.10.2015 16:41, Lane Wigley (lwigley) wrote: > Just to wrap this thread up, I found a paper that addressed my original > question. It looked at the impact of mixing different TCP implementations and > one of the calculations was the standard deviation of the congestion window > with various algorithms. All of the newer ones have lower standard deviation > than Reno. > > http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6742797 > > - Lane > > *From:*Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] > *Sent:* Wednesday, October 07, 2015 7:35 AM > *To:* Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org > *Subject:* RE: TCP evolution impact on router buffers > > Regarding 1: Sure, it doesnt make any sense to add architecture-specific > guidelines. I just wonder if the text could mention a bit more explicitly > that a number of potential router components and algorithms (e.g., DiffServ) > are omitted (or abstracted) to keep the guidelines generic and simple. > > Regarding 2: Yes, given the references mentioned by Lane, Id suggest to > somehow replace the wording and is to be set to the bandwidth-delay > product, e.g., by a MAY. > > But I leave this to the authors this is not really in TCPM scope, and I > dont have cycles to follow the AQM WG. > > Michael > > *From:*Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] > *Sent:* Wednesday, October 07, 2015 12:17 PM > *To:* Scharf, Michael (Michael); Lane Wigley (lwigley); tcpm@ietf.org > > *Subject:* RE: TCP evolution impact on router buffers > > Hi, > > 1-Regarding the CIR/PIR > > IMHO this is out of the scope of the characterization guidelines. However, if > you think that we should introduce more information on the MPLS/IP router > architectures please let us know. The only problem being that we want to be > quite generic and do not want to focus on specific architectures. > > 2-Regarding buffer sizes > > Since buffer sizing is not the only problem of evaluating AQMs, we have > looked for RFC clearly stipulating how the adequate buffer size can be > found. As said in RFC7597: > > > > The optimum buffer size is a function of > > operational requirements and should generally be sized to be > > sufficient to buffer the largest normal traffic burst that is > > expected. This size depends on the amount and burstiness of traffic > > arriving at the queue and the rate at which traffic leaves the queue. > > > > In the characterization draft, the recommendation on the buffer size is not a > MUST. > > If you think that we should more clearly say that the size of the buffer MAY > be set to the BDP, let us know. > > In its current state, it has been voluntary that the draft does not impose > any buffer size. > > Cheers, > > Nicolas > > *De :*Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] > *Envoy :* vendredi 2 octobre 2015 19:06 > * :* Kuhn Nicolas; Lane Wigley (lwigley); tcpm@ietf.org > *Objet :* RE: TCP evolution impact on router buffers > > Regarding Section 7 in [2], I am surprised that a potential impact of CIR/PIR > is not mentioned. Unfortunately, I have only a very limited understanding of > MPLS/IP router architectures, but the way bursts hit an AQM could IMHO depend > on that. > > As [2] is mentioned, one question related to the discussion below. From [2]: > > 3.2. Buffer size > > The size of the buffers should be carefully chosen, and is to be set > > to the bandwidth-delay product; the bandwidth being the bottleneck > > capacity and the delay the largest RTT in the considered network. > > This means of the order of 10GB of buffer size for a 400G router interface, > right? Wouldnt it make sense to relax the statement and is to be set a bit? > > Michael > > *From:*Kuhn Nicolas [mailto:Nicolas.Kuhn@cnes.fr] > *Sent:* Wednesday, September 30, 2015 6:41 PM > *To:* Lane Wigley (lwigley); Scharf, Michael (Michael); tcpm@ietf.org > > *Subject:* RE: TCP evolution impact on router buffers > > Hi, > > In [1], the authors evaluate the performance of CUBIC in small buffer regime. > However, they use the Linux kernel version 2.6.18 (where the IW was 3 and not > 10). Also, for any tests on CUBIC, small buffers and IW3/10, you may be > interested in looking at the scenario 7. Burst absorption of the AQM > Characterization Guidelines of the AQM WG [2]. Concerning TCP pacing, there > is an on-going document on a pacing solution in the slow-start that you may > want to have a look and comment [3]. > > I hope this helps, > > Cheers, > > Nicolas > > [1] Jain, S.; Raina, G., "An experimental evaluation of CUBIC TCP in a small > buffer regime," in /Communications (NCC), 2011 National Conference on/ , > vol., no., pp.1-5, 28-30 Jan. 2011 > doi: 10.1109/NCC.2011.5734779 > URL: > http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5734779&isnumber=5734693 > > [2] https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-08 > > [3] https://tools.ietf.org/html/draft-sallantin-tcpm-initial-spreading-01 > > *De :*tcpm [mailto:tcpm-bounces@ietf.org] *De la part de* Lane Wigley (lwigley) > *Envoy :* mercredi 30 septembre 2015 15:10 > * :* Scharf, Michael (Michael); tcpm@ietf.org > *Objet :* Re: [tcpm] TCP evolution impact on router buffers > > This paper from 2008 shows no drops with a 5 msec buffer on a > heavily-congested core link. Low levels of drops start to show up around 2.5 > msec. This was conducted at Level 3 to test the BDP / sqrt # flows proposal > from Stanford. > > Experimental Study of Router Buffer Sizing Neda Beheshti, Yashar Ganjali, > Monia Ghobadi, Nick McKeown, and Geoff Salmon > > They also explored TCP pacing as a way to further reduce buffers. Im trying > to assess if CUBIC results in some pacing effect by decoupling the window > growth from acks. > > - Lane > > *From:*Scharf, Michael (Michael) [mailto:michael.scharf@alcatel-lucent.com] > *Sent:* Tuesday, September 29, 2015 6:56 PM > *To:* Lane Wigley (lwigley); tcpm@ietf.org > *Subject:* RE: TCP evolution impact on router buffers > > This question may also be in scope of the AQM WG, and perhaps ICCRG, but > since the communities overlap, I avoid cross-posting > > Unfortunately, I dont have any recent pointer or own data. But I am aware of > examples for router buffer dimensioning based on tests with a single TCP > connection, using Reno. In those examples, the dimension of the buffers and > possibly other related parameters (two-color/three-color meters/policiers, > WRED slopes, H-QoS, etc.) are configured to achieve full line speed for a > single TCP connection. As to be expected, this typically comes down to the > old BDP rule of thumb. But it is rather obvious that a single bulk data Reno > connection is not necessarily a realistic workload these days. So, the test > methodology can be one relevant aspect of the dimensioning problem. > > I believe that a modern TCP stack with a high-speed congestion control such > as CUBIC will typically need less buffer than the outcome of this > dimensioning method and the publications 10 years ago. As far as I know, most > modern TCP stacks operate well over a very wide range of buffer sizes, and > the buffers can be relatively small. To me, the key question is the > latency-throughput tradeoff, and there may not be a one-fits-all answer to > that one. > > Personally, Id also be interested in more recent pointers on router buffer > dimensioning. I could even see some value in documenting dimensioning > guidelines used in practice, if there was a way to generalize them. > > Michael > > *From:*tcpm [mailto:tcpm-bounces@ietf.org] *On Behalf Of *Lane Wigley (lwigley) > *Sent:* Friday, September 25, 2015 5:41 PM > *To:* tcpm@ietf.org > *Subject:* [tcpm] TCP evolution impact on router buffers > > Im trying to track down some research or opinions on how the more recent TCP > schemes impact router buffer needs. There are a number of publications from > Stanford and Georgia Tech from about 10 years ago, and Im trying to assess > how changes to the algorithms (e.g. CUBIC) and parameters (initial congestion > window) deployed since then may influence those findings in the direction of > more or less buffering being needed. > > Id appreciate any input and pointers. Thanks. > > - Lane > > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Fri Oct 9 02:22:04 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CC3BA1B3007 for ; Fri, 9 Oct 2015 02:22:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.909 X-Spam-Level: X-Spam-Status: No, score=-6.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mBLxGWhp7C5T for ; Fri, 9 Oct 2015 02:21:59 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EDB5B1B2FF2 for ; Fri, 9 Oct 2015 02:21:58 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (unknown [135.239.2.42]) by Websense Email Security Gateway with ESMTPS id 6A7B81974184D; Fri, 9 Oct 2015 09:21:55 +0000 (GMT) Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id t999LtNw003410 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 9 Oct 2015 11:21:56 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.03.0195.001; Fri, 9 Oct 2015 11:21:32 +0200 From: "Scharf, Michael (Michael)" To: Joe Touch Thread-Topic: [aqm] TCP ACK Suppression Thread-Index: AQHRAAfbmdqr5Xk9lE2VWzCubgsRu55eiGYAgAD9UYCAAnWAgIAABLaAgAAFoICAANvX0A== Date: Fri, 9 Oct 2015 09:21:30 +0000 Message-ID: <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: multipart/alternative; boundary="_000_655C07320163294895BBADA28372AF5D484FCA01FR712WXCHMBA15z_" MIME-Version: 1.0 Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Greg White , "tcpm@ietf.org" , David Lang Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 09:22:02 -0000 --_000_655C07320163294895BBADA28372AF5D484FCA01FR712WXCHMBA15z_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 W1JlbW92aW5nIEFRTSwgYWRkaW5nIFRDUE1dDQoNCkhpIEpvZSwNCg0KSSBkb27igJl0IGZpbmQg dGhpcyB0ZXh0IGluIFJGQyA3OTMsIGJ1dCBpdCBpcyBjb250YWluZWQgaW4gUkZDIDI1ODEsIHdo aWNoIGlzIG9ic29sZXRlIGJ5IFJGQyA1NjgxLg0KDQpSRkMgNTY4MSBzdGF0ZXM6DQoNCjQuMi4g IEdlbmVyYXRpbmcgQWNrbm93bGVkZ21lbnRzDQoNCiAgIFRoZSBkZWxheWVkIEFDSyBhbGdvcml0 aG0gc3BlY2lmaWVkIGluIFtSRkMxMTIyXSBTSE9VTEQgYmUgdXNlZCBieSBhDQogICBUQ1AgcmVj ZWl2ZXIuICBXaGVuIHVzaW5nIGRlbGF5ZWQgQUNLcywgYSBUQ1AgcmVjZWl2ZXIgTVVTVCBOT1QN CiAgIGV4Y2Vzc2l2ZWx5IGRlbGF5IGFja25vd2xlZGdtZW50cy4gIFNwZWNpZmljYWxseSwgYW4g QUNLIFNIT1VMRCBiZQ0KICAgZ2VuZXJhdGVkIGZvciBhdCBsZWFzdCBldmVyeSBzZWNvbmQgZnVs bC1zaXplZCBzZWdtZW50LCBhbmQgTVVTVCBiZQ0KICAgZ2VuZXJhdGVkIHdpdGhpbiA1MDAgbXMg b2YgdGhlIGFycml2YWwgb2YgdGhlIGZpcnN0IHVuYWNrbm93bGVkZ2VkDQogICBwYWNrZXQuDQoN CiAgIFRoZSByZXF1aXJlbWVudCB0aGF0IGFuIEFDSyAiU0hPVUxEIiBiZSBnZW5lcmF0ZWQgZm9y IGF0IGxlYXN0IGV2ZXJ5DQogICBzZWNvbmQgZnVsbC1zaXplZCBzZWdtZW50IGlzIGxpc3RlZCBp biBbUkZDMTEyMl0gaW4gb25lIHBsYWNlIGFzIGENCiAgIFNIT1VMRCBhbmQgYW5vdGhlciBhcyBh IE1VU1QuICBIZXJlIHdlIHVuYW1iaWd1b3VzbHkgc3RhdGUgaXQgaXMgYQ0KICAgU0hPVUxELiAg V2UgYWxzbyBlbXBoYXNpemUgdGhhdCB0aGlzIGlzIGEgU0hPVUxELCBtZWFuaW5nIHRoYXQgYW4N CiAgIGltcGxlbWVudG9yIHNob3VsZCBpbmRlZWQgb25seSBkZXZpYXRlIGZyb20gdGhpcyByZXF1 aXJlbWVudCBhZnRlcg0KICAgY2FyZWZ1bCBjb25zaWRlcmF0aW9uIG9mIHRoZSBpbXBsaWNhdGlv bnMuICBTZWUgdGhlIGRpc2N1c3Npb24gb2YNCiAgICJTdHJldGNoIEFDSyB2aW9sYXRpb24iIGlu IFtSRkMyNTI1XSBhbmQgdGhlIHJlZmVyZW5jZXMgdGhlcmVpbiBmb3IgYQ0KICAgZGlzY3Vzc2lv biBvZiB0aGUgcG9zc2libGUgcGVyZm9ybWFuY2UgcHJvYmxlbXMgd2l0aCBnZW5lcmF0aW5nIEFD S3MNCiAgIGxlc3MgZnJlcXVlbnRseSB0aGFuIGV2ZXJ5IHNlY29uZCBmdWxsLXNpemVkIHNlZ21l bnQuDQoNClRodXMsIGFjY29yZGluZyB0byBvdXIgY3VycmVudCBzdGFuZGFyZHMgZGV2aWF0aW9u IGZyb20gdGhpcyBTSE9VTEQgaXMgcG9zc2libGUg4oCdYWZ0ZXIgY2FyZWZ1bCBjb25zaWRlcmF0 aW9uIG9mIHRoZSBpbXBsaWNhdGlvbnPigJ0uDQoNCk1pY2hhZWwNCg0KDQpGcm9tOiBhcW0gW21h aWx0bzphcW0tYm91bmNlc0BpZXRmLm9yZ10gT24gQmVoYWxmIE9mIFl1Y2h1bmcgQ2hlbmcNClNl bnQ6IFRodXJzZGF5LCBPY3RvYmVyIDA4LCAyMDE1IDExOjUyIFBNDQpUbzogRGF2aWQgTGFuZw0K Q2M6IExBVVRFTlNDSExBRUdFUiwgV29sZnJhbSAoV29sZnJhbSk7IEdyZWcgV2hpdGU7IGFxbUBp ZXRmLm9yZzsgSm9lIFRvdWNoDQpTdWJqZWN0OiBSZTogW2FxbV0gVENQIEFDSyBTdXBwcmVzc2lv bg0KDQoNCg0KT24gVGh1LCBPY3QgOCwgMjAxNSBhdCAyOjMxIFBNLCBEYXZpZCBMYW5nIDxkYXZp ZEBsYW5nLmhtPG1haWx0bzpkYXZpZEBsYW5nLmhtPj4gd3JvdGU6DQpPbiBUaHUsIDggT2N0IDIw MTUsIEpvZSBUb3VjaCB3cm90ZToNCk9uIDEwLzcvMjAxNSAxMjo0MiBBTSwgTEFVVEVOU0NITEFF R0VSLCBXb2xmcmFtIChXb2xmcmFtKSB3cm90ZToNCi4uLg0KSXMgdGhpcyB0b3BpYyBhZGRyZXNz ZWQgaW4gc29tZSBSRkMgYWxyZWFkeT8NCg0KSXQncyBhIGRpcmVjdCB2aW9sYXRpb24gb2YgUkZD NzkzLCB3aGljaCBleHBlY3RzIG9uZSBBQ0sgZm9yIGV2ZXJ5IHR3bw0Kc2VnbWVudHM6DQoNCjQu MiBHZW5lcmF0aW5nIEFja25vd2xlZGdtZW50cw0KDQogIFRoZSBkZWxheWVkIEFDSyBhbGdvcml0 aG0gc3BlY2lmaWVkIGluIFtCcmE4OV0gU0hPVUxEIGJlIHVzZWQgYnkgYQ0KICBUQ1AgcmVjZWl2 ZXIuICBXaGVuIHVzZWQsIGEgVENQIHJlY2VpdmVyIE1VU1QgTk9UIGV4Y2Vzc2l2ZWx5IGRlbGF5 DQogIGFja25vd2xlZGdtZW50cy4gIFNwZWNpZmljYWxseSwgYW4gQUNLIFNIT1VMRCBiZSBnZW5l cmF0ZWQgZm9yIGF0DQogIGxlYXN0IGV2ZXJ5IHNlY29uZCBmdWxsLXNpemVkIHNlZ21lbnQsIGFu ZCBNVVNUIGJlIGdlbmVyYXRlZCB3aXRoaW4NCiAgNTAwIG1zIG9mIHRoZSBhcnJpdmFsIG9mIHRo ZSBmaXJzdCB1bmFja25vd2xlZGdlZCBwYWNrZXQuDQoNCmFjdHVhbGx5LCB0aGlzIGlzIG9ubHkg YSB2aW9sYXRpb24gb2YgdGhlIFNIT1VMRCBzZWN0aW9uLCBub3QgdGhlIE1VU1Qgc2VjdGlvbi4N Cg0KQW5kIGlmIHRoZSBBY2sgcGFja2V0cyBhcmUgZ29pbmcgdG8gYXJyaXZlIGF0IHdpcmUtc3Bl ZWQgYW55d2F5IChkdWUgdG8gb3RoZXIgY2F1c2VzKSwgaXMgdGhlcmUgcmVhbGx5IGFuIGFkdmFu dGFnZSB0byBoYXZpbmcgMzIgYWNrIHBhY2tldHMgYXJyaXZpbmcgb25lIGFmdGVyIHRoZSBvdGhl ciBpbnN0ZWFkIG9mIG1ha2luZyBpdCBzbyB0aGF0IHRoZSBmaXJzdCBhY2sgcGFja2V0ICh3aGlj aCBhcnJpdmVzIGF0IHRoZSBzYW1lIHRpbWUpIGNhbiBhY2sgZXZlcnl0aGluZz8NCg0KSSB3b3Vs ZCBsaWtlIHRvIGFkZCB0aGF0IEdSTyBhbHNvIGNhdXNlIHN0cmV0Y2hlZCBBQ0tzIChhY2tpbmcg dXAgdG8gNjRLQiksIGFuZCBHUk8gaXMgY3J1Y2lhbCB0byByZWR1Y2UgY3ljbGVzIGZvciArMTBH YnBzIHRyYW5zZmVycyBvbiBzb21lIGFyY2hpdGVjdHVyZXMuDQoNCg0KQW5kIGlmIHRoZXJlIGlz IHN1Y2ggYW4gYWR2YW50YWdlLCBkb2VzIGl0IG91dHdlaWdodCB0aGUgZGlzYWR2YW50YWdlcyB0 aGF0IHRoZSBleHRyYSBhY2sgcGFja2V0cyBjYXVzZSBieSBjYXVzaW5nIGhpZ2hseSBhc3ltbWV0 cmljIGxpbmtzIHRvIGJlIG92ZXJsb2FkZWQgYW5kIGRyb3AgcGFja2V0cz8NCg0KRGF2aWQgTGFu Zw0KDQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQph cW0gbWFpbGluZyBsaXN0DQphcW1AaWV0Zi5vcmc8bWFpbHRvOmFxbUBpZXRmLm9yZz4NCmh0dHBz Oi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vYXFtDQoNCg== --_000_655C07320163294895BBADA28372AF5D484FCA01FR712WXCHMBA15z_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTIgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m YWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAy IDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpUYWhvbWE7DQoJcGFub3NlLTE6MiAxMSA2 IDQgMyA1IDQgNCAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNvTm9ybWFsLCBs aS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBjbTsNCgltYXJnaW4tYm90dG9t Oi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGltZXMgTmV3IFJv bWFuIiwic2VyaWYiO30NCmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsNCgl7bXNvLXN0eWxlLXBy aW9yaXR5Ojk5Ow0KCWNvbG9yOmJsdWU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQph OnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJsaW5rRm9sbG93ZWQNCgl7bXNvLXN0eWxlLXByaW9yaXR5 Ojk5Ow0KCWNvbG9yOnB1cnBsZTsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lO30NCnNwYW4u aG9lbnpiDQoJe21zby1zdHlsZS1uYW1lOmhvZW56Yjt9DQpzcGFuLkVtYWlsU3R5bGUxOA0KCXtt c28tc3R5bGUtdHlwZTpwZXJzb25hbC1yZXBseTsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsInNh bnMtc2VyaWYiOw0KCWNvbG9yOiMxRjQ5N0Q7fQ0KLk1zb0NocERlZmF1bHQNCgl7bXNvLXN0eWxl LXR5cGU6ZXhwb3J0LW9ubHk7fQ0KQHBhZ2UgV29yZFNlY3Rpb24xDQoJe3NpemU6NjEyLjBwdCA3 OTIuMHB0Ow0KCW1hcmdpbjo3Mi4wcHQgNzIuMHB0IDcyLjBwdCA3Mi4wcHQ7fQ0KZGl2LldvcmRT ZWN0aW9uMQ0KCXtwYWdlOldvcmRTZWN0aW9uMTt9DQotLT48L3N0eWxlPjwhLS1baWYgZ3RlIG1z byA5XT48eG1sPg0KPG86c2hhcGVkZWZhdWx0cyB2OmV4dD0iZWRpdCIgc3BpZG1heD0iMTAyNiIg Lz4NCjwveG1sPjwhW2VuZGlmXS0tPjwhLS1baWYgZ3RlIG1zbyA5XT48eG1sPg0KPG86c2hhcGVs YXlvdXQgdjpleHQ9ImVkaXQiPg0KPG86aWRtYXAgdjpleHQ9ImVkaXQiIGRhdGE9IjEiIC8+DQo8 L286c2hhcGVsYXlvdXQ+PC94bWw+PCFbZW5kaWZdLS0+DQo8L2hlYWQ+DQo8Ym9keSBsYW5nPSJF Ti1VUyIgbGluaz0iYmx1ZSIgdmxpbms9InB1cnBsZSI+DQo8ZGl2IGNsYXNzPSJXb3JkU2VjdGlv bjEiPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7 Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2Nv bG9yOiMxRjQ5N0QiPltSZW1vdmluZyBBUU0sIGFkZGluZyBUQ1BNXTxvOnA+PC9vOnA+PC9zcGFu PjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0 O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztj b2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv Tm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtD YWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+SGkgSm9l LDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxl PSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7 c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48 L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtm b250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29s b3I6IzFGNDk3RCI+SSBkb27igJl0IGZpbmQgdGhpcyB0ZXh0IGluIFJGQyA3OTMsIGJ1dCBpdCBp cyBjb250YWluZWQgaW4gUkZDIDI1ODEsIHdoaWNoIGlzIG9ic29sZXRlIGJ5IFJGQyA1NjgxLjxv OnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJm b250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fu cy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+ DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250 LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6 IzFGNDk3RCI+UkZDIDU2ODEgc3RhdGVzOjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNz PSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZx dW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48 bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBz dHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZx dW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+NC4yLiZuYnNwOyBHZW5lcmF0aW5n IEFja25vd2xlZGdtZW50czxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGli cmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNw OzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9u dC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMt c2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+Jm5ic3A7Jm5ic3A7IFRoZSBkZWxheWVkIEFDSyBh bGdvcml0aG0gc3BlY2lmaWVkIGluIFtSRkMxMTIyXSBTSE9VTEQgYmUgdXNlZCBieSBhPG86cD48 L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQt c2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNl cmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPiZuYnNwOyZuYnNwOyBUQ1AgcmVjZWl2ZXIuJm5ic3A7 IFdoZW4gdXNpbmcgZGVsYXllZCBBQ0tzLCBhIFRDUCByZWNlaXZlciBNVVNUIE5PVDxvOnA+PC9v OnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNp emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJp ZiZxdW90Oztjb2xvcjojMUY0OTdEIj4mbmJzcDsmbmJzcDsgZXhjZXNzaXZlbHkgZGVsYXkgYWNr bm93bGVkZ21lbnRzLiZuYnNwOyBTcGVjaWZpY2FsbHksIGFuIEFDSyBTSE9VTEQgYmU8bzpwPjwv bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1z aXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2Vy aWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+Jm5ic3A7Jm5ic3A7IGdlbmVyYXRlZCBmb3IgYXQgbGVh c3QgZXZlcnkgc2Vjb25kIGZ1bGwtc2l6ZWQgc2VnbWVudCwgYW5kIE1VU1QgYmU8bzpwPjwvbzpw Pjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXpl OjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYm cXVvdDs7Y29sb3I6IzFGNDk3RCI+Jm5ic3A7Jm5ic3A7IGdlbmVyYXRlZCB3aXRoaW4gNTAwIG1z IG9mIHRoZSBhcnJpdmFsIG9mIHRoZSBmaXJzdCB1bmFja25vd2xlZGdlZDxvOnA+PC9vOnA+PC9z cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEu MHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90 Oztjb2xvcjojMUY0OTdEIj4mbmJzcDsmbmJzcDsgcGFja2V0LjxvOnA+PC9vOnA+PC9zcGFuPjwv cD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2Zv bnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xv cjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9y bWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxp YnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+Jm5ic3A7Jm5i c3A7IFRoZSByZXF1aXJlbWVudCB0aGF0IGFuIEFDSyAmcXVvdDtTSE9VTEQmcXVvdDsgYmUgZ2Vu ZXJhdGVkIGZvciBhdCBsZWFzdCBldmVyeTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNz PSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZx dW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj4m bmJzcDsmbmJzcDsgc2Vjb25kIGZ1bGwtc2l6ZWQgc2VnbWVudCBpcyBsaXN0ZWQgaW4gW1JGQzEx MjJdIGluIG9uZSBwbGFjZSBhcyBhPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1z b05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7 Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPiZuYnNw OyZuYnNwOyBTSE9VTEQgYW5kIGFub3RoZXIgYXMgYSBNVVNULiZuYnNwOyBIZXJlIHdlIHVuYW1i aWd1b3VzbHkgc3RhdGUgaXQgaXMgYTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90 O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj4mbmJz cDsmbmJzcDsgU0hPVUxELiZuYnNwOyBXZSBhbHNvIGVtcGhhc2l6ZSB0aGF0IHRoaXMgaXMgYSBT SE9VTEQsIG1lYW5pbmcgdGhhdCBhbjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90 O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj4mbmJz cDsmbmJzcDsgaW1wbGVtZW50b3Igc2hvdWxkIGluZGVlZCBvbmx5IGRldmlhdGUgZnJvbSB0aGlz IHJlcXVpcmVtZW50IGFmdGVyPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2Fs aWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPiZuYnNwOyZu YnNwOyBjYXJlZnVsIGNvbnNpZGVyYXRpb24gb2YgdGhlIGltcGxpY2F0aW9ucy4mbmJzcDsgU2Vl IHRoZSBkaXNjdXNzaW9uIG9mPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2Fs aWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPiZuYnNwOyZu YnNwOyAmcXVvdDtTdHJldGNoIEFDSyB2aW9sYXRpb24mcXVvdDsgaW4gW1JGQzI1MjVdIGFuZCB0 aGUgcmVmZXJlbmNlcyB0aGVyZWluIGZvciBhPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xh c3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6 JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0Qi PiZuYnNwOyZuYnNwOyBkaXNjdXNzaW9uIG9mIHRoZSBwb3NzaWJsZSBwZXJmb3JtYW5jZSBwcm9i bGVtcyB3aXRoIGdlbmVyYXRpbmcgQUNLczxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNz PSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZx dW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj4m bmJzcDsmbmJzcDsgbGVzcyBmcmVxdWVudGx5IHRoYW4gZXZlcnkgc2Vjb25kIGZ1bGwtc2l6ZWQg c2VnbWVudC48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3Bh biBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7 LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48 L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox MS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1 b3Q7O2NvbG9yOiMxRjQ5N0QiPlRodXMsIGFjY29yZGluZyB0byBvdXIgY3VycmVudCBzdGFuZGFy ZHMgZGV2aWF0aW9uIGZyb20gdGhpcyBTSE9VTEQgaXMgcG9zc2libGUg4oCdYWZ0ZXIgY2FyZWZ1 bCBjb25zaWRlcmF0aW9uIG9mIHRoZSBpbXBsaWNhdGlvbnPigJ0uPG86cD48L286cD48L3NwYW4+ PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7 Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2Nv bG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29O b3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0Nh bGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj5NaWNoYWVs PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9 ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtz YW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwv cD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2Zv bnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xv cjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8ZGl2IHN0eWxlPSJib3Jk ZXI6bm9uZTtib3JkZXItdG9wOnNvbGlkICNCNUM0REYgMS4wcHQ7cGFkZGluZzozLjBwdCAwY20g MGNtIDBjbSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXpl OjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtUYWhvbWEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZx dW90OyI+RnJvbTo8L3NwYW4+PC9iPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQt ZmFtaWx5OiZxdW90O1RhaG9tYSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7Ij4gYXFtIFtt YWlsdG86YXFtLWJvdW5jZXNAaWV0Zi5vcmddDQo8Yj5PbiBCZWhhbGYgT2YgPC9iPll1Y2h1bmcg Q2hlbmc8YnI+DQo8Yj5TZW50OjwvYj4gVGh1cnNkYXksIE9jdG9iZXIgMDgsIDIwMTUgMTE6NTIg UE08YnI+DQo8Yj5Ubzo8L2I+IERhdmlkIExhbmc8YnI+DQo8Yj5DYzo8L2I+IExBVVRFTlNDSExB RUdFUiwgV29sZnJhbSAoV29sZnJhbSk7IEdyZWcgV2hpdGU7IGFxbUBpZXRmLm9yZzsgSm9lIFRv dWNoPGJyPg0KPGI+U3ViamVjdDo8L2I+IFJlOiBbYXFtXSBUQ1AgQUNLIFN1cHByZXNzaW9uPG86 cD48L286cD48L3NwYW4+PC9wPg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZu YnNwOzwvbzpwPjwvcD4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwv bzpwPjwvcD4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwv cD4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5PbiBUaHUsIE9jdCA4LCAyMDE1IGF0IDI6 MzEgUE0sIERhdmlkIExhbmcgJmx0OzxhIGhyZWY9Im1haWx0bzpkYXZpZEBsYW5nLmhtIiB0YXJn ZXQ9Il9ibGFuayI+ZGF2aWRAbGFuZy5obTwvYT4mZ3Q7IHdyb3RlOjxvOnA+PC9vOnA+PC9wPg0K PHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1hcmdpbi1ib3R0b206MTIuMHB0Ij5PbiBUaHUs IDggT2N0IDIwMTUsIEpvZSBUb3VjaCB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPk9uIDEwLzcvMjAxNSAxMjo0MiBBTSwgTEFVVEVOU0NITEFFR0VSLCBXb2xmcmFt IChXb2xmcmFtKSB3cm90ZTo8YnI+DQouLi48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29O b3JtYWwiPklzIHRoaXMgdG9waWMgYWRkcmVzc2VkIGluIHNvbWUgUkZDIGFscmVhZHk/PG86cD48 L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48YnI+DQpJdCdzIGEgZGlyZWN0IHZpb2xh dGlvbiBvZiBSRkM3OTMsIHdoaWNoIGV4cGVjdHMgb25lIEFDSyBmb3IgZXZlcnkgdHdvPGJyPg0K c2VnbWVudHM6PGJyPg0KPGJyPg0KNC4yIEdlbmVyYXRpbmcgQWNrbm93bGVkZ21lbnRzPGJyPg0K PGJyPg0KJm5ic3A7IFRoZSBkZWxheWVkIEFDSyBhbGdvcml0aG0gc3BlY2lmaWVkIGluIFtCcmE4 OV0gU0hPVUxEIGJlIHVzZWQgYnkgYTxicj4NCiZuYnNwOyBUQ1AgcmVjZWl2ZXIuJm5ic3A7IFdo ZW4gdXNlZCwgYSBUQ1AgcmVjZWl2ZXIgTVVTVCBOT1QgZXhjZXNzaXZlbHkgZGVsYXk8YnI+DQom bmJzcDsgYWNrbm93bGVkZ21lbnRzLiZuYnNwOyBTcGVjaWZpY2FsbHksIGFuIEFDSyBTSE9VTEQg YmUgZ2VuZXJhdGVkIGZvciBhdDxicj4NCiZuYnNwOyBsZWFzdCBldmVyeSBzZWNvbmQgZnVsbC1z aXplZCBzZWdtZW50LCBhbmQgTVVTVCBiZSBnZW5lcmF0ZWQgd2l0aGluPGJyPg0KJm5ic3A7IDUw MCBtcyBvZiB0aGUgYXJyaXZhbCBvZiB0aGUgZmlyc3QgdW5hY2tub3dsZWRnZWQgcGFja2V0Ljxv OnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PGJyPg0KYWN0dWFsbHksIHRoaXMg aXMgb25seSBhIHZpb2xhdGlvbiBvZiB0aGUgU0hPVUxEIHNlY3Rpb24sIG5vdCB0aGUgTVVTVCBz ZWN0aW9uLjxicj4NCjxicj4NCkFuZCBpZiB0aGUgQWNrIHBhY2tldHMgYXJlIGdvaW5nIHRvIGFy cml2ZSBhdCB3aXJlLXNwZWVkIGFueXdheSAoZHVlIHRvIG90aGVyIGNhdXNlcyksIGlzIHRoZXJl IHJlYWxseSBhbiBhZHZhbnRhZ2UgdG8gaGF2aW5nIDMyIGFjayBwYWNrZXRzIGFycml2aW5nIG9u ZSBhZnRlciB0aGUgb3RoZXIgaW5zdGVhZCBvZiBtYWtpbmcgaXQgc28gdGhhdCB0aGUgZmlyc3Qg YWNrIHBhY2tldCAod2hpY2ggYXJyaXZlcyBhdCB0aGUgc2FtZSB0aW1lKSBjYW4NCiBhY2sgZXZl cnl0aGluZz88bzpwPjwvbzpwPjwvcD4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpw PiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkkg d291bGQgbGlrZSB0byBhZGQgdGhhdCBHUk8gYWxzbyBjYXVzZSBzdHJldGNoZWQgQUNLcyAoYWNr aW5nIHVwIHRvIDY0S0IpLCBhbmQgR1JPIGlzIGNydWNpYWwgdG8gcmVkdWNlIGN5Y2xlcyBmb3Ig JiM0MzsxMEdicHMgdHJhbnNmZXJzIG9uIHNvbWUgYXJjaGl0ZWN0dXJlcy48bzpwPjwvbzpwPjwv cD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+ PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6 c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0 OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PGJyPg0KQW5k IGlmIHRoZXJlIGlzIHN1Y2ggYW4gYWR2YW50YWdlLCBkb2VzIGl0IG91dHdlaWdodCB0aGUgZGlz YWR2YW50YWdlcyB0aGF0IHRoZSBleHRyYSBhY2sgcGFja2V0cyBjYXVzZSBieSBjYXVzaW5nIGhp Z2hseSBhc3ltbWV0cmljIGxpbmtzIHRvIGJlIG92ZXJsb2FkZWQgYW5kIGRyb3AgcGFja2V0cz88 c3BhbiBzdHlsZT0iY29sb3I6Izg4ODg4OCI+PGJyPg0KPGJyPg0KPHNwYW4gY2xhc3M9ImhvZW56 YiI+RGF2aWQgTGFuZzwvc3Bhbj48L3NwYW4+PG86cD48L286cD48L3A+DQo8ZGl2Pg0KPGRpdj4N CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxicj4NCjxicj4NCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fPGJyPg0KYXFtIG1haWxpbmcgbGlzdDxicj4NCjxhIGhy ZWY9Im1haWx0bzphcW1AaWV0Zi5vcmciIHRhcmdldD0iX2JsYW5rIj5hcW1AaWV0Zi5vcmc8L2E+ PGJyPg0KPGEgaHJlZj0iaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9hcW0i IHRhcmdldD0iX2JsYW5rIj5odHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2Fx bTwvYT48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rp dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+DQo8 L2Rpdj4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9odG1sPg0K --_000_655C07320163294895BBADA28372AF5D484FCA01FR712WXCHMBA15z_-- From nobody Fri Oct 9 06:44:44 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 150AA1B3E81 for ; Fri, 9 Oct 2015 06:44:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7sdQtElnpL7M for ; Fri, 9 Oct 2015 06:44:41 -0700 (PDT) Received: from atl4mhob12.myregisteredsite.com (atl4mhob12.myregisteredsite.com [209.17.115.50]) by ietfa.amsl.com (Postfix) with ESMTP id AE73F1B3E71 for ; Fri, 9 Oct 2015 06:44:41 -0700 (PDT) Received: from mailpod.hostingplatform.com ([10.30.71.207]) by atl4mhob12.myregisteredsite.com (8.14.4/8.14.4) with ESMTP id t99Die4W013146 for ; Fri, 9 Oct 2015 09:44:40 -0400 Received: (qmail 10293 invoked by uid 0); 9 Oct 2015 13:44:40 -0000 X-TCPREMOTEIP: 69.81.157.131 X-Authenticated-UID: wes@mti-systems.com Received: from unknown (HELO ?192.168.1.103?) (wes@mti-systems.com@69.81.157.131) by 0 with ESMTPA; 9 Oct 2015 13:44:40 -0000 To: "Scharf, Michael (Michael)" , Joe Touch References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> From: Wesley Eddy Organization: MTI Systems Message-ID: <5617C4BF.6040504@mti-systems.com> Date: Fri, 9 Oct 2015 09:44:31 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Greg White , "tcpm@ietf.org" , David Lang Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 13:44:43 -0000 On 10/9/2015 5:21 AM, Scharf, Michael (Michael) wrote: > [Removing AQM, adding TCPM] > > > > Hi Joe, > > > > I dont find this text in RFC 793, but it is contained in RFC 2581, > which is obsolete by RFC 5681. > > To confuse matters further, there's also 1122 text with requirements language: A TCP SHOULD implement a delayed ACK, but an ACK should not be excessively delayed; in particular, the delay MUST be less than 0.5 seconds, and in a stream of full-sized segments there SHOULD be an ACK for at least every second segment. This probably belongs in 793bis. In my view, normative acknowledgement behavior belongs in the base spec rather than particular congestion control descriptions, since there may be more than CC that depends upon it or makes assumptions about it. -- Wes Eddy MTI Systems From nobody Fri Oct 9 08:21:33 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B998E1A6FFF for ; Fri, 9 Oct 2015 08:21:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lOmWjuCEiHws for ; Fri, 9 Oct 2015 08:21:28 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EDE9C1A8726 for ; Fri, 9 Oct 2015 08:21:27 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (unknown [135.239.2.42]) by Websense Email Security Gateway with ESMTPS id 0BB8B326FB49E; Fri, 9 Oct 2015 15:21:23 +0000 (GMT) Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id t99FLORe010009 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 9 Oct 2015 17:21:24 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.03.0195.001; Fri, 9 Oct 2015 17:21:24 +0200 From: "Scharf, Michael (Michael)" To: Wesley Eddy , Joe Touch Thread-Topic: [tcpm] [aqm] TCP ACK Suppression Thread-Index: AQHRAAfbmdqr5Xk9lE2VWzCubgsRu55eiGYAgAD9UYCAAnWAgIAABLaAgAAFoICAANvX0IAALkSAgAA4eTA= Date: Fri, 9 Oct 2015 15:21:23 +0000 Message-ID: <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617C4BF.6040504@mti-systems.com> In-Reply-To: <5617C4BF.6040504@mti-systems.com> Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Greg White , "tcpm@ietf.org" , David Lang Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 15:21:31 -0000 Yes, I have realized today myself that 793bis lacks that section. I agree t= hat ACK generation belongs into 793bis. 793bis actually may open an opportunity to rethink the exact wording that e= xplains this SHOULD. Personally, I believe replacing this SHOULD is not an = option for a bis document. But if deviations from that SHOULD are widely deployed in the meantime, 793= bis could possibly comment on that. Also, having some informational referen= ce could perhaps be an option.=20 Michael -----Original Message----- From: Wesley Eddy [mailto:wes@mti-systems.com]=20 Sent: Friday, October 09, 2015 3:45 PM To: Scharf, Michael (Michael); Joe Touch Cc: LAUTENSCHLAEGER, Wolfram (Wolfram); Greg White; tcpm@ietf.org; David La= ng Subject: Re: [tcpm] [aqm] TCP ACK Suppression On 10/9/2015 5:21 AM, Scharf, Michael (Michael) wrote: > [Removing AQM, adding TCPM] >=20 > =20 >=20 > Hi Joe, >=20 > =20 >=20 > I don't find this text in RFC 793, but it is contained in RFC 2581,=20 > which is obsolete by RFC 5681. >=20 > =20 To confuse matters further, there's also 1122 text with requirements language: A TCP SHOULD implement a delayed ACK, but an ACK should not be excessively delayed; in particular, the delay MUST be less than 0.5 seconds, and in a stream of full-sized segments there SHOULD be an ACK for at least every second segment. This probably belongs in 793bis. In my view, normative acknowledgement beh= avior belongs in the base spec rather than particular congestion control de= scriptions, since there may be more than CC that depends upon it or makes a= ssumptions about it. -- Wes Eddy MTI Systems From nobody Fri Oct 9 10:21:51 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 24F111B48EF for ; Fri, 9 Oct 2015 10:21:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -114.51 X-Spam-Level: X-Spam-Status: No, score=-114.51 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eHrmggzlG2t4 for ; Fri, 9 Oct 2015 10:21:46 -0700 (PDT) Received: from rcdn-iport-8.cisco.com (rcdn-iport-8.cisco.com [173.37.86.79]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A54AF1B48EB for ; Fri, 9 Oct 2015 10:21:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=18079; q=dns/txt; s=iport; t=1444411306; x=1445620906; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=RFSwbh6w37BMHQyWxlwmeopnDHf6ZM9+Ha62YkU/NSI=; b=QT4fz8QF0682IhQS+sbn1ifyjRM404D7gEgoT/Qckkpz32b24r003S3o TsoMVok99OakkuHXWztEHSQ9GW2L9E9iu83JR9uvRy3zMoi1IYJyZqCty 0Xjm6v9Gjlscn749vtELGfYdxlEPgpbLCs0eRm+AxRlKSUuQBCffGRF90 8=; X-Files: signature.asc : 833 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0A3BgCf9hdW/4MNJK1egllNVG4GrDGCTI5xgVoXAQmCcoIKfwKBRjkTAQEBAQEBAYEKhCYBAQEDAQEBASBLCwULAgEGAg4DBAEBAScDAgIhBgsTAQkIAgQOBQ6ICwMKCA2SKZ02jmkNhR8BAQEBAQEBAQEBAQEBAQEBAQEBAQETBIkDgm6CUIISASoGAQaCYzGBFAEEhgMMkAMBgk2BYYZ3BoFugViHXosAh0gBIwI+hAJxhmSBBgEBAQ X-IronPort-AV: E=Sophos;i="5.17,658,1437436800"; d="asc'?scan'208,217";a="34265201" Received: from alln-core-1.cisco.com ([173.36.13.131]) by rcdn-iport-8.cisco.com with ESMTP; 09 Oct 2015 17:21:45 +0000 Received: from XCH-ALN-015.cisco.com (xch-aln-015.cisco.com [173.36.7.25]) by alln-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id t99HLjGR026813 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Fri, 9 Oct 2015 17:21:45 GMT Received: from xch-aln-013.cisco.com (173.36.7.23) by XCH-ALN-015.cisco.com (173.36.7.25) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Fri, 9 Oct 2015 12:21:42 -0500 Received: from xch-aln-013.cisco.com ([173.36.7.23]) by XCH-ALN-013.cisco.com ([173.36.7.23]) with mapi id 15.00.1104.000; Fri, 9 Oct 2015 12:21:41 -0500 From: "Fred Baker (fred)" To: Daniel Havey Thread-Topic: [tcpm] ACK Suppression Thread-Index: AQHRArZnkv35z1pT90iz8FgGSEswmp5jvHQA Date: Fri, 9 Oct 2015 17:21:41 +0000 Message-ID: <1A2199DA-6638-4E0A-A71B-4E376ECA0EC0@cisco.com> References: <1381697972.1428458.1444411039879.JavaMail.yahoo@mail.yahoo.com> In-Reply-To: <1381697972.1428458.1444411039879.JavaMail.yahoo@mail.yahoo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.19.64.119] Content-Type: multipart/signed; boundary="Apple-Mail=_CD02012E-7FA2-4FF8-9C87-CD9C26E49810"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: Cc: David Lang , "dpreed@reed.com" , "tcpm@ietf.org" , Jonathan Morton Subject: Re: [tcpm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 17:21:49 -0000 --Apple-Mail=_CD02012E-7FA2-4FF8-9C87-CD9C26E49810 Content-Type: multipart/alternative; boundary="Apple-Mail=_A808343A-E563-4062-A3BD-748B6A9BC2B4" --Apple-Mail=_A808343A-E563-4062-A3BD-748B6A9BC2B4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Oct 9, 2015, at 10:17 AM, Daniel Havey wrote: >=20 > This sounds like an awful idea. What if there were congestion marks = in those ACKs? What if the ACK has data in it? ECN is an issue. I would expect a piggy-backed ack is treated as a data = message. > From: "gorry@erg.abdn.ac.uk" > To: Fred Baker (fred) > Cc: David Lang ; Jonathan Morton = ; "dpreed@reed.com" ; = "tcpm@ietf.org" ; "aqm@ietf.org" > Sent: Thursday, October 8, 2015 8:03 PM > Subject: Re: [aqm] [tcpm] ACK Suppression >=20 > RFC3449 pointed to some of the reasons & problems with manipulating = TCP > ACKs, and could provide a useful background if people needed to get up = to > speed on the history... >=20 > Gorry >=20 > > I'm not sure why this discussion is happening on aqm@ instead of = tcpm@... > > I have added cpm to the cc line, and would recommend that anyone > > responding to this thread do the same and remove aqm@. = > > > > On Oct 7, 2015, at 2:13 PM, David Lang > wrote: > >> So things that reduce the flow of acks can result in very real = benefits > >> to users. > > > > Dumb question of the month. What would it take to see wide = deployment of > > RFC 5690? That would result in the data/ack ratio being reduced, on > > average, to whatever amount had been negotiated. > > > > Summary for those that haven't read it - TCP implementations today > > generally ack every other packet, with caveats for isolated or final = data > > packets. This proposal allows consenting adults to change that = ratio, > > acking every third or fourth packet, or every tenth. > > > > If ack reductions are so very valuable, what's the chance of doing = that on > > an end to end basis instead of in the network? > > > >> On Oct 7, 2015, at 2:13 PM, David Lang > wrote: > >> On Wed, 7 Oct 2015, Jonathan Morton wrote: > >>>> On 7 Oct, 2015, at 23:40, Agarwal, Anil > > >>>> wrote: > >>>>> Since the cable modem link will lead to clumped ACKs the = difference > >>>>> between sending 100 ACKs vs. 1 ACK is probably not that big... > >>>>> (except w.r.t. reliability). > >>>> The difference may not be big in the spacing of new packets that = a > >>>> sender will send, unless the sender implements some sort of = pacing or > >>>> if the return link is very thin. > >>> > >>>> But with ABC, there will be a difference in the amount of cwnd > >>>> increase at the sender. > >>> > >>> There is also a potential difference for detecting packet loss in = the > >>> forward direction. It=C3=A2=E2=82=AC=E2=84=A2s entirely possible = that thinning would cause > >>> a DupAck condition to be recognised only after three MAC grants in = the > >>> reverse direction have elapsed, rather than one. Receivers are > >>> REQUIRED to send an ack for every received packet under these > >>> conditions, but that would be subverted by the modem. AckCC would = not > >>> induce this effect, because the receiver would still produce the = extra > >>> acks as required. > >>> > >>> Packet loss causes head-of-line blocking at the application level, > >>> which is perceived as latency and jerkiness by the end-user, until = the > >>> lost packet is retransmitted and actually arrives. Hence the = addition > >>> of two MAC grant delays (60ms?) may make the difference between an > >>> imperceptible problem and a noticeable one. > >> > >> and excessive ack traffic causes congestion and results in packet = loss > >> on real-world hightly asymmetric links. > >> > >> So things that reduce the flow of acks can result in very real = benefits > >> to users. > >> > >> David Lang_______________________________________________ > >> aqm mailing list > >> aqm@ietf.org > >> https://www.ietf.org/mailman/listinfo/aqm = > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm = >=20 >=20 >=20 > > >=20 >=20 > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm = >=20 --Apple-Mail=_A808343A-E563-4062-A3BD-748B6A9BC2B4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On Oct 9, 2015, at 10:17 AM, Daniel Havey <dhavey@yahoo.com> = wrote:

This sounds like an = awful idea.  What if there were congestion marks in those = ACKs?  What if the ACK has data in = it?

ECN is an issue. I = would expect a piggy-backed ack is treated as a data message.

From: "gorry@erg.abdn.ac.uk" = <gorry@erg.abdn.ac.uk>
To: Fred Baker = (fred) <fred@cisco.com>
Cc: David Lang <david@lang.hm>; Jonathan = Morton <chromatix99@gmail.com>; "dpreed@reed.com" <dpreed@reed.com>; "tcpm@ietf.org" <tcpm@ietf.org>; "aqm@ietf.org" <aqm@ietf.org>
Sent: Thursday, October 8, 2015 8:03 PM
Subject: Re: [aqm] [tcpm] ACK Suppression

RFC3449 pointed to some of the = reasons & problems with manipulating TCP
ACKs, and could provide a useful background if people needed = to get up to
speed on the history...

Gorry

> I'm not = sure why this discussion is happening on aqm@ instead of tcpm@...
> I have added = cpm to the cc line, and would recommend that anyone
> responding to this thread do the same and remove aqm@.
>
> On Oct 7, 2015, at 2:13 PM, David Lang <david@lang.hm> wrote:
>> So things that reduce the flow of = acks can result in very real benefits
>> to users.
>
> Dumb question of the month. What would it = take to see wide deployment of
> RFC = 5690? That would result in the data/ack ratio being reduced, on
> average, to whatever amount had been = negotiated.
>
> Summary for those that haven't read it - TCP = implementations today
> generally ack = every other packet, with caveats for isolated or final data
> packets. This proposal allows consenting = adults to change that ratio,
> acking = every third or fourth packet, or every tenth.
>
> If ack reductions are = so very valuable, what's the chance of doing that on
> an end to end basis instead of in the network?
>
>> On = Oct 7, 2015, at 2:13 PM, David Lang <david@lang.hm> wrote:
>> On Wed, 7 Oct 2015, Jonathan Morton wrote:
>>>> On 7 Oct, 2015, at 23:40, = Agarwal, Anil <Anil.Agarwal@viasat.com>
>>>> wrote:
>>>>> Since the cable modem link will lead to = clumped ACKs the difference
>>>>> between sending 100 ACKs vs. 1 ACK is = probably not that big...
>>>>>= (except w.r.t. reliability).
>>>> The difference may not be big in the spacing = of new packets that a
>>>> = sender will send, unless the sender implements some sort of pacing or
>>>> if the return link is very = thin.
>>>
>>>> But with ABC, there will be a difference in = the amount of cwnd
>>>> = increase at the sender.
>>>
>>> There is also a potential = difference for detecting packet loss in the
>>> forward direction.  It=C3=A2=E2=82=AC=E2=84=A2= s entirely possible that thinning would cause
>>> a DupAck condition to be recognised only after = three MAC grants in the
>>> = reverse direction have elapsed, rather than one.  Receivers are
>>> REQUIRED to send an ack for every = received packet under these
>>> = conditions, but that would be subverted by the modem.  AckCC would = not
>>> induce this effect, = because the receiver would still produce the extra
>>> acks as required.
>>>
>>> Packet = loss causes head-of-line blocking at the application level,
>>> which is perceived as latency and = jerkiness by the end-user, until the
>>> lost packet is retransmitted and actually = arrives.  Hence the addition
>>> of two MAC grant delays (60ms?) may make the = difference between an
>>> = imperceptible problem and a noticeable one.
>>
>> and excessive = ack traffic causes congestion and results in packet loss
>> on real-world hightly asymmetric links.
>>
>> = So things that reduce the flow of acks can result in very real = benefits
>> to users.
>>
>> David = Lang_______________________________________________
>> aqm mailing list
>>= aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>
> = _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm



>


_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm


=

= --Apple-Mail=_A808343A-E563-4062-A3BD-748B6A9BC2B4-- --Apple-Mail=_CD02012E-7FA2-4FF8-9C87-CD9C26E49810 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQIVAwUBVhf3w0ayAOS/EQ8MAQIMqxAAjkb9wQNzbqI+3gyhDPUugFq2VmcAVou/ IVDUxtZ8yfp+/f8ifIkeJdmoKCq1dH2165lVSXr62VoZhYNnyODnmZ14EYciUsS4 7ddcGre/CgXox/cSAKByCb+YtR2x3ftOARc7gs518JtoUdlypYheaGw61v/2idky JNcRFVFfdFdB78gj2d60ohUWDC8/PYu9/iq/zGBuJyLEJZ/IhzLL62OVRshP8n2o 8eoXO44SUNtvDfSXEXZXvxIAWYwtxXqVvCyVWrT3jrvj6oWfXHRuFfSAJ7Iljmip 3FTMP3L/95R71up98iZkSoYCyGBOoTVsQSHWJZvEmnYL+KmqqQEnFDIORd6fy7G9 9vazf3om0nqC7aPGU+YjHjJn7IzSxRom1eUwtwPLqhxqE6NKrBGANNTznP6OOYYj Rgevr5J4hV0MQyhW5sGyyjgmO9UkwS7ItXP4h8RfstqVJ721JtISM83f0AUyVLxt MUXpa0tJ8r85tjcNNmGrEefRCjE5/k1bcI97Y9UYLjDmYlMH2D0EfD1SPMl1VuFr 8MSM8WqFLZGHGud7KsnznD8NL7u5D2SdGEd1Bsusp3oi5HiThmYfpW0aOewdkuZ0 1HPh8H8OtgtbRh8KBLhBuctyzeuJL5MGudKdSZmFflC5npf0Ag/rVyGR4W8KQHjQ Uv7OnD7pKBk= =4r07 -----END PGP SIGNATURE----- --Apple-Mail=_CD02012E-7FA2-4FF8-9C87-CD9C26E49810-- From nobody Fri Oct 9 10:29:58 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C77811B490F for ; Fri, 9 Oct 2015 10:29:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UEW_V40n-lFh for ; Fri, 9 Oct 2015 10:29:55 -0700 (PDT) Received: from fruitcake.ICSI.Berkeley.EDU (fruitcake.ICSI.Berkeley.EDU [192.150.186.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2D0A11B4907 for ; Fri, 9 Oct 2015 10:29:54 -0700 (PDT) Received: from lawyers.icir.org (envoy.icir.org [192.150.187.30]) by fruitcake.ICSI.Berkeley.EDU (8.12.11.20060614/8.12.11) with ESMTP id t99HTfT0008033; Fri, 9 Oct 2015 10:29:42 -0700 (PDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id 545212D6D8F2; Fri, 9 Oct 2015 13:29:40 -0400 (EDT) To: "Fred Baker \(fred\)" From: Mark Allman In-Reply-To: <1A2199DA-6638-4E0A-A71B-4E376ECA0EC0@cisco.com> Organization: International Computer Science Institute (ICSI) Song-of-the-Day: Down on the Corner X-URL-0: http://www.icir.org/mallman-files/Document39284.doc X-URL-1: http://www.icir.org/mallman-files/Document10720.html X-URL-2: http://www.icir.org/mallman-files/Document98322.pdf MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-------459435943823349593450"; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Fri, 09 Oct 2015 13:29:40 -0400 Message-ID: <43116.1444411780@lawyers.icir.org> Sender: mallman@icir.org Archived-At: Cc: David Lang , Jonathan Morton , "dpreed@reed.com" , "tcpm@ietf.org" , Daniel Havey Subject: Re: [tcpm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: mallman@icir.org List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 17:29:57 -0000 --=-------459435943823349593450 Content-Type: text/plain Content-Transfer-Encoding: quoted-printable I am not following along much, but why would that stop me?! :) > This sounds like an awful idea. What if there were congestion > marks in those ACKs? What if the ACK has data in it? >=20 > ECN is an issue. I would expect a piggy-backed ack is treated as a > data message. (1) The point of delayed ACKs is to piggyback ACKs on data packets. So, when a host has data to send and would naturally do so then the ACK will get sent out with that data. (2) At least RFC 5681 (and previous versions) have been careful to note that when there is a hole in the sequence space of the arriving packets that ACKs should not be delayed. I.e., duplicate ACKs or ACKs with SACK information should be sent immediately as they are a possible indication of congestion. (3) I would think the spirit of (2) would also apply to ECN, but I am not sure that is written down anywhere (perhaps the ECN spec?). I.e., reflecting congestion marks should not be delayed. allman =2D- http://www.icir.org/mallman/ --=-------459435943823349593450 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlYX+YEACgkQWyrrWs4yIs62PgCfQncOlsjp/MczvlrysjhL4TmL oMQAoJ0gOCpEmTAvCwbOFNhTfx6FHsPZ =1rGU -----END PGP SIGNATURE----- --=-------459435943823349593450-- From nobody Fri Oct 9 10:34:16 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 46D811A92E3 for ; Fri, 9 Oct 2015 10:34:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KcFhBvDKQgpQ for ; Fri, 9 Oct 2015 10:34:14 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 592F31A92DD for ; Fri, 9 Oct 2015 10:34:14 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t99HWouL028475 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 10:32:51 -0700 (PDT) To: "Scharf, Michael (Michael)" References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> From: Joe Touch Message-ID: <5617FA42.6060306@isi.edu> Date: Fri, 9 Oct 2015 10:32:50 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: David Lang , "tcpm@ietf.org" , touch@isi.edu, "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Greg White Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 17:34:15 -0000 On 10/9/2015 2:21 AM, Scharf, Michael (Michael) wrote: > [Removing AQM, adding TCPM] > > > > Hi Joe, > > > > I don’t find this text in RFC 793, but it is contained in RFC 2581, > which is obsolete by RFC 5681. I think I pasted it from 1122, but had a tab open to 793 and mistook it for being there as well. ... > 4.2. Generating Acknowledgments > > The delayed ACK algorithm specified in [RFC1122] SHOULD be used by a > TCP receiver. When using delayed ACKs, a TCP receiver MUST NOT > excessively delay acknowledgments. Specifically, an ACK SHOULD be > generated for at least every second full-sized segment, and MUST be > generated within 500 ms of the arrival of the first unacknowledged > packet. ... > Thus, according to our current standards deviation from this SHOULD is > possible ”after careful consideration of the implications”. Agreed. And "careful consideration" does not consist of "I deployed it silently, didn't track what happened, and didn't hear whether anyone had any problems I wasn't listening for". Joe From nobody Fri Oct 9 10:35:14 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A2AAA1A92E3 for ; Fri, 9 Oct 2015 10:35:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P2Y1wt3DSDkv for ; Fri, 9 Oct 2015 10:35:11 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C4D651A92AB for ; Fri, 9 Oct 2015 10:35:11 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t99HYbd9028840 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 10:34:37 -0700 (PDT) To: "Scharf, Michael (Michael)" , Wesley Eddy References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617C4BF.6040504@mti-systems.com> <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> From: Joe Touch Message-ID: <5617FAAD.1050404@isi.edu> Date: Fri, 9 Oct 2015 10:34:37 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Greg White , "tcpm@ietf.org" , David Lang , touch@isi.edu Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 17:35:12 -0000 On 10/9/2015 8:21 AM, Scharf, Michael (Michael) wrote: > 793bis actually may open an opportunity to rethink the exact wording > that explains this SHOULD. Personally, I believe replacing this > SHOULD is not an option for a bis document. Yes, and as I noted, any SHOULD needs to include an explanation of the kind of situation that was prevented picking MUST or the conditions under which the SHOULD might be ignored. Otherwise, SHOULD becomes equivalent to MAY, or - in the current environment - "MIGHT, UNTIL IT INTERFERES WITH MY REVENUE MODEL". Joe From nobody Fri Oct 9 10:36:41 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F06601AC3CF for ; Fri, 9 Oct 2015 10:36:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -114.511 X-Spam-Level: X-Spam-Status: No, score=-114.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 53G38ECiLevx for ; Fri, 9 Oct 2015 10:36:38 -0700 (PDT) Received: from rcdn-iport-4.cisco.com (rcdn-iport-4.cisco.com [173.37.86.75]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6FFAA1A92FE for ; Fri, 9 Oct 2015 10:36:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3016; q=dns/txt; s=iport; t=1444412198; x=1445621798; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=OKIw9YlOxji6jXY8xr26SF+daTCU9sDsuzkMLW1japM=; b=TE+itEQ1qoeFdfC0nZomD3WY70mcpUmdcw4bBM6240WW2oyNk39INpI5 u1j0Ia7XZkTvXnxR7o8o2KE4cpQDBO8Z8voyH0b645uFXbE/CbmxLb8CL yUm3njLkgWtREPOgwlBmPo3P10akyJwjYNWhYr02w7VkH1o4nMupjCYPj 0=; X-Files: signature.asc : 833 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0BMAwDK+hdW/5ldJa1egyaBQga9YA6BWoMTggp/AoFGOBQBAQEBAQEBfwuEJgEBAQMBeQULAgEIGC4yJQIEDgUOiBgIxAcBAQEBAQEBAQEBAQEBAQEBAQEBAQEXiQOCboUNB4MagRQFlhIBgk2BYYhrm34BHwFDhAJxhmSBBgEBAQ X-IronPort-AV: E=Sophos;i="5.17,658,1437436800"; d="asc'?scan'208";a="36075206" Received: from rcdn-core-2.cisco.com ([173.37.93.153]) by rcdn-iport-4.cisco.com with ESMTP; 09 Oct 2015 17:36:37 +0000 Received: from XCH-RCD-014.cisco.com (xch-rcd-014.cisco.com [173.37.102.24]) by rcdn-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id t99Hab8M009074 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Fri, 9 Oct 2015 17:36:37 GMT Received: from xch-aln-013.cisco.com (173.36.7.23) by XCH-RCD-014.cisco.com (173.37.102.24) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Fri, 9 Oct 2015 12:36:33 -0500 Received: from xch-aln-013.cisco.com ([173.36.7.23]) by XCH-ALN-013.cisco.com ([173.36.7.23]) with mapi id 15.00.1104.000; Fri, 9 Oct 2015 12:36:33 -0500 From: "Fred Baker (fred)" To: "mallman@icir.org" Thread-Topic: [tcpm] ACK Suppression Thread-Index: AQHRArgVbKKvNxvPckWQZdjW4Lw4gJ5jwJMA Date: Fri, 9 Oct 2015 17:36:33 +0000 Message-ID: References: <43116.1444411780@lawyers.icir.org> In-Reply-To: <43116.1444411780@lawyers.icir.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.19.64.119] Content-Type: multipart/signed; boundary="Apple-Mail=_AE4E22F1-B27F-4F71-B168-7FC184679100"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Archived-At: Cc: David Lang , Jonathan Morton , "dpreed@reed.com" , "tcpm@ietf.org" , Daniel Havey Subject: Re: [tcpm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 17:36:40 -0000 --Apple-Mail=_AE4E22F1-B27F-4F71-B168-7FC184679100 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Oct 9, 2015, at 10:29 AM, mallman@icir.org wrote: >=20 >=20 > I am not following along much, but why would that stop me?! :) >=20 >> This sounds like an awful idea. What if there were congestion >> marks in those ACKs? What if the ACK has data in it? >>=20 >> ECN is an issue. I would expect a piggy-backed ack is treated as a >> data message. >=20 > (1) The point of delayed ACKs is to piggyback ACKs on data packets. > So, when a host has data to send and would naturally do so then > the ACK will get sent out with that data. >=20 > (2) At least RFC 5681 (and previous versions) have been careful to > note that when there is a hole in the sequence space of the > arriving packets that ACKs should not be delayed. I.e., > duplicate ACKs or ACKs with SACK information should be sent > immediately as they are a possible indication of congestion. >=20 > (3) I would think the spirit of (2) would also apply to ECN, but I > am not sure that is written down anywhere (perhaps the ECN > spec?). I.e., reflecting congestion marks should not be > delayed. I think the question here relates to DOCSIS specifications, which = basically encourage Cable Modems to separately queue acks from data = traffic, and delete an older untranmsitted ack from the queue when a = subsuming ack is enqueued. When the modem gets a transmission grant, it = sends all of the data it has but, as a result, a potentially-small = subset of the acks. The intended effect is to change the data:ack ratio = from 2:1 to many:1 in high transmission rate contexts. I'm not convinced that the two algorithms mesh perfectly. = (understatement) --Apple-Mail=_AE4E22F1-B27F-4F71-B168-7FC184679100 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQIVAwUBVhf7OkayAOS/EQ8MAQJX/A//WixcVj4HMHINXb8UpHgHWQcUN9slkUSm nFH/otOP4rywacyzzZFrv6rsHtRd39mSaP5u1OHMfTSqsvGr8t7K4i/kRmIl7kKi YZKLAyC4AoDORaCqhxyuIoo0re3hOBtIwlgi2Fpk2RPNSAUhaM/XosSK31W2o36R n5OZRLhUsZoJQeW2lf4f1y0AepFjaoug075f/yl9AM565W7Paz7uSGoGxWawdHIV wKelIMmY2VpVm9OkaAYpulluKq2tmOmymjzpesOuUMkRjoajgbVuPb7FNthNwHby ADQAPi+y4AbbgaPfBEah0LeWI9bKkR06wLXPMpoYcQDaLNJQrn8nmVSXOuHyAyVs Z1pS8SrFxsQsywLvPCdwJ/LNZP9OhF8eLiXvASn9szcRvvlYJFKsrIuAxWCYQOgS pQHdMuJWmpQtabbqR5S1v6TZ59ZX5dMw4Dwap5lRV8caot4IR/ZwWhOtMq3N4e7r 9+w6mO/+ZzCWR725mlrExXoziCxJpP9xYU9AaYVxt6PapgqHaky0jpiUJ/9msl2X hnFpLAk2tokLKGbAltXn9lzlranZUgmkTGahTp4XShR+N7o2hotIXRuZWwjNOjc6 oWyCTIx2gKABSJp/RNEs22JCCu5kny4GJH5CRRHw0NskusP6/8Faax6jLqj4Bbme Utj4yblSl1c= =GhX/ -----END PGP SIGNATURE----- --Apple-Mail=_AE4E22F1-B27F-4F71-B168-7FC184679100-- From nobody Fri Oct 9 10:59:32 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4CE921B2C30 for ; Fri, 9 Oct 2015 10:59:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.611 X-Spam-Level: X-Spam-Status: No, score=-2.611 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PNrjY5OWwfJ8 for ; Fri, 9 Oct 2015 10:59:29 -0700 (PDT) Received: from mta-us-west-01.viasat.com (mta-us-west-01.viasat.com [8.37.96.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5F6D91B2C1E for ; Fri, 9 Oct 2015 10:59:29 -0700 (PDT) Received: from pps.filterd (VCASPAM01.hq.corp.viasat.com [127.0.0.1]) by VCASPAM01.hq.corp.viasat.com (8.15.0.59/8.15.0.59) with SMTP id t99HuqKh021788; Fri, 9 Oct 2015 17:59:25 GMT From: "Agarwal, Anil" To: "Fred Baker (fred)" , "mallman@icir.org" Thread-Topic: [tcpm] ACK Suppression Thread-Index: AQHRArgTkv35z1pT90iz8FgGSEswmp5j0TmA//+eFhA= Date: Fri, 9 Oct 2015 17:59:24 +0000 Message-ID: <7A2801D5E40DD64A85E38DF22117852C88129F58@wdc1exchmbxp05.hq.corp.viasat.com> References: <43116.1444411780@lawyers.icir.org> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2015-10-09_13:, , signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 kscore.is_bulkscore=0 kscore.compositescore=1 compositescore=0.9 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 kscore.is_spamscore=0 rbsscore=0.9 spamscore=0 urlsuspectscore=0.9 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1507310000 definitions=main-1510090221 Archived-At: Cc: David Lang , Jonathan Morton , "dpreed@reed.com" , "tcpm@ietf.org" , Daniel Havey Subject: Re: [tcpm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 17:59:31 -0000 I presume, that "AKC Suppression", assuming it is a good thing to do, requi= res some rules for - Handling multiple ACK segments some of which may contain data, timestamps,= TCP-ECN info,=20 FIN, SACK, different receive window sizes, AO, ... Handling multiple ACK segments interspersed with large data packets of oth= er TCP connections, in the same queue. Presumably, the DOCSIS folks have designed such rules. And the performance implications of doing so, besides the obvious benefit o= f reducing return-link congestion. Regards, Anil -----Original Message----- From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Fred Baker (fred) Sent: Friday, October 09, 2015 1:37 PM To: mallman@icir.org Cc: David Lang; Jonathan Morton; dpreed@reed.com; tcpm@ietf.org; Daniel Hav= ey Subject: Re: [tcpm] ACK Suppression > On Oct 9, 2015, at 10:29 AM, mallman@icir.org wrote: >=20 >=20 > I am not following along much, but why would that stop me?! :) >=20 >> This sounds like an awful idea. What if there were congestion >> marks in those ACKs? What if the ACK has data in it? >>=20 >> ECN is an issue. I would expect a piggy-backed ack is treated as a=20 >> data message. >=20 > (1) The point of delayed ACKs is to piggyback ACKs on data packets. > So, when a host has data to send and would naturally do so then > the ACK will get sent out with that data. >=20 > (2) At least RFC 5681 (and previous versions) have been careful to > note that when there is a hole in the sequence space of the > arriving packets that ACKs should not be delayed. I.e., > duplicate ACKs or ACKs with SACK information should be sent > immediately as they are a possible indication of congestion. >=20 > (3) I would think the spirit of (2) would also apply to ECN, but I > am not sure that is written down anywhere (perhaps the ECN > spec?). I.e., reflecting congestion marks should not be > delayed. I think the question here relates to DOCSIS specifications, which basically= encourage Cable Modems to separately queue acks from data traffic, and del= ete an older untranmsitted ack from the queue when a subsuming ack is enque= ued. When the modem gets a transmission grant, it sends all of the data it = has but, as a result, a potentially-small subset of the acks. The intended = effect is to change the data:ack ratio from 2:1 to many:1 in high transmiss= ion rate contexts. I'm not convinced that the two algorithms mesh perfectly. (understatement) From nobody Fri Oct 9 10:59:38 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E1C951B2C3C; Fri, 9 Oct 2015 10:59:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cW1-Dag_9N_x; Fri, 9 Oct 2015 10:59:32 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 050D51B2C1E; Fri, 9 Oct 2015 10:59:32 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t99HwpN0005976 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 10:58:52 -0700 (PDT) To: David Lang References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <5616E42D.5090402@isi.edu> <5616FAFA.5020707@isi.edu> From: Joe Touch X-Enigmail-Draft-Status: N1110 Message-ID: <5618005A.8070303@isi.edu> Date: Fri, 9 Oct 2015 10:58:50 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Greg White , "tcpm@ietf.org" , "aqm@ietf.org" , touch@isi.edu Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 17:59:33 -0000 Dave in particular wanted some specific reasons this is a bad idea as presented. Here is my summary. The rest of this discussion should happen on TCPM. Joe --- 1) *you* shouldn't be using a mechanism that destroys information for others whether the timestamps or ACK stream spacing has any meaning is for the receiver - not you - to decide 2) *you* don't know where your mechanism will have an impact those clumped ACKs might be spaced out further downstream even if you have a rule of "I'll only gather 3 ACKs", you can't know if another box - including yours - downstream might gather those composite ACKs further 3) you claim this might be safe *if* AQM is widely deployed but *you* don't appear to be making that determination *before* deploying your approach also, AQM is in the opposite direction, and unless you deploy this with enough state to track the fact that your box is seeing traffic in both directions, you shouldn't be turning it on at all As others have noted, the SHOULD of ACK requirements can be exceeded, but only with careful consideration of the potential impact. That careful consideration does not consist of "I turned it on and I didn't hear anyone scream". --- From nobody Fri Oct 9 11:04:09 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E9D681B4926 for ; Fri, 9 Oct 2015 11:04:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q1zjy-WcRjbW for ; Fri, 9 Oct 2015 11:04:07 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6C87B1B4924 for ; Fri, 9 Oct 2015 11:04:07 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t99I3fCO007396 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 11:03:41 -0700 (PDT) To: "Agarwal, Anil" , "Fred Baker (fred)" , "mallman@icir.org" References: <43116.1444411780@lawyers.icir.org> <7A2801D5E40DD64A85E38DF22117852C88129F58@wdc1exchmbxp05.hq.corp.viasat.com> From: Joe Touch Message-ID: <5618017D.4010401@isi.edu> Date: Fri, 9 Oct 2015 11:03:41 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <7A2801D5E40DD64A85E38DF22117852C88129F58@wdc1exchmbxp05.hq.corp.viasat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: David Lang , "tcpm@ietf.org" , touch@isi.edu, "dpreed@reed.com" , Jonathan Morton , Daniel Havey Subject: Re: [tcpm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 18:04:09 -0000 Not to mention what to do if you see TCP option fields you don't understand (i.e., not to simply copy them). Please don't forget that anything that tampers with a TCP segment is taking on the responsibility of RFC1122 - i.e., it becomes a complete proxy for the correct operation of TCP. Joe On 10/9/2015 10:59 AM, Agarwal, Anil wrote: > I presume, that "AKC Suppression", assuming it is a good thing to do, requires some rules for - > Handling multiple ACK segments some of which may contain data, timestamps, TCP-ECN info, > FIN, SACK, different receive window sizes, AO, ... > Handling multiple ACK segments interspersed with large data packets of other TCP connections, in the same queue. > Presumably, the DOCSIS folks have designed such rules. > > And the performance implications of doing so, besides the obvious benefit of reducing return-link congestion. > > Regards, > Anil > > -----Original Message----- > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Fred Baker (fred) > Sent: Friday, October 09, 2015 1:37 PM > To: mallman@icir.org > Cc: David Lang; Jonathan Morton; dpreed@reed.com; tcpm@ietf.org; Daniel Havey > Subject: Re: [tcpm] ACK Suppression > > >> On Oct 9, 2015, at 10:29 AM, mallman@icir.org wrote: >> >> >> I am not following along much, but why would that stop me?! :) >> >>> This sounds like an awful idea. What if there were congestion >>> marks in those ACKs? What if the ACK has data in it? >>> >>> ECN is an issue. I would expect a piggy-backed ack is treated as a >>> data message. >> >> (1) The point of delayed ACKs is to piggyback ACKs on data packets. >> So, when a host has data to send and would naturally do so then >> the ACK will get sent out with that data. >> >> (2) At least RFC 5681 (and previous versions) have been careful to >> note that when there is a hole in the sequence space of the >> arriving packets that ACKs should not be delayed. I.e., >> duplicate ACKs or ACKs with SACK information should be sent >> immediately as they are a possible indication of congestion. >> >> (3) I would think the spirit of (2) would also apply to ECN, but I >> am not sure that is written down anywhere (perhaps the ECN >> spec?). I.e., reflecting congestion marks should not be >> delayed. > > I think the question here relates to DOCSIS specifications, which basically encourage Cable Modems to separately queue acks from data traffic, and delete an older untranmsitted ack from the queue when a subsuming ack is enqueued. When the modem gets a transmission grant, it sends all of the data it has but, as a result, a potentially-small subset of the acks. The intended effect is to change the data:ack ratio from 2:1 to many:1 in high transmission rate contexts. > > I'm not convinced that the two algorithms mesh perfectly. (understatement) > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Fri Oct 9 11:42:40 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0CC911B49F9 for ; Fri, 9 Oct 2015 11:42:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4RyQo1zySeLI for ; Fri, 9 Oct 2015 11:42:25 -0700 (PDT) Received: from smtp85.iad3a.emailsrvr.com (smtp85.iad3a.emailsrvr.com [173.203.187.85]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ED8011B2D14 for ; Fri, 9 Oct 2015 11:42:24 -0700 (PDT) Received: from smtp11.relay.iad3a.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp11.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 100F61003C2; Fri, 9 Oct 2015 14:42:24 -0400 (EDT) Received: from app58.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp11.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id DFE311003E9; Fri, 9 Oct 2015 14:42:23 -0400 (EDT) X-Sender-Id: dpreed@reed.com Received: from app58.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by 0.0.0.0:25 (trex/5.4.2); Fri, 09 Oct 2015 18:42:24 GMT Received: from reed.com (localhost.localdomain [127.0.0.1]) by app58.wa-webapps.iad3a (Postfix) with ESMTP id CA3BE380061; Fri, 9 Oct 2015 14:42:23 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@reed.com, from: dpreed@reed.com) with HTTP; Fri, 9 Oct 2015 14:42:23 -0400 (EDT) Date: Fri, 9 Oct 2015 14:42:23 -0400 (EDT) From: dpreed@reed.com To: "Joe Touch" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20151009144223000000_66927" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: <5618017D.4010401@isi.edu> References: <43116.1444411780@lawyers.icir.org> <7A2801D5E40DD64A85E38DF22117852C88129F58@wdc1exchmbxp05.hq.corp.viasat.co m> <5618017D.4010401@isi.edu> X-Auth-ID: dpreed@reed.com Message-ID: <1444416143.826519921@apps.rackspace.com> X-Mailer: webmail/11.6.1-RC Archived-At: Cc: David Lang , touch@isi.edu, "mallman@icir.org" , "tcpm@ietf.org" , =?utf-8?Q?Fred_Baker_=28fred=29?= , Jonathan Morton , Daniel Havey Subject: Re: [tcpm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 18:42:33 -0000 ------=_20151009144223000000_66927 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AIt's interesting. I think the term "bikeshedding" has started to apply = here. A couple minor comments:=0A =0A1. The idea that one should put *new*= features into routers because of unmaintained/unmaintainable endpoint stac= ks seems like an argument that is very stretched. Why do people who can't = afford to update their endpoint gear for whatever reason buy new routers in= stead? Since that old gear was probably designed for slow networks (< 1 Mb= /sec end-to-end max) why would they generate a large enough percentage of t= raffic to be worth special detection, etc.? I'd like to see the actual dat= a that such situations exist and that new routers would be used there.=0A = =0A2. The search for "features" to put into routers has always been driven = by some kind of vague "value add" concept. But it has rarely been a path to= successfully making the Internet experience better. I could give a list o= f cases where it was "plausible" that you could sell a few routers that sol= ve a hypothetical issue where it was observed that "hey, we could do this t= hing at the router level, sometimes". There are probably Ethernet switches= that claim to reduce email spam, for example, by detecting SMTP, reading t= he messages, using a bayesian classifier, and sending RST messages to both = endpoints. (actually there are, a company called Sandvine sells stuff like= that). These are rarely mainstream opportunities to make the Internet bet= ter, and it's worth wondering why they don't get adopted. The basic answer = is that they don't work well, and there are better ways to achieve the goal= , usually by rethinking the problem consistently with the design principles= of the Internet.=0A=0A=0AOn Friday, October 9, 2015 2:03pm, "Joe Touch" said:=0A=0A=0A=0A> Not to mention what to do if you see TCP o= ption fields you don't=0A> understand (i.e., not to simply copy them).=0A> = =0A> Please don't forget that anything that tampers with a TCP segment is= =0A> taking on the responsibility of RFC1122 - i.e., it becomes a complete= =0A> proxy for the correct operation of TCP.=0A> =0A> Joe=0A> =0A> On 10/9/= 2015 10:59 AM, Agarwal, Anil wrote:=0A> > I presume, that "AKC Suppression"= , assuming it is a good thing to do,=0A> requires some rules for -=0A> > Ha= ndling multiple ACK segments some of which may contain data, timestamps,=0A= > TCP-ECN info,=0A> > FIN, SACK, different receive window sizes, AO, ...=0A= > > Handling multiple ACK segments interspersed with large data packets of = other=0A> TCP connections, in the same queue.=0A> > Presumably, the DOCSIS = folks have designed such rules.=0A> >=0A> > And the performance implication= s of doing so, besides the obvious benefit of=0A> reducing return-link cong= estion.=0A> >=0A> > Regards,=0A> > Anil=0A> >=0A> > -----Original Message--= ---=0A> > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Fred Baker= (fred)=0A> > Sent: Friday, October 09, 2015 1:37 PM=0A> > To: mallman@icir= .org=0A> > Cc: David Lang; Jonathan Morton; dpreed@reed.com; tcpm@ietf.org;= Daniel=0A> Havey=0A> > Subject: Re: [tcpm] ACK Suppression=0A> >=0A> >=0A>= >> On Oct 9, 2015, at 10:29 AM, mallman@icir.org wrote:=0A> >>=0A> >>=0A> = >> I am not following along much, but why would that stop me?! :)=0A> >>=0A= > >>> This sounds like an awful idea. What if there were congestion=0A> >>>= marks in those ACKs? What if the ACK has data in it?=0A> >>>=0A> >>> ECN i= s an issue. I would expect a piggy-backed ack is treated as a=0A> >>> data = message.=0A> >>=0A> >> (1) The point of delayed ACKs is to piggyback ACKs o= n data packets.=0A> >> So, when a host has data to send and would naturally= do so then=0A> >> the ACK will get sent out with that data.=0A> >>=0A> >> = (2) At least RFC 5681 (and previous versions) have been careful to=0A> >> n= ote that when there is a hole in the sequence space of the=0A> >> arriving = packets that ACKs should not be delayed. I.e.,=0A> >> duplicate ACKs or ACK= s with SACK information should be sent=0A> >> immediately as they are a pos= sible indication of congestion.=0A> >>=0A> >> (3) I would think the spirit = of (2) would also apply to ECN, but I=0A> >> am not sure that is written do= wn anywhere (perhaps the ECN=0A> >> spec?). I.e., reflecting congestion mar= ks should not be=0A> >> delayed.=0A> >=0A> > I think the question here rela= tes to DOCSIS specifications, which basically=0A> encourage Cable Modems to= separately queue acks from data traffic, and delete an=0A> older untranmsi= tted ack from the queue when a subsuming ack is enqueued. When the=0A> mode= m gets a transmission grant, it sends all of the data it has but, as a resu= lt,=0A> a potentially-small subset of the acks. The intended effect is to c= hange the=0A> data:ack ratio from 2:1 to many:1 in high transmission rate c= ontexts.=0A> >=0A> > I'm not convinced that the two algorithms mesh perfect= ly. (understatement)=0A> >=0A> > __________________________________________= _____=0A> > tcpm mailing list=0A> > tcpm@ietf.org=0A> > https://www.ietf.or= g/mailman/listinfo/tcpm=0A> >=0A> ------=_20151009144223000000_66927 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

It's= interesting.  I think the term "bikeshedding" has started to apply he= re.  A couple minor comments:

=0A

 = ;

=0A

1. The idea that one should put *new* = features into routers because of unmaintained/unmaintainable endpoint stack= s seems like an argument that is very stretched.  Why do people who ca= n't afford to update their endpoint gear for whatever reason buy new router= s instead?  Since that old gear was probably designed for slow network= s (< 1 Mb/sec end-to-end max) why would they generate a large enough per= centage of traffic to be worth special detection, etc.?  I'd like to s= ee the actual data that such situations exist and that new routers would be= used there.

=0A

 

=0A

2. The search for "features" to put into routers has always = been driven by some kind of vague "value add" concept. But it has rarely be= en a path to successfully making the Internet experience better.  I co= uld give a list of cases where it was "plausible" that you could sell a few= routers that solve a hypothetical issue where it was observed that "hey, w= e could do this thing at the router level, sometimes".  There are prob= ably Ethernet switches that claim to reduce email spam, for example, by det= ecting SMTP, reading the messages, using a bayesian classifier, and sending= RST messages to both endpoints.  (actually there are, a company = called Sandvine sells stuff like that).  These are rarely mainstream o= pportunities to make the Internet better, and it's worth wondering why they= don't get adopted. The basic answer is that they don't work well, and ther= e are better ways to achieve the goal, usually by rethinking the problem co= nsistently with the design principles of the Internet.

=0A=0A



On Friday, October 9, 2015 2:03pm, "Joe Touch" <touch= @isi.edu> said:

=0A
=0A> Not to mention what to do if you see TCP op= tion fields you don't
> understand (i.e., not to simply copy them).=
>
> Please don't forget that anything that tampers with a= TCP segment is
> taking on the responsibility of RFC1122 - i.e., i= t becomes a complete
> proxy for the correct operation of TCP.
>
> Joe
>
> On 10/9/2015 10:59 AM, Agarwal, A= nil wrote:
> > I presume, that "AKC Suppression", assuming it is= a good thing to do,
> requires some rules for -
> > Han= dling multiple ACK segments some of which may contain data, timestamps,
> TCP-ECN info,
> > FIN, SACK, different receive window siz= es, AO, ...
> > Handling multiple ACK segments interspersed with= large data packets of other
> TCP connections, in the same queue.<= br />> > Presumably, the DOCSIS folks have designed such rules.
= > >
> > And the performance implications of doing so, besi= des the obvious benefit of
> reducing return-link congestion.
= > >
> > Regards,
> > Anil
> >
&= gt; > -----Original Message-----
> > From: tcpm [mailto:tcpm-= bounces@ietf.org] On Behalf Of Fred Baker (fred)
> > Sent: Frida= y, October 09, 2015 1:37 PM
> > To: mallman@icir.org
> &= gt; Cc: David Lang; Jonathan Morton; dpreed@reed.com; tcpm@ietf.org; Daniel=
> Havey
> > Subject: Re: [tcpm] ACK Suppression
&g= t; >
> >
> >> On Oct 9, 2015, at 10:29 AM, mall= man@icir.org wrote:
> >>
> >>
> >>= ; I am not following along much, but why would that stop me?! :)
> = >>
> >>> This sounds like an awful idea. What if the= re were congestion
> >>> marks in those ACKs? What if the = ACK has data in it?
> >>>
> >>> ECN is an= issue. I would expect a piggy-backed ack is treated as a
> >>= ;> data message.
> >>
> >> (1) The point of = delayed ACKs is to piggyback ACKs on data packets.
> >> So, w= hen a host has data to send and would naturally do so then
> >&g= t; the ACK will get sent out with that data.
> >>
> &= gt;> (2) At least RFC 5681 (and previous versions) have been careful to<= br />> >> note that when there is a hole in the sequence space of = the
> >> arriving packets that ACKs should not be delayed. I.= e.,
> >> duplicate ACKs or ACKs with SACK information should = be sent
> >> immediately as they are a possible indication of= congestion.
> >>
> >> (3) I would think the sp= irit of (2) would also apply to ECN, but I
> >> am not sure t= hat is written down anywhere (perhaps the ECN
> >> spec?). I.= e., reflecting congestion marks should not be
> >> delayed.> >
> > I think the question here relates to DOCSIS sp= ecifications, which basically
> encourage Cable Modems to separatel= y queue acks from data traffic, and delete an
> older untranmsitted= ack from the queue when a subsuming ack is enqueued. When the
> mo= dem gets a transmission grant, it sends all of the data it has but, as a re= sult,
> a potentially-small subset of the acks. The intended effect= is to change the
> data:ack ratio from 2:1 to many:1 in high trans= mission rate contexts.
> >
> > I'm not convinced that= the two algorithms mesh perfectly. (understatement)
> >
&g= t; > _______________________________________________
> > tcpm= mailing list
> > tcpm@ietf.org
> > https://www.ietf.= org/mailman/listinfo/tcpm
> >
>

=0A
 ------=_20151009144223000000_66927-- From nobody Fri Oct 9 11:59:27 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4A2AA1B4A64; Fri, 9 Oct 2015 11:59:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n2Ux3YwP04Hk; Fri, 9 Oct 2015 11:59:23 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 250061B4927; Fri, 9 Oct 2015 11:59:23 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t99Iwdeg022720 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 11:58:40 -0700 (PDT) To: David Lang References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <5616E42D.5090402@isi.edu> <5616FAFA.5020707@isi.edu> <5618005A.8070303@isi.edu> From: Joe Touch Message-ID: <56180E5F.9030501@isi.edu> Date: Fri, 9 Oct 2015 11:58:39 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "tcpm@ietf.org" , "aqm@ietf.org" , touch@isi.edu Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 18:59:24 -0000 On 10/9/2015 11:52 AM, David Lang wrote: > On Fri, 9 Oct 2015, Joe Touch wrote: > >> Dave in particular wanted some specific reasons this is a bad idea as >> presented. Here is my summary. >> >> The rest of this discussion should happen on TCPM. >> >> Joe >> >> --- >> >> 1) *you* shouldn't be using a mechanism that destroys information for >> others >> >> whether the timestamps or ACK stream spacing has any >> meaning is for the receiver - not you - to decide > > The temporal spacing is already lost. You don't know that won't be restored downstream. >> 2) *you* don't know where your mechanism will have an impact >> >> those clumped ACKs might be spaced out further >> downstream >> >> even if you have a rule of "I'll only gather 3 ACKs", >> you can't know if another box - including yours - >> downstream might gather those composite ACKs further >> >> 3) you claim this might be safe *if* AQM is widely deployed >> >> but *you* don't appear to be making that determination >> *before* deploying your approach >> >> also, AQM is in the opposite direction, and unless >> you deploy this with enough state to track the fact >> that your box is seeing traffic in both directions, >> you shouldn't be turning it on at all > > this discussion started in the context of AQM. AQM is needed in both > directions, it's not a one-direction only thing. Needed, but you don't *know* it's deployed on both path directions. The reverse path could be taking a different route. >> As others have noted, the SHOULD of ACK requirements can be exceeded, >> but only with careful consideration of the potential impact. That >> careful consideration does not consist of "I turned it on and I didn't >> hear anyone scream". > > that's not the only consideration here. There's also the consideration > that the ACK stream is already badly distorted. It's not distorted at all - it's queued up and waiting its turn. > As it turns out (as discussed on the other thread), the RFC actually > recommends what we was proposed (with some additional nuances) Assuming that you know what you can't know and assuming that you don't care whether your actions affect what others have a right to assume. Joe From nobody Fri Oct 9 13:04:48 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 969D01B2C75; Fri, 9 Oct 2015 13:04:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G1neBXBh_xXU; Fri, 9 Oct 2015 13:04:45 -0700 (PDT) Received: from fruitcake.ICSI.Berkeley.EDU (fruitcake.ICSI.Berkeley.EDU [192.150.186.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1BF051B2C72; Fri, 9 Oct 2015 13:04:43 -0700 (PDT) Received: from lawyers.icir.org (envoy.icir.org [192.150.187.30]) by fruitcake.ICSI.Berkeley.EDU (8.12.11.20060614/8.12.11) with ESMTP id t99K4LLL024998; Fri, 9 Oct 2015 13:04:23 -0700 (PDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id C7FB22D712F3; Fri, 9 Oct 2015 16:04:19 -0400 (EDT) To: Joe Touch From: Mark Allman In-Reply-To: <5618005A.8070303@isi.edu> Organization: International Computer Science Institute (ICSI) Song-of-the-Day: Down on the Corner X-URL-0: http://www.icir.org/mallman-files/Document67298.doc MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-------459435943823349593450"; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Fri, 09 Oct 2015 16:04:19 -0400 Message-ID: <70335.1444421059@lawyers.icir.org> Sender: mallman@icir.org Archived-At: Cc: David Lang , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "tcpm@ietf.org" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: mallman@icir.org List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 20:04:46 -0000 --=-------459435943823349593450 Content-Type: text/plain > 1) *you* shouldn't be using a mechanism that destroys information for others > 2) *you* don't know where your mechanism will have an impact > 3) you claim this might be safe *if* AQM is widely deployed tl;dr summary: myopia is why we can't have nice things allman -- http://www.icir.org/mallman/ --=-------459435943823349593450 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlYYHcMACgkQWyrrWs4yIs4wMwCeOrRvKSF4PgjC/OwonFyFrInO ZvMAniQv2MhPnUGuxNCHW72NQODS33Q6 =1hRi -----END PGP SIGNATURE----- --=-------459435943823349593450-- From nobody Fri Oct 9 13:07:46 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 78BE91B2CB5; Fri, 9 Oct 2015 13:07:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.31 X-Spam-Level: X-Spam-Status: No, score=-6.31 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_22=0.6, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id St7Q1WXtVTc8; Fri, 9 Oct 2015 13:07:41 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7E1321B2CB3; Fri, 9 Oct 2015 13:07:41 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t99K7L63015433 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 13:07:21 -0700 (PDT) To: mallman@icir.org References: <70335.1444421059@lawyers.icir.org> From: Joe Touch Message-ID: <56181E78.5030302@isi.edu> Date: Fri, 9 Oct 2015 13:07:20 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <70335.1444421059@lawyers.icir.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: David Lang , "tcpm@ietf.org" , touch@isi.edu, "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 20:07:42 -0000 On 10/9/2015 1:04 PM, Mark Allman wrote: > >> 1) *you* shouldn't be using a mechanism that destroys information for others >> 2) *you* don't know where your mechanism will have an impact >> 3) you claim this might be safe *if* AQM is widely deployed > > tl;dr summary: myopia is why we can't have nice things Actually, it's "be conservative in what you do, be liberal in what you accept". Joe From nobody Fri Oct 9 13:46:33 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E26611AC436 for ; Fri, 9 Oct 2015 13:46:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.002 X-Spam-Level: X-Spam-Status: No, score=-2.002 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6tFFXnZFauFW for ; Fri, 9 Oct 2015 13:46:29 -0700 (PDT) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0137.outbound.protection.outlook.com [207.46.100.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9B2201AC430 for ; Fri, 9 Oct 2015 13:46:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=vI2rHiAmyeiUYFhgHNLyh/yet0luxW7DlT/5sDqwJ/I=; b=ONHXqwbfqalTdHfKrig7jPyAei7b7qQhCbrHEpmcrVMWstkdZrkxt61fvCMm64C98idEPING1FavFDVqkQE5PRUIgPGjpfPZ2KIG7KzP71rwKFrfRtGNGdBRnTh38Jq1jgYkogdL7IHCTbAbaU7olZL/TS2/rGdSpvfEyDRN7yM= Received: from DM2PR0301MB0655.namprd03.prod.outlook.com (10.160.96.17) by DM2PR0301MB0656.namprd03.prod.outlook.com (10.160.96.18) with Microsoft SMTP Server (TLS) id 15.1.293.16; Fri, 9 Oct 2015 20:46:27 +0000 Received: from DM2PR0301MB0655.namprd03.prod.outlook.com ([10.160.96.17]) by DM2PR0301MB0655.namprd03.prod.outlook.com ([10.160.96.17]) with mapi id 15.01.0293.007; Fri, 9 Oct 2015 20:46:27 +0000 From: Christian Huitema To: "dpreed@reed.com" , Joe Touch Thread-Topic: [tcpm] ACK management (was Re:ACKSuppression) Thread-Index: AdEC0DJAUExyujMYQpW3U3lc4Kpv9g== Date: Fri, 9 Oct 2015 20:46:27 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=huitema@microsoft.com; x-originating-ip: [2001:4898:80e8:4::106] x-microsoft-exchange-diagnostics: 1; DM2PR0301MB0656; 5:Be/cLQAK0+pGaEhSebGrvQ00ieXZ55FhsouEgztxVQDCXk4V9LCcuTBmYwfY2uridGEjScXLwt/S9wA15pbkRg7NoU7In/1C9hObDUF938mcV7qAdHbrPuM5pqnI0LvYPIzNBbMF8c3a1bjeoZ9icw==; 24:Nhhe2Rqo44cLj8diGkFKmSlRX0Q5FWc9e7ZHh9MqMELdaQ0g3961TZhq9fYAQLjlqoDydg+xAGtqUGXkmThZaUYC+c2jakieF0JbBvmUz1g=; 20:eH82Q1fyidBpQmRw5/fgo0wC6b3SE0VrqydgFtYVGFNkObB7q8+xvKOI65vEoH24LkMvd53z75vEhpqvcREYlA== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM2PR0301MB0656; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(270544422350281); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425024)(601004)(2401047)(5005006)(8121501046)(520078)(3002001)(61426024)(61427024); SRVR:DM2PR0301MB0656; BCL:0; PCL:0; RULEID:; SRVR:DM2PR0301MB0656; x-forefront-prvs: 0724FCD4CD x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(189002)(199003)(8990500004)(46102003)(77096005)(33656002)(19580395003)(106356001)(101416001)(2501003)(102836002)(15975445007)(10290500002)(10090500001)(5003600100002)(5005710100001)(10400500002)(105586002)(92566002)(5002640100001)(87936001)(2171001)(76576001)(5008740100001)(5001960100002)(189998001)(74316001)(40100003)(54356999)(122556002)(86612001)(99286002)(81156007)(5001920100001)(2900100001)(86362001)(5001770100001)(5007970100001)(11100500001)(50986999)(64706001)(97736004)(5004730100002)(3826002); DIR:OUT; SFP:1102; SCL:1; SRVR:DM2PR0301MB0656; H:DM2PR0301MB0655.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Oct 2015 20:46:27.1795 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR0301MB0656 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] ACK management (was Re:ACKSuppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 20:46:32 -0000 V2hpbGUgSSBhZ3JlZSB3aXRoIG1hbnkgb2YgeW91IHRoYXQgdGhlIEFRTSBncm91cCBoYXMgbm8g YnVzaW5lc3Mgc3RhbmRhcmRpemluZyBteW9waWMgY3Jvc3MgbGF5ZXJzIHZpb2xhdGlvbiwgSSB3 b3VsZCBsaWtlIHRvIHN3aXRjaCB0aGUgZm9jdXMgdG8gIndoYXQgc2hvdWxkIFRDUCBkbz8iIA0K DQpUaGUgIkFDSyBzdXBwcmVzc2lvbiIgaGFja3MgYXJlIGFjdHVhbGx5IGluIHJlc3BvbnNlIHRv IHR3byBpc3N1ZXM6IEFDSyBjb21wcmVzc2lvbiwgYW5kIEFDSyBjb25nZXN0aW9uLg0KDQpBQ0sg Y29tcHJlc3Npb24gaGFwcGVucyB3aGVuIHNwYWNlZC1vdXQgQUNLcyBnZXQgc29tZWhvdyBkZWxp dmVyZWQgYmFjayB0byBiYWNrLiBUaGlzIGlzIG5vdCBhIG5ld2x5IGRpc2NvdmVyZWQgcGhlbm9t ZW5vbiwgc2VlIHRoZSBTSUdDT01NIDkxIHBhcGVyIGJ5IExpeGlhIFpoYW5nLCBTY290dCBTaGVu a2VyIGFuZCBEYXZlIENsYXJrOiBodHRwOi8vd2ViLmNzLnVjbGEuZWR1L35saXhpYS9wYXBlcnMv OTFTSUdDT01NLnBkZi4gSXQgc2VlbXMgdG8gaGFwcGVuIGEgbG90IGluIGFjY2VzcyBuZXR3b3Jr cyB3aGVyZSB0aGUgTUFDIHByb3ZpZGVzIGVhY2ggc3RhdGlvbiBpbiB0dXJuIHdpdGggYSBsb25n ICJ0cmFuc21pc3Npb24gc2xvdCIgLS0gc2xvdHMgdGhhdCBjYW4gY2FycnkgbWFueSBwYWNrZXRz LiBUaGlzIGluY2x1ZGVzIGhpZ2ggc3BlZWQgV2ktRmkgbmV0d29ya3MsIERPQ1NJUyBtb2RlbXMs IGFuZCBwcm9iYWJseSBtb3JlLg0KDQpBQ0sgY29uZ2VzdGlvbiBoYXBwZW5zIHdoZW4gdGhlIEFD SyB0cmFmZmljIHNhdHVyYXRlcyBhIGxpbmsuIEEgdHlwaWNhbCBleGFtcGxlIHdvdWxkIGJlIGFu IGFzeW1tZXRyaWMgY29uZmlndXJhdGlvbiwgZS5nLiBhIGhpZ2ggc3BlZWQgZG93bmxpbmsgYW5k IGEgdmVyeSBsb3cgc3BlZWQgdXBsaW5rLiBTdXBwb3NlIGZvciBleGFtcGxlIGEgZG93bmxpbmsg Y2FwYWJsZSBvZiAxR2Jwcy4gVGhhdCdzIGFib3V0IDgwLDAwMCAxLjVLQiBwYWNrZXRzIHBlciBz ZWNvbmQsIGNhdXNpbmcgNDAsMDAwIEFDS1MgcGVyIHNlY29uZCBvbiB0aGUgdXBsaW5rLiBUaGF0 J3MgY2xvc2UgdG8gMjAgTWJwcyBvZiB0cmFmZmljLCBwb3RlbnRpYWxseSBlbm91Z2ggdG8gc2F0 dXJhdGUgYSBuYXJyb3cgdXBsaW5rLg0KDQpTaG91bGQgd2UgbW9kaWZ5IHRoZSBlbmQtdG8tZW5k IFRDUCBhbGdvcml0aG0gdG8gZGVhbCB3aXRoIHRob3NlIGlzc3Vlcz8gSG93Pw0KDQotLSBDaHJp c3RpYW4gSHVpdGVtYQ0KDQoNCg0K From nobody Fri Oct 9 14:27:55 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7C4A31ACDB4 for ; Fri, 9 Oct 2015 14:27:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5UmMHu2ikX2l for ; Fri, 9 Oct 2015 14:27:53 -0700 (PDT) Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 587BF1ACDB1 for ; Fri, 9 Oct 2015 14:27:53 -0700 (PDT) Received: from [128.9.184.173] ([128.9.184.173]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id t99LRWtN028876 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 14:27:34 -0700 (PDT) To: Christian Huitema , "dpreed@reed.com" References: From: Joe Touch X-Enigmail-Draft-Status: N1110 Message-ID: <56183141.30407@isi.edu> Date: Fri, 9 Oct 2015 14:27:29 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-MailScanner-ID: t99LRWtN028876 X-ISI-4-69-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu Subject: Re: [tcpm] ACK management (was Re:ACKSuppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 21:27:54 -0000 On 10/9/2015 1:46 PM, Christian Huitema wrote: > The "ACK suppression" hacks are actually in response to two issues: ACK compression, and ACK congestion. ... > Should we modify the end-to-end TCP algorithm to deal with those issues? How? Assuming other mitigations were already being used (packet compression in particular), dealing with this inside TCP would require either that a solution support all link capacity ratios or that TCP would detect and react to these particular assymetries. ACK congestion can be detected by tracking gaps in the received ACKs (i.e., larger than 2*MSS jumps). ACK compression would be detected by seeing a large number of back-to-back ACKs. The trouble lies with: a) signaling the other end to send fewer ACKs not too hard, e.g., using a new TCP option, but has to be deployed b) pacing the local bursts TCP can/should already be doing this, e.g., it should amortize the ACKs received over the measured RTT and not send segments much faster than the average rate this ought to be safe to leave active all the time So the only real question is "when do you know you should send fewer ACKs?" The side sending those ACKs doesn't have a unilateral way to deduce this; we'd have to add a new feedback signal into TCP, AFAICT. The real issue is that TCP is a feedback system. Turning down the feedback loop makes it more granular, which makes it potentially subject to wider oscillations - e.g., when a ACK is lost. I'm not sure that's a good thing to do in general, and I don't think 20Mbps of feedback unreasonable for a 1Gbps link. Joe From nobody Fri Oct 9 14:45:42 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1DFF91B4D27 for ; Fri, 9 Oct 2015 14:45:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EDmOd3Hbn97i for ; Fri, 9 Oct 2015 14:45:38 -0700 (PDT) Received: from smtp69.iad3a.emailsrvr.com (smtp69.iad3a.emailsrvr.com [173.203.187.69]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D73AD1B4D26 for ; Fri, 9 Oct 2015 14:45:37 -0700 (PDT) Received: from smtp17.relay.iad3a.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp17.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id C6E3418036D; Fri, 9 Oct 2015 17:45:36 -0400 (EDT) Received: from app47.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp17.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id B7322180103; Fri, 9 Oct 2015 17:45:36 -0400 (EDT) X-Sender-Id: dpreed@reed.com Received: from app47.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by 0.0.0.0:25 (trex/5.4.2); Fri, 09 Oct 2015 21:45:36 GMT Received: from reed.com (localhost.localdomain [127.0.0.1]) by app47.wa-webapps.iad3a (Postfix) with ESMTP id A38193800A7; Fri, 9 Oct 2015 17:45:36 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@reed.com, from: dpreed@reed.com) with HTTP; Fri, 9 Oct 2015 17:45:36 -0400 (EDT) Date: Fri, 9 Oct 2015 17:45:36 -0400 (EDT) From: dpreed@reed.com To: "Joe Touch" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20151009174536000000_16795" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: <56183141.30407@isi.edu> References: <56183141.30407@isi.edu> X-Auth-ID: dpreed@reed.com Message-ID: <1444427136.667910253@apps.rackspace.com> X-Mailer: webmail/11.6.1-RC Archived-At: Cc: Christian Huitema , "tcpm@ietf.org" , touch@isi.edu Subject: Re: [tcpm] =?utf-8?q?ACK_management_=28was_Re=3AACKSuppression=29?= X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 21:45:40 -0000 ------=_20151009174536000000_16795 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AWell, if the forward bottleneck link and the reverse path bottleneck in = the TCP feedback loop are asymmetric, one could certainly use ECN marking (= or something close to that) as the means to identify ACKs that are encounte= ring congestion (meaning the queue into the link carrying the ACK builds up= to an undesirable level).=0A =0AECN markings on packets that only ACK woul= d be useful to learn about these conditions. The approach used to handle c= ongestion in the high-traffic direction can be used for ACK management - "i= f there is congestion, tell the sender to slow down". Slowing down ACKs wo= uld then be an independent function, but the figure of merit is the same - = avoided queueing delay on the path carrying the ACKs.=0A =0AI don't know wh= ere there are common paths with a high degree of asymmetry. Most access pl= ant is becoming more symmetric as we know. But presumably all TCP stacks s= hould evolve to handle this sort of thing eventually.=0A=0A=0AOn Friday, Oc= tober 9, 2015 5:27pm, "Joe Touch" said:=0A=0A=0A=0A> =0A> = =0A> On 10/9/2015 1:46 PM, Christian Huitema wrote:=0A> > The "ACK suppress= ion" hacks are actually in response to two issues: ACK=0A> compression, and= ACK congestion.=0A> ...=0A> > Should we modify the end-to-end TCP algorith= m to deal with those issues?=0A> How?=0A> =0A> Assuming other mitigations w= ere already being used (packet compression=0A> in particular), dealing with= this inside TCP would require either that a=0A> solution support all link = capacity ratios or that TCP would detect and=0A> react to these particular = assymetries.=0A> =0A> ACK congestion can be detected by tracking gaps in th= e received ACKs=0A> (i.e., larger than 2*MSS jumps).=0A> =0A> ACK compressi= on would be detected by seeing a large number of=0A> back-to-back ACKs.=0A>= =0A> The trouble lies with:=0A> =0A> a) signaling the other end to send fe= wer ACKs=0A> not too hard, e.g., using a new TCP option,=0A> but has to be = deployed=0A> =0A> b) pacing the local bursts=0A> TCP can/should already be = doing this, e.g., it should=0A> amortize the ACKs received over the measure= d RTT and not=0A> send segments much faster than the average rate=0A> =0A> = this ought to be safe to leave active all the time=0A> =0A> So the only rea= l question is "when do you know you should send fewer=0A> ACKs?" The side s= ending those ACKs doesn't have a unilateral way to=0A> deduce this; we'd ha= ve to add a new feedback signal into TCP, AFAICT.=0A> =0A> The real issue i= s that TCP is a feedback system. Turning down the=0A> feedback loop makes i= t more granular, which makes it potentially subject=0A> to wider oscillatio= ns - e.g., when a ACK is lost. I'm not sure that's a=0A> good thing to do i= n general, and I don't think 20Mbps of feedback=0A> unreasonable for a 1Gbp= s link.=0A> =0A> Joe=0A> ------=_20151009174536000000_16795 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Well= , if the forward bottleneck link and the reverse path bottleneck in the TCP= feedback loop are asymmetric, one could certainly use ECN marking (or some= thing close to that) as the means to identify ACKs that are encountering co= ngestion (meaning the queue into the link carrying the ACK builds up to an = undesirable level).

=0A

 

=0A

ECN markings on packets that only ACK would be useful= to learn about these conditions.  The approach used to handle congest= ion in the high-traffic direction can be used for ACK management - "if ther= e is congestion, tell the sender to slow down".  Slowing down ACKs wou= ld then be an independent function, but the figure of merit is the same - a= voided queueing delay on the path carrying the ACKs.

=0A

 

=0A

I don't know where t= here are common paths with a high degree of asymmetry.  Most access pl= ant is becoming more symmetric as we know.  But presumably all TCP sta= cks should evolve to handle this sort of thing eventually.

=0A=0A



On Friday, October 9, 2015 5:27pm, "Joe Touch" <t= ouch@isi.edu> said:

=0A
= =0A

>
>
> On 10/9/2015 1:4= 6 PM, Christian Huitema wrote:
> > The "ACK suppression" hacks a= re actually in response to two issues: ACK
> compression, and ACK c= ongestion.
> ...
> > Should we modify the end-to-end TCP= algorithm to deal with those issues?
> How?
>
> A= ssuming other mitigations were already being used (packet compression
= > in particular), dealing with this inside TCP would require either that= a
> solution support all link capacity ratios or that TCP would de= tect and
> react to these particular assymetries.
>
&= gt; ACK congestion can be detected by tracking gaps in the received ACKs> (i.e., larger than 2*MSS jumps).
>
> ACK compressi= on would be detected by seeing a large number of
> back-to-back ACK= s.
>
> The trouble lies with:
>
> a) sign= aling the other end to send fewer ACKs
> not too hard, e.g., using = a new TCP option,
> but has to be deployed
>
> b) = pacing the local bursts
> TCP can/should already be doing this, e.g= ., it should
> amortize the ACKs received over the measured RTT and= not
> send segments much faster than the average rate
> > this ought to be safe to leave active all the time
>
> So the only real question is "when do you know you should send fewer<= br />> ACKs?" The side sending those ACKs doesn't have a unilateral way = to
> deduce this; we'd have to add a new feedback signal into TCP, = AFAICT.
>
> The real issue is that TCP is a feedback syste= m. Turning down the
> feedback loop makes it more granular, which m= akes it potentially subject
> to wider oscillations - e.g., when a = ACK is lost. I'm not sure that's a
> good thing to do in general, a= nd I don't think 20Mbps of feedback
> unreasonable for a 1Gbps link= .
>
> Joe
>

=0A
 ------=_20151009174536000000_16795-- From nobody Fri Oct 9 14:52:40 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DC26B1ACEF4 for ; Fri, 9 Oct 2015 14:52:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K-66LgN7WYtS for ; Fri, 9 Oct 2015 14:52:37 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3745F1ACEF2 for ; Fri, 9 Oct 2015 14:52:37 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t99LqFYO001584 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 14:52:16 -0700 (PDT) To: dpreed@reed.com References: <56183141.30407@isi.edu> <1444427136.667910253@apps.rackspace.com> From: Joe Touch Message-ID: <5618370F.7080909@isi.edu> Date: Fri, 9 Oct 2015 14:52:15 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <1444427136.667910253@apps.rackspace.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: Christian Huitema , "tcpm@ietf.org" , touch@isi.edu Subject: Re: [tcpm] ACK management (was Re:ACKSuppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 21:52:39 -0000 That sounds like a plan - basically, if ACKs are being lost (and not data), then sending fewer ACKs is a gamble that can be taken (or not) at the discretion of that TCP. The risk of taking that gamble is higher oscillation with loss, but that's something the endpoint can figure out too. Joe On 10/9/2015 2:45 PM, dpreed@reed.com wrote: > Well, if the forward bottleneck link and the reverse path bottleneck in > the TCP feedback loop are asymmetric, one could certainly use ECN > marking (or something close to that) as the means to identify ACKs that > are encountering congestion (meaning the queue into the link carrying > the ACK builds up to an undesirable level). > > > > ECN markings on packets that only ACK would be useful to learn about > these conditions. The approach used to handle congestion in the > high-traffic direction can be used for ACK management - "if there is > congestion, tell the sender to slow down". Slowing down ACKs would then > be an independent function, but the figure of merit is the same - > avoided queueing delay on the path carrying the ACKs. > > > > I don't know where there are common paths with a high degree of > asymmetry. Most access plant is becoming more symmetric as we know. > But presumably all TCP stacks should evolve to handle this sort of > thing eventually. > > > > On Friday, October 9, 2015 5:27pm, "Joe Touch" said: > >> >> >> On 10/9/2015 1:46 PM, Christian Huitema wrote: >> > The "ACK suppression" hacks are actually in response to two issues: ACK >> compression, and ACK congestion. >> ... >> > Should we modify the end-to-end TCP algorithm to deal with those issues? >> How? >> >> Assuming other mitigations were already being used (packet compression >> in particular), dealing with this inside TCP would require either that a >> solution support all link capacity ratios or that TCP would detect and >> react to these particular assymetries. >> >> ACK congestion can be detected by tracking gaps in the received ACKs >> (i.e., larger than 2*MSS jumps). >> >> ACK compression would be detected by seeing a large number of >> back-to-back ACKs. >> >> The trouble lies with: >> >> a) signaling the other end to send fewer ACKs >> not too hard, e.g., using a new TCP option, >> but has to be deployed >> >> b) pacing the local bursts >> TCP can/should already be doing this, e.g., it should >> amortize the ACKs received over the measured RTT and not >> send segments much faster than the average rate >> >> this ought to be safe to leave active all the time >> >> So the only real question is "when do you know you should send fewer >> ACKs?" The side sending those ACKs doesn't have a unilateral way to >> deduce this; we'd have to add a new feedback signal into TCP, AFAICT. >> >> The real issue is that TCP is a feedback system. Turning down the >> feedback loop makes it more granular, which makes it potentially subject >> to wider oscillations - e.g., when a ACK is lost. I'm not sure that's a >> good thing to do in general, and I don't think 20Mbps of feedback >> unreasonable for a 1Gbps link. >> >> Joe >> > From nobody Fri Oct 9 14:54:21 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F16841ACEFD for ; Fri, 9 Oct 2015 14:54:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id InY6HsBTdaKv for ; Fri, 9 Oct 2015 14:54:13 -0700 (PDT) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A9D581ACF1D for ; Fri, 9 Oct 2015 14:54:09 -0700 (PDT) Received: by padhy16 with SMTP id hy16so97191528pad.1 for ; Fri, 09 Oct 2015 14:54:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:in-reply-to:references :content-type:mime-version:content-transfer-encoding; bh=2EJF122VoFEGGZog4VKvMsS+IvSf1Wsgj1mby2wBii8=; b=YHjR7wE63/mcnmQqcgRCsY4n4B16hmQLSYsYm1R0Tl5F2AW65+WqtrCCyRXBDpYlGU whP0/gmoxCUHmkCO/zzUDnmhX3fY7rI4UJKGZEpFoL2zHLGYO4EvA969o8rYcKc7E4Ty tH69Ji/AT8MBOzf9dU9ZG3NB7RV6l+SlmdO2jnyaWltkcho+kVMxoPdOZiHGlSfkisRm DmY7cZh5r3kKDaRaNPnHPdTVSO54ahULHcqxcSwdL5baeXTUJHlZ2ayazW/MNuAZAxdi n21dRAtVX082BWc7jxz0KX0AQdYHh7U0X1TghHE832E71GlaEnBOUykTt2Mjr1Bhp5Q/ 140w== X-Received: by 10.68.131.6 with SMTP id oi6mr17911352pbb.3.1444427649280; Fri, 09 Oct 2015 14:54:09 -0700 (PDT) Received: from ?IPv6:2620:0:1000:3e02:d412:fd3:2300:c13f? ([2620:0:1000:3e02:d412:fd3:2300:c13f]) by smtp.gmail.com with ESMTPSA id po4sm4292346pbb.64.2015.10.09.14.54.08 (version=TLSv1.2 cipher=AES128-GCM-SHA256 bits=128/128); Fri, 09 Oct 2015 14:54:08 -0700 (PDT) Message-ID: <1444427647.27760.92.camel@edumazet-glaptop2.roam.corp.google.com> From: Eric Dumazet To: Joe Touch Date: Fri, 09 Oct 2015 14:54:07 -0700 In-Reply-To: <56183141.30407@isi.edu> References: <56183141.30407@isi.edu> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Archived-At: Cc: Christian Huitema , "dpreed@reed.com" , "tcpm@ietf.org" Subject: Re: [tcpm] ACK management (was Re:ACKSuppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 21:54:15 -0000 On Fri, 2015-10-09 at 14:27 -0700, Joe Touch wrote: > The real issue is that TCP is a feedback system. Turning down the > feedback loop makes it more granular, which makes it potentially subject > to wider oscillations - e.g., when a ACK is lost. I'm not sure that's a > good thing to do in general, and I don't think 20Mbps of feedback > unreasonable for a 1Gbps link. ACK load should be counted in number of packets, not bandwidth (rarely an issue). 20 Mbits sounds small, but this is about 40,000 ACK per second, and this sounds a lot more. Multiply this load by 10 (10 Gbit link speed) Is is really needed to get ~ 400,000 ack per second if a flow is sending at 10Gbits ? What useful info could the sender get from this ACK flood, given a ridiculous time budget to process them (and continue to cook/feed tx packets) We (TCP stacks) spend considerable amount of energy processing ACK packets all over the world. What a waste. LRO/GRO aggregates up to 45 MSS per 'super packet', so effectively hosts are already doing ACK suppression. That still is about 18,000 ACK per second at 10Gbit. More than enough. In many cases, one ACK per ms is enough to maintain ACK clocking (look also at TSO defer at sender side) and proper RTT measures, but this only matters for flows with a throughput above 25 Mbits (2000 packets per second) From nobody Fri Oct 9 15:02:12 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4818A1B4DA5 for ; Fri, 9 Oct 2015 15:02:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.226 X-Spam-Level: X-Spam-Status: No, score=0.226 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4tZM28mNWyXq for ; Fri, 9 Oct 2015 15:02:10 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 070FB1B4D39 for ; Fri, 9 Oct 2015 15:02:08 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id t99M26Ow015219; Fri, 9 Oct 2015 16:02:06 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Fri, 09 Oct 2015 16:02:06 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([::1]) by EXCHANGE.cablelabs.com ([::1]) with mapi id 14.03.0224.002; Fri, 9 Oct 2015 16:02:03 -0600 From: Greg White To: Joe Touch , "Scharf, Michael (Michael)" Thread-Topic: [aqm] TCP ACK Suppression Thread-Index: AQHRAAfaJ/73TrlDIUOGwELhN8Uffp5eqfGAgAFh4oCAAnWAgIAABLaAgAAFoICAAMCfAIAAiUcA///mpgA= Date: Fri, 9 Oct 2015 22:02:02 +0000 Message-ID: References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617FA42.6060306@isi.edu> In-Reply-To: <5617FA42.6060306@isi.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.5.6.150930 x-originating-ip: [10.4.11.28] Content-Type: text/plain; charset="iso-8859-1" Content-ID: <467429F2627252439957A2C50C0810A4@cablelabs.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , David Lang , "tcpm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:02:11 -0000 As I said earlier in this thread, I'm listening (and it seems others are too). While I can't take credit for the initial deployments in DOCSIS close to 15 years ago, I did have a hand in adding the recommendations in the DOCSIS 3.0 spec nearly 10 years ago. I am genuinely interested in specific problems that have occurred as a result of this practice, so that their impact can be weighed against the established benefits. So far, it seems the objections have been mostly philosophical, which doesn't necessarily negate them, but it is harder to weigh philosophy against measurable benefit. There have been some inflammatory (and uninformed) statements made in this thread about cable ISPs and the motivations for this practice, but I think the truth is simply that engineers try to make things work better. This is a case where a fairly simple optimization really appears to make things work better for the observable universe. If there are specific cases where things are made worse, then perhaps some further guidance can be given to implementers in a RFC3449bis in order to avoid those problems. In terms of the deleterious impact that this has on the world, I'm unclear where we stand. If intelligently discarding superfluous ACKs resulted in worse performance for the TCP session, it would never have been deployed (so I need more help understanding the arguments about it destroying information that TCP needs). Delivery of IP packets in real networks is rarely guaranteed, and transports need to deal with missing packets. The status quo until recently was that a single drop-tail queue with buffering equivalent to the expected worst case Internet RTT (or I've even heard some vendors doubling that for good measure) was considered the right thing to do. >From an end-to-end perspective, that seems good, simple, and predictable. Things work. They may not work really well, but they work. TCP (per RFC) isn't perfect. In fact it has contributed significantly to the problems with network performance, and is very difficult to change, so the equipment industry has tried to find ways to make the network work better (ACK prioritization, ACK suppression, AQM, flow queuing, etc.), largely by being more intelligent about dropping packets. AQM adds intelligence and decision making into the network that some find objectionable (and, despite what RFC7567 says, all AQMs make an implicit assumption that the majority of traffic causing a standing queue is TCP that will respond to loss in a certain way), but today it is widely considered to be better than the alternative. In the big picture this is definitely a tradeoff (simple and predictable network vs performant network). But for device vendors (and network operators) good pragmatic solutions that provide high performance are likely going to win out over philosophical stances that don't. If TCP gets fixed so it only sends on the order of 1 ACK/ms, or it doesn't bloat queues, then the need for these sorts of pragmatic solutions goes away and people stop doing them, but until then? -Greg On 10/9/15, 11:32 AM, "Joe Touch" wrote: > > >On 10/9/2015 2:21 AM, Scharf, Michael (Michael) wrote: >> [Removing AQM, adding TCPM] >>=20 >> =20 >>=20 >> Hi Joe, >>=20 >> =20 >>=20 >> I don=B9t find this text in RFC 793, but it is contained in RFC 2581, >> which is obsolete by RFC 5681. > >I think I pasted it from 1122, but had a tab open to 793 and mistook it >for being there as well. > >... >> 4.2. Generating Acknowledgments >>=20 >> The delayed ACK algorithm specified in [RFC1122] SHOULD be used by a >> TCP receiver. When using delayed ACKs, a TCP receiver MUST NOT >> excessively delay acknowledgments. Specifically, an ACK SHOULD be >> generated for at least every second full-sized segment, and MUST be >> generated within 500 ms of the arrival of the first unacknowledged >> packet. >... >> Thus, according to our current standards deviation from this SHOULD is >> possible =B2after careful consideration of the implications=B2. > >Agreed. And "careful consideration" does not consist of "I deployed it >silently, didn't track what happened, and didn't hear whether anyone had >any problems I wasn't listening for". > >Joe From nobody Fri Oct 9 15:02:36 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E3071B4DA8; Fri, 9 Oct 2015 15:02:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.226 X-Spam-Level: X-Spam-Status: No, score=0.226 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZubdVXzTN3DH; Fri, 9 Oct 2015 15:02:33 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 64ADB1B4D39; Fri, 9 Oct 2015 15:02:33 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id t99M2RS6015240; Fri, 9 Oct 2015 16:02:27 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Fri, 09 Oct 2015 16:02:27 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([::1]) by EXCHANGE.cablelabs.com ([::1]) with mapi id 14.03.0224.002; Fri, 9 Oct 2015 16:02:24 -0600 From: Greg White To: "mallman@icir.org" , Joe Touch Thread-Topic: [tcpm] [aqm] TCP ACK Suppression Thread-Index: AQHRAs27IO4Ju/dItUi+m4vQRfWi/p5jtsQA Date: Fri, 9 Oct 2015 22:02:24 +0000 Message-ID: References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> In-Reply-To: <70335.1444421059@lawyers.icir.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.5.6.150930 x-originating-ip: [10.4.11.28] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: David Lang , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "tcpm@ietf.org" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:02:34 -0000 On 10/9/15, 2:04 PM, "Mark Allman" wrote: > >> 1) *you* shouldn't be using a mechanism that destroys information for >>others >> 2) *you* don't know where your mechanism will have an impact >> 3) you claim this might be safe *if* AQM is widely deployed > >tl;dr summary: myopia is why we can't have nice things Too true. DOCSIS would have been much cleaner if we didn't have to deal with the fallout from the myopic TCP designers. :-P > >allman > > >-- >http://www.icir.org/mallman/ > > > From nobody Fri Oct 9 15:08:36 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 622761B4D75 for ; Fri, 9 Oct 2015 15:08:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Rvb-bjazhXB8 for ; Fri, 9 Oct 2015 15:08:34 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 46C201ACF1B for ; Fri, 9 Oct 2015 15:08:34 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t99M87cm003851 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 15:08:07 -0700 (PDT) To: Eric Dumazet References: <56183141.30407@isi.edu> <1444427647.27760.92.camel@edumazet-glaptop2.roam.corp.google.com> From: Joe Touch Message-ID: <56183AC7.3070204@isi.edu> Date: Fri, 9 Oct 2015 15:08:07 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <1444427647.27760.92.camel@edumazet-glaptop2.roam.corp.google.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: Christian Huitema , "dpreed@reed.com" , "tcpm@ietf.org" , touch@isi.edu Subject: Re: [tcpm] ACK management (was Re:ACKSuppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:08:35 -0000 On 10/9/2015 2:54 PM, Eric Dumazet wrote: ... > ACK load should be counted in number of packets, not bandwidth (rarely > an issue). 20 Mbits sounds small, but this is about 40,000 ACK per > second, and this sounds a lot more. > > Multiply this load by 10 (10 Gbit link speed) > > Is is really needed to get ~ 400,000 ack per second if a flow is sending > at 10Gbits ? What useful info could the sender get from this ACK flood, > given a ridiculous time budget to process them (and continue to > cook/feed tx packets) If this were from a single flow, then yes, this is silly - the TCP ought to be sending larger data packets and fewer ACKs. But that's a highly specialized case and also not necessarily one that needs to be optimized inside the network... > We (TCP stacks) spend considerable amount of energy processing ACK > packets all over the world. What a waste. How silly is it that UPS collects signatures for billions of packages a year when only a very small percentage are ever challenged? > LRO/GRO aggregates up to 45 MSS per 'super packet', so effectively hosts > are already doing ACK suppression. As I already noted, it's also not all safe. But aggregating at the receiver is a much different issue than doing so in the network or at the transmitter. When the receiver aggregates, it knows that the aggregate result will not have a higher impact if lost (because it won't be lost). Joe From nobody Fri Oct 9 15:10:38 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C0921B4DAE; Fri, 9 Oct 2015 15:10:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.31 X-Spam-Level: X-Spam-Status: No, score=-6.31 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_22=0.6, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 196X0oH95NLQ; Fri, 9 Oct 2015 15:10:35 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2178F1B4DB4; Fri, 9 Oct 2015 15:10:35 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t99MAH4j004008 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 15:10:17 -0700 (PDT) To: Greg White , "mallman@icir.org" References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> From: Joe Touch Message-ID: <56183B49.4000506@isi.edu> Date: Fri, 9 Oct 2015 15:10:17 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: David Lang , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "tcpm@ietf.org" , "aqm@ietf.org" , touch@isi.edu Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:10:36 -0000 On 10/9/2015 3:02 PM, Greg White wrote: > > > On 10/9/15, 2:04 PM, "Mark Allman" wrote: > >> >>> 1) *you* shouldn't be using a mechanism that destroys information for >>> others >>> 2) *you* don't know where your mechanism will have an impact >>> 3) you claim this might be safe *if* AQM is widely deployed >> >> tl;dr summary: myopia is why we can't have nice things > > Too true. DOCSIS would have been much cleaner if we didn't have to deal > with the fallout from the myopic TCP designers. :-P Wouldn't it have been cleaner with more appropriate network provisioning? Why are you blaming TCP? A DOCSIS router ought to route. If it left the packets alone, IMO we'd all be better off. Joe From nobody Fri Oct 9 15:18:57 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CC65A1B4AA0 for ; Fri, 9 Oct 2015 15:18:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.002 X-Spam-Level: X-Spam-Status: No, score=-2.002 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MxtUM5y7yXpr for ; Fri, 9 Oct 2015 15:18:52 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bon0786.outbound.protection.outlook.com [IPv6:2a01:111:f400:fc10::1:786]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 76D481B2D21 for ; Fri, 9 Oct 2015 15:18:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=G00fiph5K3e/ycRdHowNz3mDvN7BYRiD4RcVETWrj+E=; b=KfaD8jvi+olqMRcrlvgCXa/tICXqb4N0At31ptz36B6kFcHXxnycpSxYJOlntwhOKr5xyTGFQosTrKRdCp47TMDT12qbHF5gppazoDIf9TWPxsDMHNTpquyPF8xDPd+Wh9xZxSZkCSdmZjMmxGHhxjI6tI4d6J2/brQm2EXIJIk= Received: from BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) by BN1PR0301MB0641.namprd03.prod.outlook.com (10.160.171.14) with Microsoft SMTP Server (TLS) id 15.1.293.16; Fri, 9 Oct 2015 22:18:35 +0000 Received: from BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.173]) by BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.153]) with mapi id 15.01.0293.007; Fri, 9 Oct 2015 22:18:35 +0000 From: Praveen Balasubramanian To: Joe Touch , Eric Dumazet Thread-Topic: [tcpm] ACK management (was Re:ACKSuppression) Thread-Index: AdEC0DJAUExyujMYQpW3U3lc4Kpv9gACRqKAAADuH4AAAH0sgAAAKcFQ Date: Fri, 9 Oct 2015 22:18:35 +0000 Message-ID: References: <56183141.30407@isi.edu> <1444427647.27760.92.camel@edumazet-glaptop2.roam.corp.google.com> <56183AC7.3070204@isi.edu> In-Reply-To: <56183AC7.3070204@isi.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com; x-originating-ip: [2001:4898:80e8:5::584] x-microsoft-exchange-diagnostics: 1; BN1PR0301MB0641; 5:+CbM04s45UV+t/CX/T4FSr0POxNKdAZtCIT7gbWhXY52FefkO9hLcybGAMGfO4Jg+izm5UKx2hyIuID3iH11WiQ1NYFRNN2F+uqo50XpFcxppFu0QHBDQOQ2LNftHPVxZLRi/YBZqJrboILPxPCfYQ==; 24:K/PZV4vYF+diSkIeyNOvguvbnJKVUzQ2bDgQYkoM1t9ADL+d/qvaNMAwe85Hd3qZHEoxGDJRlsvGTbYxtSA+2ZPW8/NlFtNnitXHRTWWTJg=; 20:ThI1Z3cuusAG5gM3WHRNgXRYPMqcNKAaRkJ0+73YQzcgYDLbfSrYfhLg95hmBLWjAXIY1VGT/BaHhWXmChrV9w== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN1PR0301MB0641; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(108003899814671); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425024)(601004)(2401047)(5005006)(520078)(8121501046)(3002001)(61426024)(61427024); SRVR:BN1PR0301MB0641; BCL:0; PCL:0; RULEID:; SRVR:BN1PR0301MB0641; x-forefront-prvs: 0724FCD4CD x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(377454003)(13464003)(24454002)(189002)(479174004)(199003)(46102003)(5007970100001)(92566002)(11100500001)(40100003)(102836002)(97736004)(189998001)(74316001)(15975445007)(33656002)(64706001)(5001960100002)(10290500002)(86362001)(99286002)(2171001)(87936001)(8990500004)(5008740100001)(10090500001)(2950100001)(10400500002)(86612001)(50986999)(5002640100001)(5005710100001)(106356001)(81156007)(2900100001)(19580405001)(19580395003)(105586002)(5004730100002)(54356999)(93886004)(76576001)(122556002)(5001770100001)(5003600100002)(101416001)(5001920100001)(76176999)(3826002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR0301MB0641; H:BN1PR03MB008.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Oct 2015 22:18:35.2199 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR0301MB0641 Archived-At: Cc: Christian Huitema , "dpreed@reed.com" , "tcpm@ietf.org" Subject: Re: [tcpm] ACK management (was Re:ACKSuppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:18:56 -0000 LRO/GRO should ideally not coalesce pure ACKs. Also LRO/GRO should be desig= ned in a way that the host TCP/IP stack knows how many segments/bytes were = coalesced and can do the right accounting for modifying TCP state variables= . What I am saying is that LRO/GRO can be an information preserving packet = transformation if so designed.=20 I think letting the network suppress ACKs without end host knowledge can ha= ve bad consequences. But I am in favor of end host optimization that can re= duce ACKs generated at receiver in conjunction with use of ABC at sender.=20 -----Original Message----- From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Joe Touch Sent: Friday, October 9, 2015 3:08 PM To: Eric Dumazet Cc: Christian Huitema ; dpreed@reed.com; tcpm@ietf.o= rg; touch@isi.edu Subject: Re: [tcpm] ACK management (was Re:ACKSuppression) On 10/9/2015 2:54 PM, Eric Dumazet wrote: ... > ACK load should be counted in number of packets, not bandwidth (rarely=20 > an issue). 20 Mbits sounds small, but this is about 40,000 ACK per=20 > second, and this sounds a lot more. >=20 > Multiply this load by 10 (10 Gbit link speed) >=20 > Is is really needed to get ~ 400,000 ack per second if a flow is=20 > sending at 10Gbits ? What useful info could the sender get from this=20 > ACK flood, given a ridiculous time budget to process them (and=20 > continue to cook/feed tx packets) If this were from a single flow, then yes, this is silly - the TCP ought to= be sending larger data packets and fewer ACKs. But that's a highly special= ized case and also not necessarily one that needs to be optimized inside th= e network... > We (TCP stacks) spend considerable amount of energy processing ACK=20 > packets all over the world. What a waste. How silly is it that UPS collects signatures for billions of packages a yea= r when only a very small percentage are ever challenged? > LRO/GRO aggregates up to 45 MSS per 'super packet', so effectively=20 > hosts are already doing ACK suppression. As I already noted, it's also not all safe. But aggregating at the receiver is a much different issue than doing so in = the network or at the transmitter. When the receiver aggregates, it knows t= hat the aggregate result will not have a higher impact if lost (because it = won't be lost). Joe _______________________________________________ tcpm mailing list tcpm@ietf.org https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri Oct 9 15:24:13 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B2FC1B4DBF for ; Fri, 9 Oct 2015 15:24:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0jgkKsyeRnTb for ; Fri, 9 Oct 2015 15:24:10 -0700 (PDT) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 09E311B4DB7 for ; Fri, 9 Oct 2015 15:24:09 -0700 (PDT) Received: by padhy16 with SMTP id hy16so97751250pad.1 for ; Fri, 09 Oct 2015 15:24:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:in-reply-to:references :content-type:mime-version:content-transfer-encoding; bh=nSGCMc67VnkFyxzk30cPSHJcNGH3UdjOrH7/sdvrQsw=; b=LyoJ65aip7mGGaHb8QUQ97ytFsNG8uka4SKBdHyyYuQJz7Akk69cxKvoLZtyQE7V39 UOgAPkihmvISa9twDX3L9RmkUIMzXGGcKo/eKSSG2EV+x8q5RBJsB4PAjGu898zIMNT8 UBc4XZMR9P6HUDXANrqLgLBoR9xUkLpgOLPfCJpWojRLL4sHIyq9abjIcf8ftrFZYfr2 /rKMZLX1Dg3WbACzWM8V3raj9VccCTZE/YBtcTG+KbsTXoZ0ZuiW3mWP7x8jleaCAwG2 jVm66uMpHfzHv3ODDorqtF9lOJOuskaHhSWiXbdoJt8N7uDYxQU9CJlE1aLUd75sajnQ Rtrw== X-Received: by 10.68.129.231 with SMTP id nz7mr18005070pbb.53.1444429449600; Fri, 09 Oct 2015 15:24:09 -0700 (PDT) Received: from ?IPv6:2620:0:1000:3e02:d412:fd3:2300:c13f? ([2620:0:1000:3e02:d412:fd3:2300:c13f]) by smtp.gmail.com with ESMTPSA id ol3sm4382865pbb.49.2015.10.09.15.24.08 (version=TLSv1.2 cipher=AES128-GCM-SHA256 bits=128/128); Fri, 09 Oct 2015 15:24:08 -0700 (PDT) Message-ID: <1444429447.27760.102.camel@edumazet-glaptop2.roam.corp.google.com> From: Eric Dumazet To: Joe Touch Date: Fri, 09 Oct 2015 15:24:07 -0700 In-Reply-To: <56183AC7.3070204@isi.edu> References: <56183141.30407@isi.edu> <1444427647.27760.92.camel@edumazet-glaptop2.roam.corp.google.com> <56183AC7.3070204@isi.edu> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Archived-At: Cc: Christian Huitema , "dpreed@reed.com" , "tcpm@ietf.org" Subject: Re: [tcpm] ACK management (was Re:ACKSuppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:24:11 -0000 On Fri, 2015-10-09 at 15:08 -0700, Joe Touch wrote: > > On 10/9/2015 2:54 PM, Eric Dumazet wrote: > > We (TCP stacks) spend considerable amount of energy processing ACK > > packets all over the world. What a waste. > > How silly is it that UPS collects signatures for billions of packages a > year when only a very small percentage are ever challenged? It is rare UPS collects 400,000 identical signatures per second from a single customer ;) TCP receivers should not send ACK immediately. They should bet on the immediate future. I did a ~50 lines prototype patch in linux to arm a 1 ms timer, instead of immediately sending one ACK. I did this for SACK packets, trying to cope with reordering fabrics, (per packet load balancing, not caring on flow hash), but patch could be extended quite easily with a more fine grained timer (like min(rtt/10, 1ms)) If another packet(s) is(are) coming before timer expires, we avoid sending all these useless floods of ACK packets, and avoid intermediate nodes trying to be smart and do their own ACK compression. From nobody Fri Oct 9 15:24:43 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 11A661B4DBF; Fri, 9 Oct 2015 15:24:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7c2oLuxqBIf9; Fri, 9 Oct 2015 15:24:40 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5888D1B4DB7; Fri, 9 Oct 2015 15:24:40 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t99MOJYS006232 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 15:24:19 -0700 (PDT) To: David Lang References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> From: Joe Touch X-Enigmail-Draft-Status: N1110 Message-ID: <56183E93.1010308@isi.edu> Date: Fri, 9 Oct 2015 15:24:19 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:24:42 -0000 On 10/9/2015 3:16 PM, David Lang wrote: ... >> Wouldn't it have been cleaner with more appropriate network provisioning? > > "more appropriate network provisioning" is not always going to result in > more bandwidth the way you want it to. > > If there is established infrastructure that can handle X in one > direction and 100X in the other direction, but "appropriate network > provisioning" requires that the ratio never be more than 3x, it's not > going to magically increase bandwidth in one direction, all it can do is > cap bandwidth in the other direction (throwing away capacity) Sure, but we're dealing with a problem that arises when the ratio can't support 40:1. That's quite an asymmetry except in extreme cases where we already know extreme measures are required (e.g., satcom with telco backchannels). >> Why are you blaming TCP? > > first off, didn't you see the smiley? > > But treating your question as serious, it's because as RFC3449 shows, > highly assymetric networks or networks with other traffic on them don't > work with TCP the way you want it to work. So the option is to modify > TCP or tinker with the packets to leverage less commonly used features > of TCP (or features that are implicit in the design rather than explicit) The former is always on the table - that's what TCPM is for. The latter has always been problematic, exactly because it's subject to tragedy-of-the-commons. TCP is a contract between the endpoints, and whenever a middlebox starts modifying those packets, IMO it becomes a party to that contract that needs to be taken very seriously (smiley faces aside ;-) >> A DOCSIS router ought to route. If it left the packets alone, IMO we'd >> all be better off. > > So you would rather have people getting slower downloads than have > network equipment modify packets? I would rather people get slower downloads than break TCP in ways we won't see for 5 years or inhibit the extension of TCP in new ways that might be even more beneficial. > Those are the real-world choices. There were other cable modem companies that tried similar "tricks" in the past and got in a lot of hot water for making a box that went a LOT faster for *their* customers. TCP works when everyone plays by the same conservative rules. There has always been a choice to break those rules for individual gain, and there has always been a reason we try very hard to avoid that. Joe From nobody Fri Oct 9 15:39:44 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 532D71B2D83; Fri, 9 Oct 2015 15:39:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 79c6LhQqA2By; Fri, 9 Oct 2015 15:39:40 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2AA511B2D7F; Fri, 9 Oct 2015 15:39:40 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t99MdAeH008748 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 15:39:10 -0700 (PDT) To: David Lang References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> From: Joe Touch Message-ID: <5618420E.9040609@isi.edu> Date: Fri, 9 Oct 2015 15:39:10 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:39:41 -0000 On 10/9/2015 3:31 PM, David Lang wrote: > On Fri, 9 Oct 2015, Joe Touch wrote: > >> On 10/9/2015 3:16 PM, David Lang wrote: >> ... >>>> Wouldn't it have been cleaner with more appropriate network >>>> provisioning? >>> >>> "more appropriate network provisioning" is not always going to result in >>> more bandwidth the way you want it to. >>> >>> If there is established infrastructure that can handle X in one >>> direction and 100X in the other direction, but "appropriate network >>> provisioning" requires that the ratio never be more than 3x, it's not >>> going to magically increase bandwidth in one direction, all it can do is >>> cap bandwidth in the other direction (throwing away capacity) >> >> Sure, but we're dealing with a problem that arises when the ratio can't >> support 40:1. That's quite an asymmetry except in extreme cases where we >> already know extreme measures are required (e.g., satcom with telco >> backchannels). > > that's not the only situation we're talking about here. > > Also, keep in mind that your 40:1 ratio assumes that there is no traffic > going the other direction. FWIW, I wasn't - I was assuming that 40:1 is fine if what you intend to support is browsing only. > If you have one person trying to watch streaming video while another > person is uploading pictures to facebook, you can run into trouble at > much more even ratios. Restated, you run into trouble because you sold a service you didn't provision for ;-) (or, more to the point, you took a gamble that you could sell a useful service with a particular assumption about traffic ratios, and that assumption no longer holds) Again, IMO TCP isn't to be tampered with merely to support a business model, and that's all I see so far. Joe From nobody Fri Oct 9 15:54:01 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B457A1B4E0A for ; Fri, 9 Oct 2015 15:54:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.161 X-Spam-Level: X-Spam-Status: No, score=-2.161 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, STOX_REPLY_TYPE=0.439] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WLSrYW9HAA0k for ; Fri, 9 Oct 2015 15:53:58 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 78DD81B4E0B for ; Fri, 9 Oct 2015 15:53:58 -0700 (PDT) Received: from srichardlxp2 ([213.143.121.76]) by mail.gmx.com (mrgmx101) with ESMTPSA (Nemesis) id 0MNdxu-1ZefBb2DCE-007A7f; Sat, 10 Oct 2015 00:53:13 +0200 Message-ID: <96FC85CD93C14C7FADC014391BAFE03E@srichardlxp2> From: "Richard Scheffenegger" To: "Scharf, Michael \(Michael\)" , "Wesley Eddy" , "Joe Touch" References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617C4BF.6040504@mti-systems.com> <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> Date: Sat, 10 Oct 2015 00:52:54 +0200 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-Provags-ID: V03:K0:WXqrEO9jZBPsfPpCNFpSZ1cwb+UDakSoH9uMWdokOJCfA3mT5v7 4+g5Vj+ZSTXWYtXGvVEKwHhpwUJA4jm+GaBIVrUSSq/5a31YifCEL+WDM8i58PTbRxtKEMC v/U/mp+VVYdW11HifsZ4eSmeFq0A38jNnFISMzlt+Ap3MGT4TvQux5NaFsy5CoJ/tlog4AO y7ARjJL15aVl8XgTo+CBA== X-UI-Out-Filterresults: notjunk:1;V01:K0:uI6/+2OzFB0=:RMdOmT4zlorI98sE9TMbSm yehlRWp3TIgJarypoiJ8SE+PJFATxwvum5ERKy18GkhjqkiM3qVqbG1E9IyZxWPqRoTffTklK ZomGcgio1r24k5rE2cRIoGUqIJEzLuz4T2OLdL+WZblwAsD+SRJGy7zw0mWSLr7bSJkzcVjAp oxbZjno0DK1g7PwG+2WmLp5Ylyr2+4zKU+SLWSAt+yzqv3jagavjWaGu5hKVr81wYpUaaMna/ H/TSB/BDzduHmmVCH25ZzBI0F6jkwZQ7YxWYlcR8wcyYCRFWCxglUJVToHXaQgA9z9GeEdErb S74aR87UEtrHrAnYwnduNYCCP3RMBxzqGEVNWpL9/xyL8Lt/l2WrrfMzzoQBR9NEayALijlkb SH2j67NJO/QAHhjOrNuFANVhEK7dlVM4kqtXDx6HBL6pqIZEcBCOPb0jDliqUWJ4Qk6mJAt0U fn/ruXfP8zP21egAorVsvLuAeTO83c9SJQ9kLXBSgkkqvOdaxsd287opsaPVMLq6hXqHMR3kN Bx+w2+6KiwNspdKt9T3gXDQKjJH7k2ItoDNcMjlR5ujYJiyyqLbl3FBGp+03SYo3Y4iaAO/WN LXpx/2JGwQVgs5AvDoMAI7weTnhOcxgMJrG/ozGudhAaOvynDDianIT+o9zD9JWGsKOT1pxTC ZZGkXWnLjkTZelOG06J9UPq03CScd2LmN/rKh9IeTVoNPmuaiSiATJh0cEL7BdX+iX7GZ05PH bkL0zgAuLnTqkvFZFlunqQXjjKcPM7oKKer1rJbqCk7F7jxbMCUorU2alOyubrm+tORg0vVtn VLDLEuu Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , tcpm@ietf.org, David Lang Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:54:00 -0000 This might also be an opportunity to go more into the reasoning for the 500ms maximum... Many implementations interpret "the delay MUST be less than 0.5 seconds" to trigger the delayed ACK timer at 500ms (with this time a fixed, compile-time value) - while it has been shown, that making that dynamic (or a based on RTT and pps might be a more efficient approach (especially during loss at end-of-stream), depending on the environment [1]. [I thought to have read a recent paper on tuning the delayed ACK timer down to 1ms in datacenter environments, but cann't find that now]. Best regards, Richard [1] Altman, Eitan, and Tania Jimnez. "Novel delayed ACK techniques for improving TCP performance in multihop wireless networks." Personal Wireless Communications. Springer Berlin Heidelberg, 2003. ----- Original Message ----- From: "Scharf, Michael (Michael)" To: "Wesley Eddy" ; "Joe Touch" Cc: "LAUTENSCHLAEGER, Wolfram (Wolfram)" ; "Greg White" ; ; "David Lang" Sent: Friday, October 09, 2015 5:21 PM Subject: Re: [tcpm] [aqm] TCP ACK Suppression > Yes, I have realized today myself that 793bis lacks that section. I agree > that ACK generation belongs into 793bis. > > 793bis actually may open an opportunity to rethink the exact wording that > explains this SHOULD. Personally, I believe replacing this SHOULD is not > an option for a bis document. > > But if deviations from that SHOULD are widely deployed in the meantime, > 793bis could possibly comment on that. Also, having some informational > reference could perhaps be an option. > > Michael > > > -----Original Message----- > From: Wesley Eddy [mailto:wes@mti-systems.com] > Sent: Friday, October 09, 2015 3:45 PM > To: Scharf, Michael (Michael); Joe Touch > Cc: LAUTENSCHLAEGER, Wolfram (Wolfram); Greg White; tcpm@ietf.org; David > Lang > Subject: Re: [tcpm] [aqm] TCP ACK Suppression > > On 10/9/2015 5:21 AM, Scharf, Michael (Michael) wrote: >> [Removing AQM, adding TCPM] >> >> >> >> Hi Joe, >> >> >> >> I don't find this text in RFC 793, but it is contained in RFC 2581, >> which is obsolete by RFC 5681. >> >> > > > To confuse matters further, there's also 1122 text with requirements > language: > > A TCP SHOULD implement a delayed ACK, but an ACK should not > be excessively delayed; in particular, the delay MUST be > less than 0.5 seconds, and in a stream of full-sized > segments there SHOULD be an ACK for at least every second > segment. > > This probably belongs in 793bis. In my view, normative acknowledgement > behavior belongs in the base spec rather than particular congestion > control descriptions, since there may be more than CC that depends upon it > or makes assumptions about it. > > > -- > Wes Eddy > MTI Systems > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Fri Oct 9 15:59:59 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6B8301B4E24 for ; Fri, 9 Oct 2015 15:59:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.388 X-Spam-Level: X-Spam-Status: No, score=-1.388 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gQSK9R8iCPll for ; Fri, 9 Oct 2015 15:59:56 -0700 (PDT) Received: from mail-vk0-x232.google.com (mail-vk0-x232.google.com [IPv6:2607:f8b0:400c:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1007A1B4E23 for ; Fri, 9 Oct 2015 15:59:56 -0700 (PDT) Received: by vkha6 with SMTP id a6so3206116vkh.2 for ; Fri, 09 Oct 2015 15:59:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=mrPaRjB0HIqqnmUiN0TokdOBuNhqD6fHuiqcawfEyd8=; b=DO/4G+SbJT8CCC2MPd823A33n19EgXNIC9Bege1AGA9a7fUg1QPgsL55277ei9ZuW/ FVmEPc/iUhOhrd1S7JIX/rJrBsA7FDZCmDlcQju1+7E4Wnl+rG4l+uUlvmuZmFnq6yaT MrLzZGX14ivf6BI7OKSXNoAP50BCpRpseiENWUzGtZIePGF7tO2tFabQeh6IfLGSOcFt f/2zZjWgwuwWLUv+DewP9LZGb18NpXE35EacxOOgV5nwkfEWeHQ6meQxvO9S+e39Lazn 0NOVGXtni/RHK2c/0R3gLGBpIVrWiN3CMfgAr34Ul4I3mOY5u3bapLBv+0ubg+Y1+/RY hNTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=mrPaRjB0HIqqnmUiN0TokdOBuNhqD6fHuiqcawfEyd8=; b=mvH3TIHYoeCOht+JeTADBaEdhcnbmFgWDkB1ZgiOJIkDzg1LqEe1q3A0KMe4EZ8G2r 8+mu7h5QgdM1sCnfDo+qOvpBjxDruVmxX/C7ErHg3IMgD5pcORnvm+hiHjRAaRyZii6s k7b8L6FIXhEEQLx33CAojuLUA53Gc1MowUJTdfkiGG5HWD2pSxDOKn3PoKR20ci4wqvg fOrDvOvERpMmFvaipc/fExRtItixQLPciY8LCj6gks3meVO1sRJAL5yx6ytlX6oJOg9z wK/eeVxqQOl5KxD2n3EnT0b8VyY628f/IbCerRs09OPioT/n/kOTXqq5Y0Uo4gAW5xl0 5Zow== X-Gm-Message-State: ALoCoQm9JlNLXlrFXwOqFqWaQV1WsQFW36G6/mh08DT3hpmPS/LpbcJJZds6xZwCODYtDK8z6ifT X-Received: by 10.31.2.18 with SMTP id 18mr603758vkc.145.1444431594221; Fri, 09 Oct 2015 15:59:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.153.134 with HTTP; Fri, 9 Oct 2015 15:59:14 -0700 (PDT) In-Reply-To: <43116.1444411780@lawyers.icir.org> References: <1A2199DA-6638-4E0A-A71B-4E376ECA0EC0@cisco.com> <43116.1444411780@lawyers.icir.org> From: Yuchung Cheng Date: Fri, 9 Oct 2015 15:59:14 -0700 Message-ID: To: "mallman@icir.org" Content-Type: multipart/alternative; boundary=001a11466c2e13ab8d0521b3f051 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 22:59:57 -0000 --001a11466c2e13ab8d0521b3f051 Content-Type: text/plain; charset=UTF-8 On Fri, Oct 9, 2015 at 10:29 AM, Mark Allman wrote: > > I am not following along much, but why would that stop me?! :) > > > This sounds like an awful idea. What if there were congestion > > marks in those ACKs? What if the ACK has data in it? > > > > ECN is an issue. I would expect a piggy-backed ack is treated as a > > data message. > > (1) The point of delayed ACKs is to piggyback ACKs on data packets. > So, when a host has data to send and would naturally do so then > the ACK will get sent out with that data. > > (2) At least RFC 5681 (and previous versions) have been careful to > note that when there is a hole in the sequence space of the > arriving packets that ACKs should not be delayed. I.e., > duplicate ACKs or ACKs with SACK information should be sent > immediately as they are a possible indication of congestion. > > (3) I would think the spirit of (2) would also apply to ECN, but I > am not sure that is written down anywhere (perhaps the ECN > spec?). I.e., reflecting congestion marks should not be > delayed. > Gorry pointed out earlier: https://tools.ietf.org/html/rfc3449#section-5.2.1 As Wes and Michael pointed out, this ack policy is discussed in too many RFCs (5681, 1112, 3449). > allman > > > -- > http://www.icir.org/mallman/ > > > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > > --001a11466c2e13ab8d0521b3f051 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Fri, Oct 9, 2015 at 10:29 AM, Mark Allman <mallman@icir.org><= /span> wrote:

I am not following along much, but why would that stop me?! :)

>=C2=A0 =C2=A0 =C2=A0This sounds like an awful idea. What if there were = congestion
>=C2=A0 =C2=A0 =C2=A0marks in those ACKs? What if the ACK has data in it= ?
>
> ECN is an issue. I would expect a piggy-backed ack is treated as a
> data message.

(1) The point of delayed ACKs is to piggyback ACKs on data packets.<= br> =C2=A0 =C2=A0 So, when a host has data to send and would naturally do so th= en
=C2=A0 =C2=A0 the ACK will get sent out with that data.

(2) At least RFC 5681 (and previous versions) have been careful to
=C2=A0 =C2=A0 note that when there is a hole in the sequence space of the =C2=A0 =C2=A0 arriving packets that ACKs should not be delayed.=C2=A0 I.e.,=
=C2=A0 =C2=A0 duplicate ACKs or ACKs with SACK information should be sent =C2=A0 =C2=A0 immediately as they are a possible indication of congestion.<= br>
(3) I would think the spirit of (2) would also apply to ECN, but I
=C2=A0 =C2=A0 am not sure that is written down anywhere (perhaps the ECN =C2=A0 =C2=A0 spec?).=C2=A0 I.e., reflecting congestion marks should not be=
=C2=A0 =C2=A0 delayed.

As = Wes and Michael pointed out, this ack policy is discussed in too many RFCs = (5681, 1112, 3449).


allman


--
http://www.icir.org/mallman/




_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm


--001a11466c2e13ab8d0521b3f051-- From nobody Fri Oct 9 16:26:03 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EC6571B4E37 for ; Fri, 9 Oct 2015 16:26:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.792 X-Spam-Level: X-Spam-Status: No, score=-1.792 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_DKIM_INVALID=0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mu5H59Tcs-2x for ; Fri, 9 Oct 2015 16:25:58 -0700 (PDT) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0106.outbound.protection.outlook.com [65.55.169.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CCAAE1B4E33 for ; Fri, 9 Oct 2015 16:25:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=qj0MXYGxVxfwqd8Qi4A5Mtumx1tUXXUElkK1K/6z78Q=; b=GIgGp4tAZNZC5ZK+sOu5L06kOlWS0eeKJuBkN9OuGty7qZMwlv3knQ4eSzYxHU9O/cwv6kGNQf9ZML2pVpAQbqOlEMPQtvA58S6m7y8hb7q4BI7J2WdjqhDsVdSpE9c4LYwCuJdF5b3WGlN/5e9vHQNLsEcM2kapcBieif3AjoQ= Received: from BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) by BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) with Microsoft SMTP Server (TLS) id 15.1.293.16; Fri, 9 Oct 2015 23:25:49 +0000 Received: from BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.173]) by BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.153]) with mapi id 15.01.0293.007; Fri, 9 Oct 2015 23:25:49 +0000 From: Praveen Balasubramanian To: Richard Scheffenegger , "Scharf, Michael (Michael)" , Wesley Eddy , "Joe Touch" Thread-Topic: [tcpm] [aqm] TCP ACK Suppression Thread-Index: AQHRAAfbmdqr5Xk9lE2VWzCubgsRu55eiGYAgAD9UYCAAnWAgIAABLaAgAAFoICAANvX0IAAT8uAgAAbEYCAAH6ACoAAADpg Date: Fri, 9 Oct 2015 23:25:49 +0000 Message-ID: References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617C4BF.6040504@mti-systems.com> <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> <96FC85CD93C14C7FADC014391BAFE03E@srichardlxp2> In-Reply-To: <96FC85CD93C14C7FADC014391BAFE03E@srichardlxp2> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com; x-originating-ip: [2001:4898:80e8:5::584] x-microsoft-exchange-diagnostics: 1; BN1PR03MB008; 5:IzcPxuWl6Mj7WZDp/6aRTbyPi45OnJ80WwT1eWTyT9/GLFXkeHuLOfQauMfGSQaAa/z4ND3bhwO1WYXm2Bpl7SlFaQlmZdng4wyWnVXT2w5oUQHP/y98kn2+/76u+CHHUKojHFGa9OF5hejJI6W+8Q==; 24:uLKactS/SOtq2Np4HQjvP+p9zZ44tWB5ME+ET4tsJ/5DGf8CuspuCPItKZgMBwUfWgxRoP+0lUF+wuE3Ghaits8KyaxI3545+TJQT5Dhifg=; 20:Z2XY5vL6RxpLqRPNiDS1en5uiK3cFHTtn5Y098XO/vYkmilpkIC7Ism85PZPAY7DGlssIl818fLQ0VKb0A33GA== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN1PR03MB008; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425024)(601004)(2401047)(520078)(8121501046)(5005006)(3002001)(61426024)(61427024); SRVR:BN1PR03MB008; BCL:0; PCL:0; RULEID:; SRVR:BN1PR03MB008; x-forefront-prvs: 0724FCD4CD x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(55674003)(189002)(24454002)(479174004)(13464003)(377454003)(199003)(74316001)(11100500001)(2900100001)(50986999)(87936001)(10290500002)(2171001)(76576001)(86612001)(92566002)(5005710100001)(86362001)(189998001)(10090500001)(5008740100001)(15975445007)(5001960100002)(81156007)(102836002)(8990500004)(5001770100001)(33656002)(10400500002)(5004730100002)(5007970100001)(93886004)(97736004)(2950100001)(106356001)(122556002)(5003600100002)(46102003)(5002640100001)(64706001)(54356999)(99286002)(101416001)(76176999)(106116001)(40100003)(105586002)(19580405001)(19580395003)(7059030)(3826002)(217873001); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR03MB008; H:BN1PR03MB008.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Oct 2015 23:25:49.3715 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR03MB008 Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , David Lang , "tcpm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 23:26:02 -0000 Agreed that delayed ACK timeout should be a function of RTT than a hardcode= d value. Windows 8 onwards this is the behavior for high latency connection= s. Windows Server uses a 1 msec delayed ACK timeout for low latency (datace= nter) connections.=20 And while we are on the topic of delayed ACKs, the bad interaction between = Nagling and delayed ACKs is well known. John Nagle's comment in HN which su= ggests a per connection logic to throttle delayed ACKs: "If you're going to do that, it's a good time to fix delayed ACK as well. A= s is well known, the interaction between delayed ACKs and the Nagle algorit= hm is awful. This is a historical accident; I implemented one, the Berkeley= guys implemented the other, and by the time I found out I was out of netwo= rking and involved with a small startup called Autodesk. So that never got = fixed. Here's how to think about that. A delayed ACK is a bet. You're betti= ng that there will be a reply, upon with an ACK can be piggybacked, before = the fixed timer runs out. If the fixed timer runs out, and you have to send= an ACK as a separate message, you lost the bet. Current TCP implementation= s will happily lose that bet forever without turning off the ACK delay. Tha= t's just wrong. The right answer is to track wins and losses on delayed and= non-delayed ACKs. Don't turn on ACK delay unless you're sending a lot of n= on-delayed ACKs closely followed by packets on which the ACK could have bee= n piggybacked. Turn it off when a delayed ACK has to be sent." -----Original Message----- From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Richard Scheffenegge= r Sent: Friday, October 9, 2015 3:53 PM To: Scharf, Michael (Michael) ; Wesley E= ddy ; Joe Touch Cc: LAUTENSCHLAEGER, Wolfram (Wolfram) ; tcpm@ietf.org; David Lang Subject: Re: [tcpm] [aqm] TCP ACK Suppression This might also be an opportunity to go more into the reasoning for the 500= ms maximum... Many implementations interpret "the delay MUST be less than 0.5 seconds" t= o trigger the delayed ACK timer at 500ms (with this time a fixed, compile-t= ime value) - while it has been shown, that making that dynamic (or a based on R= TT and pps might be a more efficient approach (especially during loss at en= d-of-stream), depending on the environment [1]. [I thought to have read a recent paper on tuning the delayed ACK timer down= to 1ms in datacenter environments, but cann't find that now]. Best regards, Richard [1] Altman, Eitan, and Tania Jim=E9nez. "Novel delayed ACK techniques for i= mproving TCP performance in multihop wireless networks." Personal Wireless = Communications. Springer Berlin Heidelberg, 2003. ----- Original Message ----- From: "Scharf, Michael (Michael)" To: "Wesley Eddy" ; "Joe Touch" Cc: "LAUTENSCHLAEGER, Wolfram (Wolfram)"=20 ; "Greg White"=20 ; ; "David Lang" Sent: Friday, October 09, 2015 5:21 PM Subject: Re: [tcpm] [aqm] TCP ACK Suppression > Yes, I have realized today myself that 793bis lacks that section. I agree= =20 > that ACK generation belongs into 793bis. > > 793bis actually may open an opportunity to rethink the exact wording that= =20 > explains this SHOULD. Personally, I believe replacing this SHOULD is not= =20 > an option for a bis document. > > But if deviations from that SHOULD are widely deployed in the meantime,=20 > 793bis could possibly comment on that. Also, having some informational=20 > reference could perhaps be an option. > > Michael > > > -----Original Message----- > From: Wesley Eddy [mailto:wes@mti-systems.com] > Sent: Friday, October 09, 2015 3:45 PM > To: Scharf, Michael (Michael); Joe Touch > Cc: LAUTENSCHLAEGER, Wolfram (Wolfram); Greg White; tcpm@ietf.org; David= =20 > Lang > Subject: Re: [tcpm] [aqm] TCP ACK Suppression > > On 10/9/2015 5:21 AM, Scharf, Michael (Michael) wrote: >> [Removing AQM, adding TCPM] >> >> >> >> Hi Joe, >> >> >> >> I don't find this text in RFC 793, but it is contained in RFC 2581, >> which is obsolete by RFC 5681. >> >> > > > To confuse matters further, there's also 1122 text with requirements > language: > > A TCP SHOULD implement a delayed ACK, but an ACK should not > be excessively delayed; in particular, the delay MUST be > less than 0.5 seconds, and in a stream of full-sized > segments there SHOULD be an ACK for at least every second > segment. > > This probably belongs in 793bis. In my view, normative acknowledgement=20 > behavior belongs in the base spec rather than particular congestion=20 > control descriptions, since there may be more than CC that depends upon i= t=20 > or makes assumptions about it. > > > -- > Wes Eddy > MTI Systems > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm >=20 _______________________________________________ tcpm mailing list tcpm@ietf.org https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri Oct 9 16:30:51 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B04731B4E4E for ; Fri, 9 Oct 2015 16:30:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Cr0NPm3e3W1a for ; Fri, 9 Oct 2015 16:30:47 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 969651B4E45 for ; Fri, 9 Oct 2015 16:30:47 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t99NU3dF015297 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 16:30:03 -0700 (PDT) To: Praveen Balasubramanian , Richard Scheffenegger , "Scharf, Michael (Michael)" , Wesley Eddy References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617C4BF.6040504@mti-systems.com> <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> <96FC85CD93C14C7FADC014391BAFE03E@srichardlxp2> From: Joe Touch Message-ID: <56184DFB.1080905@isi.edu> Date: Fri, 9 Oct 2015 16:30:03 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , David Lang , "tcpm@ietf.org" , touch@isi.edu Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2015 23:30:49 -0000 On 10/9/2015 4:25 PM, Praveen Balasubramanian wrote: > And while we are on the topic of delayed ACKs, the bad interaction > between Nagling and delayed ACKs is well known. Nagle should be disabled except for very specific protocols such as telnet using ASCII, i.e., those that a) generate large numbers of sub-MTU packets b) do NOT use multi-byte interactions (which could be split across segments and delayed due to the interaction referred to above) Joe From nobody Fri Oct 9 17:02:01 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A12DB1B4F29; Fri, 9 Oct 2015 17:01:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SAFdjDeB1_Ba; Fri, 9 Oct 2015 17:01:56 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 74B441B4F2A; Fri, 9 Oct 2015 17:01:56 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t9A01Jhq027697 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 17:01:19 -0700 (PDT) To: David Lang References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> From: Joe Touch Message-ID: <5618554F.3080103@isi.edu> Date: Fri, 9 Oct 2015 17:01:19 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Oct 2015 00:01:57 -0000 FWIW, I'm using "you" metaphorically, both here and in other posts. I.e., whomever is pushing this solution is supporting a business model that's broken. I don't much care whether the proponents are employed by that business or not. There is always tension between commercial interests (and proponents) and the model of the Internet. The Internet presents a dangerous opportunity for incredible gain without responsibility. The tragedy of the commons is very real, and this is a good example of it. It doesn't matter whether the solutions are being pushed by those profiting or by those who see only the local, short-term benefit rather then the potential long term impact. As has been discussed on TCPM, there may be ways to reduce the impact of ACKs on the net without having an intermediate device interfere. Let's have that discussion there and work together on a solution that benefits everyone - avoiding unnecessary load on the net AND preserving the E2E semantics of TCP so it can continue to evolve for future capabilities. Joe On 10/9/2015 4:31 PM, David Lang wrote: > On Fri, 9 Oct 2015, Joe Touch wrote: > >>> If you have one person trying to watch streaming video while another >>> person is uploading pictures to facebook, you can run into trouble at >>> much more even ratios. >> >> Restated, you run into trouble because you sold a service you didn't >> provision for ;-) >> >> (or, more to the point, you took a gamble that you could sell a useful >> service with a particular assumption about traffic ratios, and that >> assumption no longer holds) >> >> Again, IMO TCP isn't to be tampered with merely to support a business >> model, and that's all I see so far. > > Where did you get the idea that I work for an ISP, let alone a cable ISP > or had any influence in designing or building such networks. I don't. > > You are assigning motives here that are clearly false. > > As I keep re-stating, this is not a cable/DOCSIS issue, it applies to > all sorts of things, with Wifi and Cellular being the newest (and > arguably biggest) sources of the network flow getting quantomized. > > David Lang From nobody Fri Oct 9 17:19:25 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 20F0C1B29B0 for ; Fri, 9 Oct 2015 17:19:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.388 X-Spam-Level: X-Spam-Status: No, score=-1.388 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6PbEz09zPXhe for ; Fri, 9 Oct 2015 17:19:21 -0700 (PDT) Received: from mail-vk0-x229.google.com (mail-vk0-x229.google.com [IPv6:2607:f8b0:400c:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6AF9F1B29AD for ; Fri, 9 Oct 2015 17:19:21 -0700 (PDT) Received: by vkha6 with SMTP id a6so3865062vkh.2 for ; Fri, 09 Oct 2015 17:19:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=MVVBaMItYc1qZ63fdmTB7+gvyhyPeOVcu0UW6UJBTuY=; b=Ky/dsF8nfH+g9SNIc9YNMXG5Oeo6zokVvWtyVnNn7LFNBl+bCNnTfi2M1VThG5vJQ5 s3PMiSr54bjEHr/BfDT9gMYLldod/VpMDN0dQOjBmNgd00mweOcDxEIaldnR9knXscO/ kwboxV612wIpRkuf6v+zKvi2oSAOXIHpf7rHmid1K9a5eKddZ5Eao9+SfzAmqmM/lVFj VNpMJPx5mr3/k72zY2Z+YRAFtSEtzi/NuujEmtZOHEanuEX8gTWjm45LwOMwoa3BL++j 56bJqbdmAImv9svSLXgMsfR/VBI98stEzR12OpAfLE4mcwHIQaNTZYIzSXYP9qhNLNs1 30kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=MVVBaMItYc1qZ63fdmTB7+gvyhyPeOVcu0UW6UJBTuY=; b=QaGa1sW3tltdKOjMr2F72qhhBUvambgFwQv2BZDLhjfrGWQM7o/FREfJ7js9rikFoE 9HZPFU772N05yl8wCVFpZsZnCp6zsIBSZePXwUTTllTrcz4L8Eg3ohMPXd6Ml/1ETmHZ ixkY3mGEV1yHDzM420PqOM2LoovNGPlAAkBUfqmpl0cZso1VboOehbbskKixldxVJgia 821hUQlKbrjWnSG8Thh9QuGkCWnbkRj9SHkFLi9QJIcZWCCb3U7TJ1wrytF1vnogvzpL tEpu9aBH6RveDly6bxb9/AeCxAam7eFO5k0k2nhI+aMq2ZdHxT3Yp30K53FSzYCTSRS5 qXGQ== X-Gm-Message-State: ALoCoQlN6pc9C9hB93eRVRBdCnTRELO52Uf4FPNLxXyhvI8j+9yQwPdTnr7iQfEdUSe2WQk8Taxl X-Received: by 10.31.6.20 with SMTP id 20mr3898973vkg.158.1444436360344; Fri, 09 Oct 2015 17:19:20 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.153.134 with HTTP; Fri, 9 Oct 2015 17:18:40 -0700 (PDT) In-Reply-To: References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617C4BF.6040504@mti-systems.com> <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> <96FC85CD93C14C7FADC014391BAFE03E@srichardlxp2> From: Yuchung Cheng Date: Fri, 9 Oct 2015 17:18:40 -0700 Message-ID: To: Praveen Balasubramanian Content-Type: multipart/alternative; boundary=001a1143d7ca28fa800521b50cb7 Archived-At: Cc: David Lang , "tcpm@ietf.org" , Joe Touch , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Oct 2015 00:19:24 -0000 --001a1143d7ca28fa800521b50cb7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Fri, Oct 9, 2015 at 4:25 PM, Praveen Balasubramanian wrote: > Agreed that delayed ACK timeout should be a function of RTT than a > hardcoded value. Windows 8 onwards this is the behavior for high latency > connections. Windows Server uses a 1 msec delayed ACK timeout for low > latency (datacenter) connections. > > And while we are on the topic of delayed ACKs, the bad interaction betwee= n > Nagling and delayed ACKs is well known. John Nagle's comment in HN which > suggests a per connection logic to throttle delayed ACKs: > "If you're going to do that, it's a good time to fix delayed ACK as well. > As is well known, the interaction between delayed ACKs and the Nagle > algorithm is awful. This is a historical accident; I implemented one, the > Berkeley guys implemented the other, and by the time I found out I was ou= t > of networking and involved with a small startup called Autodesk. So that > never got fixed. Here's how to think about that. A delayed ACK is a bet. > You're betting that there will be a reply, upon with an ACK can be > piggybacked, before the fixed timer runs out. If the fixed timer runs out= , > and you have to send an ACK as a separate message, you lost the bet. > Current TCP implementations will happily lose that bet forever without > turning off the ACK delay. That's just wrong. The right answer is to trac= k > wins and losses on delayed and non-delayed ACKs. Don't turn on ACK delay > unless you're sending a lot of non-delayed ACKs closely followed by packe= ts > on which the ACK could have been piggybacked. Turn it off when a delayed > ACK has to be sent." > Well .. Mr. Nagle will be glad to learn that Linux has implemented his (or similar) idea for years. It's called QUICK_ACK. The decision window is a bit short and can use some tuning. But it is not "never" fixed. Agree on that 500ms is ridiculous. > -----Original Message----- > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Richard > Scheffenegger > Sent: Friday, October 9, 2015 3:53 PM > To: Scharf, Michael (Michael) ; Wesley > Eddy ; Joe Touch > Cc: LAUTENSCHLAEGER, Wolfram (Wolfram) < > wolfram.lautenschlaeger@alcatel-lucent.com>; tcpm@ietf.org; David Lang < > david@lang.hm> > Subject: Re: [tcpm] [aqm] TCP ACK Suppression > > This might also be an opportunity to go more into the reasoning for the > 500ms maximum... > > Many implementations interpret "the delay MUST be less than 0.5 seconds" > to trigger the delayed ACK timer at 500ms (with this time a fixed, > compile-time > value) - while it has been shown, that making that dynamic (or a based on > RTT and pps might be a more efficient approach (especially during loss at > end-of-stream), depending on the environment [1]. > > [I thought to have read a recent paper on tuning the delayed ACK timer > down to 1ms in datacenter environments, but cann't find that now]. > > Best regards, > Richard > > > > > [1] Altman, Eitan, and Tania Jim=C3=A9nez. "Novel delayed ACK techniques = for > improving TCP performance in multihop wireless networks." Personal Wirele= ss > Communications. Springer Berlin Heidelberg, 2003. > > > ----- Original Message ----- > From: "Scharf, Michael (Michael)" > To: "Wesley Eddy" ; "Joe Touch" > Cc: "LAUTENSCHLAEGER, Wolfram (Wolfram)" > ; "Greg White" > ; ; "David Lang" > Sent: Friday, October 09, 2015 5:21 PM > Subject: Re: [tcpm] [aqm] TCP ACK Suppression > > > > Yes, I have realized today myself that 793bis lacks that section. I agr= ee > > that ACK generation belongs into 793bis. > > > > 793bis actually may open an opportunity to rethink the exact wording th= at > > explains this SHOULD. Personally, I believe replacing this SHOULD is no= t > > an option for a bis document. > > > > But if deviations from that SHOULD are widely deployed in the meantime, > > 793bis could possibly comment on that. Also, having some informational > > reference could perhaps be an option. > > > > Michael > > > > > > -----Original Message----- > > From: Wesley Eddy [mailto:wes@mti-systems.com] > > Sent: Friday, October 09, 2015 3:45 PM > > To: Scharf, Michael (Michael); Joe Touch > > Cc: LAUTENSCHLAEGER, Wolfram (Wolfram); Greg White; tcpm@ietf.org; Davi= d > > Lang > > Subject: Re: [tcpm] [aqm] TCP ACK Suppression > > > > On 10/9/2015 5:21 AM, Scharf, Michael (Michael) wrote: > >> [Removing AQM, adding TCPM] > >> > >> > >> > >> Hi Joe, > >> > >> > >> > >> I don't find this text in RFC 793, but it is contained in RFC 2581, > >> which is obsolete by RFC 5681. > >> > >> > > > > > > To confuse matters further, there's also 1122 text with requirements > > language: > > > > A TCP SHOULD implement a delayed ACK, but an ACK should not > > be excessively delayed; in particular, the delay MUST be > > less than 0.5 seconds, and in a stream of full-sized > > segments there SHOULD be an ACK for at least every second > > segment. > > > > This probably belongs in 793bis. In my view, normative acknowledgement > > behavior belongs in the base spec rather than particular congestion > > control descriptions, since there may be more than CC that depends upon > it > > or makes assumptions about it. > > > > > > -- > > Wes Eddy > > MTI Systems > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm > > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > --001a1143d7ca28fa800521b50cb7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Fri, Oct 9, 2015 at 4:25 PM, Praveen Balasubramanian <pravb@micro= soft.com> wrote:
Agreed t= hat delayed ACK timeout should be a function of RTT than a hardcoded value.= Windows 8 onwards this is the behavior for high latency connections. Windo= ws Server uses a 1 msec delayed ACK timeout for low latency (datacenter) co= nnections.

And while we are on the topic of delayed ACKs, the bad interaction between = Nagling and delayed ACKs is well known. John Nagle's comment in HN whic= h suggests a per connection logic to throttle delayed ACKs:
"If you're going to do that, it's a good time to fix delayed A= CK as well. As is well known, the interaction between delayed ACKs and the = Nagle algorithm is awful. This is a historical accident; I implemented one,= the Berkeley guys implemented the other, and by the time I found out I was= out of networking and involved with a small startup called Autodesk. So th= at never got fixed. Here's how to think about that. A delayed ACK is a = bet. You're betting that there will be a reply, upon with an ACK can be= piggybacked, before the fixed timer runs out. If the fixed timer runs out,= and you have to send an ACK as a separate message, you lost the bet. Curre= nt TCP implementations will happily lose that bet forever without turning o= ff the ACK delay. That's just wrong. The right answer is to track wins = and losses on delayed and non-delayed ACKs. Don't turn on ACK delay unl= ess you're sending a lot of non-delayed ACKs closely followed by packet= s on which the ACK could have been piggybacked. Turn it off when a delayed = ACK has to be sent."
Well .. Mr. Nagle will be gl= ad to learn that Linux has implemented his (or similar) idea for years. It&= #39;s called QUICK_ACK. The decision window is a bit short and can use some= tuning. But it is not "never" fixed.

Ag= ree on that 500ms is ridiculous.


-----Original Message-----
From: tcpm [mailto:tcpm-bounces@ie= tf.org] On Behalf Of Richard Scheffenegger
Sent: Friday, October 9, 2015 3:53 PM
To: Scharf, Michael (Michael) <michael.scharf@alcatel-lucent.com>; Wesley Eddy <wes@mti-systems.com>; Joe Touch &= lt;touch@isi.edu>
Cc: LAUTENSCHLAEGER, Wolfram (Wolfram) <wolfram.lautenschlaeger@alcatel-lucent.com>; tcpm@ietf.org; David Lang <<= a href=3D"mailto:david@lang.hm">david@lang.hm>
Subject: Re: [tcpm] [aqm] TCP ACK Suppression

This might also be an opportunity to go more into the reasoning for the 500= ms maximum...

Many implementations interpret=C2=A0 "the delay MUST be less than 0.5 = seconds" to trigger the delayed ACK timer at 500ms (with this time a f= ixed, compile-time
value) - while it has been shown, that making that dynamic (or a based on R= TT and pps might be a more efficient approach (especially during loss at en= d-of-stream), depending on the environment [1].

[I thought to have read a recent paper on tuning the delayed ACK timer down= to 1ms in datacenter environments, but cann't find that now].

Best regards,
=C2=A0 Richard




[1] Altman, Eitan, and Tania Jim=C3=A9nez. "Novel delayed ACK techniqu= es for improving TCP performance in multihop wireless networks." Perso= nal Wireless Communications. Springer Berlin Heidelberg, 2003.


----- Original Message -----
From: "Scharf, Michael (Michael)" <michael.scharf@alcatel-lucent.com>
To: "Wesley Eddy" <wes@= mti-systems.com>; "Joe Touch" <touch@isi.edu>
Cc: "LAUTENSCHLAEGER, Wolfram (Wolfram)"
<wolfram.l= autenschlaeger@alcatel-lucent.com>; "Greg White"
<g.white@cablelabs.com>;= <tcpm@ietf.org>; "David La= ng" <david@lang.hm>
Sent: Friday, October 09, 2015 5:21 PM
Subject: Re: [tcpm] [aqm] TCP ACK Suppression


> Yes, I have realized today myself that 793bis lacks that section. I ag= ree
> that ACK generation belongs into 793bis.
>
> 793bis actually may open an opportunity to rethink the exact wording t= hat
> explains this SHOULD. Personally, I believe replacing this SHOULD is n= ot
> an option for a bis document.
>
> But if deviations from that SHOULD are widely deployed in the meantime= ,
> 793bis could possibly comment on that. Also, having some informational=
> reference could perhaps be an option.
>
> Michael
>
>
> -----Original Message-----
> From: Wesley Eddy [mailto:wes@m= ti-systems.com]
> Sent: Friday, October 09, 2015 3:45 PM
> To: Scharf, Michael (Michael); Joe Touch
> Cc: LAUTENSCHLAEGER, Wolfram (Wolfram); Greg White; tcpm@ietf.org; David
> Lang
> Subject: Re: [tcpm] [aqm] TCP ACK Suppression
>
> On 10/9/2015 5:21 AM, Scharf, Michael (Michael) wrote:
>> [Removing AQM, adding TCPM]
>>
>>
>>
>> Hi Joe,
>>
>>
>>
>> I don't find this text in RFC 793, but it is contained in RFC = 2581,
>> which is obsolete by RFC 5681.
>>
>>
>
>
> To confuse matters further, there's also 1122 text with requiremen= ts
> language:
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 A TCP SHOULD implement a dela= yed ACK, but an ACK should not
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 be excessively delayed; in pa= rticular, the delay MUST be
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 less than 0.5 seconds, and in= a stream of full-sized
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 segments there SHOULD be an A= CK for at least every second
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 segment.
>
> This probably belongs in 793bis.=C2=A0 In my view, normative acknowled= gement
> behavior belongs in the base spec rather than particular congestion > control descriptions, since there may be more than CC that depends upo= n it
> or makes assumptions about it.
>
>
> --
> Wes Eddy
> MTI Systems
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm

--001a1143d7ca28fa800521b50cb7-- From nobody Fri Oct 9 17:40:01 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E91C21B2E27; Fri, 9 Oct 2015 17:39:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6oOjnguDEfGF; Fri, 9 Oct 2015 17:39:59 -0700 (PDT) Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F01341B2E04; Fri, 9 Oct 2015 17:39:58 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id t9A0dWQf021332 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 17:39:32 -0700 (PDT) To: David Lang References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> From: Joe Touch X-Enigmail-Draft-Status: N1110 Message-ID: <56185E44.9050702@isi.edu> Date: Fri, 9 Oct 2015 17:39:32 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-MailScanner-ID: t9A0dWQf021332 X-ISI-4-69-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Oct 2015 00:40:00 -0000 On 10/9/2015 5:22 PM, David Lang wrote: > You don't want to acknowlege it, but TCP is broken in the face of > excessive buffering. Arguably, buffering was broken and failed to provide the feedback to TCP (see next paragraph) > TCPM isn't fixing that, grassroots efforts are > developing the fixes and AQM is formalizing the results. TCPM fixed it in 2001 by providing the flags for ECN, which was enabled by default in Windows since 2012. ALTQ support for ECN has been around for nearly that long. What's changed? Not the TCP reaction (except in extreme cases such as for datacenters) but the router algs. And getting the router algs into routers - esp. home devices. > TCP is also broken in the face of current and within-the-next-year > future wifi technologies. AQM helps some here, and there is again > outside efforts to address the problem. TCPM hasn't done anything that > I've heard of other than possibly say that the network technology is > broken and shouldn't be used. You might start tracking the TCPM list. We've been talking today about how to reduce the ACKs by direct action of the source. Joe From nobody Fri Oct 9 20:06:04 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8F9BC1B5426 for ; Fri, 9 Oct 2015 20:06:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WlYEkiIRpABQ for ; Fri, 9 Oct 2015 20:06:02 -0700 (PDT) Received: from fruitcake.ICSI.Berkeley.EDU (fruitcake.ICSI.Berkeley.EDU [192.150.186.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CB6E91B5416 for ; Fri, 9 Oct 2015 20:06:02 -0700 (PDT) Received: from lawyers.icir.org (envoy.icir.org [192.150.187.30]) by fruitcake.ICSI.Berkeley.EDU (8.12.11.20060614/8.12.11) with ESMTP id t9A360qJ002804; Fri, 9 Oct 2015 20:06:01 -0700 (PDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id 667C52D74802; Fri, 9 Oct 2015 23:06:00 -0400 (EDT) To: Yuchung Cheng From: Mark Allman In-Reply-To: Organization: International Computer Science Institute (ICSI) Song-of-the-Day: Down on the Corner X-URL-0: http://www.icir.org/mallman-files/Document64429.pdf MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-------459435943823349593450"; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Fri, 09 Oct 2015 23:06:00 -0400 Message-ID: <20864.1444446360@lawyers.icir.org> Sender: mallman@icir.org Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: mallman@icir.org List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Oct 2015 03:06:03 -0000 --=-------459435943823349593450 Content-Type: text/plain > this ack policy is discussed in too many RFCs (5681, 1112, 3449). And his answer was to put it in one more! :) Everything about TCP is discussed in too many RFCs. Who can keep track?! allman --=-------459435943823349593450 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlYYgJgACgkQWyrrWs4yIs43/wCfZ0zYewZyCvNqpAVbeXG4uvJx SG4AniwC9Ri1cCIsSP4PFW0fKK0Vn+u1 =vdoK -----END PGP SIGNATURE----- --=-------459435943823349593450-- From nobody Fri Oct 9 21:20:14 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E23CD1B55B5; Fri, 9 Oct 2015 21:20:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jrpuvt8DrifM; Fri, 9 Oct 2015 21:20:11 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 88EF31B55B6; Fri, 9 Oct 2015 21:20:06 -0700 (PDT) Received: from [192.168.1.156] (cpe-172-250-225-10.socal.res.rr.com [172.250.225.10]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t9A4JIw5009245 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 21:19:32 -0700 (PDT) To: David Lang References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> From: Joe Touch Message-ID: <561891C3.90004@isi.edu> Date: Fri, 9 Oct 2015 21:19:15 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Oct 2015 04:20:12 -0000 On 10/9/2015 7:14 PM, David Lang wrote: > The problem with proposals like that is just like RFC3449 says, the > source can't know what the network looks like to decide if there is a > benefit to reducing ACKs. To understand your position, is your conclusion that if it benefits a single place in the network, then that's enough of a view to know that it's safe and beneficial throughout the network? Joe From nobody Fri Oct 9 23:24:51 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5EC9C1B30A7; Fri, 9 Oct 2015 23:24:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lWgoZUU5X1QE; Fri, 9 Oct 2015 23:24:47 -0700 (PDT) Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 753EF1B30A5; Fri, 9 Oct 2015 23:24:47 -0700 (PDT) Received: from [192.168.1.156] (cpe-172-250-225-10.socal.res.rr.com [172.250.225.10]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id t9A6OC0p025858 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 9 Oct 2015 23:24:14 -0700 (PDT) To: David Lang References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> From: Joe Touch X-Enigmail-Draft-Status: N1110 Message-ID: <5618AF0A.4010101@isi.edu> Date: Fri, 9 Oct 2015 23:24:10 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-MailScanner-ID: t9A6OC0p025858 X-ISI-4-69-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Oct 2015 06:24:48 -0000 On 10/9/2015 10:58 PM, David Lang wrote: > RFC3449 doesn't completely address the current situation, but it > provides a very good place to start, and it seems to me that the > solitions it explores to address the conerns that it (and you) raise are > actually being addressed pretty completely. There are still some areas > to talk about (ECN interaction for example) and we wshould be talking > about those issues rather than arguing that the proposal violates holy > writ. I'm OK with the recommendations for the endpoints to modify their ACK behavior, but still am concerned that stripping out ACKs in the middle of the network is problematic for a number of reasons, only some of which are addressed in RFC3449 (which also recommends against deploying them in the general Internet, so if that's your justification then it's very clearly in direct opposition). Joe From nobody Sat Oct 10 10:23:15 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C54511B4201 for ; Sat, 10 Oct 2015 10:23:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dlz2RKysnUbO for ; Sat, 10 Oct 2015 10:23:11 -0700 (PDT) Received: from smtp93.iad3a.emailsrvr.com (smtp93.iad3a.emailsrvr.com [173.203.187.93]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 33BFD1B41FF for ; Sat, 10 Oct 2015 10:23:11 -0700 (PDT) Received: from smtp20.relay.iad3a.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp20.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 13AD31800D7; Sat, 10 Oct 2015 13:23:10 -0400 (EDT) Received: from app36.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp20.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id F215C180097; Sat, 10 Oct 2015 13:23:09 -0400 (EDT) X-Sender-Id: dpreed@reed.com Received: from app36.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by 0.0.0.0:25 (trex/5.4.2); Sat, 10 Oct 2015 17:23:10 GMT Received: from reed.com (localhost.localdomain [127.0.0.1]) by app36.wa-webapps.iad3a (Postfix) with ESMTP id DE91C3800DA; Sat, 10 Oct 2015 13:23:09 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@reed.com, from: dpreed@reed.com) with HTTP; Sat, 10 Oct 2015 13:23:09 -0400 (EDT) Date: Sat, 10 Oct 2015 13:23:09 -0400 (EDT) From: dpreed@reed.com To: "Eric Dumazet" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20151010132309000000_28623" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: <1444429447.27760.102.camel@edumazet-glaptop2.roam.corp.google.com> References: <56183141.30407@isi.edu> <1444427647.27760.92.camel@edumazet-glaptop2.roam.corp.google.com> <56183AC7.3070204@isi.edu> <1444429447.27760.102.camel@edumazet-glaptop2.roam.corp.google.com> X-Auth-ID: dpreed@reed.com Message-ID: <1444497789.90931278@apps.rackspace.com> X-Mailer: webmail/11.6.1-RC Archived-At: Cc: Christian Huitema , "tcpm@ietf.org" , Joe Touch Subject: Re: [tcpm] =?utf-8?q?ACK_management_=28was_Re=3AACKSuppression=29?= X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Oct 2015 17:23:13 -0000 ------=_20151010132309000000_28623 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AThe idea of "dallying" (waiting a small amount of time before sending an= ACK in hope that there will be a piggyback opportunity) was invented, as f= ar as I know, by my colleague Dave Clark in is original expermental TCP imp= lementation on the Xerox Alto (written by Dave in BCPL). That was done aro= und 1978, shortly after Xerox donated about 20 Altos to MIT's Lab for Compu= ter Science.=0A =0ASo it is an old idea, and in some cases it helps with th= roughput- in particular, when there is a bidirectional flow of data (as hap= pens in protocols like SMTP), or where the destination has the ability to b= uffer a large number of early TCP segments.=0A =0AHowever, at high speed, (= 10 GigE, for example), this "dallying" typically requires the receiving hos= t to take an extra interrupt per ack - basically a timer interrupt needs to= be set up to go off after the dally delay expires. In most OS's, the over= head in timer management code to insert a new event into the queue, operate= on the timer, handle the fact that the interrupt might arrive on a differe= nt cpu core than the one that originally received the packet, ... well, yo= u get the idea.=0A =0AMaking a statement that "dallying for a millisecond" = should always be done doesn't take into account the costs of doing this, vs= . the benefit, which is situation dependent. Most ACKs also adjust the rec= eive window, which reflects into the flow control loop the actual rate at w= hich the receiving process consumes the data. Keeping the window "constant= -sized" and no more than the measured RTT * observed bottleneck rate is als= o beneficial, because it enables fair usage of the bottleneck link, without= oscillations due to receiving process consumption being coupled into the b= ottleneck link queue size control. More ACKs can adjust the receive windo= w more gradually.=0A =0A(it should be noted that Dave also named the so-cal= led "silly window syndrome" using his BCPL experimental TCP at about the sa= me time. Silly window is one of the simplest issues that arise from tuning= the receive window to track consumption and avoid intermediate packet buil= dup)=0A=0A=0AOn Friday, October 9, 2015 6:24pm, "Eric Dumazet" said:=0A=0A=0A=0A> On Fri, 2015-10-09 at 15:08 -0700, Joe Touc= h wrote:=0A> >=0A> > On 10/9/2015 2:54 PM, Eric Dumazet wrote:=0A> =0A> > >= We (TCP stacks) spend considerable amount of energy processing ACK=0A> > >= packets all over the world. What a waste.=0A> >=0A> > How silly is it that= UPS collects signatures for billions of packages a=0A> > year when only a = very small percentage are ever challenged?=0A> =0A> It is rare UPS collects= 400,000 identical signatures per second from a=0A> single customer ;)=0A> = =0A> =0A> TCP receivers should not send ACK immediately.=0A> They should be= t on the immediate future.=0A> =0A> I did a ~50 lines prototype patch in li= nux to arm a 1 ms timer,=0A> instead of immediately sending one ACK.=0A> = =0A> I did this for SACK packets, trying to cope with reordering fabrics,= =0A> (per packet load balancing, not caring on flow hash),=0A> but patch co= uld be extended quite easily with a more fine grained timer=0A> (like min(r= tt/10, 1ms))=0A> =0A> If another packet(s) is(are) coming before timer expi= res, we avoid=0A> sending all these useless floods of ACK packets, and avoi= d intermediate=0A> nodes trying to be smart and do their own ACK compressio= n.=0A> =0A> =0A> =0A> ------=_20151010132309000000_28623 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

The = idea of "dallying" (waiting a small amount of time before sending an ACK in= hope that there will be a piggyback opportunity) was invented, as far as I= know, by my colleague Dave Clark in is original expermental TCP implementa= tion on the Xerox Alto (written by Dave in BCPL).  That was done aroun= d 1978, shortly after Xerox donated about 20 Altos to MIT's Lab for Compute= r Science.

=0A

 

=0A

So it is an old idea, and in some cases it helps with throughp= ut- in particular, when there is a bidirectional flow of data (as happens i= n protocols like SMTP), or where the destination has the ability to buffer = a large number of early TCP segments.

=0A

&n= bsp;

=0A

However, at high speed, (10 GigE, f= or example), this "dallying" typically requires the receiving host to take = an extra interrupt per ack - basically a timer interrupt needs to be set up= to go off after the dally delay expires.  In most OS's, the overhead = in timer management code to insert a new event into the queue, operate on t= he timer, handle the fact that the interrupt might arrive on a different cp= u core than the one that originally received the packet, ...  well, yo= u get the idea.

=0A

 

=0A

Making a statement that "dallying for a millisecond" shou= ld always be done doesn't take into account the costs of doing this, vs. th= e benefit, which is situation dependent.  Most ACKs also adjust the re= ceive window, which reflects into the flow control loop the actual rate at = which the receiving process consumes the data.  Keeping the window "co= nstant-sized" and no more than the measured RTT * observed bottleneck rate = is also beneficial, because it enables fair usage of the bottleneck link, w= ithout oscillations due to receiving process consumption being coupled into= the bottleneck link queue size control.   More ACKs can adjust the re= ceive window more gradually.

=0A

 

= =0A

(it should be noted that Dave also named th= e so-called "silly window syndrome" using his BCPL experimental TCP at abou= t the same time.  Silly window is one of the simplest issues that aris= e from tuning the receive window to track consumption and avoid intermediat= e packet buildup)

=0A=0A



On Friday, October= 9, 2015 6:24pm, "Eric Dumazet" <eric.dumazet@gmail.com> said:
<= br />

=0A
=0A

, touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2015 00:57:06 -0000 On 10/10/2015 4:06 PM, Dave Taht wrote: > I just wanted to say that I'm with dlang here... aggregated and > asymmetric transmits in wifi, cell, and cable, are here to stay. Deal > with it. Right until they're encrypted, then we'll be back to square 1. Time to deal with the case where the net isn't involved in tampering with E2E connections, IMO. Joe From nobody Sat Oct 10 22:14:23 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 757031AD0A3; Sat, 10 Oct 2015 22:14:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 719hviQVR2EV; Sat, 10 Oct 2015 22:14:19 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 167EA1AD0A1; Sat, 10 Oct 2015 22:14:19 -0700 (PDT) Received: from [192.168.1.156] (cpe-172-250-225-10.socal.res.rr.com [172.250.225.10]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t9B5DnYg021732 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Sat, 10 Oct 2015 22:13:51 -0700 (PDT) To: David Lang , Jonathan Morton References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> From: Joe Touch Message-ID: <5619F00A.2040009@isi.edu> Date: Sat, 10 Oct 2015 22:13:46 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2015 05:14:20 -0000 On 10/10/2015 8:45 PM, David Lang wrote: > But I actually view this as a tragedy of the commons situation. Sending > the extra packets doesn't hurt you, but it does mean that you are using > more airtime than you need, and that hurts everyone (including, > eventually, you) So, basically, you appreciate the tragedy of the commons when it's your tragedy and their commons, but not the other way around? (using 'sed') s/sending the extra/altering the transport/ s/using more airtime/altering E2E semantics/ Joe From nobody Sun Oct 11 07:41:29 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3CB071A8A68 for ; Sun, 11 Oct 2015 07:41:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.909 X-Spam-Level: X-Spam-Status: No, score=-6.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8DwTtRwmWbIK for ; Sun, 11 Oct 2015 07:41:24 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1762E1A8A67 for ; Sun, 11 Oct 2015 07:41:23 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 66F305114BA96; Sun, 11 Oct 2015 14:41:18 +0000 (GMT) Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t9BEfKjG010315 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 11 Oct 2015 16:41:20 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.03.0195.001; Sun, 11 Oct 2015 16:41:20 +0200 From: "Scharf, Michael (Michael)" To: Yuchung Cheng , Praveen Balasubramanian , Wesley Eddy Thread-Topic: Delayed ACKs vs. Nagle (was: RE: [tcpm] [aqm] TCP ACK Suppression) Thread-Index: AQHRBDLkoeCMEiaauUmhCORc+sdhww== Date: Sun, 11 Oct 2015 14:41:20 +0000 Message-ID: <655C07320163294895BBADA28372AF5D484FFC3F@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617C4BF.6040504@mti-systems.com> <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> <96FC85CD93C14C7FADC014391BAFE03E@srichardlxp2> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: multipart/alternative; boundary="_000_655C07320163294895BBADA28372AF5D484FFC3FFR712WXCHMBA15z_" MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" Subject: [tcpm] Delayed ACKs vs. Nagle (was: RE: [aqm] TCP ACK Suppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2015 14:41:28 -0000 --_000_655C07320163294895BBADA28372AF5D484FFC3FFR712WXCHMBA15z_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 QXMgZmFyIGFzIEkgcmVjYWxsIChXZXMgY2FuIGNvcnJlY3QgbWUgaWYgSSBhbSB3cm9uZykgd2Ug aGF2ZSBhbiBvcGVuIGlzc3VlIGluIDc5M2JpcyByZWdhcmRpbmcgdGhhdCBpc3N1ZS4NCg0KU2Vl IGRyYWZ0LWlldGYtdGNwbS1yZmM3OTNiaXMtMDEgU2VjdGlvbiA0Og0KDQogICAxMS4gIGxvb2sg YXQgcG9zc2libGUgbWVudGlvbiBvZiBkcmFmdC1taW5zaGFsbC1uYWdsZSAoZS5nLiBhcyBpbg0K ICAgICAgICBMaW51eCkNCg0KVGhlIGNoYWxsZW5nZSBpcyB0aGF0IGZvciA3OTNiaXMgd2Ugd291 bGQgbmVlZCBhIFByb3Bvc2VkIFN0YW5kYXJkIFJGQyBzcGVjaWZ5aW5nIGhvdyB0byBzb3J0IG91 dCB0aGlzIGludGVyYWN0aW9uLCB1bmxlc3Mgd2UgZXh0ZW5kIHRoZSBzY29wZSBvZiA3OTNiaXMg YmV5b25kIHdoYXQgd2UgYWdyZWVkIHVwb24gKGkuZS4sIG9ubHkgY2xhcmlmaWNhdGlvbiwgbm8g cHJvdG9jb2wgY2hhbmdlcykuDQoNCkkgaGF2ZSBwZXJzb25hbGx5IG5ldmVyIGxvb2tlZCBhdCBo b3cgdGhhdCBsb2dpYyBpcyBhY3R1YWxseSBpbXBsZW1lbnRlZCBpbiBkaWZmZXJlbnQgc3RhY2tz LCBhbmQgSSBhY3R1YWxseSBkb27igJl0IGtub3cgaWYgdGhlcmUgaGFzIGV2ZXIgYmVlbiBjb21t dW5pdHkgY29uc2Vuc3VzIG9uIGhvdyB0aGUgZXhhY3Qgc29sdXRpb24gdG8gdGhhdCBpbnRlcmFj dGlvbiBzaG91bGQgbG9vayBsaWtlLg0KDQpCdXQgaWYgd2UgaGF2ZSBhIGdhcCB0byB3aGF0IGlz IHdpZGVseSBpbXBsZW1lbnRlZCwgYW5kIGlmIGl0IHdhcyByZWFsaXN0aWMgdG8gcHVibGlzaCBh IFBTIFJGQyBiZWZvcmUgY29tcGxldGluZyA3OTNiaXMgKEkgZ3Vlc3MgaXQgY291bGQgYmUgYSAy LXBhZ2UgUkZDKSwgSeKAmWQgYXJndWUgaW4gZmF2b3Igb2YgYSDigJxmYXN0IHRyYWNr4oCdIFBT IHB1YmxpY2F0aW9uLiAoV2VsbCwgd2l0aCDigJhmYXN04oCZIGJlaW5nIHJlbGF0aXZlIHRvIG91 ciBub3JtYWwgUFMgcHVibGljYXRpb24gc3BlZWQuKQ0KDQpNaWNoYWVsDQoNCg0KRnJvbTogWXVj aHVuZyBDaGVuZyBbbWFpbHRvOnljaGVuZ0Bnb29nbGUuY29tXQ0KU2VudDogU2F0dXJkYXksIE9j dG9iZXIgMTAsIDIwMTUgMjoxOSBBTQ0KVG86IFByYXZlZW4gQmFsYXN1YnJhbWFuaWFuDQpDYzog UmljaGFyZCBTY2hlZmZlbmVnZ2VyOyBTY2hhcmYsIE1pY2hhZWwgKE1pY2hhZWwpOyBXZXNsZXkg RWRkeTsgSm9lIFRvdWNoOyBMQVVURU5TQ0hMQUVHRVIsIFdvbGZyYW0gKFdvbGZyYW0pOyBEYXZp ZCBMYW5nOyB0Y3BtQGlldGYub3JnDQpTdWJqZWN0OiBSZTogW3RjcG1dIFthcW1dIFRDUCBBQ0sg U3VwcHJlc3Npb24NCg0KDQoNCk9uIEZyaSwgT2N0IDksIDIwMTUgYXQgNDoyNSBQTSwgUHJhdmVl biBCYWxhc3VicmFtYW5pYW4gPHByYXZiQG1pY3Jvc29mdC5jb208bWFpbHRvOnByYXZiQG1pY3Jv c29mdC5jb20+PiB3cm90ZToNCkFncmVlZCB0aGF0IGRlbGF5ZWQgQUNLIHRpbWVvdXQgc2hvdWxk IGJlIGEgZnVuY3Rpb24gb2YgUlRUIHRoYW4gYSBoYXJkY29kZWQgdmFsdWUuIFdpbmRvd3MgOCBv bndhcmRzIHRoaXMgaXMgdGhlIGJlaGF2aW9yIGZvciBoaWdoIGxhdGVuY3kgY29ubmVjdGlvbnMu IFdpbmRvd3MgU2VydmVyIHVzZXMgYSAxIG1zZWMgZGVsYXllZCBBQ0sgdGltZW91dCBmb3IgbG93 IGxhdGVuY3kgKGRhdGFjZW50ZXIpIGNvbm5lY3Rpb25zLg0KDQpBbmQgd2hpbGUgd2UgYXJlIG9u IHRoZSB0b3BpYyBvZiBkZWxheWVkIEFDS3MsIHRoZSBiYWQgaW50ZXJhY3Rpb24gYmV0d2VlbiBO YWdsaW5nIGFuZCBkZWxheWVkIEFDS3MgaXMgd2VsbCBrbm93bi4gSm9obiBOYWdsZSdzIGNvbW1l bnQgaW4gSE4gd2hpY2ggc3VnZ2VzdHMgYSBwZXIgY29ubmVjdGlvbiBsb2dpYyB0byB0aHJvdHRs ZSBkZWxheWVkIEFDS3M6DQoiSWYgeW91J3JlIGdvaW5nIHRvIGRvIHRoYXQsIGl0J3MgYSBnb29k IHRpbWUgdG8gZml4IGRlbGF5ZWQgQUNLIGFzIHdlbGwuIEFzIGlzIHdlbGwga25vd24sIHRoZSBp bnRlcmFjdGlvbiBiZXR3ZWVuIGRlbGF5ZWQgQUNLcyBhbmQgdGhlIE5hZ2xlIGFsZ29yaXRobSBp cyBhd2Z1bC4gVGhpcyBpcyBhIGhpc3RvcmljYWwgYWNjaWRlbnQ7IEkgaW1wbGVtZW50ZWQgb25l LCB0aGUgQmVya2VsZXkgZ3V5cyBpbXBsZW1lbnRlZCB0aGUgb3RoZXIsIGFuZCBieSB0aGUgdGlt ZSBJIGZvdW5kIG91dCBJIHdhcyBvdXQgb2YgbmV0d29ya2luZyBhbmQgaW52b2x2ZWQgd2l0aCBh IHNtYWxsIHN0YXJ0dXAgY2FsbGVkIEF1dG9kZXNrLiBTbyB0aGF0IG5ldmVyIGdvdCBmaXhlZC4g SGVyZSdzIGhvdyB0byB0aGluayBhYm91dCB0aGF0LiBBIGRlbGF5ZWQgQUNLIGlzIGEgYmV0LiBZ b3UncmUgYmV0dGluZyB0aGF0IHRoZXJlIHdpbGwgYmUgYSByZXBseSwgdXBvbiB3aXRoIGFuIEFD SyBjYW4gYmUgcGlnZ3liYWNrZWQsIGJlZm9yZSB0aGUgZml4ZWQgdGltZXIgcnVucyBvdXQuIElm IHRoZSBmaXhlZCB0aW1lciBydW5zIG91dCwgYW5kIHlvdSBoYXZlIHRvIHNlbmQgYW4gQUNLIGFz IGEgc2VwYXJhdGUgbWVzc2FnZSwgeW91IGxvc3QgdGhlIGJldC4gQ3VycmVudCBUQ1AgaW1wbGVt ZW50YXRpb25zIHdpbGwgaGFwcGlseSBsb3NlIHRoYXQgYmV0IGZvcmV2ZXIgd2l0aG91dCB0dXJu aW5nIG9mZiB0aGUgQUNLIGRlbGF5LiBUaGF0J3MganVzdCB3cm9uZy4gVGhlIHJpZ2h0IGFuc3dl ciBpcyB0byB0cmFjayB3aW5zIGFuZCBsb3NzZXMgb24gZGVsYXllZCBhbmQgbm9uLWRlbGF5ZWQg QUNLcy4gRG9uJ3QgdHVybiBvbiBBQ0sgZGVsYXkgdW5sZXNzIHlvdSdyZSBzZW5kaW5nIGEgbG90 IG9mIG5vbi1kZWxheWVkIEFDS3MgY2xvc2VseSBmb2xsb3dlZCBieSBwYWNrZXRzIG9uIHdoaWNo IHRoZSBBQ0sgY291bGQgaGF2ZSBiZWVuIHBpZ2d5YmFja2VkLiBUdXJuIGl0IG9mZiB3aGVuIGEg ZGVsYXllZCBBQ0sgaGFzIHRvIGJlIHNlbnQuIg0KV2VsbCAuLiBNci4gTmFnbGUgd2lsbCBiZSBn bGFkIHRvIGxlYXJuIHRoYXQgTGludXggaGFzIGltcGxlbWVudGVkIGhpcyAob3Igc2ltaWxhcikg aWRlYSBmb3IgeWVhcnMuIEl0J3MgY2FsbGVkIFFVSUNLX0FDSy4gVGhlIGRlY2lzaW9uIHdpbmRv dyBpcyBhIGJpdCBzaG9ydCBhbmQgY2FuIHVzZSBzb21lIHR1bmluZy4gQnV0IGl0IGlzIG5vdCAi bmV2ZXIiIGZpeGVkLg0KDQpBZ3JlZSBvbiB0aGF0IDUwMG1zIGlzIHJpZGljdWxvdXMuDQoNCg0K LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IHRjcG0gW21haWx0bzp0Y3BtLWJvdW5j ZXNAaWV0Zi5vcmc8bWFpbHRvOnRjcG0tYm91bmNlc0BpZXRmLm9yZz5dIE9uIEJlaGFsZiBPZiBS aWNoYXJkIFNjaGVmZmVuZWdnZXINClNlbnQ6IEZyaWRheSwgT2N0b2JlciA5LCAyMDE1IDM6NTMg UE0NClRvOiBTY2hhcmYsIE1pY2hhZWwgKE1pY2hhZWwpIDxtaWNoYWVsLnNjaGFyZkBhbGNhdGVs LWx1Y2VudC5jb208bWFpbHRvOm1pY2hhZWwuc2NoYXJmQGFsY2F0ZWwtbHVjZW50LmNvbT4+OyBX ZXNsZXkgRWRkeSA8d2VzQG10aS1zeXN0ZW1zLmNvbTxtYWlsdG86d2VzQG10aS1zeXN0ZW1zLmNv bT4+OyBKb2UgVG91Y2ggPHRvdWNoQGlzaS5lZHU8bWFpbHRvOnRvdWNoQGlzaS5lZHU+Pg0KQ2M6 IExBVVRFTlNDSExBRUdFUiwgV29sZnJhbSAoV29sZnJhbSkgPHdvbGZyYW0ubGF1dGVuc2NobGFl Z2VyQGFsY2F0ZWwtbHVjZW50LmNvbTxtYWlsdG86d29sZnJhbS5sYXV0ZW5zY2hsYWVnZXJAYWxj YXRlbC1sdWNlbnQuY29tPj47IHRjcG1AaWV0Zi5vcmc8bWFpbHRvOnRjcG1AaWV0Zi5vcmc+OyBE YXZpZCBMYW5nIDxkYXZpZEBsYW5nLmhtPG1haWx0bzpkYXZpZEBsYW5nLmhtPj4NClN1YmplY3Q6 IFJlOiBbdGNwbV0gW2FxbV0gVENQIEFDSyBTdXBwcmVzc2lvbg0KDQpUaGlzIG1pZ2h0IGFsc28g YmUgYW4gb3Bwb3J0dW5pdHkgdG8gZ28gbW9yZSBpbnRvIHRoZSByZWFzb25pbmcgZm9yIHRoZSA1 MDBtcyBtYXhpbXVtLi4uDQoNCk1hbnkgaW1wbGVtZW50YXRpb25zIGludGVycHJldCAgInRoZSBk ZWxheSBNVVNUIGJlIGxlc3MgdGhhbiAwLjUgc2Vjb25kcyIgdG8gdHJpZ2dlciB0aGUgZGVsYXll ZCBBQ0sgdGltZXIgYXQgNTAwbXMgKHdpdGggdGhpcyB0aW1lIGEgZml4ZWQsIGNvbXBpbGUtdGlt ZQ0KdmFsdWUpIC0gd2hpbGUgaXQgaGFzIGJlZW4gc2hvd24sIHRoYXQgbWFraW5nIHRoYXQgZHlu YW1pYyAob3IgYSBiYXNlZCBvbiBSVFQgYW5kIHBwcyBtaWdodCBiZSBhIG1vcmUgZWZmaWNpZW50 IGFwcHJvYWNoIChlc3BlY2lhbGx5IGR1cmluZyBsb3NzIGF0IGVuZC1vZi1zdHJlYW0pLCBkZXBl bmRpbmcgb24gdGhlIGVudmlyb25tZW50IFsxXS4NCg0KW0kgdGhvdWdodCB0byBoYXZlIHJlYWQg YSByZWNlbnQgcGFwZXIgb24gdHVuaW5nIHRoZSBkZWxheWVkIEFDSyB0aW1lciBkb3duIHRvIDFt cyBpbiBkYXRhY2VudGVyIGVudmlyb25tZW50cywgYnV0IGNhbm4ndCBmaW5kIHRoYXQgbm93XS4N Cg0KQmVzdCByZWdhcmRzLA0KICBSaWNoYXJkDQoNCg0KDQoNClsxXSBBbHRtYW4sIEVpdGFuLCBh bmQgVGFuaWEgSmltw6luZXouICJOb3ZlbCBkZWxheWVkIEFDSyB0ZWNobmlxdWVzIGZvciBpbXBy b3ZpbmcgVENQIHBlcmZvcm1hbmNlIGluIG11bHRpaG9wIHdpcmVsZXNzIG5ldHdvcmtzLiIgUGVy c29uYWwgV2lyZWxlc3MgQ29tbXVuaWNhdGlvbnMuIFNwcmluZ2VyIEJlcmxpbiBIZWlkZWxiZXJn LCAyMDAzLg0KDQoNCi0tLS0tIE9yaWdpbmFsIE1lc3NhZ2UgLS0tLS0NCkZyb206ICJTY2hhcmYs IE1pY2hhZWwgKE1pY2hhZWwpIiA8bWljaGFlbC5zY2hhcmZAYWxjYXRlbC1sdWNlbnQuY29tPG1h aWx0bzptaWNoYWVsLnNjaGFyZkBhbGNhdGVsLWx1Y2VudC5jb20+Pg0KVG86ICJXZXNsZXkgRWRk eSIgPHdlc0BtdGktc3lzdGVtcy5jb208bWFpbHRvOndlc0BtdGktc3lzdGVtcy5jb20+PjsgIkpv ZSBUb3VjaCIgPHRvdWNoQGlzaS5lZHU8bWFpbHRvOnRvdWNoQGlzaS5lZHU+Pg0KQ2M6ICJMQVVU RU5TQ0hMQUVHRVIsIFdvbGZyYW0gKFdvbGZyYW0pIg0KPHdvbGZyYW0ubGF1dGVuc2NobGFlZ2Vy QGFsY2F0ZWwtbHVjZW50LmNvbTxtYWlsdG86d29sZnJhbS5sYXV0ZW5zY2hsYWVnZXJAYWxjYXRl bC1sdWNlbnQuY29tPj47ICJHcmVnIFdoaXRlIg0KPGcud2hpdGVAY2FibGVsYWJzLmNvbTxtYWls dG86Zy53aGl0ZUBjYWJsZWxhYnMuY29tPj47IDx0Y3BtQGlldGYub3JnPG1haWx0bzp0Y3BtQGll dGYub3JnPj47ICJEYXZpZCBMYW5nIiA8ZGF2aWRAbGFuZy5obTxtYWlsdG86ZGF2aWRAbGFuZy5o bT4+DQpTZW50OiBGcmlkYXksIE9jdG9iZXIgMDksIDIwMTUgNToyMSBQTQ0KU3ViamVjdDogUmU6 IFt0Y3BtXSBbYXFtXSBUQ1AgQUNLIFN1cHByZXNzaW9uDQoNCg0KPiBZZXMsIEkgaGF2ZSByZWFs aXplZCB0b2RheSBteXNlbGYgdGhhdCA3OTNiaXMgbGFja3MgdGhhdCBzZWN0aW9uLiBJIGFncmVl DQo+IHRoYXQgQUNLIGdlbmVyYXRpb24gYmVsb25ncyBpbnRvIDc5M2Jpcy4NCj4NCj4gNzkzYmlz IGFjdHVhbGx5IG1heSBvcGVuIGFuIG9wcG9ydHVuaXR5IHRvIHJldGhpbmsgdGhlIGV4YWN0IHdv cmRpbmcgdGhhdA0KPiBleHBsYWlucyB0aGlzIFNIT1VMRC4gUGVyc29uYWxseSwgSSBiZWxpZXZl IHJlcGxhY2luZyB0aGlzIFNIT1VMRCBpcyBub3QNCj4gYW4gb3B0aW9uIGZvciBhIGJpcyBkb2N1 bWVudC4NCj4NCj4gQnV0IGlmIGRldmlhdGlvbnMgZnJvbSB0aGF0IFNIT1VMRCBhcmUgd2lkZWx5 IGRlcGxveWVkIGluIHRoZSBtZWFudGltZSwNCj4gNzkzYmlzIGNvdWxkIHBvc3NpYmx5IGNvbW1l bnQgb24gdGhhdC4gQWxzbywgaGF2aW5nIHNvbWUgaW5mb3JtYXRpb25hbA0KPiByZWZlcmVuY2Ug Y291bGQgcGVyaGFwcyBiZSBhbiBvcHRpb24uDQo+DQo+IE1pY2hhZWwNCj4NCj4NCj4gLS0tLS1P cmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogV2VzbGV5IEVkZHkgW21haWx0bzp3ZXNAbXRp LXN5c3RlbXMuY29tPG1haWx0bzp3ZXNAbXRpLXN5c3RlbXMuY29tPl0NCj4gU2VudDogRnJpZGF5 LCBPY3RvYmVyIDA5LCAyMDE1IDM6NDUgUE0NCj4gVG86IFNjaGFyZiwgTWljaGFlbCAoTWljaGFl bCk7IEpvZSBUb3VjaA0KPiBDYzogTEFVVEVOU0NITEFFR0VSLCBXb2xmcmFtIChXb2xmcmFtKTsg R3JlZyBXaGl0ZTsgdGNwbUBpZXRmLm9yZzxtYWlsdG86dGNwbUBpZXRmLm9yZz47IERhdmlkDQo+ IExhbmcNCj4gU3ViamVjdDogUmU6IFt0Y3BtXSBbYXFtXSBUQ1AgQUNLIFN1cHByZXNzaW9uDQo+ DQo+IE9uIDEwLzkvMjAxNSA1OjIxIEFNLCBTY2hhcmYsIE1pY2hhZWwgKE1pY2hhZWwpIHdyb3Rl Og0KPj4gW1JlbW92aW5nIEFRTSwgYWRkaW5nIFRDUE1dDQo+Pg0KPj4NCj4+DQo+PiBIaSBKb2Us DQo+Pg0KPj4NCj4+DQo+PiBJIGRvbid0IGZpbmQgdGhpcyB0ZXh0IGluIFJGQyA3OTMsIGJ1dCBp dCBpcyBjb250YWluZWQgaW4gUkZDIDI1ODEsDQo+PiB3aGljaCBpcyBvYnNvbGV0ZSBieSBSRkMg NTY4MS4NCj4+DQo+Pg0KPg0KPg0KPiBUbyBjb25mdXNlIG1hdHRlcnMgZnVydGhlciwgdGhlcmUn cyBhbHNvIDExMjIgdGV4dCB3aXRoIHJlcXVpcmVtZW50cw0KPiBsYW5ndWFnZToNCj4NCj4gICAg ICAgICAgICBBIFRDUCBTSE9VTEQgaW1wbGVtZW50IGEgZGVsYXllZCBBQ0ssIGJ1dCBhbiBBQ0sg c2hvdWxkIG5vdA0KPiAgICAgICAgICAgIGJlIGV4Y2Vzc2l2ZWx5IGRlbGF5ZWQ7IGluIHBhcnRp Y3VsYXIsIHRoZSBkZWxheSBNVVNUIGJlDQo+ICAgICAgICAgICAgbGVzcyB0aGFuIDAuNSBzZWNv bmRzLCBhbmQgaW4gYSBzdHJlYW0gb2YgZnVsbC1zaXplZA0KPiAgICAgICAgICAgIHNlZ21lbnRz IHRoZXJlIFNIT1VMRCBiZSBhbiBBQ0sgZm9yIGF0IGxlYXN0IGV2ZXJ5IHNlY29uZA0KPiAgICAg ICAgICAgIHNlZ21lbnQuDQo+DQo+IFRoaXMgcHJvYmFibHkgYmVsb25ncyBpbiA3OTNiaXMuICBJ biBteSB2aWV3LCBub3JtYXRpdmUgYWNrbm93bGVkZ2VtZW50DQo+IGJlaGF2aW9yIGJlbG9uZ3Mg aW4gdGhlIGJhc2Ugc3BlYyByYXRoZXIgdGhhbiBwYXJ0aWN1bGFyIGNvbmdlc3Rpb24NCj4gY29u dHJvbCBkZXNjcmlwdGlvbnMsIHNpbmNlIHRoZXJlIG1heSBiZSBtb3JlIHRoYW4gQ0MgdGhhdCBk ZXBlbmRzIHVwb24gaXQNCj4gb3IgbWFrZXMgYXNzdW1wdGlvbnMgYWJvdXQgaXQuDQo+DQo+DQo+ IC0tDQo+IFdlcyBFZGR5DQo+IE1USSBTeXN0ZW1zDQo+DQo+IF9fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fDQo+IHRjcG0gbWFpbGluZyBsaXN0DQo+IHRjcG1A aWV0Zi5vcmc8bWFpbHRvOnRjcG1AaWV0Zi5vcmc+DQo+IGh0dHBzOi8vd3d3LmlldGYub3JnL21h aWxtYW4vbGlzdGluZm8vdGNwbQ0KPg0KDQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fXw0KdGNwbSBtYWlsaW5nIGxpc3QNCnRjcG1AaWV0Zi5vcmc8bWFpbHRv OnRjcG1AaWV0Zi5vcmc+DQpodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3Rj cG0NCg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCnRj cG0gbWFpbGluZyBsaXN0DQp0Y3BtQGlldGYub3JnPG1haWx0bzp0Y3BtQGlldGYub3JnPg0KaHR0 cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby90Y3BtDQoNCg== --_000_655C07320163294895BBADA28372AF5D484FFC3FFR712WXCHMBA15z_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTIgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m YWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAy IDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpUYWhvbWE7DQoJcGFub3NlLTE6MiAxMSA2 IDQgMyA1IDQgNCAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNvTm9ybWFsLCBs aS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBjbTsNCgltYXJnaW4tYm90dG9t Oi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGltZXMgTmV3IFJv bWFuIiwic2VyaWYiO30NCmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsNCgl7bXNvLXN0eWxlLXBy aW9yaXR5Ojk5Ow0KCWNvbG9yOmJsdWU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQph OnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJsaW5rRm9sbG93ZWQNCgl7bXNvLXN0eWxlLXByaW9yaXR5 Ojk5Ow0KCWNvbG9yOnB1cnBsZTsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lO30NCnAuTXNv QWNldGF0ZSwgbGkuTXNvQWNldGF0ZSwgZGl2Lk1zb0FjZXRhdGUNCgl7bXNvLXN0eWxlLXByaW9y aXR5Ojk5Ow0KCW1zby1zdHlsZS1saW5rOiJTcHJlY2hibGFzZW50ZXh0IFpjaG4iOw0KCW1hcmdp bjowY207DQoJbWFyZ2luLWJvdHRvbTouMDAwMXB0Ow0KCWZvbnQtc2l6ZTo4LjBwdDsNCglmb250 LWZhbWlseToiVGFob21hIiwic2Fucy1zZXJpZiI7fQ0Kc3Bhbi5FLU1haWxGb3JtYXR2b3JsYWdl MTcNCgl7bXNvLXN0eWxlLXR5cGU6cGVyc29uYWwtcmVwbHk7DQoJZm9udC1mYW1pbHk6IkNhbGli cmkiLCJzYW5zLXNlcmlmIjsNCgljb2xvcjojMUY0OTdEO30NCnNwYW4uU3ByZWNoYmxhc2VudGV4 dFpjaG4NCgl7bXNvLXN0eWxlLW5hbWU6IlNwcmVjaGJsYXNlbnRleHQgWmNobiI7DQoJbXNvLXN0 eWxlLXByaW9yaXR5Ojk5Ow0KCW1zby1zdHlsZS1saW5rOlNwcmVjaGJsYXNlbnRleHQ7DQoJZm9u dC1mYW1pbHk6IlRhaG9tYSIsInNhbnMtc2VyaWYiO30NCi5Nc29DaHBEZWZhdWx0DQoJe21zby1z dHlsZS10eXBlOmV4cG9ydC1vbmx5O30NCkBwYWdlIFdvcmRTZWN0aW9uMQ0KCXtzaXplOjYxMi4w cHQgNzkyLjBwdDsNCgltYXJnaW46NzIuMHB0IDcyLjBwdCA3Mi4wcHQgNzIuMHB0O30NCmRpdi5X b3JkU2VjdGlvbjENCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQ0KLS0+PC9zdHlsZT48IS0tW2lmIGd0 ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlZGVmYXVsdHMgdjpleHQ9ImVkaXQiIHNwaWRtYXg9IjEw MjYiIC8+DQo8L3htbD48IVtlbmRpZl0tLT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNo YXBlbGF5b3V0IHY6ZXh0PSJlZGl0Ij4NCjxvOmlkbWFwIHY6ZXh0PSJlZGl0IiBkYXRhPSIxIiAv Pg0KPC9vOnNoYXBlbGF5b3V0PjwveG1sPjwhW2VuZGlmXS0tPg0KPC9oZWFkPg0KPGJvZHkgbGFu Zz0iREUiIGxpbms9ImJsdWUiIHZsaW5rPSJwdXJwbGUiPg0KPGRpdiBjbGFzcz0iV29yZFNlY3Rp b24xIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9u dC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMt c2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+QXMgZmFyIGFzIEkgcmVjYWxsIChXZXMgY2FuIGNv cnJlY3QgbWUgaWYgSSBhbSB3cm9uZykgd2UgaGF2ZSBhbiBvcGVuIGlzc3VlIGluIDc5M2JpcyBy ZWdhcmRpbmcgdGhhdCBpc3N1ZS48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv Tm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1m YW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMx RjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTom cXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+ U2VlIGRyYWZ0LWlldGYtdGNwbS1yZmM3OTNiaXMtMDEgU2VjdGlvbiA0OjxvOnA+PC9vOnA+PC9z cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0i Zm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3Nh bnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9w Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNp emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJp ZiZxdW90Oztjb2xvcjojMUY0OTdEIj4mbmJzcDsmbmJzcDsgMTEuJm5ic3A7IGxvb2sgYXQgcG9z c2libGUgbWVudGlvbiBvZiBkcmFmdC1taW5zaGFsbC1uYWdsZSAoZS5nLiBhcyBpbjxvOnA+PC9v OnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBz dHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZx dW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IExpbnV4KTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBw dDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7 Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1z b05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQt ZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjoj MUY0OTdEIj5UaGUgY2hhbGxlbmdlIGlzIHRoYXQgZm9yIDc5M2JpcyB3ZSB3b3VsZCBuZWVkIGEg UHJvcG9zZWQgU3RhbmRhcmQgUkZDIHNwZWNpZnlpbmcgaG93IHRvIHNvcnQgb3V0IHRoaXMgaW50 ZXJhY3Rpb24sIHVubGVzcyB3ZSBleHRlbmQgdGhlIHNjb3BlDQogb2YgNzkzYmlzIGJleW9uZCB3 aGF0IHdlIGFncmVlZCB1cG9uIChpLmUuLCBvbmx5IGNsYXJpZmljYXRpb24sIG5vIHByb3RvY29s IGNoYW5nZXMpLjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz cGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVv dDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+PG86 cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFu Zz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGli cmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj5JIGhhdmUgcGVy c29uYWxseSBuZXZlciBsb29rZWQgYXQgaG93IHRoYXQgbG9naWMgaXMgYWN0dWFsbHkgaW1wbGVt ZW50ZWQgaW4gZGlmZmVyZW50IHN0YWNrcywgYW5kIEkgYWN0dWFsbHkgZG9u4oCZdCBrbm93IGlm IHRoZXJlIGhhcyBldmVyIGJlZW4NCiBjb21tdW5pdHkgY29uc2Vuc3VzIG9uIGhvdyB0aGUgZXhh Y3Qgc29sdXRpb24gdG8gdGhhdCBpbnRlcmFjdGlvbiBzaG91bGQgbG9vayBsaWtlLg0KPG86cD48 L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMi IHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDss JnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwv c3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9 ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtz YW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPkJ1dCBpZiB3ZSBoYXZlIGEgZ2FwIHRvIHdo YXQgaXMgd2lkZWx5IGltcGxlbWVudGVkLCBhbmQgaWYgaXQgd2FzIHJlYWxpc3RpYyB0byBwdWJs aXNoIGEgUFMgUkZDIGJlZm9yZSBjb21wbGV0aW5nIDc5M2JpcyAoSSBndWVzcyBpdCBjb3VsZCBi ZSBhDQogMi1wYWdlIFJGQyksIEnigJlkIGFyZ3VlIGluIGZhdm9yIG9mIGEg4oCcZmFzdCB0cmFj a+KAnSBQUyBwdWJsaWNhdGlvbi4gKFdlbGwsIHdpdGgg4oCYZmFzdOKAmSBiZWluZyByZWxhdGl2 ZSB0byBvdXIgbm9ybWFsIFBTIHB1YmxpY2F0aW9uIHNwZWVkLik8bzpwPjwvbzpwPjwvc3Bhbj48 L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQt c2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNl cmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjEx LjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVv dDs7Y29sb3I6IzFGNDk3RCI+TWljaGFlbDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNz PSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtm b250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29s b3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFt aWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0 OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8ZGl2IHN0eWxlPSJib3JkZXI6bm9u ZTtib3JkZXItbGVmdDpzb2xpZCBibHVlIDEuNXB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNC4wcHQi Pg0KPGRpdj4NCjxkaXYgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci10b3A6c29saWQgI0I1QzRE RiAxLjBwdDtwYWRkaW5nOjMuMHB0IDBjbSAwY20gMGNtIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1RhaG9t YSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7Ij5Gcm9tOjwvc3Bhbj48L2I+PHNwYW4gc3R5 bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGFob21hJnF1b3Q7LCZxdW90 O3NhbnMtc2VyaWYmcXVvdDsiPiBZdWNodW5nIENoZW5nIFttYWlsdG86eWNoZW5nQGdvb2dsZS5j b21dDQo8YnI+DQo8Yj5TZW50OjwvYj4gU2F0dXJkYXksIE9jdG9iZXIgMTAsIDIwMTUgMjoxOSBB TTxicj4NCjxiPlRvOjwvYj4gUHJhdmVlbiBCYWxhc3VicmFtYW5pYW48YnI+DQo8Yj5DYzo8L2I+ IFJpY2hhcmQgU2NoZWZmZW5lZ2dlcjsgU2NoYXJmLCBNaWNoYWVsIChNaWNoYWVsKTsgV2VzbGV5 IEVkZHk7IEpvZSBUb3VjaDsgTEFVVEVOU0NITEFFR0VSLCBXb2xmcmFtIChXb2xmcmFtKTsgRGF2 aWQgTGFuZzsgdGNwbUBpZXRmLm9yZzxicj4NCjxiPlN1YmplY3Q6PC9iPiBSZTogW3RjcG1dIFth cW1dIFRDUCBBQ0sgU3VwcHJlc3Npb248bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8L2Rpdj4NCjwv ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0K PHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPHAgY2xh c3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1z b05vcm1hbCI+T24gRnJpLCBPY3QgOSwgMjAxNSBhdCA0OjI1IFBNLCBQcmF2ZWVuIEJhbGFzdWJy YW1hbmlhbiAmbHQ7PGEgaHJlZj0ibWFpbHRvOnByYXZiQG1pY3Jvc29mdC5jb20iIHRhcmdldD0i X2JsYW5rIj5wcmF2YkBtaWNyb3NvZnQuY29tPC9hPiZndDsgd3JvdGU6PG86cD48L286cD48L3A+ DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5BZ3JlZWQgdGhhdCBkZWxheWVkIEFDSyB0aW1lb3V0IHNo b3VsZCBiZSBhIGZ1bmN0aW9uIG9mIFJUVCB0aGFuIGEgaGFyZGNvZGVkIHZhbHVlLiBXaW5kb3dz IDggb253YXJkcyB0aGlzIGlzIHRoZSBiZWhhdmlvciBmb3IgaGlnaCBsYXRlbmN5IGNvbm5lY3Rp b25zLiBXaW5kb3dzIFNlcnZlciB1c2VzIGEgMSBtc2VjIGRlbGF5ZWQgQUNLIHRpbWVvdXQgZm9y IGxvdyBsYXRlbmN5IChkYXRhY2VudGVyKSBjb25uZWN0aW9ucy48YnI+DQo8YnI+DQpBbmQgd2hp bGUgd2UgYXJlIG9uIHRoZSB0b3BpYyBvZiBkZWxheWVkIEFDS3MsIHRoZSBiYWQgaW50ZXJhY3Rp b24gYmV0d2VlbiBOYWdsaW5nIGFuZCBkZWxheWVkIEFDS3MgaXMgd2VsbCBrbm93bi4gSm9obiBO YWdsZSdzIGNvbW1lbnQgaW4gSE4gd2hpY2ggc3VnZ2VzdHMgYSBwZXIgY29ubmVjdGlvbiBsb2dp YyB0byB0aHJvdHRsZSBkZWxheWVkIEFDS3M6PGJyPg0KJnF1b3Q7SWYgeW91J3JlIGdvaW5nIHRv IGRvIHRoYXQsIGl0J3MgYSBnb29kIHRpbWUgdG8gZml4IGRlbGF5ZWQgQUNLIGFzIHdlbGwuIEFz IGlzIHdlbGwga25vd24sIHRoZSBpbnRlcmFjdGlvbiBiZXR3ZWVuIGRlbGF5ZWQgQUNLcyBhbmQg dGhlIE5hZ2xlIGFsZ29yaXRobSBpcyBhd2Z1bC4gVGhpcyBpcyBhIGhpc3RvcmljYWwgYWNjaWRl bnQ7IEkgaW1wbGVtZW50ZWQgb25lLCB0aGUgQmVya2VsZXkgZ3V5cyBpbXBsZW1lbnRlZCB0aGUg b3RoZXIsIGFuZA0KIGJ5IHRoZSB0aW1lIEkgZm91bmQgb3V0IEkgd2FzIG91dCBvZiBuZXR3b3Jr aW5nIGFuZCBpbnZvbHZlZCB3aXRoIGEgc21hbGwgc3RhcnR1cCBjYWxsZWQgQXV0b2Rlc2suIFNv IHRoYXQgbmV2ZXIgZ290IGZpeGVkLiBIZXJlJ3MgaG93IHRvIHRoaW5rIGFib3V0IHRoYXQuIEEg ZGVsYXllZCBBQ0sgaXMgYSBiZXQuIFlvdSdyZSBiZXR0aW5nIHRoYXQgdGhlcmUgd2lsbCBiZSBh IHJlcGx5LCB1cG9uIHdpdGggYW4gQUNLIGNhbiBiZSBwaWdneWJhY2tlZCwNCiBiZWZvcmUgdGhl IGZpeGVkIHRpbWVyIHJ1bnMgb3V0LiBJZiB0aGUgZml4ZWQgdGltZXIgcnVucyBvdXQsIGFuZCB5 b3UgaGF2ZSB0byBzZW5kIGFuIEFDSyBhcyBhIHNlcGFyYXRlIG1lc3NhZ2UsIHlvdSBsb3N0IHRo ZSBiZXQuIEN1cnJlbnQgVENQIGltcGxlbWVudGF0aW9ucyB3aWxsIGhhcHBpbHkgbG9zZSB0aGF0 IGJldCBmb3JldmVyIHdpdGhvdXQgdHVybmluZyBvZmYgdGhlIEFDSyBkZWxheS4gVGhhdCdzIGp1 c3Qgd3JvbmcuIFRoZSByaWdodA0KIGFuc3dlciBpcyB0byB0cmFjayB3aW5zIGFuZCBsb3NzZXMg b24gZGVsYXllZCBhbmQgbm9uLWRlbGF5ZWQgQUNLcy4gRG9uJ3QgdHVybiBvbiBBQ0sgZGVsYXkg dW5sZXNzIHlvdSdyZSBzZW5kaW5nIGEgbG90IG9mIG5vbi1kZWxheWVkIEFDS3MgY2xvc2VseSBm b2xsb3dlZCBieSBwYWNrZXRzIG9uIHdoaWNoIHRoZSBBQ0sgY291bGQgaGF2ZSBiZWVuIHBpZ2d5 YmFja2VkLiBUdXJuIGl0IG9mZiB3aGVuIGEgZGVsYXllZCBBQ0sgaGFzIHRvIGJlIHNlbnQuJnF1 b3Q7PG86cD48L286cD48L3A+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+V2VsbCAuLiBN ci4gTmFnbGUgd2lsbCBiZSBnbGFkIHRvIGxlYXJuIHRoYXQgTGludXggaGFzIGltcGxlbWVudGVk IGhpcyAob3Igc2ltaWxhcikgaWRlYSBmb3IgeWVhcnMuIEl0J3MgY2FsbGVkIFFVSUNLX0FDSy4g VGhlIGRlY2lzaW9uIHdpbmRvdyBpcyBhIGJpdCBzaG9ydCBhbmQgY2FuIHVzZSBzb21lIHR1bmlu Zy4gQnV0IGl0IGlzIG5vdCAmcXVvdDtuZXZlciZxdW90OyBmaXhlZC48bzpwPjwvbzpwPjwvcD4N CjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9w Pg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+QWdyZWUgb24gdGhhdCA1MDBt cyBpcyByaWRpY3Vsb3VzLjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9 Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHN0 eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6 MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXJpZ2h0OjBjbSI+DQo8 ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxicj4NCi0tLS0tT3JpZ2luYWwgTWVz c2FnZS0tLS0tPGJyPg0KRnJvbTogdGNwbSBbbWFpbHRvOjxhIGhyZWY9Im1haWx0bzp0Y3BtLWJv dW5jZXNAaWV0Zi5vcmciPnRjcG0tYm91bmNlc0BpZXRmLm9yZzwvYT5dIE9uIEJlaGFsZiBPZiBS aWNoYXJkIFNjaGVmZmVuZWdnZXI8YnI+DQpTZW50OiBGcmlkYXksIE9jdG9iZXIgOSwgMjAxNSAz OjUzIFBNPGJyPg0KVG86IFNjaGFyZiwgTWljaGFlbCAoTWljaGFlbCkgJmx0OzxhIGhyZWY9Im1h aWx0bzptaWNoYWVsLnNjaGFyZkBhbGNhdGVsLWx1Y2VudC5jb20iPm1pY2hhZWwuc2NoYXJmQGFs Y2F0ZWwtbHVjZW50LmNvbTwvYT4mZ3Q7OyBXZXNsZXkgRWRkeSAmbHQ7PGEgaHJlZj0ibWFpbHRv Ondlc0BtdGktc3lzdGVtcy5jb20iPndlc0BtdGktc3lzdGVtcy5jb208L2E+Jmd0OzsgSm9lIFRv dWNoICZsdDs8YSBocmVmPSJtYWlsdG86dG91Y2hAaXNpLmVkdSI+dG91Y2hAaXNpLmVkdTwvYT4m Z3Q7PGJyPg0KQ2M6IExBVVRFTlNDSExBRUdFUiwgV29sZnJhbSAoV29sZnJhbSkgJmx0OzxhIGhy ZWY9Im1haWx0bzp3b2xmcmFtLmxhdXRlbnNjaGxhZWdlckBhbGNhdGVsLWx1Y2VudC5jb20iPndv bGZyYW0ubGF1dGVuc2NobGFlZ2VyQGFsY2F0ZWwtbHVjZW50LmNvbTwvYT4mZ3Q7Ow0KPGEgaHJl Zj0ibWFpbHRvOnRjcG1AaWV0Zi5vcmciPnRjcG1AaWV0Zi5vcmc8L2E+OyBEYXZpZCBMYW5nICZs dDs8YSBocmVmPSJtYWlsdG86ZGF2aWRAbGFuZy5obSI+ZGF2aWRAbGFuZy5obTwvYT4mZ3Q7PGJy Pg0KU3ViamVjdDogUmU6IFt0Y3BtXSBbYXFtXSBUQ1AgQUNLIFN1cHByZXNzaW9uPGJyPg0KPGJy Pg0KVGhpcyBtaWdodCBhbHNvIGJlIGFuIG9wcG9ydHVuaXR5IHRvIGdvIG1vcmUgaW50byB0aGUg cmVhc29uaW5nIGZvciB0aGUgNTAwbXMgbWF4aW11bS4uLjxicj4NCjxicj4NCk1hbnkgaW1wbGVt ZW50YXRpb25zIGludGVycHJldCZuYnNwOyAmcXVvdDt0aGUgZGVsYXkgTVVTVCBiZSBsZXNzIHRo YW4gMC41IHNlY29uZHMmcXVvdDsgdG8gdHJpZ2dlciB0aGUgZGVsYXllZCBBQ0sgdGltZXIgYXQg NTAwbXMgKHdpdGggdGhpcyB0aW1lIGEgZml4ZWQsIGNvbXBpbGUtdGltZTxicj4NCnZhbHVlKSAt IHdoaWxlIGl0IGhhcyBiZWVuIHNob3duLCB0aGF0IG1ha2luZyB0aGF0IGR5bmFtaWMgKG9yIGEg YmFzZWQgb24gUlRUIGFuZCBwcHMgbWlnaHQgYmUgYSBtb3JlIGVmZmljaWVudCBhcHByb2FjaCAo ZXNwZWNpYWxseSBkdXJpbmcgbG9zcyBhdCBlbmQtb2Ytc3RyZWFtKSwgZGVwZW5kaW5nIG9uIHRo ZSBlbnZpcm9ubWVudCBbMV0uPGJyPg0KPGJyPg0KW0kgdGhvdWdodCB0byBoYXZlIHJlYWQgYSBy ZWNlbnQgcGFwZXIgb24gdHVuaW5nIHRoZSBkZWxheWVkIEFDSyB0aW1lciBkb3duIHRvIDFtcyBp biBkYXRhY2VudGVyIGVudmlyb25tZW50cywgYnV0IGNhbm4ndCBmaW5kIHRoYXQgbm93XS48YnI+ DQo8YnI+DQpCZXN0IHJlZ2FyZHMsPGJyPg0KJm5ic3A7IFJpY2hhcmQ8YnI+DQo8YnI+DQo8YnI+ DQo8YnI+DQo8YnI+DQpbMV0gQWx0bWFuLCBFaXRhbiwgYW5kIFRhbmlhIEppbcOpbmV6LiAmcXVv dDtOb3ZlbCBkZWxheWVkIEFDSyB0ZWNobmlxdWVzIGZvciBpbXByb3ZpbmcgVENQIHBlcmZvcm1h bmNlIGluIG11bHRpaG9wIHdpcmVsZXNzIG5ldHdvcmtzLiZxdW90OyBQZXJzb25hbCBXaXJlbGVz cyBDb21tdW5pY2F0aW9ucy4gU3ByaW5nZXIgQmVybGluIEhlaWRlbGJlcmcsIDIwMDMuPGJyPg0K PGJyPg0KPGJyPg0KLS0tLS0gT3JpZ2luYWwgTWVzc2FnZSAtLS0tLTxicj4NCkZyb206ICZxdW90 O1NjaGFyZiwgTWljaGFlbCAoTWljaGFlbCkmcXVvdDsgJmx0OzxhIGhyZWY9Im1haWx0bzptaWNo YWVsLnNjaGFyZkBhbGNhdGVsLWx1Y2VudC5jb20iPm1pY2hhZWwuc2NoYXJmQGFsY2F0ZWwtbHVj ZW50LmNvbTwvYT4mZ3Q7PGJyPg0KVG86ICZxdW90O1dlc2xleSBFZGR5JnF1b3Q7ICZsdDs8YSBo cmVmPSJtYWlsdG86d2VzQG10aS1zeXN0ZW1zLmNvbSI+d2VzQG10aS1zeXN0ZW1zLmNvbTwvYT4m Z3Q7OyAmcXVvdDtKb2UgVG91Y2gmcXVvdDsgJmx0OzxhIGhyZWY9Im1haWx0bzp0b3VjaEBpc2ku ZWR1Ij50b3VjaEBpc2kuZWR1PC9hPiZndDs8YnI+DQpDYzogJnF1b3Q7TEFVVEVOU0NITEFFR0VS LCBXb2xmcmFtIChXb2xmcmFtKSZxdW90Ozxicj4NCiZsdDs8YSBocmVmPSJtYWlsdG86d29sZnJh bS5sYXV0ZW5zY2hsYWVnZXJAYWxjYXRlbC1sdWNlbnQuY29tIj53b2xmcmFtLmxhdXRlbnNjaGxh ZWdlckBhbGNhdGVsLWx1Y2VudC5jb208L2E+Jmd0OzsgJnF1b3Q7R3JlZyBXaGl0ZSZxdW90Ozxi cj4NCiZsdDs8YSBocmVmPSJtYWlsdG86Zy53aGl0ZUBjYWJsZWxhYnMuY29tIj5nLndoaXRlQGNh YmxlbGFicy5jb208L2E+Jmd0OzsgJmx0OzxhIGhyZWY9Im1haWx0bzp0Y3BtQGlldGYub3JnIj50 Y3BtQGlldGYub3JnPC9hPiZndDs7ICZxdW90O0RhdmlkIExhbmcmcXVvdDsgJmx0OzxhIGhyZWY9 Im1haWx0bzpkYXZpZEBsYW5nLmhtIj5kYXZpZEBsYW5nLmhtPC9hPiZndDs8YnI+DQpTZW50OiBG cmlkYXksIE9jdG9iZXIgMDksIDIwMTUgNToyMSBQTTxicj4NClN1YmplY3Q6IFJlOiBbdGNwbV0g W2FxbV0gVENQIEFDSyBTdXBwcmVzc2lvbjxicj4NCjxicj4NCjxicj4NCiZndDsgWWVzLCBJIGhh dmUgcmVhbGl6ZWQgdG9kYXkgbXlzZWxmIHRoYXQgNzkzYmlzIGxhY2tzIHRoYXQgc2VjdGlvbi4g SSBhZ3JlZTxicj4NCiZndDsgdGhhdCBBQ0sgZ2VuZXJhdGlvbiBiZWxvbmdzIGludG8gNzkzYmlz Ljxicj4NCiZndDs8YnI+DQomZ3Q7IDc5M2JpcyBhY3R1YWxseSBtYXkgb3BlbiBhbiBvcHBvcnR1 bml0eSB0byByZXRoaW5rIHRoZSBleGFjdCB3b3JkaW5nIHRoYXQ8YnI+DQomZ3Q7IGV4cGxhaW5z IHRoaXMgU0hPVUxELiBQZXJzb25hbGx5LCBJIGJlbGlldmUgcmVwbGFjaW5nIHRoaXMgU0hPVUxE IGlzIG5vdDxicj4NCiZndDsgYW4gb3B0aW9uIGZvciBhIGJpcyBkb2N1bWVudC48YnI+DQomZ3Q7 PGJyPg0KJmd0OyBCdXQgaWYgZGV2aWF0aW9ucyBmcm9tIHRoYXQgU0hPVUxEIGFyZSB3aWRlbHkg ZGVwbG95ZWQgaW4gdGhlIG1lYW50aW1lLDxicj4NCiZndDsgNzkzYmlzIGNvdWxkIHBvc3NpYmx5 IGNvbW1lbnQgb24gdGhhdC4gQWxzbywgaGF2aW5nIHNvbWUgaW5mb3JtYXRpb25hbDxicj4NCiZn dDsgcmVmZXJlbmNlIGNvdWxkIHBlcmhhcHMgYmUgYW4gb3B0aW9uLjxicj4NCiZndDs8YnI+DQom Z3Q7IE1pY2hhZWw8YnI+DQomZ3Q7PGJyPg0KJmd0Ozxicj4NCiZndDsgLS0tLS1PcmlnaW5hbCBN ZXNzYWdlLS0tLS08YnI+DQomZ3Q7IEZyb206IFdlc2xleSBFZGR5IFttYWlsdG86PGEgaHJlZj0i bWFpbHRvOndlc0BtdGktc3lzdGVtcy5jb20iPndlc0BtdGktc3lzdGVtcy5jb208L2E+XTxicj4N CiZndDsgU2VudDogRnJpZGF5LCBPY3RvYmVyIDA5LCAyMDE1IDM6NDUgUE08YnI+DQomZ3Q7IFRv OiBTY2hhcmYsIE1pY2hhZWwgKE1pY2hhZWwpOyBKb2UgVG91Y2g8YnI+DQomZ3Q7IENjOiBMQVVU RU5TQ0hMQUVHRVIsIFdvbGZyYW0gKFdvbGZyYW0pOyBHcmVnIFdoaXRlOyA8YSBocmVmPSJtYWls dG86dGNwbUBpZXRmLm9yZyI+DQp0Y3BtQGlldGYub3JnPC9hPjsgRGF2aWQ8YnI+DQomZ3Q7IExh bmc8YnI+DQomZ3Q7IFN1YmplY3Q6IFJlOiBbdGNwbV0gW2FxbV0gVENQIEFDSyBTdXBwcmVzc2lv bjxicj4NCiZndDs8YnI+DQomZ3Q7IE9uIDEwLzkvMjAxNSA1OjIxIEFNLCBTY2hhcmYsIE1pY2hh ZWwgKE1pY2hhZWwpIHdyb3RlOjxicj4NCiZndDsmZ3Q7IFtSZW1vdmluZyBBUU0sIGFkZGluZyBU Q1BNXTxicj4NCiZndDsmZ3Q7PGJyPg0KJmd0OyZndDs8YnI+DQomZ3Q7Jmd0Ozxicj4NCiZndDsm Z3Q7IEhpIEpvZSw8YnI+DQomZ3Q7Jmd0Ozxicj4NCiZndDsmZ3Q7PGJyPg0KJmd0OyZndDs8YnI+ DQomZ3Q7Jmd0OyBJIGRvbid0IGZpbmQgdGhpcyB0ZXh0IGluIFJGQyA3OTMsIGJ1dCBpdCBpcyBj b250YWluZWQgaW4gUkZDIDI1ODEsPGJyPg0KJmd0OyZndDsgd2hpY2ggaXMgb2Jzb2xldGUgYnkg UkZDIDU2ODEuPGJyPg0KJmd0OyZndDs8YnI+DQomZ3Q7Jmd0Ozxicj4NCiZndDs8YnI+DQomZ3Q7 PGJyPg0KJmd0OyBUbyBjb25mdXNlIG1hdHRlcnMgZnVydGhlciwgdGhlcmUncyBhbHNvIDExMjIg dGV4dCB3aXRoIHJlcXVpcmVtZW50czxicj4NCiZndDsgbGFuZ3VhZ2U6PGJyPg0KJmd0Ozxicj4N CiZndDsmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyBBIFRDUCBTSE9V TEQgaW1wbGVtZW50IGEgZGVsYXllZCBBQ0ssIGJ1dCBhbiBBQ0sgc2hvdWxkIG5vdDxicj4NCiZn dDsmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyBiZSBleGNlc3NpdmVs eSBkZWxheWVkOyBpbiBwYXJ0aWN1bGFyLCB0aGUgZGVsYXkgTVVTVCBiZTxicj4NCiZndDsmbmJz cDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyBsZXNzIHRoYW4gMC41IHNlY29u ZHMsIGFuZCBpbiBhIHN0cmVhbSBvZiBmdWxsLXNpemVkPGJyPg0KJmd0OyZuYnNwOyAmbmJzcDsg Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IHNlZ21lbnRzIHRoZXJlIFNIT1VMRCBiZSBhbiBB Q0sgZm9yIGF0IGxlYXN0IGV2ZXJ5IHNlY29uZDxicj4NCiZndDsmbmJzcDsgJm5ic3A7ICZuYnNw OyAmbmJzcDsgJm5ic3A7ICZuYnNwOyBzZWdtZW50Ljxicj4NCiZndDs8YnI+DQomZ3Q7IFRoaXMg cHJvYmFibHkgYmVsb25ncyBpbiA3OTNiaXMuJm5ic3A7IEluIG15IHZpZXcsIG5vcm1hdGl2ZSBh Y2tub3dsZWRnZW1lbnQ8YnI+DQomZ3Q7IGJlaGF2aW9yIGJlbG9uZ3MgaW4gdGhlIGJhc2Ugc3Bl YyByYXRoZXIgdGhhbiBwYXJ0aWN1bGFyIGNvbmdlc3Rpb248YnI+DQomZ3Q7IGNvbnRyb2wgZGVz Y3JpcHRpb25zLCBzaW5jZSB0aGVyZSBtYXkgYmUgbW9yZSB0aGFuIENDIHRoYXQgZGVwZW5kcyB1 cG9uIGl0PGJyPg0KJmd0OyBvciBtYWtlcyBhc3N1bXB0aW9ucyBhYm91dCBpdC48YnI+DQomZ3Q7 PGJyPg0KJmd0Ozxicj4NCiZndDsgLS08YnI+DQomZ3Q7IFdlcyBFZGR5PGJyPg0KJmd0OyBNVEkg U3lzdGVtczxicj4NCiZndDs8YnI+DQomZ3Q7IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fPGJyPg0KJmd0OyB0Y3BtIG1haWxpbmcgbGlzdDxicj4NCiZndDsg PGEgaHJlZj0ibWFpbHRvOnRjcG1AaWV0Zi5vcmciPnRjcG1AaWV0Zi5vcmc8L2E+PGJyPg0KJmd0 OyA8YSBocmVmPSJodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG0iIHRh cmdldD0iX2JsYW5rIj5odHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG08 L2E+PGJyPg0KJmd0Ozxicj4NCjxicj4NCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fPGJyPg0KdGNwbSBtYWlsaW5nIGxpc3Q8YnI+DQo8YSBocmVmPSJtYWls dG86dGNwbUBpZXRmLm9yZyI+dGNwbUBpZXRmLm9yZzwvYT48YnI+DQo8YSBocmVmPSJodHRwczov L3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG0iIHRhcmdldD0iX2JsYW5rIj5odHRw czovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG08L2E+PGJyPg0KPGJyPg0KX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188YnI+DQp0Y3BtIG1h aWxpbmcgbGlzdDxicj4NCjxhIGhyZWY9Im1haWx0bzp0Y3BtQGlldGYub3JnIj50Y3BtQGlldGYu b3JnPC9hPjxicj4NCjxhIGhyZWY9Imh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGlu Zm8vdGNwbSIgdGFyZ2V0PSJfYmxhbmsiPmh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlz dGluZm8vdGNwbTwvYT48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVv dGU+DQo8L2Rpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0K PC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2JvZHk+DQo8L2h0bWw+DQo= --_000_655C07320163294895BBADA28372AF5D484FFC3FFR712WXCHMBA15z_-- From nobody Sun Oct 11 09:29:38 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 68A641A03D5 for ; Sun, 11 Oct 2015 09:29:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ExVI9dedEqt1 for ; Sun, 11 Oct 2015 09:29:34 -0700 (PDT) Received: from mail-vk0-x22a.google.com (mail-vk0-x22a.google.com [IPv6:2607:f8b0:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2CFBA1A037F for ; Sun, 11 Oct 2015 09:29:34 -0700 (PDT) Received: by vkgc187 with SMTP id c187so11804860vkg.3 for ; Sun, 11 Oct 2015 09:29:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=oryCgVvGoCzO749jHxkn05Y2qChcOr8pGrgCCuKUprs=; b=R+Oj8VlIGhFu5HcQhr31MzGmipop+Ax4O3zYuhtRK8iKO554ScpCrwbkBYfWTgNbc6 +v8QnUnOLUeb4tiMaHvFp0lJhAXTddE5m34T4SvV5T6tTsLElCxYo8ByeqXH6d9DvDsp FO+hKOJxbQSbV9VPsuQRTLPc+XM/s18Ub/cYZx/KLc3188L1gnuqzWrP37VAJSXrBvdr 1OMETgbvCuWO5+JfFbslCpfB7mOAiSJv2wN0EBJ0VEYqoTzcliLt1G0zRSQneK/pWWUE c2jHNVy+zh5KxX1BUyD8gtsVDH6+2beQPpn+axzZ6884oXRZunPvzUolMgrVnJg9MVmh Sl6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=oryCgVvGoCzO749jHxkn05Y2qChcOr8pGrgCCuKUprs=; b=cNUm0D7qotvPPYXXMMC12Z53wrxoBZrym8FUJVDKEgzxOH8bcqRtES9x3WaXpvVnZB hB8TOet5IEhYnnK/pOlMjiJTAG5Ku22dS2R2DgpTudRHGITGu1zBZ/L5TonhgAd757Gr 2NtqxAsI5lt2p0y2FNur7UEjOqpfKM/lp0xQECo+cFwGMnW4ltUNL5OcfwDSxNbPDE87 DgxsCsMOxdDUPvJgleU+1QN85wOfj8NeFXY19cpQzUT8scQ2/Yh5i5cpJe9eO6ZLnaPA A7CtfIRBb+b0BxpD9dkPKkMADGJpiys1BPG0yyE+39r6ljMiNlyJdwarUzix6vrKuT+S Tjhw== X-Gm-Message-State: ALoCoQlZfLiT6ob93AWjhxwthf18r3qcxfxsM8g0N07vaJqBlBAzUzJYRpqrFa7h2q81VDD75t1j X-Received: by 10.31.185.12 with SMTP id j12mr14066489vkf.98.1444580973214; Sun, 11 Oct 2015 09:29:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.153.134 with HTTP; Sun, 11 Oct 2015 09:28:53 -0700 (PDT) In-Reply-To: <655C07320163294895BBADA28372AF5D484FFC3F@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617C4BF.6040504@mti-systems.com> <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> <96FC85CD93C14C7FADC014391BAFE03E@srichardlxp2> <655C07320163294895BBADA28372AF5D484FFC3F@FR712WXCHMBA15.zeu.alcatel-lucent.com> From: Yuchung Cheng Date: Sun, 11 Oct 2015 09:28:53 -0700 Message-ID: To: "Scharf, Michael (Michael)" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Delayed ACKs vs. Nagle (was: RE: [aqm] TCP ACK Suppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2015 16:29:36 -0000 On Sun, Oct 11, 2015 at 7:41 AM, Scharf, Michael (Michael) wrote: > As far as I recall (Wes can correct me if I am wrong) we have an open iss= ue > in 793bis regarding that issue. > > > > See draft-ietf-tcpm-rfc793bis-01 Section 4: > > > > 11. look at possible mention of draft-minshall-nagle (e.g. as in > > Linux) > > > > The challenge is that for 793bis we would need a Proposed Standard RFC > specifying how to sort out this interaction, unless we extend the scope o= f > 793bis beyond what we agreed upon (i.e., only clarification, no protocol > changes). > > > > I have personally never looked at how that logic is actually implemented = in > different stacks, and I actually don=E2=80=99t know if there has ever bee= n community > consensus on how the exact solution to that interaction should look like. > > > > But if we have a gap to what is widely implemented, and if it was realist= ic > to publish a PS RFC before completing 793bis (I guess it could be a 2-pag= e > RFC), I=E2=80=99d argue in favor of a =E2=80=9Cfast track=E2=80=9D PS pub= lication. (Well, with > =E2=80=98fast=E2=80=99 being relative to our normal PS publication speed.= ) I am not sure what your concern is: are you concerned Minshall's revision to Nagle's algorithm on the sender would interact badly with Mr. Nagle's idea to improve delayed-ACKs at the receiver? > > > > Michael > > > > > > From: Yuchung Cheng [mailto:ycheng@google.com] > Sent: Saturday, October 10, 2015 2:19 AM > To: Praveen Balasubramanian > Cc: Richard Scheffenegger; Scharf, Michael (Michael); Wesley Eddy; Joe > Touch; LAUTENSCHLAEGER, Wolfram (Wolfram); David Lang; tcpm@ietf.org > Subject: Re: [tcpm] [aqm] TCP ACK Suppression > > > > > > > > On Fri, Oct 9, 2015 at 4:25 PM, Praveen Balasubramanian > wrote: > > Agreed that delayed ACK timeout should be a function of RTT than a hardco= ded > value. Windows 8 onwards this is the behavior for high latency connection= s. > Windows Server uses a 1 msec delayed ACK timeout for low latency > (datacenter) connections. > > And while we are on the topic of delayed ACKs, the bad interaction betwee= n > Nagling and delayed ACKs is well known. John Nagle's comment in HN which > suggests a per connection logic to throttle delayed ACKs: > "If you're going to do that, it's a good time to fix delayed ACK as well.= As > is well known, the interaction between delayed ACKs and the Nagle algorit= hm > is awful. This is a historical accident; I implemented one, the Berkeley > guys implemented the other, and by the time I found out I was out of > networking and involved with a small startup called Autodesk. So that nev= er > got fixed. Here's how to think about that. A delayed ACK is a bet. You're > betting that there will be a reply, upon with an ACK can be piggybacked, > before the fixed timer runs out. If the fixed timer runs out, and you hav= e > to send an ACK as a separate message, you lost the bet. Current TCP > implementations will happily lose that bet forever without turning off th= e > ACK delay. That's just wrong. The right answer is to track wins and losse= s > on delayed and non-delayed ACKs. Don't turn on ACK delay unless you're > sending a lot of non-delayed ACKs closely followed by packets on which th= e > ACK could have been piggybacked. Turn it off when a delayed ACK has to be > sent." > > Well .. Mr. Nagle will be glad to learn that Linux has implemented his (o= r > similar) idea for years. It's called QUICK_ACK. The decision window is a = bit > short and can use some tuning. But it is not "never" fixed. > > > > Agree on that 500ms is ridiculous. > > > > > -----Original Message----- > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Richard Scheffeneg= ger > Sent: Friday, October 9, 2015 3:53 PM > To: Scharf, Michael (Michael) ; Wesley > Eddy ; Joe Touch > Cc: LAUTENSCHLAEGER, Wolfram (Wolfram) > ; tcpm@ietf.org; David Lang > > Subject: Re: [tcpm] [aqm] TCP ACK Suppression > > This might also be an opportunity to go more into the reasoning for the > 500ms maximum... > > Many implementations interpret "the delay MUST be less than 0.5 seconds"= to > trigger the delayed ACK timer at 500ms (with this time a fixed, compile-t= ime > value) - while it has been shown, that making that dynamic (or a based on > RTT and pps might be a more efficient approach (especially during loss at > end-of-stream), depending on the environment [1]. > > [I thought to have read a recent paper on tuning the delayed ACK timer do= wn > to 1ms in datacenter environments, but cann't find that now]. > > Best regards, > Richard > > > > > [1] Altman, Eitan, and Tania Jim=C3=A9nez. "Novel delayed ACK techniques = for > improving TCP performance in multihop wireless networks." Personal Wirele= ss > Communications. Springer Berlin Heidelberg, 2003. > > > ----- Original Message ----- > From: "Scharf, Michael (Michael)" > To: "Wesley Eddy" ; "Joe Touch" > Cc: "LAUTENSCHLAEGER, Wolfram (Wolfram)" > ; "Greg White" > ; ; "David Lang" > Sent: Friday, October 09, 2015 5:21 PM > Subject: Re: [tcpm] [aqm] TCP ACK Suppression > > >> Yes, I have realized today myself that 793bis lacks that section. I agre= e >> that ACK generation belongs into 793bis. >> >> 793bis actually may open an opportunity to rethink the exact wording tha= t >> explains this SHOULD. Personally, I believe replacing this SHOULD is not >> an option for a bis document. >> >> But if deviations from that SHOULD are widely deployed in the meantime, >> 793bis could possibly comment on that. Also, having some informational >> reference could perhaps be an option. >> >> Michael >> >> >> -----Original Message----- >> From: Wesley Eddy [mailto:wes@mti-systems.com] >> Sent: Friday, October 09, 2015 3:45 PM >> To: Scharf, Michael (Michael); Joe Touch >> Cc: LAUTENSCHLAEGER, Wolfram (Wolfram); Greg White; tcpm@ietf.org; David >> Lang >> Subject: Re: [tcpm] [aqm] TCP ACK Suppression >> >> On 10/9/2015 5:21 AM, Scharf, Michael (Michael) wrote: >>> [Removing AQM, adding TCPM] >>> >>> >>> >>> Hi Joe, >>> >>> >>> >>> I don't find this text in RFC 793, but it is contained in RFC 2581, >>> which is obsolete by RFC 5681. >>> >>> >> >> >> To confuse matters further, there's also 1122 text with requirements >> language: >> >> A TCP SHOULD implement a delayed ACK, but an ACK should not >> be excessively delayed; in particular, the delay MUST be >> less than 0.5 seconds, and in a stream of full-sized >> segments there SHOULD be an ACK for at least every second >> segment. >> >> This probably belongs in 793bis. In my view, normative acknowledgement >> behavior belongs in the base spec rather than particular congestion >> control descriptions, since there may be more than CC that depends upon = it >> or makes assumptions about it. >> >> >> -- >> Wes Eddy >> MTI Systems >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >> > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > > From nobody Sun Oct 11 11:18:27 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6C1461B2C41 for ; Sun, 11 Oct 2015 11:18:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PeuSB9xueQ0P for ; Sun, 11 Oct 2015 11:18:24 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7003B1B2C3B for ; Sun, 11 Oct 2015 11:18:24 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 08AAA3609A154; Sun, 11 Oct 2015 18:18:19 +0000 (GMT) Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t9BIILNL009383 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 11 Oct 2015 20:18:22 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.03.0195.001; Sun, 11 Oct 2015 20:18:21 +0200 From: "Scharf, Michael (Michael)" To: "draft-ietf-tcpm-dctcp@tools.ietf.org" Thread-Topic: Wording on enabling DCTCP in draft-ietf-tcpm-dctcp Thread-Index: AdEEUTThiGgw+TVnST6f8Mttkqg+nw== Date: Sun, 11 Oct 2015 18:18:20 +0000 Message-ID: <655C07320163294895BBADA28372AF5D484FFECB@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" Subject: [tcpm] Wording on enabling DCTCP in draft-ietf-tcpm-dctcp X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2015 18:18:26 -0000 Hi, During IETF 93 we had a quick discussion on how to negotiate DCTCP (see htt= p://www.ietf.org/proceedings/93/minutes/minutes-93-tcpm). Given that the WG= aims for a very fast publication of this I-D, I'd like to trigger a follow= -up discussion on what the document should state regarding enabling DCTCP.= =20 As an individual contributor to TCPM, I believe that it would be good if th= e document addresses the enablement (and perhaps also the implicit negotiat= ion) in a dedicated section; right now this issue is spread across differen= t parts of the document. Specifically, my personal view would be: * Abstract =3D> I think the abstract should better explain the focus on controlled d= omains. Also, it could mention coexistence with standard TCP as an issue. F= or the IESG processing, it could also help to stress the status as informat= ional (e.g., by "This informational memo..."). * Section 1 Introduction It is recommended that DCTCP be deployed in a datacenter environment where the endpoints and the switching fabric are under a single administrative domain. This memo also discusses deployment issues related to the coexistence of DCTCP and conventional TCP, the lack of a negotiating mechanism between sender and receiver, and presents some possible mitigations. =20 =3D> I wonder whether to add a more explicit warning sign such as "The pro= tocol is not meant for uncontrolled deployment in the global Internet", or = a wording like the statement in Section 4. * Section 4 Implementation Issues The implementation must also decide when to use DCTCP. Datacenter servers may need to communicate with endpoints outside the datacenter, where DCTCP is unsuitable or unsupported. Thus, a global configuration setting to enable DCTCP will generally not suffice. DCTCP may be configured based on the IP address of the remote endpoint. Microsoft Windows Server 2012 (and later) also supports automatic selection of DCTCP if the estimated RTT is less than 10 msec and ECN is successfully negotiated, under the assumption that if the RTT is low, then the two endpoints are likely on the same datacenter network. =3D> To me, whether to enable DCTCP or not is not an "implementation issu= e" only. I think this warrants a dedicated section. I'd also suggest to spl= it the problem statement and *possible* solutions from the actual heuristic= in Windows Server 2012 into two paragraphs. This could probably be achieve= d by a purely editorial change only. * Section 6 Known Issues DCTCP provides no mechanism for negotiating its use. Thus, there is additional management and configuration overhead required to ensure that DCTCP is not used with non-DCTCP endpoints. The affect of using DCTCP with a standard ECN endpoint has been analyzed in [ODCTCP][BSDCAN]. Furthermore, it is possible that other implementations may also modify [RFC3168] behavior without negotiation, causing further interoperability issues. =3D> The requirement to detect DCTCP endpoints seems related to the enabl= ing discussion above. At least, I think it would be valuable to have an int= ernal cross-reference between the sections discussing the two aspects. Thanks Michael From nobody Sun Oct 11 14:18:46 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E1F921B2DBC; Sun, 11 Oct 2015 14:18:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SxriNMKJ5vut; Sun, 11 Oct 2015 14:18:42 -0700 (PDT) Received: from bifrost.lang.hm (mail.lang.hm [64.81.33.126]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D32581B2C13; Sun, 11 Oct 2015 14:18:42 -0700 (PDT) Received: from asgard.lang.hm (asgard.lang.hm [10.0.0.100]) by bifrost.lang.hm (8.13.4/8.13.4/Debian-3) with ESMTP id t9BLISnX008079; Sun, 11 Oct 2015 14:18:28 -0700 Date: Sun, 11 Oct 2015 14:18:28 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: Joe Touch In-Reply-To: <5619F00A.2040009@isi.edu> Message-ID: References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: Cc: "tcpm@ietf.org" , "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Jonathan Morton , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2015 21:18:45 -0000 On Sat, 10 Oct 2015, Joe Touch wrote: > On 10/10/2015 8:45 PM, David Lang wrote: >> But I actually view this as a tragedy of the commons situation. Sending >> the extra packets doesn't hurt you, but it does mean that you are using >> more airtime than you need, and that hurts everyone (including, >> eventually, you) > > So, basically, you appreciate the tragedy of the commons when it's your > tragedy and their commons, but not the other way around? > > (using 'sed') > > s/sending the extra/altering the transport/ > > s/using more airtime/altering E2E semantics/ I actually disagree that there is a significnt difference in the actual E2E semantics between sending a single stretched ACK and sending the full number of ACKs back-to-back. The "transmit burst" issue will happen in either case. But you will notice that in both cases, I am in favor of reducing the number of packets on the wire. Packets not send can't interfere with other traffic. If ACK packets are delayed to the point that additional ACK packets have caught up with them, they are not providing the rate signalling signal that you are looking for. Question: Does anyone know if current server TCP stacks implement the TCP byte counting that RFC3449 talks about? David Lang From nobody Sun Oct 11 14:28:45 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 082BE1B2DE0; Sun, 11 Oct 2015 14:28:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JfNa04Kv_Zdr; Sun, 11 Oct 2015 14:28:43 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1076B1B2DDD; Sun, 11 Oct 2015 14:28:43 -0700 (PDT) Received: from [128.9.184.173] ([128.9.184.173]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t9BLSVtG028023 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Sun, 11 Oct 2015 14:28:31 -0700 (PDT) To: David Lang References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> From: Joe Touch X-Enigmail-Draft-Status: N1110 Message-ID: <561AD47B.9030602@isi.edu> Date: Sun, 11 Oct 2015 14:28:27 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Jonathan Morton , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2015 21:28:44 -0000 On 10/11/2015 2:18 PM, David Lang wrote: > But you will notice that in both cases, I am in favor of reducing the > number of packets on the wire. Packets not send can't interfere with > other traffic. The degenerate case of this approach is a circuit. Take a look at Morris's 1997 ICNP paper to see what happens when you end up with one - or sometimes less than one - packet per round trip time. One of the reasons we use packets is to provide more timely, fine-grained feedback between endpoints. Aggregating them at the source or in the network (rather than at the receiver) can amplify reaction to packet loss (when the aggregate ACK is lost) and increases discretization effects in the TCP algorithms. Joe From nobody Sun Oct 11 15:49:48 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0A5271B2E98; Sun, 11 Oct 2015 15:49:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v89X5Ykw8eeZ; Sun, 11 Oct 2015 15:49:41 -0700 (PDT) Received: from bifrost.lang.hm (mail.lang.hm [64.81.33.126]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7D57B1B2E9A; Sun, 11 Oct 2015 15:49:41 -0700 (PDT) Received: from asgard.lang.hm (asgard.lang.hm [10.0.0.100]) by bifrost.lang.hm (8.13.4/8.13.4/Debian-3) with ESMTP id t9BMnPZs008510; Sun, 11 Oct 2015 15:49:25 -0700 Date: Sun, 11 Oct 2015 15:49:25 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: Joe Touch In-Reply-To: <561AD47B.9030602@isi.edu> Message-ID: References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> <561AD47B.9030602@isi.edu> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: Cc: "tcpm@ietf.org" , "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Jonathan Morton , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2015 22:49:45 -0000 On Sun, 11 Oct 2015, Joe Touch wrote: > On 10/11/2015 2:18 PM, David Lang wrote: >> But you will notice that in both cases, I am in favor of reducing the >> number of packets on the wire. Packets not send can't interfere with >> other traffic. > > The degenerate case of this approach is a circuit. Take a look at > Morris's 1997 ICNP paper to see what happens when you end up with one - > or sometimes less than one - packet per round trip time. > > One of the reasons we use packets is to provide more timely, > fine-grained feedback between endpoints. Aggregating them at the source > or in the network (rather than at the receiver) can amplify reaction to > packet loss (when the aggregate ACK is lost) the issue of amplified reaction to packet loss is a valid one, but packet loss is still pretty rare, so the proposal of putting a duplicate stretched ack into the queue for the next transmit window when you aggregate packets in the current transmit window should largely mitigate this. In addition, Since cablemodems have been doing this for 15 years, we should be able to get people to look at real-world traffic and tell us how much of a problem this is. > and increases > discretization effects in the TCP algorithms. This is where I disagree with you. The ACK packets are already collapsed time-wise. Whatever timing they had when they were generated, the fact that they are all sitting in the same queue at the same time means taht that timing has been lost, Nobody is adding latency by trying to recreate the time spacing, so the discretization (or as I've been calling it quantitization) of the traffic is happening even if every ACK packet is sent. My contention is that since this is already happening, consolidating the ACK packets into stretched ACKs doesn't make this any worse, and it saves network bandwidth (and decreases latency to the extent that data is acked faster than waiting for the entire chain or original ACKs to get through, especially if that would take multiple transmit windows). As a result, thinning the ACKs is kinder to the network. I am in no way advocating delaying any packet that could be sent, just combining those that have already been delayed. We're talking about ACK packets here, but any protocol that sends short TCP packets is potentially fair game (TCP/SSH for example). If the packets are all sitting in the same queue, they should be able to be combined. This happens on the sending server if multiple writes to the socket happen before the packet has been sent, it should be able to happen on the network as well (further reducing overhead by sending the same data with fewer packets) David Lang From nobody Sun Oct 11 17:10:57 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3444D1B2F1F; Sun, 11 Oct 2015 17:10:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xiLPvnFO-5ZO; Sun, 11 Oct 2015 17:10:53 -0700 (PDT) Received: from bifrost.lang.hm (mail.lang.hm [64.81.33.126]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D0F521B2F22; Sun, 11 Oct 2015 17:10:53 -0700 (PDT) Received: from asgard.lang.hm (asgard.lang.hm [10.0.0.100]) by bifrost.lang.hm (8.13.4/8.13.4/Debian-3) with ESMTP id t9C0AbJX008928; Sun, 11 Oct 2015 17:10:37 -0700 Date: Sun, 11 Oct 2015 17:10:37 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: Jonathan Morton In-Reply-To: Message-ID: References: <5618005A.8070303@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> <561AD47B.9030602@isi.edu> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Archived-At: Cc: "tcpm@ietf.org" , Joe Touch , "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 00:10:55 -0000 On Mon, 12 Oct 2015, Jonathan Morton wrote: >> My contention is that since this is already happening, consolidating the > ACK packets into stretched ACKs doesn't make this any worse, and it saves > network bandwidth (and decreases latency to the extent that data is acked > faster than waiting for the entire chain or original ACKs to get through, > especially if that would take multiple transmit windows). As a result, > thinning the ACKs is kinder to the network. > > I agree, *iff* they are not DupACKs signalling packet loss. Do existing > and future cable modems take that subtle distinction into account? I don't know, and RFC3449 section 5.2.1 lists a couple other conditions to watch out for. But this discussion started with what AQM algorithms can/should do when they see multiple ACKs in their queue (and I added wifi as part of make-wifi-fast). So we are talking about what should be put in future code. The cable modems are just an example we can draw from of a past implementation of similar ideas. hopefully future cable modems will use a more generic AQM algorithm instead of the hard-coded optimizations they've had in the past. It's almost always a win when a generic solution can replace a specific solution :-) David Lang From nobody Mon Oct 12 05:05:08 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 672351AD067 for ; Mon, 12 Oct 2015 05:05:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C-8AIUtIU6kU for ; Mon, 12 Oct 2015 05:04:58 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D76A61AD06C for ; Mon, 12 Oct 2015 05:04:57 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 4D26AFDA0B62C; Mon, 12 Oct 2015 12:04:53 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t9CC4qlF018760 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 12 Oct 2015 14:04:54 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Mon, 12 Oct 2015 14:04:52 +0200 From: "Scharf, Michael (Michael)" To: Yuchung Cheng Thread-Topic: Delayed ACKs vs. Nagle (was: RE: [tcpm] [aqm] TCP ACK Suppression) Thread-Index: AQHRBDLkoeCMEiaauUmhCORc+sdhw55mWeiAgAFikoA= Date: Mon, 12 Oct 2015 12:04:52 +0000 Message-ID: <655C07320163294895BBADA28372AF5D48503648@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <0A452E1DADEF254C9A7AC1969B8781284A7D9B66@FR712WXCHMBA13.zeu.alcatel-lucent.com> <5616DCD9.8@isi.edu> <655C07320163294895BBADA28372AF5D484FCA01@FR712WXCHMBA15.zeu.alcatel-lucent.com> <5617C4BF.6040504@mti-systems.com> <655C07320163294895BBADA28372AF5D484FD4EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> <96FC85CD93C14C7FADC014391BAFE03E@srichardlxp2> <655C07320163294895BBADA28372AF5D484FFC3F@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.39] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Delayed ACKs vs. Nagle (was: RE: [aqm] TCP ACK Suppression) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 12:05:04 -0000 QWgsIHNvcnJ5LCBJIG1peGVkIHVwIHRoZSB0d28gcHJvYmxlbXMuIA0KDQpNeSBnZW5lcmFsIGNv bmNlcm4gaXMgdG8gYXZvaWQgdGhhdCA3OTNiaXMgaXMgYWxyZWFkeSBvdXRkYXRlZCBhdCB0aGUg dGltZSBvZiBwdWJsaWNhdGlvbi4gZHJhZnQtbWluc2hhbGwtbmFnbGUgbWF5IG9yIG1heSBub3Qg YmUgb25lIGV4YW1wbGUgaW4gd2hpY2ggVENQTSBsYWNrcyBhIHNtYWxsIFBTL0JDUCBkb2N1bWVu dC4gQXMgSSBzYWlkIGFscmVhZHksIEkgaGF2ZSBub3QgY2hlY2tlZCB0aGlzIHBhcnRpY3VsYXIg YXJlYSBteXNlbGYgc28gZmFyLg0KDQpPdXQtb2YtbXktaGVhZCwgSSBkb24ndCByZWNhbGwgYW55 IHJlY2VudCBSRkMgdGhhdCByZWFsbHkgZGVhbHMgd2l0aCBEZWxBQ0sgdHVuaW5nIGFzIGRpc2N1 c3NlZCBiZWxvdy4gSWYgdGhhdCBzb3J0IG9mIGhldXJpc3RpYyBpcyBkb25lIGJ5IG1vc3Qgc3Rh Y2tzIHRoZXNlIGRheXMsIGl0IGNvdWxkIHBlcmhhcHMgbWFrZSBzZW5zZSB0byBkb2N1bWVudCB0 aGF0LCBpZiB0aGUgY29tbXVuaXR5IGNvdWxkIGUuZy4gYWdyZWUgb24gY2VydGFpbiBoaWdoLWxl dmVsIGNvbnN0cmFpbnRzIChlLmcuLCBkb2VzIGl0IG1ha2Ugc2Vuc2UgdG8gcmVkdWNlIHRoZSB0 aW1lciBiZWxvdyAxbXM/KS4NCg0KQnV0IHRoaXMgaXMganVzdCBicmFpbnN0b3JtaW5nLCBhbmQg bXkgdmlldyBhcyBpbmRpdmlkdWFsIGVuZ2luZWVyIG9ubHkuLi4NCg0KTWljaGFlbA0KDQoNCi0t LS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBZdWNodW5nIENoZW5nIFttYWlsdG86eWNo ZW5nQGdvb2dsZS5jb21dIA0KU2VudDogU3VuZGF5LCBPY3RvYmVyIDExLCAyMDE1IDY6MjkgUE0N ClRvOiBTY2hhcmYsIE1pY2hhZWwgKE1pY2hhZWwpDQpDYzogUHJhdmVlbiBCYWxhc3VicmFtYW5p YW47IFdlc2xleSBFZGR5OyB0Y3BtQGlldGYub3JnDQpTdWJqZWN0OiBSZTogRGVsYXllZCBBQ0tz IHZzLiBOYWdsZSAod2FzOiBSRTogW3RjcG1dIFthcW1dIFRDUCBBQ0sgU3VwcHJlc3Npb24pDQoN Ck9uIFN1biwgT2N0IDExLCAyMDE1IGF0IDc6NDEgQU0sIFNjaGFyZiwgTWljaGFlbCAoTWljaGFl bCkgPG1pY2hhZWwuc2NoYXJmQGFsY2F0ZWwtbHVjZW50LmNvbT4gd3JvdGU6DQo+IEFzIGZhciBh cyBJIHJlY2FsbCAoV2VzIGNhbiBjb3JyZWN0IG1lIGlmIEkgYW0gd3JvbmcpIHdlIGhhdmUgYW4g b3BlbiANCj4gaXNzdWUgaW4gNzkzYmlzIHJlZ2FyZGluZyB0aGF0IGlzc3VlLg0KPg0KPg0KPg0K PiBTZWUgZHJhZnQtaWV0Zi10Y3BtLXJmYzc5M2Jpcy0wMSBTZWN0aW9uIDQ6DQo+DQo+DQo+DQo+ ICAgIDExLiAgbG9vayBhdCBwb3NzaWJsZSBtZW50aW9uIG9mIGRyYWZ0LW1pbnNoYWxsLW5hZ2xl IChlLmcuIGFzIGluDQo+DQo+ICAgICAgICAgTGludXgpDQo+DQo+DQo+DQo+IFRoZSBjaGFsbGVu Z2UgaXMgdGhhdCBmb3IgNzkzYmlzIHdlIHdvdWxkIG5lZWQgYSBQcm9wb3NlZCBTdGFuZGFyZCBS RkMgDQo+IHNwZWNpZnlpbmcgaG93IHRvIHNvcnQgb3V0IHRoaXMgaW50ZXJhY3Rpb24sIHVubGVz cyB3ZSBleHRlbmQgdGhlIA0KPiBzY29wZSBvZiA3OTNiaXMgYmV5b25kIHdoYXQgd2UgYWdyZWVk IHVwb24gKGkuZS4sIG9ubHkgY2xhcmlmaWNhdGlvbiwgDQo+IG5vIHByb3RvY29sIGNoYW5nZXMp Lg0KPg0KPg0KPg0KPiBJIGhhdmUgcGVyc29uYWxseSBuZXZlciBsb29rZWQgYXQgaG93IHRoYXQg bG9naWMgaXMgYWN0dWFsbHkgDQo+IGltcGxlbWVudGVkIGluIGRpZmZlcmVudCBzdGFja3MsIGFu ZCBJIGFjdHVhbGx5IGRvbuKAmXQga25vdyBpZiB0aGVyZSANCj4gaGFzIGV2ZXIgYmVlbiBjb21t dW5pdHkgY29uc2Vuc3VzIG9uIGhvdyB0aGUgZXhhY3Qgc29sdXRpb24gdG8gdGhhdCBpbnRlcmFj dGlvbiBzaG91bGQgbG9vayBsaWtlLg0KPg0KPg0KPg0KPiBCdXQgaWYgd2UgaGF2ZSBhIGdhcCB0 byB3aGF0IGlzIHdpZGVseSBpbXBsZW1lbnRlZCwgYW5kIGlmIGl0IHdhcyANCj4gcmVhbGlzdGlj IHRvIHB1Ymxpc2ggYSBQUyBSRkMgYmVmb3JlIGNvbXBsZXRpbmcgNzkzYmlzIChJIGd1ZXNzIGl0 IA0KPiBjb3VsZCBiZSBhIDItcGFnZSBSRkMpLCBJ4oCZZCBhcmd1ZSBpbiBmYXZvciBvZiBhIOKA nGZhc3QgdHJhY2vigJ0gUFMgDQo+IHB1YmxpY2F0aW9uLiAoV2VsbCwgd2l0aCDigJhmYXN04oCZ IGJlaW5nIHJlbGF0aXZlIHRvIG91ciBub3JtYWwgUFMgDQo+IHB1YmxpY2F0aW9uIHNwZWVkLikN CkkgYW0gbm90IHN1cmUgd2hhdCB5b3VyIGNvbmNlcm4gaXM6IGFyZSB5b3UgY29uY2VybmVkIE1p bnNoYWxsJ3MgcmV2aXNpb24gdG8gTmFnbGUncyBhbGdvcml0aG0gb24gdGhlIHNlbmRlciB3b3Vs ZCBpbnRlcmFjdCBiYWRseSB3aXRoIE1yLiBOYWdsZSdzIGlkZWEgdG8gaW1wcm92ZSBkZWxheWVk LUFDS3MgYXQgdGhlIHJlY2VpdmVyPw0KDQo+DQo+DQo+DQo+IE1pY2hhZWwNCj4NCj4NCj4NCj4N Cj4NCj4gRnJvbTogWXVjaHVuZyBDaGVuZyBbbWFpbHRvOnljaGVuZ0Bnb29nbGUuY29tXQ0KPiBT ZW50OiBTYXR1cmRheSwgT2N0b2JlciAxMCwgMjAxNSAyOjE5IEFNDQo+IFRvOiBQcmF2ZWVuIEJh bGFzdWJyYW1hbmlhbg0KPiBDYzogUmljaGFyZCBTY2hlZmZlbmVnZ2VyOyBTY2hhcmYsIE1pY2hh ZWwgKE1pY2hhZWwpOyBXZXNsZXkgRWRkeTsgSm9lIA0KPiBUb3VjaDsgTEFVVEVOU0NITEFFR0VS LCBXb2xmcmFtIChXb2xmcmFtKTsgRGF2aWQgTGFuZzsgdGNwbUBpZXRmLm9yZw0KPiBTdWJqZWN0 OiBSZTogW3RjcG1dIFthcW1dIFRDUCBBQ0sgU3VwcHJlc3Npb24NCj4NCj4NCj4NCj4NCj4NCj4N Cj4NCj4gT24gRnJpLCBPY3QgOSwgMjAxNSBhdCA0OjI1IFBNLCBQcmF2ZWVuIEJhbGFzdWJyYW1h bmlhbiANCj4gPHByYXZiQG1pY3Jvc29mdC5jb20+IHdyb3RlOg0KPg0KPiBBZ3JlZWQgdGhhdCBk ZWxheWVkIEFDSyB0aW1lb3V0IHNob3VsZCBiZSBhIGZ1bmN0aW9uIG9mIFJUVCB0aGFuIGEgDQo+ IGhhcmRjb2RlZCB2YWx1ZS4gV2luZG93cyA4IG9ud2FyZHMgdGhpcyBpcyB0aGUgYmVoYXZpb3Ig Zm9yIGhpZ2ggbGF0ZW5jeSBjb25uZWN0aW9ucy4NCj4gV2luZG93cyBTZXJ2ZXIgdXNlcyBhIDEg bXNlYyBkZWxheWVkIEFDSyB0aW1lb3V0IGZvciBsb3cgbGF0ZW5jeQ0KPiAoZGF0YWNlbnRlcikg Y29ubmVjdGlvbnMuDQo+DQo+IEFuZCB3aGlsZSB3ZSBhcmUgb24gdGhlIHRvcGljIG9mIGRlbGF5 ZWQgQUNLcywgdGhlIGJhZCBpbnRlcmFjdGlvbiANCj4gYmV0d2VlbiBOYWdsaW5nIGFuZCBkZWxh eWVkIEFDS3MgaXMgd2VsbCBrbm93bi4gSm9obiBOYWdsZSdzIGNvbW1lbnQgDQo+IGluIEhOIHdo aWNoIHN1Z2dlc3RzIGEgcGVyIGNvbm5lY3Rpb24gbG9naWMgdG8gdGhyb3R0bGUgZGVsYXllZCBB Q0tzOg0KPiAiSWYgeW91J3JlIGdvaW5nIHRvIGRvIHRoYXQsIGl0J3MgYSBnb29kIHRpbWUgdG8g Zml4IGRlbGF5ZWQgQUNLIGFzIA0KPiB3ZWxsLiBBcyBpcyB3ZWxsIGtub3duLCB0aGUgaW50ZXJh Y3Rpb24gYmV0d2VlbiBkZWxheWVkIEFDS3MgYW5kIHRoZSANCj4gTmFnbGUgYWxnb3JpdGhtIGlz IGF3ZnVsLiBUaGlzIGlzIGEgaGlzdG9yaWNhbCBhY2NpZGVudDsgSSBpbXBsZW1lbnRlZCANCj4g b25lLCB0aGUgQmVya2VsZXkgZ3V5cyBpbXBsZW1lbnRlZCB0aGUgb3RoZXIsIGFuZCBieSB0aGUg dGltZSBJIGZvdW5kIA0KPiBvdXQgSSB3YXMgb3V0IG9mIG5ldHdvcmtpbmcgYW5kIGludm9sdmVk IHdpdGggYSBzbWFsbCBzdGFydHVwIGNhbGxlZCANCj4gQXV0b2Rlc2suIFNvIHRoYXQgbmV2ZXIg Z290IGZpeGVkLiBIZXJlJ3MgaG93IHRvIHRoaW5rIGFib3V0IHRoYXQuIEEgDQo+IGRlbGF5ZWQg QUNLIGlzIGEgYmV0LiBZb3UncmUgYmV0dGluZyB0aGF0IHRoZXJlIHdpbGwgYmUgYSByZXBseSwg dXBvbiANCj4gd2l0aCBhbiBBQ0sgY2FuIGJlIHBpZ2d5YmFja2VkLCBiZWZvcmUgdGhlIGZpeGVk IHRpbWVyIHJ1bnMgb3V0LiBJZiANCj4gdGhlIGZpeGVkIHRpbWVyIHJ1bnMgb3V0LCBhbmQgeW91 IGhhdmUgdG8gc2VuZCBhbiBBQ0sgYXMgYSBzZXBhcmF0ZSANCj4gbWVzc2FnZSwgeW91IGxvc3Qg dGhlIGJldC4gQ3VycmVudCBUQ1AgaW1wbGVtZW50YXRpb25zIHdpbGwgaGFwcGlseSANCj4gbG9z ZSB0aGF0IGJldCBmb3JldmVyIHdpdGhvdXQgdHVybmluZyBvZmYgdGhlIEFDSyBkZWxheS4gVGhh dCdzIGp1c3QgDQo+IHdyb25nLiBUaGUgcmlnaHQgYW5zd2VyIGlzIHRvIHRyYWNrIHdpbnMgYW5k IGxvc3NlcyBvbiBkZWxheWVkIGFuZCANCj4gbm9uLWRlbGF5ZWQgQUNLcy4gRG9uJ3QgdHVybiBv biBBQ0sgZGVsYXkgdW5sZXNzIHlvdSdyZSBzZW5kaW5nIGEgbG90IA0KPiBvZiBub24tZGVsYXll ZCBBQ0tzIGNsb3NlbHkgZm9sbG93ZWQgYnkgcGFja2V0cyBvbiB3aGljaCB0aGUgQUNLIGNvdWxk IA0KPiBoYXZlIGJlZW4gcGlnZ3liYWNrZWQuIFR1cm4gaXQgb2ZmIHdoZW4gYSBkZWxheWVkIEFD SyBoYXMgdG8gYmUgc2VudC4iDQo+DQo+IFdlbGwgLi4gTXIuIE5hZ2xlIHdpbGwgYmUgZ2xhZCB0 byBsZWFybiB0aGF0IExpbnV4IGhhcyBpbXBsZW1lbnRlZCBoaXMgDQo+IChvcg0KPiBzaW1pbGFy KSBpZGVhIGZvciB5ZWFycy4gSXQncyBjYWxsZWQgUVVJQ0tfQUNLLiBUaGUgZGVjaXNpb24gd2lu ZG93IGlzIA0KPiBhIGJpdCBzaG9ydCBhbmQgY2FuIHVzZSBzb21lIHR1bmluZy4gQnV0IGl0IGlz IG5vdCAibmV2ZXIiIGZpeGVkLg0KPg0KPg0KPg0KPiBBZ3JlZSBvbiB0aGF0IDUwMG1zIGlzIHJp ZGljdWxvdXMuDQo+DQo+DQo+DQo+DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZy b206IHRjcG0gW21haWx0bzp0Y3BtLWJvdW5jZXNAaWV0Zi5vcmddIE9uIEJlaGFsZiBPZiBSaWNo YXJkIA0KPiBTY2hlZmZlbmVnZ2VyDQo+IFNlbnQ6IEZyaWRheSwgT2N0b2JlciA5LCAyMDE1IDM6 NTMgUE0NCj4gVG86IFNjaGFyZiwgTWljaGFlbCAoTWljaGFlbCkgPG1pY2hhZWwuc2NoYXJmQGFs Y2F0ZWwtbHVjZW50LmNvbT47IA0KPiBXZXNsZXkgRWRkeSA8d2VzQG10aS1zeXN0ZW1zLmNvbT47 IEpvZSBUb3VjaCA8dG91Y2hAaXNpLmVkdT4NCj4gQ2M6IExBVVRFTlNDSExBRUdFUiwgV29sZnJh bSAoV29sZnJhbSkgDQo+IDx3b2xmcmFtLmxhdXRlbnNjaGxhZWdlckBhbGNhdGVsLWx1Y2VudC5j b20+OyB0Y3BtQGlldGYub3JnOyBEYXZpZCANCj4gTGFuZyA8ZGF2aWRAbGFuZy5obT4NCj4gU3Vi amVjdDogUmU6IFt0Y3BtXSBbYXFtXSBUQ1AgQUNLIFN1cHByZXNzaW9uDQo+DQo+IFRoaXMgbWln aHQgYWxzbyBiZSBhbiBvcHBvcnR1bml0eSB0byBnbyBtb3JlIGludG8gdGhlIHJlYXNvbmluZyBm b3IgDQo+IHRoZSA1MDBtcyBtYXhpbXVtLi4uDQo+DQo+IE1hbnkgaW1wbGVtZW50YXRpb25zIGlu dGVycHJldCAgInRoZSBkZWxheSBNVVNUIGJlIGxlc3MgdGhhbiAwLjUgDQo+IHNlY29uZHMiIHRv IHRyaWdnZXIgdGhlIGRlbGF5ZWQgQUNLIHRpbWVyIGF0IDUwMG1zICh3aXRoIHRoaXMgdGltZSBh IA0KPiBmaXhlZCwgY29tcGlsZS10aW1lDQo+IHZhbHVlKSAtIHdoaWxlIGl0IGhhcyBiZWVuIHNo b3duLCB0aGF0IG1ha2luZyB0aGF0IGR5bmFtaWMgKG9yIGEgYmFzZWQgDQo+IG9uIFJUVCBhbmQg cHBzIG1pZ2h0IGJlIGEgbW9yZSBlZmZpY2llbnQgYXBwcm9hY2ggKGVzcGVjaWFsbHkgZHVyaW5n IA0KPiBsb3NzIGF0IGVuZC1vZi1zdHJlYW0pLCBkZXBlbmRpbmcgb24gdGhlIGVudmlyb25tZW50 IFsxXS4NCj4NCj4gW0kgdGhvdWdodCB0byBoYXZlIHJlYWQgYSByZWNlbnQgcGFwZXIgb24gdHVu aW5nIHRoZSBkZWxheWVkIEFDSyB0aW1lciANCj4gZG93biB0byAxbXMgaW4gZGF0YWNlbnRlciBl bnZpcm9ubWVudHMsIGJ1dCBjYW5uJ3QgZmluZCB0aGF0IG5vd10uDQo+DQo+IEJlc3QgcmVnYXJk cywNCj4gICBSaWNoYXJkDQo+DQo+DQo+DQo+DQo+IFsxXSBBbHRtYW4sIEVpdGFuLCBhbmQgVGFu aWEgSmltw6luZXouICJOb3ZlbCBkZWxheWVkIEFDSyB0ZWNobmlxdWVzIA0KPiBmb3IgaW1wcm92 aW5nIFRDUCBwZXJmb3JtYW5jZSBpbiBtdWx0aWhvcCB3aXJlbGVzcyBuZXR3b3Jrcy4iIFBlcnNv bmFsIA0KPiBXaXJlbGVzcyBDb21tdW5pY2F0aW9ucy4gU3ByaW5nZXIgQmVybGluIEhlaWRlbGJl cmcsIDIwMDMuDQo+DQo+DQo+IC0tLS0tIE9yaWdpbmFsIE1lc3NhZ2UgLS0tLS0NCj4gRnJvbTog IlNjaGFyZiwgTWljaGFlbCAoTWljaGFlbCkiIDxtaWNoYWVsLnNjaGFyZkBhbGNhdGVsLWx1Y2Vu dC5jb20+DQo+IFRvOiAiV2VzbGV5IEVkZHkiIDx3ZXNAbXRpLXN5c3RlbXMuY29tPjsgIkpvZSBU b3VjaCIgPHRvdWNoQGlzaS5lZHU+DQo+IENjOiAiTEFVVEVOU0NITEFFR0VSLCBXb2xmcmFtIChX b2xmcmFtKSINCj4gPHdvbGZyYW0ubGF1dGVuc2NobGFlZ2VyQGFsY2F0ZWwtbHVjZW50LmNvbT47 ICJHcmVnIFdoaXRlIg0KPiA8Zy53aGl0ZUBjYWJsZWxhYnMuY29tPjsgPHRjcG1AaWV0Zi5vcmc+ OyAiRGF2aWQgTGFuZyIgPGRhdmlkQGxhbmcuaG0+DQo+IFNlbnQ6IEZyaWRheSwgT2N0b2JlciAw OSwgMjAxNSA1OjIxIFBNDQo+IFN1YmplY3Q6IFJlOiBbdGNwbV0gW2FxbV0gVENQIEFDSyBTdXBw cmVzc2lvbg0KPg0KPg0KPj4gWWVzLCBJIGhhdmUgcmVhbGl6ZWQgdG9kYXkgbXlzZWxmIHRoYXQg NzkzYmlzIGxhY2tzIHRoYXQgc2VjdGlvbi4gSSANCj4+IGFncmVlIHRoYXQgQUNLIGdlbmVyYXRp b24gYmVsb25ncyBpbnRvIDc5M2Jpcy4NCj4+DQo+PiA3OTNiaXMgYWN0dWFsbHkgbWF5IG9wZW4g YW4gb3Bwb3J0dW5pdHkgdG8gcmV0aGluayB0aGUgZXhhY3Qgd29yZGluZyANCj4+IHRoYXQgZXhw bGFpbnMgdGhpcyBTSE9VTEQuIFBlcnNvbmFsbHksIEkgYmVsaWV2ZSByZXBsYWNpbmcgdGhpcyAN Cj4+IFNIT1VMRCBpcyBub3QgYW4gb3B0aW9uIGZvciBhIGJpcyBkb2N1bWVudC4NCj4+DQo+PiBC dXQgaWYgZGV2aWF0aW9ucyBmcm9tIHRoYXQgU0hPVUxEIGFyZSB3aWRlbHkgZGVwbG95ZWQgaW4g dGhlIA0KPj4gbWVhbnRpbWUsIDc5M2JpcyBjb3VsZCBwb3NzaWJseSBjb21tZW50IG9uIHRoYXQu IEFsc28sIGhhdmluZyBzb21lIA0KPj4gaW5mb3JtYXRpb25hbCByZWZlcmVuY2UgY291bGQgcGVy aGFwcyBiZSBhbiBvcHRpb24uDQo+Pg0KPj4gTWljaGFlbA0KPj4NCj4+DQo+PiAtLS0tLU9yaWdp bmFsIE1lc3NhZ2UtLS0tLQ0KPj4gRnJvbTogV2VzbGV5IEVkZHkgW21haWx0bzp3ZXNAbXRpLXN5 c3RlbXMuY29tXQ0KPj4gU2VudDogRnJpZGF5LCBPY3RvYmVyIDA5LCAyMDE1IDM6NDUgUE0NCj4+ IFRvOiBTY2hhcmYsIE1pY2hhZWwgKE1pY2hhZWwpOyBKb2UgVG91Y2gNCj4+IENjOiBMQVVURU5T Q0hMQUVHRVIsIFdvbGZyYW0gKFdvbGZyYW0pOyBHcmVnIFdoaXRlOyB0Y3BtQGlldGYub3JnOyAN Cj4+IERhdmlkIExhbmcNCj4+IFN1YmplY3Q6IFJlOiBbdGNwbV0gW2FxbV0gVENQIEFDSyBTdXBw cmVzc2lvbg0KPj4NCj4+IE9uIDEwLzkvMjAxNSA1OjIxIEFNLCBTY2hhcmYsIE1pY2hhZWwgKE1p Y2hhZWwpIHdyb3RlOg0KPj4+IFtSZW1vdmluZyBBUU0sIGFkZGluZyBUQ1BNXQ0KPj4+DQo+Pj4N Cj4+Pg0KPj4+IEhpIEpvZSwNCj4+Pg0KPj4+DQo+Pj4NCj4+PiBJIGRvbid0IGZpbmQgdGhpcyB0 ZXh0IGluIFJGQyA3OTMsIGJ1dCBpdCBpcyBjb250YWluZWQgaW4gUkZDIDI1ODEsIA0KPj4+IHdo aWNoIGlzIG9ic29sZXRlIGJ5IFJGQyA1NjgxLg0KPj4+DQo+Pj4NCj4+DQo+Pg0KPj4gVG8gY29u ZnVzZSBtYXR0ZXJzIGZ1cnRoZXIsIHRoZXJlJ3MgYWxzbyAxMTIyIHRleHQgd2l0aCByZXF1aXJl bWVudHMNCj4+IGxhbmd1YWdlOg0KPj4NCj4+ICAgICAgICAgICAgQSBUQ1AgU0hPVUxEIGltcGxl bWVudCBhIGRlbGF5ZWQgQUNLLCBidXQgYW4gQUNLIHNob3VsZCBub3QNCj4+ICAgICAgICAgICAg YmUgZXhjZXNzaXZlbHkgZGVsYXllZDsgaW4gcGFydGljdWxhciwgdGhlIGRlbGF5IE1VU1QgYmUN Cj4+ICAgICAgICAgICAgbGVzcyB0aGFuIDAuNSBzZWNvbmRzLCBhbmQgaW4gYSBzdHJlYW0gb2Yg ZnVsbC1zaXplZA0KPj4gICAgICAgICAgICBzZWdtZW50cyB0aGVyZSBTSE9VTEQgYmUgYW4gQUNL IGZvciBhdCBsZWFzdCBldmVyeSBzZWNvbmQNCj4+ICAgICAgICAgICAgc2VnbWVudC4NCj4+DQo+ PiBUaGlzIHByb2JhYmx5IGJlbG9uZ3MgaW4gNzkzYmlzLiAgSW4gbXkgdmlldywgbm9ybWF0aXZl IA0KPj4gYWNrbm93bGVkZ2VtZW50IGJlaGF2aW9yIGJlbG9uZ3MgaW4gdGhlIGJhc2Ugc3BlYyBy YXRoZXIgdGhhbiANCj4+IHBhcnRpY3VsYXIgY29uZ2VzdGlvbiBjb250cm9sIGRlc2NyaXB0aW9u cywgc2luY2UgdGhlcmUgbWF5IGJlIG1vcmUgDQo+PiB0aGFuIENDIHRoYXQgZGVwZW5kcyB1cG9u IGl0IG9yIG1ha2VzIGFzc3VtcHRpb25zIGFib3V0IGl0Lg0KPj4NCj4+DQo+PiAtLQ0KPj4gV2Vz IEVkZHkNCj4+IE1USSBTeXN0ZW1zDQo+Pg0KPj4gX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX18NCj4+IHRjcG0gbWFpbGluZyBsaXN0DQo+PiB0Y3BtQGlldGYu b3JnDQo+PiBodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG0NCj4+DQo+ DQo+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQo+IHRj cG0gbWFpbGluZyBsaXN0DQo+IHRjcG1AaWV0Zi5vcmcNCj4gaHR0cHM6Ly93d3cuaWV0Zi5vcmcv bWFpbG1hbi9saXN0aW5mby90Y3BtDQo+DQo+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fDQo+IHRjcG0gbWFpbGluZyBsaXN0DQo+IHRjcG1AaWV0Zi5vcmcN Cj4gaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby90Y3BtDQo+DQo+DQo= From nobody Mon Oct 12 07:13:56 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B6F641B32E7; Mon, 12 Oct 2015 07:13:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.061 X-Spam-Level: X-Spam-Status: No, score=-1.061 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HELO_EQ_SE=0.35, J_CHICKENPOX_22=0.6, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I4oGaKok9ML6; Mon, 12 Oct 2015 07:13:49 -0700 (PDT) Received: from uplift.swm.pp.se (ipv6.swm.pp.se [IPv6:2a00:801::f]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2EB1A1B32E0; Mon, 12 Oct 2015 07:13:48 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id 0E78AA1; Mon, 12 Oct 2015 16:13:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1444659226; bh=pMYxo3URz8G1Gbs7nn966ybigAVdhu+79ADnF+Tsz3w=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=N29Rk39vAJ1ePrDxMKv7XRdlKMKfYEcUSKvutiLGCv97BFGyqAvv8gQiof4232itU FaNINiet7tujBW/yQvlm+r2p2d7n14EwkdIIuEAmkxckP6S90mrcyVMTtjU7cNU643 4A7/d79M5HbepmXEZv2kvHfG21lUwoNkiTJldZs8= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id 037929F; Mon, 12 Oct 2015 16:13:46 +0200 (CEST) Date: Mon, 12 Oct 2015 16:13:45 +0200 (CEST) From: Mikael Abrahamsson To: Greg White In-Reply-To: Message-ID: References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: Cc: "tcpm@ietf.org" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 14:13:55 -0000 On Fri, 9 Oct 2015, Greg White wrote: > > > On 10/9/15, 2:04 PM, "Mark Allman" wrote: > >> >>> 1) *you* shouldn't be using a mechanism that destroys information for >>> others >>> 2) *you* don't know where your mechanism will have an impact >>> 3) you claim this might be safe *if* AQM is widely deployed >> >> tl;dr summary: myopia is why we can't have nice things > > Too true. DOCSIS would have been much cleaner if we didn't have to deal > with the fallout from the myopic TCP designers. :-P So I agree that most likely, it's beneficial to have fewer ACKs. What I think people arguing against this practice are these kinds of issues: http://blog.dan.drown.org/sb6183-dropping-ipv6-traffic/ I don't think there is a solution that we all can agree on, all approaches have their benefits and drawbacks. I think the above article just shows how things can go wrong in very subtle ways. -- Mikael Abrahamsson email: swmike@swm.pp.se From nobody Mon Oct 12 08:00:50 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 04BFF1A6F07 for ; Mon, 12 Oct 2015 08:00:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.25 X-Spam-Level: X-Spam-Status: No, score=-3.25 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_DE=0.35, J_CHICKENPOX_22=0.6, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lRvl7f-Dwyjk for ; Mon, 12 Oct 2015 08:00:42 -0700 (PDT) Received: from iramx2.ira.uni-karlsruhe.de (iramx2.ira.uni-karlsruhe.de [141.3.10.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 285D31A6F03 for ; Mon, 12 Oct 2015 08:00:41 -0700 (PDT) Received: from i72vorta.tm.uni-karlsruhe.de ([141.3.71.26] helo=i72vorta.tm.kit.edu) by iramx2.ira.uni-karlsruhe.de with esmtp port 25 iface 141.3.10.81 id 1Zleac-0006bS-2q; Mon, 12 Oct 2015 17:00:38 +0200 Received: from [IPv6:::1] (ip6-localhost [IPv6:::1]) by i72vorta.tm.kit.edu (Postfix) with ESMTPS id 08EDEB01A20; Mon, 12 Oct 2015 17:00:38 +0200 (CEST) To: tcpm@ietf.org, Mikael Abrahamsson References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> From: "Bless, Roland (TM)" X-Enigmail-Draft-Status: N1110 Organization: Institute of Telematics, Karlsruhe Institute of Technology Message-ID: <561BCB15.1040804@kit.edu> Date: Mon, 12 Oct 2015 17:00:37 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-ATIS-AV: ClamAV (iramx2.ira.uni-karlsruhe.de) X-ATIS-Timestamp: iramx2.ira.uni-karlsruhe.de 1444662038. Archived-At: Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 15:00:49 -0000 Hi, On 12.10.2015 at 16:13 Mikael Abrahamsson wrote: > On Fri, 9 Oct 2015, Greg White wrote: > >> >> >> On 10/9/15, 2:04 PM, "Mark Allman" wrote: >> >>> >>>> 1) *you* shouldn't be using a mechanism that destroys information for >>>> others >>>> 2) *you* don't know where your mechanism will have an impact >>>> 3) you claim this might be safe *if* AQM is widely deployed >>> >>> tl;dr summary: myopia is why we can't have nice things >> >> Too true. DOCSIS would have been much cleaner if we didn't have to deal >> with the fallout from the myopic TCP designers. :-P > > So I agree that most likely, it's beneficial to have fewer ACKs. > > What I think people arguing against this practice are these kinds of > issues: > > http://blog.dan.drown.org/sb6183-dropping-ipv6-traffic/ > > I don't think there is a solution that we all can agree on, all > approaches have their benefits and drawbacks. I think the above article > just shows how things can go wrong in very subtle ways. Surely, broken or non-prudent implementations can easily break things. My other issue with this is that *protocol evolution* suffers. New options may not be understood by such optimization, so their deployment is inhibited, or, the ACKing strategy changes and the previously greatly working optimization is totally counter-productive. I guess that the MPTCP WG work has suffered a lot from several kinds of "built-in optimization" network gear. Moreover, if people are starting to use newer transports other than TCP, they may complain about poor performance then, because the optimization doesn't work in these cases... Regards, Roland From nobody Mon Oct 12 09:09:10 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 095FD1A8849; Mon, 12 Oct 2015 09:09:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.31 X-Spam-Level: X-Spam-Status: No, score=-1.31 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_22=0.6, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8GWJ7uh4Uw2L; Mon, 12 Oct 2015 09:09:08 -0700 (PDT) Received: from bifrost.lang.hm (mail.lang.hm [64.81.33.126]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AC7051A883F; Mon, 12 Oct 2015 09:09:08 -0700 (PDT) Received: from asgard.lang.hm (asgard.lang.hm [10.0.0.100]) by bifrost.lang.hm (8.13.4/8.13.4/Debian-3) with ESMTP id t9CG949O015849; Mon, 12 Oct 2015 09:09:04 -0700 Date: Mon, 12 Oct 2015 09:09:04 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: Mikael Abrahamsson In-Reply-To: Message-ID: References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: Cc: "tcpm@ietf.org" , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 16:09:10 -0000 On Mon, 12 Oct 2015, Mikael Abrahamsson wrote: > On Fri, 9 Oct 2015, Greg White wrote: > >> >> >> On 10/9/15, 2:04 PM, "Mark Allman" wrote: >> >>> >>>> 1) *you* shouldn't be using a mechanism that destroys information for >>>> others >>>> 2) *you* don't know where your mechanism will have an impact >>>> 3) you claim this might be safe *if* AQM is widely deployed >>> >>> tl;dr summary: myopia is why we can't have nice things >> >> Too true. DOCSIS would have been much cleaner if we didn't have to deal >> with the fallout from the myopic TCP designers. :-P > > So I agree that most likely, it's beneficial to have fewer ACKs. > > What I think people arguing against this practice are these kinds of issues: > > http://blog.dan.drown.org/sb6183-dropping-ipv6-traffic/ > > I don't think there is a solution that we all can agree on, all approaches > have their benefits and drawbacks. I think the above article just shows how > things can go wrong in very subtle ways. no question that things can go wrong in subtle ways. If network protocol design was easy, we would have had a couple protocols designed back in the early days of computing and there wouldn't be the recent rash of new protocol designs :-) But too much fear that "something may go wrong somewhere" can prevent any progress. David Lang From nobody Mon Oct 12 09:11:09 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 54FF11A8857 for ; Mon, 12 Oct 2015 09:11:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.31 X-Spam-Level: X-Spam-Status: No, score=-1.31 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_22=0.6, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qjH3KWdegoz1 for ; Mon, 12 Oct 2015 09:11:07 -0700 (PDT) Received: from bifrost.lang.hm (mail.lang.hm [64.81.33.126]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4797B1A885D for ; Mon, 12 Oct 2015 09:11:06 -0700 (PDT) Received: from asgard.lang.hm (asgard.lang.hm [10.0.0.100]) by bifrost.lang.hm (8.13.4/8.13.4/Debian-3) with ESMTP id t9CGB4cs015875; Mon, 12 Oct 2015 09:11:04 -0700 Date: Mon, 12 Oct 2015 09:11:04 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: "Bless, Roland (TM)" In-Reply-To: <561BCB15.1040804@kit.edu> Message-ID: References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <561BCB15.1040804@kit.edu> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: Cc: tcpm@ietf.org Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 16:11:08 -0000 On Mon, 12 Oct 2015, Bless, Roland (TM) wrote: > Hi, > > On 12.10.2015 at 16:13 Mikael Abrahamsson wrote: >> On Fri, 9 Oct 2015, Greg White wrote: >> >>> >>> >>> On 10/9/15, 2:04 PM, "Mark Allman" wrote: >>> >>>> >>>>> 1) *you* shouldn't be using a mechanism that destroys information for >>>>> others >>>>> 2) *you* don't know where your mechanism will have an impact >>>>> 3) you claim this might be safe *if* AQM is widely deployed >>>> >>>> tl;dr summary: myopia is why we can't have nice things >>> >>> Too true. DOCSIS would have been much cleaner if we didn't have to deal >>> with the fallout from the myopic TCP designers. :-P >> >> So I agree that most likely, it's beneficial to have fewer ACKs. >> >> What I think people arguing against this practice are these kinds of >> issues: >> >> http://blog.dan.drown.org/sb6183-dropping-ipv6-traffic/ >> >> I don't think there is a solution that we all can agree on, all >> approaches have their benefits and drawbacks. I think the above article >> just shows how things can go wrong in very subtle ways. > > Surely, broken or non-prudent implementations can easily break > things. My other issue with this is that *protocol evolution* > suffers. New options may not be understood by such optimization, > so their deployment is inhibited, or, the ACKing strategy changes > and the previously greatly working optimization is totally > counter-productive. I guess that the MPTCP WG work has suffered > a lot from several kinds of "built-in optimization" network gear. > Moreover, if people are starting to use newer transports other > than TCP, they may complain about poor performance then, because > the optimization doesn't work in these cases... New protcols should have similar optimizations built in and not require network assistance ;-) Or, if the improvement is really something that can only take place at the network layer (possibly due to the local knowledge of the queue state for example), this will just be one disadvantage of the new protocol until similar optimizations can be deployed for the new protocol. David Lang From nobody Mon Oct 12 09:44:17 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 15B5E1A1A51 for ; Mon, 12 Oct 2015 09:44:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0vwxgqZmUSJi for ; Mon, 12 Oct 2015 09:44:10 -0700 (PDT) Received: from wizmail.org (wizmail.org [IPv6:2a00:1940:107::2:0:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3436E1A1B74 for ; Mon, 12 Oct 2015 09:44:10 -0700 (PDT) Received: from [2a00:b900:109e:0:3e97:eff:feb2:e128] (helo=lap.dom.ain) by wizmail.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.86_89-e1eedf0) id 1ZlgCm-0001ql-Fj for tcpm@ietf.org (return-path ); Mon, 12 Oct 2015 16:44:08 +0000 To: tcpm@ietf.org References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> <561AD47B.9030602@isi.edu> From: Jeremy Harris Message-ID: <561BE357.4090405@wizmail.org> Date: Mon, 12 Oct 2015 17:44:07 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Pcms-Received-Sender: [2a00:b900:109e:0:3e97:eff:feb2:e128] (helo=lap.dom.ain) with esmtpsa Archived-At: Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 16:44:16 -0000 On 11/10/15 23:49, David Lang wrote: > packet loss is still pretty rare, Not in my day-job it isn't. > The ACK packets are already collapsed > time-wise. Whatever timing they had when they were generated, the fact > that they are all sitting in the same queue at the same time means taht > that timing has been lost, Unless they carried timestamps? What are you doing with TCP options (and ECN marks)? -- Cheers, Jeremy From nobody Mon Oct 12 16:08:54 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7D75E1B2B84; Mon, 12 Oct 2015 16:08:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.227 X-Spam-Level: X-Spam-Status: No, score=0.227 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, HTML_MESSAGE=0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QVTlHugHjc0o; Mon, 12 Oct 2015 16:08:50 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 2C48A1B2B7C; Mon, 12 Oct 2015 16:08:50 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id t9CN8jAp013300; Mon, 12 Oct 2015 17:08:45 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Mon, 12 Oct 2015 17:08:45 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([::1]) by EXCHANGE.cablelabs.com ([::1]) with mapi id 14.03.0224.002; Mon, 12 Oct 2015 17:08:42 -0600 From: Greg White To: Jonathan Morton , David Lang Thread-Topic: [aqm] [tcpm] TCP ACK Suppression Thread-Index: AQHRAuFHTV+VD6RCtU2qgvWw5c3NuZ5kIzsAgAACOACAAA69gIAACDeAgAAF9oCAAAS3AIAAGpuAgAAiyYCAABu/AIAABycAgAEGRICAAAclgIAACxeAgABNnoCAABiMAIABDYgAgAACy4CAABafgIAADKqAgAEmiIA= Date: Mon, 12 Oct 2015 23:08:41 +0000 Message-ID: References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> <561AD47B.9030602@isi.edu> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.5.6.150930 x-originating-ip: [10.4.11.26] Content-Type: multipart/alternative; boundary="_000_D241965754F70gwhitecablelabscom_" MIME-Version: 1.0 Archived-At: Cc: "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , "tcpm@ietf.org" , "mallman@icir.org" , "aqm@ietf.org" , Joe Touch Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 23:08:51 -0000 --_000_D241965754F70gwhitecablelabscom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable > My contention is that since this is already happening, consolidating the = ACK packets into stretched ACKs doesn't make this any worse, and it saves n= etwork bandwidth (and decreases latency to the extent that data is acked fa= ster than waiting for the entire chain or original ACKs to get through, esp= ecially if that would take multiple transmit windows). As a result, thinnin= g the ACKs is kinder to the network. I agree, *iff* they are not DupACKs signalling packet loss. Do existing an= d future cable modems take that subtle distinction into account? I presume that they are not discarding DupACKs or corrupting SACK in some w= ay (the whole point of specialized ACK handling is to ensure good TCP perfo= rmance after all), but since the algorithms are proprietary, for existing c= able modems I don't know. I will contact the developers as well as run som= e tests on a few modems to get a better idea (and maybe Mikael already has = some data from his recent testing he could share?). For future modems, I = can add an explicit reference to RFC3449 in the DOCSIS 3.1 spec just in cas= e implementers aren't aware for some reason. --_000_D241965754F70gwhitecablelabscom_ Content-Type: text/html; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable


> My contention is that since this is already happening, consolidating t= he ACK packets into stretched ACKs doesn't make this any worse, and it save= s network bandwidth (and decreases latency to the extent that data is acked= faster than waiting for the entire chain or original ACKs to get through, especially if that would take multi= ple transmit windows). As a result, thinning the ACKs is kinder to the netw= ork.

I agree, *iff* they are not DupACKs signalling packet loss.&= nbsp; Do existing and future cable modems take that subtle distinction into= account?

I presume that they are not discarding DupACKs or corrupting SACK in s= ome way (the whole point of specialized ACK handling is to ensure good TCP = performance after all), but since the algorithms are proprietary, for exist= ing cable modems I don't know.  I will contact the developers as well as run some tests on a few modems to g= et a better idea (and maybe Mikael already has some data from his recent te= sting he could share?).   For future modems, I can add an explicit ref= erence to RFC3449 in the DOCSIS 3.1 spec just in case implementers aren't aware for some reason. 


--_000_D241965754F70gwhitecablelabscom_-- From nobody Tue Oct 13 05:55:53 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 900DF1B3068; Tue, 13 Oct 2015 05:55:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PeQuaDeDciph; Tue, 13 Oct 2015 05:55:49 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 6979F1B306F; Tue, 13 Oct 2015 05:55:49 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: X-Test-IDTracker: no X-IETF-IDTracker: 6.5.0 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20151013125549.31353.52573.idtracker@ietfa.amsl.com> Date: Tue, 13 Oct 2015 05:55:49 -0700 Archived-At: Cc: tcpm@ietf.org Subject: [tcpm] I-D Action: draft-ietf-tcpm-undeployed-03.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2015 12:55:50 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the TCP Maintenance and Minor Extensions Working Group of the IETF. Title : Moving Outdated TCP Extensions and TCP-related Documents to Historic and Informational Status Authors : Alexander Zimmermann Wesley M. Eddy Lars Eggert Filename : draft-ietf-tcpm-undeployed-03.txt Pages : 8 Date : 2015-10-13 Abstract: This document reclassifies several TCP extensions and TCP-related documents that have either been superseded, have never seen widespread use, or are no longer recommended for use to "Historic" status. The affected RFCs are RFC 675, RFC 721, RFC 761, RFC 813, RFC 816, RFC 879, RFC 896, RFC 1078, and RFC 6013. Additionally, this document reclassifies RFC 700, RFC 794, RFC 814, RFC 817, RFC 872, RFC 889, RFC 964, and RFC 1071 to "Informational" status. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-tcpm-undeployed/ There's also a htmlized version available at: https://tools.ietf.org/html/draft-ietf-tcpm-undeployed-03 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-undeployed-03 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Tue Oct 13 07:55:34 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D4D2E1B45CC; Tue, 13 Oct 2015 07:55:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iUdCXtSo7_5x; Tue, 13 Oct 2015 07:55:29 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F1ED71B45CA; Tue, 13 Oct 2015 07:55:28 -0700 (PDT) Received: from [128.9.176.211] (c2-vpn02.isi.edu [128.9.176.211]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t9DEsx68006639 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 13 Oct 2015 07:55:02 -0700 (PDT) To: David Lang References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> <561AD47B.9030602@isi.edu> From: Joe Touch X-Enigmail-Draft-Status: N1110 Message-ID: <561D1B40.2060006@isi.edu> Date: Tue, 13 Oct 2015 07:54:56 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" , touch@isi.edu, "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Jonathan Morton , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2015 14:55:31 -0000 David, On 10/11/2015 3:49 PM, David Lang wrote: >> One of the reasons we use packets is to provide more timely, >> fine-grained feedback between endpoints. Aggregating them at the source >> or in the network (rather than at the receiver) can amplify reaction to >> packet loss (when the aggregate ACK is lost) > > the issue of amplified reaction to packet loss is a valid one, but > packet loss is still pretty rare, But you claim to need to drop the ACKs to avoid dropping other packets. What's to say that 1) the ACKs won't end up getting dropped in your system or 2) the ACKs won't get dropped further downstream? Again, *you* don't know. ... >> and increases >> discretization effects in the TCP algorithms. > > This is where I disagree with you. The ACK packets are already collapsed > time-wise. Time is only one element of the discretization. The other is fate sharing - dropping one aggregate ACK has the effect of dropping the entire set of un-aggregated ones. The stream of unaggregated ACKs gives TCP a robustness to drops, but you're now forcing a large burst of drops for a specific packet type. That's the problem. We've gone round this issue for a lot of messages, but the key point is that this isn't a new idea, nor are the numerous considerations and concerns. Ultimately, if you have too many packets from a protocol that reacts to congestion, the correct solution is to provide that feedback and let the ends figure out how to respond. That's the only solution that will work when (not if) new messages or encrypted headers enter the picture. These kinds of in-network hacks are what's really killing the ability of the Internet to evolve. Joe From nobody Tue Oct 13 12:09:27 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 23FF01A8902; Tue, 13 Oct 2015 12:09:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aAsJQKfOCEDZ; Tue, 13 Oct 2015 12:09:24 -0700 (PDT) Received: from bifrost.lang.hm (mail.lang.hm [64.81.33.126]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 16CCC1A8900; Tue, 13 Oct 2015 12:09:23 -0700 (PDT) Received: from asgard.lang.hm (asgard.lang.hm [10.0.0.100]) by bifrost.lang.hm (8.13.4/8.13.4/Debian-3) with ESMTP id t9DJ9CWO027513; Tue, 13 Oct 2015 12:09:12 -0700 Date: Tue, 13 Oct 2015 12:09:12 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: Joe Touch In-Reply-To: <561D1B40.2060006@isi.edu> Message-ID: References: <5618005A.8070303@isi.edu> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> <561AD47B.9030602@isi.edu> <561D1B40.2060006@isi.edu> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: Cc: "tcpm@ietf.org" , "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Jonathan Morton , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2015 19:09:26 -0000 On Tue, 13 Oct 2015, Joe Touch wrote: > David, > > On 10/11/2015 3:49 PM, David Lang wrote: >>> One of the reasons we use packets is to provide more timely, >>> fine-grained feedback between endpoints. Aggregating them at the source >>> or in the network (rather than at the receiver) can amplify reaction to >>> packet loss (when the aggregate ACK is lost) >> >> the issue of amplified reaction to packet loss is a valid one, but >> packet loss is still pretty rare, > > But you claim to need to drop the ACKs to avoid dropping other packets. > What's to say that 1) the ACKs won't end up getting dropped in your > system or 2) the ACKs won't get dropped further downstream? > > Again, *you* don't know. combining ACKs further downstream should not hurt anything (as long as the RFC3449 criteria are covered) _dropping_ ACKs without combinging them is packet loss, pure and simple. With all the implications of that. The aggregated ACK is more significant that one of the other ACKs that were combined into it, but reducing the number of packets reduces the chances of needing to drop a packet. And the suggestion that was made to dup the aggregated ACK into the next transmit slot will greatly reduce the impact of the aggregated ACK being lost. >>> and increases >>> discretization effects in the TCP algorithms. >> >> This is where I disagree with you. The ACK packets are already collapsed >> time-wise. > > Time is only one element of the discretization. The other is fate > sharing - dropping one aggregate ACK has the effect of dropping the > entire set of un-aggregated ones. The stream of unaggregated ACKs gives > TCP a robustness to drops, but you're now forcing a large burst of drops > for a specific packet type. That's the problem. this is why the suggestion to dup the aggregated ACK into the next transmit slot is such a great idea (I'd have to go back through this thread and dig up who suggested it). It almost entirely mitigates the problem of the ACK packet getting dropped. > We've gone round this issue for a lot of messages, but the key point is > that this isn't a new idea, nor are the numerous considerations and > concerns. > > Ultimately, if you have too many packets from a protocol that reacts to > congestion, the correct solution is to provide that feedback and let the > ends figure out how to respond. That's the only solution that will work > when (not if) new messages or encrypted headers enter the picture. > > These kinds of in-network hacks are what's really killing the ability of > the Internet to evolve. If people are faced with existing protocols not working well, or making local modifications that make things work, but limit the ability for the "Internet to eveolve" the local modifications are going to win every time. This is the Internet evolving, just not in the way that you want. David Lang From nobody Tue Oct 13 12:36:16 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EDDCA1A8968; Tue, 13 Oct 2015 12:36:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7tAz_SveVkKG; Tue, 13 Oct 2015 12:36:12 -0700 (PDT) Received: from bifrost.lang.hm (mail.lang.hm [64.81.33.126]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 20B251A8965; Tue, 13 Oct 2015 12:36:12 -0700 (PDT) Received: from asgard.lang.hm (asgard.lang.hm [10.0.0.100]) by bifrost.lang.hm (8.13.4/8.13.4/Debian-3) with ESMTP id t9DJa1AG027663; Tue, 13 Oct 2015 12:36:02 -0700 Date: Tue, 13 Oct 2015 12:36:01 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: Greg White In-Reply-To: Message-ID: References: <5618005A.8070303@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> <561AD47B.9030602@isi.edu> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Archived-At: Cc: "tcpm@ietf.org" , Joe Touch , "mallman@icir.org" , "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" , Jonathan Morton , "aqm@ietf.org" Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2015 19:36:14 -0000 On Mon, 12 Oct 2015, Greg White wrote: >> My contention is that since this is already happening, consolidating the ACK >> packets into stretched ACKs doesn't make this any worse, and it saves network >> bandwidth (and decreases latency to the extent that data is acked faster than >> waiting for the entire chain or original ACKs to get through, especially if >> that would take multiple transmit windows). As a result, thinning the ACKs is >> kinder to the network. > > I agree, *iff* they are not DupACKs signalling packet loss. Do existing and > future cable modems take that subtle distinction into account? > > I presume that they are not discarding DupACKs or corrupting SACK in some way > (the whole point of specialized ACK handling is to ensure good TCP performance > after all), but since the algorithms are proprietary, for existing cable > modems I don't know. I will contact the developers as well as run some tests > on a few modems to get a better idea (and maybe Mikael already has some data > from his recent testing he could share?). For future modems, I can add an > explicit reference to RFC3449 in the DOCSIS 3.1 spec just in case implementers > aren't aware for some reason. It should also say that AQM is needed on the downstream side to protect against transmit bursts. The suggestion was posted early in this thread (but not in someting I can quickly find) that to harden the connection against the aggregate ACK being lost, it should be sent again in a separate transmit window. I latched onto that and think the right thing to do is to duplicate the aggregate ACK that's created and put it at the beginning of the queue that remains after the current transmit window. If there are more ACKs in the next transmit window, it will get aggregated into them and not cause any additional traffic. If there are not any more ACKs in the next transmit window, and the first ACK was lost, it will replace it with the minimum of delay If there are not any more ACKs in the next transmit window, and the first ACK was not lost, we are now transmitting a duplicate ACK, which may be interpreted as a signal to speed up tranmission (depending on the exact timing of the arrivials of these two ACKs). While this is not good, we already need to be protected from transmit bursts, so this is just another possible cause for such bursts. David Lang From nobody Tue Oct 13 12:41:25 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 22E5C1A8974 for ; Tue, 13 Oct 2015 12:41:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Wkc_ys0NHoVg for ; Tue, 13 Oct 2015 12:41:23 -0700 (PDT) Received: from bifrost.lang.hm (mail.lang.hm [64.81.33.126]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2C8281A8966 for ; Tue, 13 Oct 2015 12:41:23 -0700 (PDT) Received: from asgard.lang.hm (asgard.lang.hm [10.0.0.100]) by bifrost.lang.hm (8.13.4/8.13.4/Debian-3) with ESMTP id t9DJfGRH027736; Tue, 13 Oct 2015 12:41:16 -0700 Date: Tue, 13 Oct 2015 12:41:16 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: Jeremy Harris In-Reply-To: <561BE357.4090405@wizmail.org> Message-ID: References: <5618005A.8070303@isi.edu> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> <561AD47B.9030602@isi.edu> <561BE357.4090405@wizmail.org> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Archived-At: Cc: tcpm@ietf.org Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2015 19:41:24 -0000 On Mon, 12 Oct 2015, Jeremy Harris wrote: > On 11/10/15 23:49, David Lang wrote: >> packet loss is still pretty rare, > > Not in my day-job it isn't. > >> The ACK packets are already collapsed >> time-wise. Whatever timing they had when they were generated, the fact >> that they are all sitting in the same queue at the same time means taht >> that timing has been lost, > > Unless they carried timestamps? What are you doing with TCP options > (and ECN marks)? This is an area that should be discussed more. RFC3449 says "uitable algorithms to support IPSec authentication, SACK, and ECN remain areas of ongoing research." I don't know what research has been done with this since 2002. The cablemodem folks have practical experience doing _something_. We should probably nail down what they are doing with SACK, ECN, etc and work through the issues that they could be causing so that we can expand on RFC3449 and say what should be done in those cases. David Lang From nobody Tue Oct 13 17:22:18 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A5C2F1A9151 for ; Tue, 13 Oct 2015 17:22:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.3 X-Spam-Level: X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dvDXL5sQv9cM for ; Tue, 13 Oct 2015 17:22:10 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B7DAE1A914D for ; Tue, 13 Oct 2015 17:22:08 -0700 (PDT) Received: from 61.153.112.87.dyn.plus.net ([87.112.153.61]:33242 helo=[192.168.0.6]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85) (envelope-from ) id 1Zm9pV-00030Q-Tx; Wed, 14 Oct 2015 01:22:06 +0100 To: Praveen Balasubramanian , tcpm IETF list , Richard Scheffenegger , =?UTF-8?Q?Mirja_K=c3=bchlewind?= References: <55F055AD.3050809@tik.ee.ethz.ch> <55F05D54.5060708@tik.ee.ethz.ch> <55FF2910.7080908@bobbriscoe.net> From: Bob Briscoe Message-ID: <561DA02D.8020001@bobbriscoe.net> Date: Wed, 14 Oct 2015 01:22:05 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------010809000509060904030800" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net Archived-At: Subject: Re: [tcpm] Fwd: Fwd: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 00:22:16 -0000 This is a multi-part message in MIME format. --------------010809000509060904030800 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Praveen, Sorry I've taken so long to reply. Inline... On 28/09/15 18:35, Praveen Balasubramanian wrote: > > Hi all, > > I reviewed this draft and have a few comments below. Overall the draft > is well written and covers all the corner cases. > > Thanks > Thankyou. We tried v hard to keep it clear, and to ensure that the logical flow was not interrupted by the corner cases. > > The section 2.3 summary immediately brings about a question around > stretch ACKs. I see that you explain the receiver requirement in > section 3.2.2 (the delayed ACK frequency limit of 6 for CE marked > packets) but it might be worth suggesting this up front. > That's a good point - as you know, section 2 is meant to be a discursive summary of the detailed rules in section 3. However, in this case, I think we rewrote 3.2.2 after we had written 2.3. So, you're right that the summary ought to give more of a hint of how this will be solved. Will do. > > Is there any model to suggest what would be the accuracy of the scheme > described in Appendix A.3 to estimate the number of marked bytes from > the ACE field? Using a new TCP option will have deployment issues. > Yes, it's important to check how functional the protocol will be for those cases where the TCP option won't get through. But no, we haven't yet modelled the accuracy of estimating marked bytes from marked packets. I don't want to spend too much time on that until someone has attempted to implement AccECN (Mirja has made a good start). Because, at least in Linux, it might be relatively easy for the sender to record the sizes of all the segments in flight, so that the sender can calculate, rather than estimate, CE marked bytes. Estimating, rather than calculating, might still be necessary for Linux, and certainly some other OSs such as FreeBSD. But I would rather understand what is possible in at least one OS before modelling this problem. At present DCTCP measures the extent of congestion in packets not bytes. But we envisaged that an algorithm like Relentless TCP might be used where, every time new CE feedback arrives, the sender just decrements cwnd by (CE bytes)/2. > With AccECN if SYN can now be safely ECN-capable then what are the > drawbacks of making it a requirement? > FIrst I need to correct the premise of your question. Just because an AccECN receiver /can/ feed back CE on a SYN, does not mean it is safe for a sender to set ECN-capable on a SYN. A feedback protocol has to feed back stuff it receives, whether or not that stuff is valid (so that the sender can know what stuff appeared at the other end in order to detect middlebox mangling as well as valid protocol transitions). Quoting section 2.5 entitled "Generic (Dumb) Reflector": Providing feedback of CE marking on the SYN also supports future scenarios in which SYNs might be ECN-enabled (without prejudging whether they ought to be). Indeed, an ECN-capable SYN is outside the scope of AccECN, which solely covers the feedback behaviour of a receiver. An ECN-capable SYN would have to be defined in an RFC that covered TCP sender behaviour. E.g. perhaps, once your DCTCP draft becomes RFC, we might work on an evolution of DCTCP for both data centres and the public Internet, as discussed in Prague. Nonetheless, to answer your question, IMO making a SYN ECN-capable has no drawbacks, /as long as/ it is accompanied by the following safety measures: a) At the SYN stage, the TCP client doesn't yet know whether the server even understands ECN. So if a SYN were ECN-capable and then it was marked CE in the network, a non-ECN capable receiver would just ignore the marking. Therefore, if the sender received a SYN-ACK that did not complete the ECN negotiation, it would have to conservatively assume that the SYN might have been CE-marked, and reduce its initial window as if it was. b) There is some concern that ECN-capable SYN floods could starve non-ECN traffic. However, all AQMs are required to switch off ECN marking if the queue starts to overload. So, before I believe an ECN-capable SYN could be a DoS risk, I would like to see someone demonstrate an ECN-capable SYN flood that does more harm than a non-ECN SYN flood. I doubt it would be possible, because before the attack could have any effect, the cause of the effect (ECN) would be turned off. Interestingly, in the recent (2015) study on ECN traversal (Trammell et al) they found ECT or CE on SYN was dropped in only 0.82% and 0.61% of cases (v4 & v6 resp.), which was (to me) surprisingly low, given RFC3168 says (without evidence) that this could be a DoS attack vector. > Regarding the requirement that "a TCP server that confirms its support > for AccECN (as described in Section 3.1) MUST also include an AccECN > TCP Option in the SYN/ACK", it seems onerous to make the supplementary > part a MUST. The server may have prior information that packets in > 3WHS are dropped on a particular path. The server may also have a knob > which is set in cases where the middleware is known to discard packets > with unknown TCP option. > Good point. We need to allow an exception where the host has cached knowledge that AccECN option black-holes (but it should test occasionally in case the black hole has been plugged). Not only for the SYN/ACK from the server, but also for the first ACK and first data segment from the client. > From practical deployment point of view section 3.3 needs to be > expanded upon to include how LSO and LRO would work in presence of the > new option. > Yes, it might be useful to write another appendix with examples of how segmentation offload might work. If you could help write that, it would be v useful (I know Richard has expertise in this area too). > > For the LSO case the host can choose to generate an extra ACK with the > option rather than piggybacking it on a large send. However even if > the option is set on an LSO send, the NIC can repeat the option in > every segment without any adverse affects. > Care! We said the following for a reason: Nonetheless, such hardware MUST attempt to preserve the timing of each ACK (for example, if it coalesced ACKs it would not be AccECN- compliant). If the large send coalesces the ACKs (whether data ACKs or pure ACKs) for a number of received segments, the other end would see possibly multiple counters incremented, representing different markings on different arriving packets. ...Ideally we wanted the feedback to make it possible for a sender to extract the order and the timing of each transition between one ECN marking and another, as they arrived at the receiver. For instance, in 2010, Mirja and I wrote a paper about how to implement rapid detection of available capacity using packet chirps . It would be really useful to be able for AccECN to support any senders trying to do something similar, but with a shallow ECN threshold like that used in DCTCP, particularly during slow-start. I'm not saying all senders would have to work like that, just that it would be nice if a generic receiver behaviour would allow some senders to. Examples like this justify what we called "change-triggered ACKs". We (the co-authors) had long internal arguments about whether to make change-triggered ACKs a MUST or a SHOULD. We decided that the draft would say 'SHOULD', but the WG might want to continue our argument. The problem: if a sender wants to rely on change-triggered ACKs to get rapid info early in a new flow, it needs to be sure that the receiver has implemented change-triggered ACKs. We wanted to keep it simple, so we didn't want this to be a negotiable option. Therefore, such a sender would want change-triggered ACKs to be a MUST. However, we didn't want to write a spec that might be impossible to implement with today's hardware. So we settled on a SHOULD. But then our chirping sender cannot be sure whether the feedback is doing what it SHOULD or not, so it has to detect what the receiver is probably doing, which wastes time - precisely what the sender doesn't want to do. So, in the end, we decided to write the SHOULD with an extra-strong plea to say you really ought to think really hard before not implementing this SHOULD - what one might call a "MUST-SHOULD"! > LRO will be complicated if data segments contain the new option. At > least in Windows, LRO does not coalesce pure ACKs. > My personal view: I don't worry if acceleration hardware cannot handle newly defined capabilities, as long as software implementations are fast enough for the new protocol to be usable and to become successful. IMO, acceleration hardware is designed to accelerate today's common operations, and no-one would expect it to always be able to accelerate a newly invented protocol. As long as AccECN is good enough to be successful, tomorrow's hardware will accelerate it. This is definitely an area where we are looking for input from people like you and others in the WG. Thanks again for your comments. We've got a few minor edits, which we will fold in with the changes promised above, assuming you, my co-authors and the WG are all happy with my responses. Cheers Bob > *From:*tcpm [mailto:tcpm-bounces@ietf.org] *On Behalf Of *Bob Briscoe > *Sent:* Sunday, September 20, 2015 2:46 PM > *To:* Mirja Kühlewind ; > tcpPrague@ietf.org > *Cc:* tcpm IETF list ; Richard Scheffenegger > > *Subject:* Re: [tcpm] Fwd: Fwd: New Version Notification for > draft-kuehlewind-tcpm-accurate-ecn-04.txt > > tcpPrague list and Mirja, > > For those of you who don't follow the IETF so closely, the reason for > not using DCTCP's original feedback scheme was because it was not > designed to cope with ACK loss. > > This is explained in the requirements for more accurate ECN feedback > that were recently agreed and published as RFC 7560 > . The appendix in that RFC gives > some examples where the original DCTCP would get highly confused by a > few ACK losses. > > In that RFC it was admitted that no-one expected that all the > requirements could be satisfied at once. > * The previous version > satisfied them all except 'simple', but there was push-back on that. > * So this time > > has gone for simple, but compromised a bit on 'low to zero overhead'. > > As Mirja said, pls read it and tell the IETF tcpm list (in cc) whether > you support this new approach, and if not, why not. > The IETF doesn't charter work if no-one is talking about it. > > Cheers > > > Bob > > On 09/09/15 17:24, Mirja Kühlewind wrote: > > Hi all, > > find below the mail that I've just sent to the tcpm list. You > might have seen this already but I ed explicitly point this out to > the tcpprague list as we partly discussed this activity already at > the meeting in Prague. > > In short AccECN can be used with DCTCP or any other (DCTCP-like) > scheme that needs more accurate information on how many ECN > markings have been received. If that's interesting for you, please > read the draft and provide feedback. Please make sure that you > also cc the tcpm list regarding all feedback that is directly on > AccECN and the draft! > > If you would like to discuss future usage of AccECN, this can > happen on this list only, or you may cc tcpm if e.g. your use case > would require changes to AccECN as currently proposed. > > Thanks! > Mirja > > > -------- Forwarded Message -------- > Subject: [tcpm] Fwd: New Version Notification for > draft-kuehlewind-tcpm-accurate-ecn-04.txt > Date: Wed, 9 Sep 2015 17:52:13 +0200 > From: Mirja Kühlewind > > To: tcpm@ietf.org Extensions > , Bob Briscoe > , Richard > Scheffenegger > > Hi all, > > we submitted a new draft for AccECN (see below). We significantly > simplified the > draft because we believe it is more important to have a solution > that is easy to > understand and therefore more easy to correctly implement, than > having a > solution that tries to fulfill all requirements but is overly > complex. > > In short, we now only overwrite the 3 TCP header bits (no > additional bits in the > TCP header are used) and use them simply for a CE packet counter, > while > bytes-wise information on all other markings is provided by a new > TCP option. In > case the option is not available, e.g. because it's blocked, this > will still > provide the needed information to react to a CE-based congestion > feedback. > However, any advanced mechanism that needs further information on > the other > markings received will only work if the option is available. > > Further, we also tried to keep the draft as short as possible. > That means the > actual normative part only has 9 pages. However, we still provide > an overview > section with some reasoning as well as some discussion on > interaction with other > mechanisms which leads in total to 25 pages (without appendix). > > I'm currently also working on an implementation of this proposed > solution. I > will announce it as soon as I'm ready! > > In any case we would like to discuss this at the next meeting. We, > the author, > think that this proposal is now the right way forward and hope > that this > simplified proposal will allows us to quickly proceed on AccECN as > the need for > it is increasing. > > Please let us know if you have any feedback on the draft! > > Mirja > > > -------- Forwarded Message -------- > Subject: New Version Notification for > draft-kuehlewind-tcpm-accurate-ecn-04.txt > Date: Sun, 06 Sep 2015 16:19:47 -0700 > From: internet-drafts@ietf.org > To: Richard Scheffenegger , > "Mirja Kühlewind" > > , Mirja Kuehlewind > > , Richard Scheffenegger > , Bob > Briscoe , Bob > Briscoe > > > A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-04.txt > has been successfully submitted by Bob Briscoe and posted to the > IETF repository. > > Name: draft-kuehlewind-tcpm-accurate-ecn > Revision: 04 > Title: More Accurate ECN Feedback in TCP > Document date: 2015-09-06 > Group: Individual Submission > Pages: 36 > URL: > https://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-04.txt > > Status: > https://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn/ > Htmlized: > https://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-04 > Diff: > https://www.ietf.org/rfcdiff?url2=draft-kuehlewind-tcpm-accurate-ecn-04 > > > Abstract: > Explicit Congestion Notification (ECN) is a mechanism where > network > nodes can mark IP packets instead of dropping them to indicate > incipient congestion to the end-points. Receivers with an ECN- > capable transport protocol feed back this information to the > sender. > ECN is specified for TCP in such a way that only one feedback > signal > can be transmitted per Round-Trip Time (RTT). Recently, new TCP > mechanisms like Congestion Exposure (ConEx) or Data Center TCP > (DCTCP) need more accurate ECN feedback information whenever more > than one marking is received in one RTT. This document > specifies an > experimental scheme to provide more than one feedback signal > per RTT > in the TCP header. Given TCP header space is scarce, it > overloads > the three existing ECN-related flags in the TCP header and > provides > additional information in a new TCP option. > > > > > Please note that it may take a couple of minutes from the time of > submission > until the htmlized version and diff are available at tools.ietf.org. > > The IETF Secretariat > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > > > > -- > ________________________________________________________________ > Bob Briscoehttp://bobbriscoe.net/ -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ --------------010809000509060904030800 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Praveen,

Sorry I've taken so long to reply.
Inline...


On 28/09/15 18:35, Praveen Balasubramanian wrote:

Hi all,

 

I reviewed this draft and have a few comments below. Overall the draft is well written and covers all the corner cases.

 

Thanks

Thankyou. We tried v hard to keep it clear, and to ensure that the logical flow was not interrupted by the corner cases.

 

 

The section 2.3 summary immediately brings about a question around stretch ACKs. I see that you explain the receiver requirement in section 3.2.2 (the delayed ACK frequency limit of 6 for CE marked packets) but it might be worth suggesting this up front.

That's a good point - as you know, section 2 is meant to be a discursive summary of the detailed rules in section 3. However, in this case, I think we rewrote 3.2.2 after we had written 2.3. So, you're right that the summary ought to give more of a hint of how this will be solved. Will do.

 

Is there any model to suggest what would be the accuracy of the scheme described in Appendix A.3 to estimate the number of marked bytes from the ACE field? Using a new TCP option will have deployment issues.

Yes, it's important to check how functional the protocol will be for those cases where the TCP option won't get through.
But no, we haven't yet modelled the accuracy of estimating marked bytes from marked packets.

I don't want to spend too much time on that until someone has attempted to implement AccECN (Mirja has made a good start). Because, at least in Linux, it might be relatively easy for the sender to record the sizes of all the segments in flight, so that the sender can calculate, rather than estimate, CE marked bytes.

Estimating, rather than calculating, might still be necessary for Linux, and certainly some other OSs such as FreeBSD. But I would rather understand what is possible in at least one OS before modelling this problem.

At present DCTCP measures the extent of congestion in packets not bytes. But we envisaged that an algorithm like Relentless TCP might be used where,  every time new CE feedback arrives, the sender just decrements cwnd by (CE bytes)/2.

 

With AccECN if SYN can now be safely ECN-capable then what are the drawbacks of making it a requirement?

FIrst I need to correct the premise of your question. Just because an AccECN receiver /can/ feed back CE on a SYN, does not mean it is safe for a sender to set ECN-capable on a SYN. A feedback protocol has to feed back stuff it receives, whether or not that stuff is valid (so that the sender can know what stuff appeared at the other end in order to detect middlebox mangling as well as valid protocol transitions).

Quoting section 2.5  entitled "Generic (Dumb) Reflector":

   Providing feedback of CE marking on the SYN also supports future
   scenarios in which SYNs might be ECN-enabled (without prejudging
   whether they ought to be).

Indeed, an ECN-capable SYN is outside the scope of AccECN, which solely covers the feedback behaviour of a receiver. An ECN-capable SYN would have to be defined in an RFC that covered TCP sender behaviour. E.g. perhaps, once your DCTCP draft becomes RFC, we might work on an evolution of DCTCP for both data centres and the public Internet, as discussed in Prague.


Nonetheless, to answer your question, IMO making a SYN ECN-capable has no drawbacks, /as long as/ it is accompanied by the following safety measures:

a) At the SYN stage, the TCP client doesn't yet know whether the server even understands ECN. So if a SYN were ECN-capable and then it was marked CE in the network, a non-ECN capable receiver would just ignore the marking. Therefore, if the sender received a SYN-ACK that did not complete the ECN negotiation, it would have to conservatively assume that the SYN might have been CE-marked, and reduce its initial window as if it was.

b) There is some concern that ECN-capable SYN floods could starve non-ECN traffic. However, all AQMs are required to switch off ECN marking if the queue starts to overload. So, before I believe an ECN-capable SYN could be a DoS risk, I would like to see someone demonstrate an ECN-capable SYN flood that does more harm than a non-ECN SYN flood. I doubt it would be possible, because before the attack could have any effect, the cause of the effect (ECN) would be turned off.

Interestingly, in the recent (2015) study on ECN traversal (Trammell et al) they found ECT or CE on SYN was dropped in only 0.82% and 0.61% of cases (v4 & v6 resp.), which was (to me) surprisingly low, given RFC3168 says (without evidence) that this could be a DoS attack vector.


 

Regarding the requirement that "a TCP server that confirms its support for AccECN (as described in Section 3.1) MUST also include an AccECN TCP Option in the SYN/ACK", it seems onerous to make the supplementary part a MUST. The server may have prior information that packets in 3WHS are dropped on a particular path. The server may also have a knob which is set in cases where the middleware is known to discard packets with unknown TCP option.

Good point. We need to allow an exception where the host has cached knowledge that AccECN option black-holes (but it should test occasionally in case the black hole has been plugged).
Not only for the SYN/ACK from the server, but also for the first ACK and first data segment from the client.

 

From practical deployment point of view section 3.3 needs to be expanded upon to include how LSO and LRO would work in presence of the new option.

Yes, it might be useful to write another appendix with examples of how segmentation offload might work. If you could help write that, it would be v useful (I know Richard has expertise in this area too).

For the LSO case the host can choose to generate an extra ACK with the option rather than piggybacking it on a large send. However even if the option is set on an LSO send, the NIC can repeat the option in every segment without any adverse affects.

Care! We said the following for a reason:

   Nonetheless, such hardware MUST attempt to preserve the timing of
   each ACK (for example, if it coalesced ACKs it would not be AccECN-
   compliant).

If the large send coalesces the ACKs (whether data ACKs or pure ACKs) for a number of received segments, the other end would see possibly multiple counters incremented, representing different markings on different arriving packets.

...Ideally we wanted the feedback to make it possible for a sender to extract the order and the timing of each transition between one ECN marking and another, as they arrived at the receiver. For instance, in 2010, Mirja and I wrote a paper about how to implement rapid detection of available capacity using packet chirps <http://www.bobbriscoe.net/pubs.html#chirp_impl>. It would be really useful to be able for AccECN to support any senders trying to do something similar, but with a shallow ECN threshold like that used in DCTCP, particularly during slow-start.

I'm not saying all senders would have to work like that, just that it would be nice if a generic receiver behaviour would allow some senders to.

Examples like this justify what we called "change-triggered ACKs". We (the co-authors) had long internal arguments about whether to make change-triggered ACKs a MUST or a SHOULD. We decided that the draft would say 'SHOULD', but the WG might want to continue our argument.

The problem: if a sender wants to rely on change-triggered ACKs to get rapid info early in a new flow, it needs to be sure that the receiver has implemented change-triggered ACKs. We wanted to keep it simple, so we didn't want this to be a negotiable option. Therefore, such a sender would want change-triggered ACKs to be a MUST. However, we didn't want to write a spec that might be impossible to implement with today's hardware. So we settled on a SHOULD. But then our chirping sender cannot be sure whether the feedback is doing what it SHOULD or not, so it has to detect what the receiver is probably doing, which wastes time - precisely what the sender doesn't want to do.

So, in the end, we decided to write the SHOULD with an extra-strong plea to say you really ought to think really hard before not implementing this SHOULD - what one might call a "MUST-SHOULD"!

LRO will be complicated if data segments contain the new option. At least in Windows, LRO does not coalesce pure ACKs.

My personal view: I don't worry if acceleration hardware cannot handle newly defined capabilities, as long as software implementations are fast enough for the new protocol to be usable and to become successful. IMO, acceleration hardware is designed to accelerate today's common operations, and no-one would expect it to always be able to accelerate a newly invented protocol. As long as AccECN is good enough to be successful, tomorrow's hardware will accelerate it.

This is definitely an area where we are looking for input from people like you and others in the WG.


Thanks again for your comments.
We've got a few minor edits, which we will fold in with the changes promised above, assuming you, my co-authors and the WG are all happy with my responses.

Cheers



Bob

 

 

 

From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Bob Briscoe
Sent: Sunday, September 20, 2015 2:46 PM
To: Mirja Kühlewind <mirja.kuehlewind@tik.ee.ethz.ch>; tcpPrague@ietf.org
Cc: tcpm IETF list <tcpm@ietf.org>; Richard Scheffenegger <rscheff@gmx.at>
Subject: Re: [tcpm] Fwd: Fwd: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt

 

tcpPrague list and Mirja,

For those of you who don't follow the IETF so closely, the reason for not using DCTCP's original feedback scheme was because it was not designed to cope with ACK loss.

This is explained in the requirements for more accurate ECN feedback that were recently agreed and published as RFC 7560 <https://tools.ietf.org/html/rfc7560>. The appendix in that RFC gives some examples where the original DCTCP would get highly confused by a few ACK losses.

In that RFC it was admitted that no-one expected that all the requirements could be satisfied at once.
* The previous version <draft-kuehlewind-tcpm-accurate-ecn-03> satisfied them all except 'simple', but there was push-back on that.
* So this time <draft-kuehlewind-tcpm-accurate-ecn-04> has gone for simple, but compromised a bit on 'low to zero overhead'.

As Mirja said, pls read it and tell the IETF tcpm list (in cc) whether you support this new approach, and if not, why not.
The IETF doesn't charter work if no-one is talking about it.

Cheers


Bob

On 09/09/15 17:24, Mirja Kühlewind wrote:

Hi all,

find below the mail that I've just sent to the tcpm list. You might have seen this already but I ed explicitly point this out to the tcpprague list as we partly discussed this activity already at the meeting in Prague.

In short AccECN can be used with DCTCP or any other (DCTCP-like) scheme that needs more accurate information on how many ECN markings have been received. If that's interesting for you, please read the draft and provide feedback. Please make sure that you also cc the tcpm list regarding all feedback that is directly on AccECN and the draft!

If you would like to discuss future usage of AccECN, this can happen on this list only, or you may cc tcpm if e.g. your use case would require changes to AccECN as currently proposed.

Thanks!
Mirja


-------- Forwarded Message --------
Subject: [tcpm] Fwd: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt
Date: Wed, 9 Sep 2015 17:52:13 +0200
From: Mirja Kühlewind <mirja.kuehlewind@tik.ee.ethz.ch>
To: tcpm@ietf.org Extensions <tcpm@ietf.org>, Bob Briscoe <ietf@bobbriscoe.net>, Richard Scheffenegger <rscheff@gmx.at>

Hi all,

we submitted a new draft for AccECN (see below). We significantly simplified the
draft because we believe it is more important to have a solution that is easy to
understand and therefore more easy to correctly implement, than having a
solution that tries to fulfill all requirements but is overly complex.

In short, we now only overwrite the 3 TCP header bits (no additional bits in the
TCP header are used) and use them simply for a CE packet counter, while
bytes-wise information on all other markings is provided by a new TCP option. In
case the option is not available, e.g. because it's blocked, this will still
provide the needed information to react to a CE-based congestion feedback.
However, any advanced mechanism that needs further information on the other
markings received will only work if the option is available.

Further, we also tried to keep the draft as short as possible. That means the
actual normative part only has 9 pages. However, we still provide an overview
section with some reasoning as well as some discussion on interaction with other
mechanisms which leads in total to 25 pages (without appendix).

I'm currently also working on an implementation of this proposed solution. I
will announce it as soon as I'm ready!

In any case we would like to discuss this at the next meeting. We, the author,
think that this proposal is now the right way forward and hope that this
simplified proposal will allows us to quickly proceed on AccECN as the need for
it is increasing.

Please let us know if you have any feedback on the draft!

Mirja


-------- Forwarded Message --------
Subject: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt
Date: Sun, 06 Sep 2015 16:19:47 -0700
From: internet-drafts@ietf.org
To: Richard Scheffenegger <rs@netapp.com>, "Mirja Kühlewind"
<mirja.kuehlewind@tik.ee.ethz.ch>, Mirja Kuehlewind
<mirja.kuehlewind@tik.ee.ethz.ch>, Richard Scheffenegger <rs@netapp.com>, Bob
Briscoe <ietf@bobbriscoe.net>, Bob Briscoe <ietf@bobbriscoe.net>


A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-04.txt
has been successfully submitted by Bob Briscoe and posted to the
IETF repository.

Name:        draft-kuehlewind-tcpm-accurate-ecn
Revision:    04
Title:        More Accurate ECN Feedback in TCP
Document date:    2015-09-06
Group:        Individual Submission
Pages:        36
URL:
https://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-04.txt
Status:         https://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn/
Htmlized:       https://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-04
Diff:
https://www.ietf.org/rfcdiff?url2=draft-kuehlewind-tcpm-accurate-ecn-04

Abstract:
    Explicit Congestion Notification (ECN) is a mechanism where network
    nodes can mark IP packets instead of dropping them to indicate
    incipient congestion to the end-points.  Receivers with an ECN-
    capable transport protocol feed back this information to the sender.
    ECN is specified for TCP in such a way that only one feedback signal
    can be transmitted per Round-Trip Time (RTT).  Recently, new TCP
    mechanisms like Congestion Exposure (ConEx) or Data Center TCP
    (DCTCP) need more accurate ECN feedback information whenever more
    than one marking is received in one RTT.  This document specifies an
    experimental scheme to provide more than one feedback signal per RTT
    in the TCP header.  Given TCP header space is scarce, it overloads
    the three existing ECN-related flags in the TCP header and provides
    additional information in a new TCP option.




Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat


_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm 



-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/
--------------010809000509060904030800-- From nobody Wed Oct 14 02:49:56 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B3E0A1B2D58; Wed, 14 Oct 2015 02:49:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QeURTU8g4djO; Wed, 14 Oct 2015 02:49:47 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 165061B2D50; Wed, 14 Oct 2015 02:49:42 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: "Spencer Dawkins" To: "The IESG" X-Test-IDTracker: no X-IETF-IDTracker: 6.5.1 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20151014094942.15086.66278.idtracker@ietfa.amsl.com> Date: Wed, 14 Oct 2015 02:49:42 -0700 Archived-At: Cc: tcpm@ietf.org Subject: [tcpm] Spencer Dawkins' Yes on draft-ietf-tcpm-rtorestart-08: (with COMMENT) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 09:49:48 -0000 Spencer Dawkins has entered the following ballot position for draft-ietf-tcpm-rtorestart-08: Yes When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.) Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html for more information about IESG DISCUSS and COMMENT positions. The document, along with other ballot positions, can be found here: https://datatracker.ietf.org/doc/draft-ietf-tcpm-rtorestart/ ---------------------------------------------------------------------- COMMENT: ---------------------------------------------------------------------- Thanks for producing this. It looks helpful. I had a few comments you might consider. Overall - I'm reading "restart the RTO timer" and its variations as an action that would *increase* the delay before retransmission, but that's backwards - RTOR is an action that can *decrease* the delay before retransmission. I suspect I'm not the only reader who would be confused on this. The word "restart" is pervasive in this document (I counted 48 occurances, including page headers), so I can't reasonably ask about using a different term, but I'm wondering if it would be clearer if the abstract said something like OLD The modification, RTO Restart (RTOR), allows the transport to restart its retransmission timer so that the effective RTO becomes more aggressive in situations where fast retransmit cannot be used. NEW The modification, RTO Restart (RTOR), allows the transport to restart its retransmission timer using a smaller delay, so that the effective RTO becomes more aggressive in situations where fast retransmit cannot be used. In the Introduction, Second, when a sender receives duplicate acknowledgments, or similar information via selective acknowledgments, the fast retransmit algorithm infers data loss and triggers a retransmission. this is describing retransmission without any conditions. A couple of sentences later, the text describes the conditions that trigger a retransmission, so the paragraph gets it right in total, but it might be clearer if this sentence said something like Second, when a sender receives duplicate acknowledgments, or similar information via selective acknowledgments, the fast retransmit algorithm suspects data loss and can trigger a retransmission. In this sentence, Further experimentation is needed to determine this and thereby move this specification from experimental to proposed standard. if you make other text changes, you might consider s/to proposed standard/to the standards track/. In a perfect world, the specification might advance beyond proposed standard, given deployment experience ... From nobody Wed Oct 14 09:53:08 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 17A971ACE3B; Wed, 14 Oct 2015 09:53:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T2112JK5LL2r; Wed, 14 Oct 2015 09:53:05 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id CF7201ACE24; Wed, 14 Oct 2015 09:53:05 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: "Benoit Claise" To: "The IESG" X-Test-IDTracker: no X-IETF-IDTracker: 6.5.1 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20151014165305.11276.58309.idtracker@ietfa.amsl.com> Date: Wed, 14 Oct 2015 09:53:05 -0700 Archived-At: Cc: tcpm@ietf.org Subject: [tcpm] Benoit Claise's No Objection on draft-ietf-tcpm-rtorestart-08: (with COMMENT) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 16:53:07 -0000 Benoit Claise has entered the following ballot position for draft-ietf-tcpm-rtorestart-08: No Objection When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.) Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html for more information about IESG DISCUSS and COMMENT positions. The document, along with other ballot positions, can be found here: https://datatracker.ietf.org/doc/draft-ietf-tcpm-rtorestart/ ---------------------------------------------------------------------- COMMENT: ---------------------------------------------------------------------- Here is Tim's OPS DIR review: This document describes an experimental modification to the TCP Retransmission Timeout (RTO) to act more aggressivly in connections that are short-lived or application limited. It's well written. The document is for both TCP and SCTCP, though primarily the TCP implementation is discussed. This is fine as it is experimental. I found one thing in the introduction: This document describes a modified sender-side algorithm for managing the TCP and SCTP retransmission timers that provides faster loss recovery I believe that it should be "provide" singular and not plural. In section 4, there is this text: The RECOMMENDED value of rrthresh is four, as this value will ensure that RTOR is only used when fast retransmit cannot be triggered. This update needs TCP implementations to track the time elapsed since the transmission of the earliest outstanding segment (T_earliest). The text is saying the implementation track time elapsed, so should it say: "With this update, TCP implementations MUST track the time elapsed..."? From nobody Wed Oct 14 10:37:22 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4EAAA1AC3D5 for ; Wed, 14 Oct 2015 10:37:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.01 X-Spam-Level: X-Spam-Status: No, score=-6.01 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_21=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VNjl5zzpE5Ib for ; Wed, 14 Oct 2015 10:37:17 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5F7AD1ACD81 for ; Wed, 14 Oct 2015 10:37:17 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 0BEA0CF934096; Wed, 14 Oct 2015 17:37:12 +0000 (GMT) Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t9EHbFXl032337 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 14 Oct 2015 19:37:15 +0200 Received: from FR711WXCHMBA05.zeu.alcatel-lucent.com ([169.254.1.213]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.03.0195.001; Wed, 14 Oct 2015 19:37:15 +0200 From: "De Schepper, Koen (Koen)" To: =?utf-8?B?TWlyamEgS8O8aGxld2luZA==?= , "tcpm@ietf.org Extensions" , Bob Briscoe , Richard Scheffenegger Thread-Topic: [tcpm] Fwd: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt Thread-Index: AQHQ6xecH9RU3iyM/EmwHF1qn1drPp5rUo5g Date: Wed, 14 Oct 2015 17:37:14 +0000 Message-ID: References: <20150906231947.7501.54950.idtracker@ietfa.amsl.com> <55F055AD.3050809@tik.ee.ethz.ch> In-Reply-To: <55F055AD.3050809@tik.ee.ethz.ch> Accept-Language: nl-BE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.39] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] Fwd: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 17:37:21 -0000 SGksDQoNCkp1c3QgZmluaXNoZWQgcmVhZGluZy9yZXZpZXdpbmcgdGhpcyBkcmFmdC4gSSB0aGlu ayBpdCBpcyByZWFsbHkgd2VsbCB3cml0dGVuLCBzaW1wbGUgYW5kIGl0IGhhcyBnb25lIGNsZWFy bHkgdGhyb3VnaCBhIGxvdCBvZiB0aG91Z2h0cyBhbHJlYWR5LiANCg0KQmVsb3cgc29tZSBxdWVz dGlvbnMgYW5kIHJlbWFya3M6DQoNCi0gSXMgaG93IHRoaXMgcHJvdG9jb2wgaXMgaW50ZW5kZWQg dG8gYmUgdXNlZCBieSBjbGFzc2ljLUVDTi1UQ1AgYW5kIERDVENQIGRvY3VtZW50ZWQgc29tZXdo ZXJlPyBJIHRoaW5rIGl0IHVzZWZ1bCB0byBkZXNjcmliZSAobm90IHN1cmUgaWYgeW91IHdhbnQg dG8gYWRkIGl0IHRvIHRoaXMgZG9jPykNCg0KLSBXaHkgdGhlIHF1aXRlIHN0cm9uZyByZXN0cmlj dGlvbnMgaW4gd2hhdCBvZmZsb2FkIGVuZ2luZXMgYXJlIGFsbG93ZWQgdG8gZG8uIElzIGl0IHVz ZWZ1bCB0byBjbGVhcmx5IHNwZWNpZnkgd2hhdCBhbiBvZmZsb2FkIGVuZ2luZSBtdXN0IGRvPyBG b3IgZXhhbXBsZSBJIGNhbiBpbWFnaW5lIGlzIGEgbGFyZ2UgbGlzdCBvZiBwYWNrZXRzIGFyZSBy ZWNlaXZlZCBpbiBhIE5JQywgdGhlIGVuZ2luZSBjYW4gY29tYmluZSBhbGwgYWNrcyBhbmQgZGF0 YSBwYWNrZXRzIGFzIGxvbmcgYXMgb25seSBvbmUgY291bnRlciBpcyBpbmNyZW1lbnRlZCwgYW5k IG9ubHkgcmVwb3J0IHRoZSBsYXN0IHZhbHVlIG9mIHRoZSBjb3VudGVyIGluIHRoZSBvcHRpb24u IEFsc28gYXMgbG9uZyBhcyBpdCBjYW4gZ3VhcmFudGVlIHRoYXQgdGhlIEFDRSBjb3VudGVyIGlz IG5vdCBpbnRlcnByZXRlZCBhcyBiZWluZyB3cmFwcGVkLCBpdCBhbHNvIGNhbiBvZmZsb2FkIGJ5 IGNvbWJpbmluZyB1cCB0byA3IHBhY2tldHMgb3IgYWNrcy4uLg0KDQotIElmIEw0UyAoRENUQ1Av VENQLVByYWd1ZSkgc2hvdWxkIHVzZSBFQ1QxLCB0aGVuIGl0IG5lZWRzIGFsd2F5cyAxMSBhbmQg c29tZXRpbWVzIDggYnl0ZXMgb3B0aW9ucy4uLiBXaHkgbm90IHB1dHRpbmcgRUNFQiBmaXJzdCwg YW5kIGFsbG93IGEgbW9yZSByZWxheGVkIHJlcXVpcmVtZW50IG9uIHNpZ25hbGluZyBFRTBCIGFu ZCBFRTFCPyBJZiBvcHRpb24gc3BhY2UgaXMgbGltaXRlZCwgYXQgbGVhc3QgRUNFQiB3aWxsIGJl IGFibGUgdG8gZ2V0IHRocm91Z2ggZW5vdWdoIHRvIGVzdGltYXRlIHRoZSBjb25nZXN0aW9uLg0K DQotIElmIEkgYW0gaW50ZXJlc3RlZCBpbiB0aGUgcHJvYmFiaWxpdHkgb2YgbWFya3MsIGp1c3Qg ZWNob2luZyB0aGUgbGFzdCByZWNlaXZlZCBJUCBtYXJrIHdvdWxkIHBlcmZlY3RseSBmaXQgdGhp cyByZXF1aXJlbWVudCAoYXNzdW1pbmcgcmFuZG9tIG1hcmtpbmcpLiBDdXJyZW50bHkgdGhpcyBp cyBub3Qgc3VwcG9ydGVkLCBhbmQgdGhpbmtpbmcgYWJvdXQgdGhpcywgeW91ciAnc2FmZScgYWxn b3JpdGhtcyB3aWxsIGdpdmUgcG90ZW50aWFsbHkgcXVpdGUgZGV2aWF0aW5nIHJlc3VsdHMsIHRh a2luZyBhbGwgcG9zc2libGUgQUNLIGxvc3Nlcy9taXNzZXMgaW50byBhY2NvdW50LiBBcmUgdGhl c2UgZnV0dXJlLXVzZSByZXF1aXJlbWVudHMgd29ydGggdGhlIG92ZXJoZWFkL2NvbXBsZXhpdHk/ DQoNCi0gTDRTIChEQ1RDUC9QcmFndWUvLi4uKSBjYW4gaGF2ZSBvZnRlbiAocmFuZG9tKSBtYXJr aW5nIHByb2JhYmlsaXRpZXMgYXJvdW5kIDUwJSwgY2F1c2luZyB0aGUgY2hhbmdlIHRyaWdnZXJl ZCBhY2tzIHRvIGFjayBhbG1vc3QgZXZlcnkgcGFja2V0ICh0aGlzIGlzIHdoYXQgdGhlIGN1cnJl bnQgRENUQ1AgaXMgZG9pbmcgdG9vLCBidXQgb24gYSBzdGVwLVJFRCBBUU0geW91IGdldCBsYXJn ZSBvbi1vZmYgc2VyaWVzLCB3aXRoIER1YWxRIHlvdSBnZXQgYSByYW5kb20gZGlzdHJpYnV0aW9u IHdpdGggbm9uZSBvciBzbWFsbCBzZXJpZXMpLiBJcyBvcmRlciBzbyBpbXBvcnRhbnQ/IE90aGVy d2lzZSwgdGhlIGFja3MgY291bGQganVzdCBjYXJyeSBFRTBCIChvciBldmVuIGltbWVkaWF0ZWx5 IEVDRUIgYXMgYWJvdmUpLCB0aGUgc2VuZGVyIGtub3dzIGhvdyBtdWNoIGJ5dGVzIGl0IGhhcyBz ZW50LCBhc3N1bWluZyBhbGwgdGhlIHJlc3QgaXMgRUNFQiAoRUUwQi9FRTFCIGFzIGFwcHJvcHJp YXRlKS4NCg0KU29tZSB0eXBvJ3MgYW5kIGNvbmZ1c2luZyB0ZXh0Og0KDQotIFRoZSB0ZXJtIGNs YXNzaWMgRUNOIGNvbmZ1c2VkIG1lIGEgZmV3IHRpbWVzLiBJIHN1cHBvc2UgeW91IG1lYW4gY2xh c3NpYyBFQ04gZWNob2luZyBwcm90b2NvbCwgbm90IHRoZSByZXNwb25zZSB0byBjb25nZXN0aW9u IGluIHRoZSBUQ1Agc2VuZGVyLiBNYXliZSBjb25zaXN0ZW50IHVzZSBvZiAnY2xhc3NpYyBFQ04g ZWNob2luZycgaXMgY2xlYXJlci4NCg0KLSBUaGUgZHJhZnQgaXMgc3RydWN0dXJlZCB2ZXJ5IGdv b2QsIGFzIEkgY291bGQgcmVhZCBzZXF1ZW5jaWFsbHkgdGhvdWdoIGl0LCBleGNlcHQgZm9yIG9u ZSBjYXNlIHdoZXJlIEkgaGFkIHRvIGxvb2sgZnVydGhlciB0byB1bmRlcnN0YW5kIHdoYXQgd2Fz IGV4YWN0bHkgbWVhbnQ6IFRoZSB0aGlyZCBwYXJhZ3JhcGggb2Ygc2VjdGlvbiAzLjEgd2FzICIu Li4gVENQIHNlcnZlciBNVVNUIHNldCBOUz0xIG9uIHRoZSBTWU4vQUNLLiIuIENoYW5naW5nIGl0 IHRvICIuLi4gVENQIHNlcnZlciBNVVNUIHNldCBhZGRpdGlvbmFsbHkgTlM9MSBvbiB0aGUgU1lO L0FDSy4iLCBhbmQgb3B0aW9uYWxseSBhZGQgdGhhdCB0aGlzIGlzIG9ubHkgd2hlbiBpbiBhY2NF Q04gbW9kZSwgd291bGQgaGF2ZSBjbGFyaWZpZWQgYWxsIGRvdWJ0cy4NCg0KLSBUaGUgc2l4dGgg cGFyYWdyYXBoIG9mIHNlY3Rpb24gMy4xIGNvbnRhaW5zIGFuIElmICh3aXRoIGNhcGl0YWwgSSkg aW4gdGhlIG1pZGRsZSBvZiB0aGUgZmlzdCBzZW50ZW5jZS4NCg0KLSBTZWN0aW9uIDMuMi4xIGhh cyAidGh5IiBpbnN0ZWFkIG9mICJ0aGV5Ii4NCg0KLSBTZWN0aW9uIDMuMi4yIGhhcyAiKG4pIiBp bnN0ZWFkIG9mICJuIiwgbGF0ZXIgaXQgYWxzbyBhcHBlYXJzIGFzICJuIiB3aXRob3V0IHRoZSBi cmFja2V0cy4NCg0KLSBTZWN0aW9uIDUgaGFzIHRyaWlnZ2VyZWQgaW5zdGVhZCBvZiB0cmlnZ2Vy ZWQNCg0KDQpSZWdhcmRzLA0KS29lbi4NCg0KDQoNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2Ut LS0tLQ0KPiBGcm9tOiB0Y3BtIFttYWlsdG86dGNwbS1ib3VuY2VzQGlldGYub3JnXSBPbiBCZWhh bGYgT2YgTWlyamEgS8O8aGxld2luZA0KPiBTZW50OiB3b2Vuc2RhZyA5IHNlcHRlbWJlciAyMDE1 IDE3OjUyDQo+IFRvOiB0Y3BtQGlldGYub3JnIEV4dGVuc2lvbnM7IEJvYiBCcmlzY29lOyBSaWNo YXJkIFNjaGVmZmVuZWdnZXINCj4gU3ViamVjdDogW3RjcG1dIEZ3ZDogTmV3IFZlcnNpb24gTm90 aWZpY2F0aW9uIGZvciBkcmFmdC1rdWVobGV3aW5kLQ0KPiB0Y3BtLWFjY3VyYXRlLWVjbi0wNC50 eHQNCj4gDQo+IEhpIGFsbCwNCj4gDQo+IHdlIHN1Ym1pdHRlZCBhIG5ldyBkcmFmdCBmb3IgQWNj RUNOIChzZWUgYmVsb3cpLiBXZSBzaWduaWZpY2FudGx5DQo+IHNpbXBsaWZpZWQgdGhlDQo+IGRy YWZ0IGJlY2F1c2Ugd2UgYmVsaWV2ZSBpdCBpcyBtb3JlIGltcG9ydGFudCB0byBoYXZlIGEgc29s dXRpb24gdGhhdA0KPiBpcyBlYXN5IHRvDQo+IHVuZGVyc3RhbmQgYW5kIHRoZXJlZm9yZSBtb3Jl IGVhc3kgdG8gY29ycmVjdGx5IGltcGxlbWVudCwgdGhhbiBoYXZpbmcNCj4gYQ0KPiBzb2x1dGlv biB0aGF0IHRyaWVzIHRvIGZ1bGZpbGwgYWxsIHJlcXVpcmVtZW50cyBidXQgaXMgb3Zlcmx5IGNv bXBsZXguDQo+IA0KPiBJbiBzaG9ydCwgd2Ugbm93IG9ubHkgb3ZlcndyaXRlIHRoZSAzIFRDUCBo ZWFkZXIgYml0cyAobm8gYWRkaXRpb25hbA0KPiBiaXRzIGluIHRoZQ0KPiBUQ1AgaGVhZGVyIGFy ZSB1c2VkKSBhbmQgdXNlIHRoZW0gc2ltcGx5IGZvciBhIENFIHBhY2tldCBjb3VudGVyLCB3aGls ZQ0KPiBieXRlcy13aXNlIGluZm9ybWF0aW9uIG9uIGFsbCBvdGhlciBtYXJraW5ncyBpcyBwcm92 aWRlZCBieSBhIG5ldyBUQ1ANCj4gb3B0aW9uLiBJbg0KPiBjYXNlIHRoZSBvcHRpb24gaXMgbm90 IGF2YWlsYWJsZSwgZS5nLiBiZWNhdXNlIGl0J3MgYmxvY2tlZCwgdGhpcyB3aWxsDQo+IHN0aWxs DQo+IHByb3ZpZGUgdGhlIG5lZWRlZCBpbmZvcm1hdGlvbiB0byByZWFjdCB0byBhIENFLWJhc2Vk IGNvbmdlc3Rpb24NCj4gZmVlZGJhY2suDQo+IEhvd2V2ZXIsIGFueSBhZHZhbmNlZCBtZWNoYW5p c20gdGhhdCBuZWVkcyBmdXJ0aGVyIGluZm9ybWF0aW9uIG9uIHRoZQ0KPiBvdGhlcg0KPiBtYXJr aW5ncyByZWNlaXZlZCB3aWxsIG9ubHkgd29yayBpZiB0aGUgb3B0aW9uIGlzIGF2YWlsYWJsZS4N Cj4gDQo+IEZ1cnRoZXIsIHdlIGFsc28gdHJpZWQgdG8ga2VlcCB0aGUgZHJhZnQgYXMgc2hvcnQg YXMgcG9zc2libGUuIFRoYXQNCj4gbWVhbnMgdGhlDQo+IGFjdHVhbCBub3JtYXRpdmUgcGFydCBv bmx5IGhhcyA5IHBhZ2VzLiBIb3dldmVyLCB3ZSBzdGlsbCBwcm92aWRlIGFuDQo+IG92ZXJ2aWV3 DQo+IHNlY3Rpb24gd2l0aCBzb21lIHJlYXNvbmluZyBhcyB3ZWxsIGFzIHNvbWUgZGlzY3Vzc2lv biBvbiBpbnRlcmFjdGlvbg0KPiB3aXRoIG90aGVyDQo+IG1lY2hhbmlzbXMgd2hpY2ggbGVhZHMg aW4gdG90YWwgdG8gMjUgcGFnZXMgKHdpdGhvdXQgYXBwZW5kaXgpLg0KPiANCj4gSSdtIGN1cnJl bnRseSBhbHNvIHdvcmtpbmcgb24gYW4gaW1wbGVtZW50YXRpb24gb2YgdGhpcyBwcm9wb3NlZA0K PiBzb2x1dGlvbi4gSQ0KPiB3aWxsIGFubm91bmNlIGl0IGFzIHNvb24gYXMgSSdtIHJlYWR5IQ0K PiANCj4gSW4gYW55IGNhc2Ugd2Ugd291bGQgbGlrZSB0byBkaXNjdXNzIHRoaXMgYXQgdGhlIG5l eHQgbWVldGluZy4gV2UsIHRoZQ0KPiBhdXRob3IsDQo+IHRoaW5rIHRoYXQgdGhpcyBwcm9wb3Nh bCBpcyBub3cgdGhlIHJpZ2h0IHdheSBmb3J3YXJkIGFuZCBob3BlIHRoYXQNCj4gdGhpcw0KPiBz aW1wbGlmaWVkIHByb3Bvc2FsIHdpbGwgYWxsb3dzIHVzIHRvIHF1aWNrbHkgcHJvY2VlZCBvbiBB Y2NFQ04gYXMgdGhlDQo+IG5lZWQgZm9yDQo+IGl0IGlzIGluY3JlYXNpbmcuDQo+IA0KPiBQbGVh c2UgbGV0IHVzIGtub3cgaWYgeW91IGhhdmUgYW55IGZlZWRiYWNrIG9uIHRoZSBkcmFmdCENCj4g DQo+IE1pcmphDQo+IA0KPiANCj4gLS0tLS0tLS0gRm9yd2FyZGVkIE1lc3NhZ2UgLS0tLS0tLS0N Cj4gU3ViamVjdDogTmV3IFZlcnNpb24gTm90aWZpY2F0aW9uIGZvciBkcmFmdC1rdWVobGV3aW5k LXRjcG0tYWNjdXJhdGUtDQo+IGVjbi0wNC50eHQNCj4gRGF0ZTogU3VuLCAwNiBTZXAgMjAxNSAx NjoxOTo0NyAtMDcwMA0KPiBGcm9tOiBpbnRlcm5ldC1kcmFmdHNAaWV0Zi5vcmcNCj4gVG86IFJp Y2hhcmQgU2NoZWZmZW5lZ2dlciA8cnNAbmV0YXBwLmNvbT4sICJNaXJqYSBLw7xobGV3aW5kIg0K PiA8bWlyamEua3VlaGxld2luZEB0aWsuZWUuZXRoei5jaD4sIE1pcmphIEt1ZWhsZXdpbmQNCj4g PG1pcmphLmt1ZWhsZXdpbmRAdGlrLmVlLmV0aHouY2g+LCBSaWNoYXJkIFNjaGVmZmVuZWdnZXIN Cj4gPHJzQG5ldGFwcC5jb20+LCBCb2INCj4gQnJpc2NvZSA8aWV0ZkBib2JicmlzY29lLm5ldD4s IEJvYiBCcmlzY29lIDxpZXRmQGJvYmJyaXNjb2UubmV0Pg0KPiANCj4gDQo+IEEgbmV3IHZlcnNp b24gb2YgSS1ELCBkcmFmdC1rdWVobGV3aW5kLXRjcG0tYWNjdXJhdGUtZWNuLTA0LnR4dA0KPiBo YXMgYmVlbiBzdWNjZXNzZnVsbHkgc3VibWl0dGVkIGJ5IEJvYiBCcmlzY29lIGFuZCBwb3N0ZWQg dG8gdGhlDQo+IElFVEYgcmVwb3NpdG9yeS4NCj4gDQo+IE5hbWU6CQlkcmFmdC1rdWVobGV3aW5k LXRjcG0tYWNjdXJhdGUtZWNuDQo+IFJldmlzaW9uOgkwNA0KPiBUaXRsZToJCU1vcmUgQWNjdXJh dGUgRUNOIEZlZWRiYWNrIGluIFRDUA0KPiBEb2N1bWVudCBkYXRlOgkyMDE1LTA5LTA2DQo+IEdy b3VwOgkJSW5kaXZpZHVhbCBTdWJtaXNzaW9uDQo+IFBhZ2VzOgkJMzYNCj4gVVJMOg0KPiBodHRw czovL3d3dy5pZXRmLm9yZy9pbnRlcm5ldC1kcmFmdHMvZHJhZnQta3VlaGxld2luZC10Y3BtLWFj Y3VyYXRlLQ0KPiBlY24tMDQudHh0DQo+IFN0YXR1czogICAgICAgICBodHRwczovL2RhdGF0cmFj a2VyLmlldGYub3JnL2RvYy9kcmFmdC1rdWVobGV3aW5kLXRjcG0tDQo+IGFjY3VyYXRlLWVjbi8N Cj4gSHRtbGl6ZWQ6ICAgICAgIGh0dHBzOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1rdWVo bGV3aW5kLXRjcG0tDQo+IGFjY3VyYXRlLWVjbi0wNA0KPiBEaWZmOg0KPiBodHRwczovL3d3dy5p ZXRmLm9yZy9yZmNkaWZmP3VybDI9ZHJhZnQta3VlaGxld2luZC10Y3BtLWFjY3VyYXRlLWVjbi0w NA0KPiANCj4gQWJzdHJhY3Q6DQo+ICAgICBFeHBsaWNpdCBDb25nZXN0aW9uIE5vdGlmaWNhdGlv biAoRUNOKSBpcyBhIG1lY2hhbmlzbSB3aGVyZSBuZXR3b3JrDQo+ICAgICBub2RlcyBjYW4gbWFy ayBJUCBwYWNrZXRzIGluc3RlYWQgb2YgZHJvcHBpbmcgdGhlbSB0byBpbmRpY2F0ZQ0KPiAgICAg aW5jaXBpZW50IGNvbmdlc3Rpb24gdG8gdGhlIGVuZC1wb2ludHMuICBSZWNlaXZlcnMgd2l0aCBh biBFQ04tDQo+ICAgICBjYXBhYmxlIHRyYW5zcG9ydCBwcm90b2NvbCBmZWVkIGJhY2sgdGhpcyBp bmZvcm1hdGlvbiB0byB0aGUNCj4gc2VuZGVyLg0KPiAgICAgRUNOIGlzIHNwZWNpZmllZCBmb3Ig VENQIGluIHN1Y2ggYSB3YXkgdGhhdCBvbmx5IG9uZSBmZWVkYmFjaw0KPiBzaWduYWwNCj4gICAg IGNhbiBiZSB0cmFuc21pdHRlZCBwZXIgUm91bmQtVHJpcCBUaW1lIChSVFQpLiAgUmVjZW50bHks IG5ldyBUQ1ANCj4gICAgIG1lY2hhbmlzbXMgbGlrZSBDb25nZXN0aW9uIEV4cG9zdXJlIChDb25F eCkgb3IgRGF0YSBDZW50ZXIgVENQDQo+ICAgICAoRENUQ1ApIG5lZWQgbW9yZSBhY2N1cmF0ZSBF Q04gZmVlZGJhY2sgaW5mb3JtYXRpb24gd2hlbmV2ZXIgbW9yZQ0KPiAgICAgdGhhbiBvbmUgbWFy a2luZyBpcyByZWNlaXZlZCBpbiBvbmUgUlRULiAgVGhpcyBkb2N1bWVudCBzcGVjaWZpZXMNCj4g YW4NCj4gICAgIGV4cGVyaW1lbnRhbCBzY2hlbWUgdG8gcHJvdmlkZSBtb3JlIHRoYW4gb25lIGZl ZWRiYWNrIHNpZ25hbCBwZXINCj4gUlRUDQo+ICAgICBpbiB0aGUgVENQIGhlYWRlci4gIEdpdmVu IFRDUCBoZWFkZXIgc3BhY2UgaXMgc2NhcmNlLCBpdCBvdmVybG9hZHMNCj4gICAgIHRoZSB0aHJl ZSBleGlzdGluZyBFQ04tcmVsYXRlZCBmbGFncyBpbiB0aGUgVENQIGhlYWRlciBhbmQgcHJvdmlk ZXMNCj4gICAgIGFkZGl0aW9uYWwgaW5mb3JtYXRpb24gaW4gYSBuZXcgVENQIG9wdGlvbi4NCj4g DQo+IA0KPiANCj4gDQo+IFBsZWFzZSBub3RlIHRoYXQgaXQgbWF5IHRha2UgYSBjb3VwbGUgb2Yg bWludXRlcyBmcm9tIHRoZSB0aW1lIG9mDQo+IHN1Ym1pc3Npb24NCj4gdW50aWwgdGhlIGh0bWxp emVkIHZlcnNpb24gYW5kIGRpZmYgYXJlIGF2YWlsYWJsZSBhdCB0b29scy5pZXRmLm9yZy4NCj4g DQo+IFRoZSBJRVRGIFNlY3JldGFyaWF0DQo+IA0KPiANCj4gX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gdGNwbSBtYWlsaW5nIGxpc3QNCj4gdGNwbUBp ZXRmLm9yZw0KPiBodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG0NCg== From nobody Wed Oct 14 10:46:49 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E774A1AD09B; Wed, 14 Oct 2015 10:46:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.01 X-Spam-Level: X-Spam-Status: No, score=-6.01 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_21=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tQ6bg1iFfWb2; Wed, 14 Oct 2015 10:46:32 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 821B71AD087; Wed, 14 Oct 2015 10:46:32 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 9D00EF21E4C48; Wed, 14 Oct 2015 17:46:27 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t9EHkUCO009227 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 14 Oct 2015 19:46:30 +0200 Received: from FR711WXCHMBA05.zeu.alcatel-lucent.com ([169.254.1.213]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Wed, 14 Oct 2015 19:46:30 +0200 From: "De Schepper, Koen (Koen)" To: "tcpPrague@ietf.org" , =?utf-8?B?TWlyamEgS8O8aGxld2luZA==?= , "tcpm@ietf.org Extensions" , Bob Briscoe , Richard Scheffenegger Thread-Topic: AccECN fallback to classic ECN, was: RE: [tcpm] Fwd: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt Thread-Index: AQHQ6xecH9RU3iyM/EmwHF1qn1drPp5rUo5ggAAlR7A= Date: Wed, 14 Oct 2015 17:46:30 +0000 Message-ID: References: <20150906231947.7501.54950.idtracker@ietfa.amsl.com> <55F055AD.3050809@tik.ee.ethz.ch> Accept-Language: nl-BE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.39] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Subject: [tcpm] AccECN fallback to classic ECN, was: RE: Fwd: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 17:46:40 -0000 QW4gYWRkaXRpb25hbCB0aG91Z2h0LCBhbHNvIHVzZWZ1bCBmb3IgdGhlIFRDUC1QcmFndWUgbGlz dDoNCg0KLSBJZiB0aGUgZ29hbCBpcyB0byByZXBsYWNlIGNsYXNzaWMgRUNOIGVjaG9pbmcgb24g dGhlIEludGVybmV0LCBpc24ndCBpdCBiZXR0ZXIgTk9UIHRvIGZhbGwgYmFjayB0byBjbGFzc2lj IEVDTiwgYnV0IGJldHRlciAoYXMgYSBNVVNUPykgYWx3YXlzIHRvIGRyb3A/IEl0IHdpbGwgYmUg YW4gaW5jZW50aXZlIHRvIHVwZ3JhZGUgb24gdGhlIEludGVybmV0IChhbmQgYXZvaWQgY2xhc3Np YyBFQ04gZWNob2luZyBiZWhhdmlvcikuIEluIHRoZSBkYXRhY2VudGVyLCBpdCBzaG91bGQgcHJv YmFibHkgZmFsbCBiYWNrIGFzc3VtaW5nIHRoZXJlIGlzIGEgbGFyZ2UgZGVwbG95bWVudCBiYXNl IG9mICdjbGFzc2ljIEVDTiBlY2hvaW5nJyBEQ1RDUHMuDQoNClJlZ2FyZHMsDQpLb2VuLg0KDQoN Cj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogRGUgU2NoZXBwZXIsIEtvZW4g KEtvZW4pDQo+IFNlbnQ6IHdvZW5zZGFnIDE0IG9rdG9iZXIgMjAxNSAxOTozNw0KPiBUbzogJ01p cmphIEvDvGhsZXdpbmQnOyB0Y3BtQGlldGYub3JnIEV4dGVuc2lvbnM7IEJvYiBCcmlzY29lOyBS aWNoYXJkDQo+IFNjaGVmZmVuZWdnZXINCj4gU3ViamVjdDogUkU6IFt0Y3BtXSBGd2Q6IE5ldyBW ZXJzaW9uIE5vdGlmaWNhdGlvbiBmb3IgZHJhZnQta3VlaGxld2luZC0NCj4gdGNwbS1hY2N1cmF0 ZS1lY24tMDQudHh0DQo+IA0KPiBIaSwNCj4gDQo+IEp1c3QgZmluaXNoZWQgcmVhZGluZy9yZXZp ZXdpbmcgdGhpcyBkcmFmdC4gSSB0aGluayBpdCBpcyByZWFsbHkgd2VsbA0KPiB3cml0dGVuLCBz aW1wbGUgYW5kIGl0IGhhcyBnb25lIGNsZWFybHkgdGhyb3VnaCBhIGxvdCBvZiB0aG91Z2h0cw0K PiBhbHJlYWR5Lg0KPiANCj4gQmVsb3cgc29tZSBxdWVzdGlvbnMgYW5kIHJlbWFya3M6DQo+IA0K PiAtIElzIGhvdyB0aGlzIHByb3RvY29sIGlzIGludGVuZGVkIHRvIGJlIHVzZWQgYnkgY2xhc3Np Yy1FQ04tVENQIGFuZA0KPiBEQ1RDUCBkb2N1bWVudGVkIHNvbWV3aGVyZT8gSSB0aGluayBpdCB1 c2VmdWwgdG8gZGVzY3JpYmUgKG5vdCBzdXJlIGlmDQo+IHlvdSB3YW50IHRvIGFkZCBpdCB0byB0 aGlzIGRvYz8pDQo+IA0KPiAtIFdoeSB0aGUgcXVpdGUgc3Ryb25nIHJlc3RyaWN0aW9ucyBpbiB3 aGF0IG9mZmxvYWQgZW5naW5lcyBhcmUgYWxsb3dlZA0KPiB0byBkby4gSXMgaXQgdXNlZnVsIHRv IGNsZWFybHkgc3BlY2lmeSB3aGF0IGFuIG9mZmxvYWQgZW5naW5lIG11c3QgZG8/DQo+IEZvciBl eGFtcGxlIEkgY2FuIGltYWdpbmUgaXMgYSBsYXJnZSBsaXN0IG9mIHBhY2tldHMgYXJlIHJlY2Vp dmVkIGluIGENCj4gTklDLCB0aGUgZW5naW5lIGNhbiBjb21iaW5lIGFsbCBhY2tzIGFuZCBkYXRh IHBhY2tldHMgYXMgbG9uZyBhcyBvbmx5DQo+IG9uZSBjb3VudGVyIGlzIGluY3JlbWVudGVkLCBh bmQgb25seSByZXBvcnQgdGhlIGxhc3QgdmFsdWUgb2YgdGhlDQo+IGNvdW50ZXIgaW4gdGhlIG9w dGlvbi4gQWxzbyBhcyBsb25nIGFzIGl0IGNhbiBndWFyYW50ZWUgdGhhdCB0aGUgQUNFDQo+IGNv dW50ZXIgaXMgbm90IGludGVycHJldGVkIGFzIGJlaW5nIHdyYXBwZWQsIGl0IGFsc28gY2FuIG9m ZmxvYWQgYnkNCj4gY29tYmluaW5nIHVwIHRvIDcgcGFja2V0cyBvciBhY2tzLi4uDQo+IA0KPiAt IElmIEw0UyAoRENUQ1AvVENQLVByYWd1ZSkgc2hvdWxkIHVzZSBFQ1QxLCB0aGVuIGl0IG5lZWRz IGFsd2F5cyAxMQ0KPiBhbmQgc29tZXRpbWVzIDggYnl0ZXMgb3B0aW9ucy4uLiBXaHkgbm90IHB1 dHRpbmcgRUNFQiBmaXJzdCwgYW5kIGFsbG93DQo+IGEgbW9yZSByZWxheGVkIHJlcXVpcmVtZW50 IG9uIHNpZ25hbGluZyBFRTBCIGFuZCBFRTFCPyBJZiBvcHRpb24gc3BhY2UNCj4gaXMgbGltaXRl ZCwgYXQgbGVhc3QgRUNFQiB3aWxsIGJlIGFibGUgdG8gZ2V0IHRocm91Z2ggZW5vdWdoIHRvDQo+ IGVzdGltYXRlIHRoZSBjb25nZXN0aW9uLg0KPiANCj4gLSBJZiBJIGFtIGludGVyZXN0ZWQgaW4g dGhlIHByb2JhYmlsaXR5IG9mIG1hcmtzLCBqdXN0IGVjaG9pbmcgdGhlIGxhc3QNCj4gcmVjZWl2 ZWQgSVAgbWFyayB3b3VsZCBwZXJmZWN0bHkgZml0IHRoaXMgcmVxdWlyZW1lbnQgKGFzc3VtaW5n IHJhbmRvbQ0KPiBtYXJraW5nKS4gQ3VycmVudGx5IHRoaXMgaXMgbm90IHN1cHBvcnRlZCwgYW5k IHRoaW5raW5nIGFib3V0IHRoaXMsDQo+IHlvdXIgJ3NhZmUnIGFsZ29yaXRobXMgd2lsbCBnaXZl IHBvdGVudGlhbGx5IHF1aXRlIGRldmlhdGluZyByZXN1bHRzLA0KPiB0YWtpbmcgYWxsIHBvc3Np YmxlIEFDSyBsb3NzZXMvbWlzc2VzIGludG8gYWNjb3VudC4gQXJlIHRoZXNlIGZ1dHVyZS0NCj4g dXNlIHJlcXVpcmVtZW50cyB3b3J0aCB0aGUgb3ZlcmhlYWQvY29tcGxleGl0eT8NCj4gDQo+IC0g TDRTIChEQ1RDUC9QcmFndWUvLi4uKSBjYW4gaGF2ZSBvZnRlbiAocmFuZG9tKSBtYXJraW5nIHBy b2JhYmlsaXRpZXMNCj4gYXJvdW5kIDUwJSwgY2F1c2luZyB0aGUgY2hhbmdlIHRyaWdnZXJlZCBh Y2tzIHRvIGFjayBhbG1vc3QgZXZlcnkNCj4gcGFja2V0ICh0aGlzIGlzIHdoYXQgdGhlIGN1cnJl bnQgRENUQ1AgaXMgZG9pbmcgdG9vLCBidXQgb24gYSBzdGVwLVJFRA0KPiBBUU0geW91IGdldCBs YXJnZSBvbi1vZmYgc2VyaWVzLCB3aXRoIER1YWxRIHlvdSBnZXQgYSByYW5kb20NCj4gZGlzdHJp YnV0aW9uIHdpdGggbm9uZSBvciBzbWFsbCBzZXJpZXMpLiBJcyBvcmRlciBzbyBpbXBvcnRhbnQ/ DQo+IE90aGVyd2lzZSwgdGhlIGFja3MgY291bGQganVzdCBjYXJyeSBFRTBCIChvciBldmVuIGlt bWVkaWF0ZWx5IEVDRUIgYXMNCj4gYWJvdmUpLCB0aGUgc2VuZGVyIGtub3dzIGhvdyBtdWNoIGJ5 dGVzIGl0IGhhcyBzZW50LCBhc3N1bWluZyBhbGwgdGhlDQo+IHJlc3QgaXMgRUNFQiAoRUUwQi9F RTFCIGFzIGFwcHJvcHJpYXRlKS4NCj4gDQo+IFNvbWUgdHlwbydzIGFuZCBjb25mdXNpbmcgdGV4 dDoNCj4gDQo+IC0gVGhlIHRlcm0gY2xhc3NpYyBFQ04gY29uZnVzZWQgbWUgYSBmZXcgdGltZXMu IEkgc3VwcG9zZSB5b3UgbWVhbg0KPiBjbGFzc2ljIEVDTiBlY2hvaW5nIHByb3RvY29sLCBub3Qg dGhlIHJlc3BvbnNlIHRvIGNvbmdlc3Rpb24gaW4gdGhlIFRDUA0KPiBzZW5kZXIuIE1heWJlIGNv bnNpc3RlbnQgdXNlIG9mICdjbGFzc2ljIEVDTiBlY2hvaW5nJyBpcyBjbGVhcmVyLg0KPiANCj4g LSBUaGUgZHJhZnQgaXMgc3RydWN0dXJlZCB2ZXJ5IGdvb2QsIGFzIEkgY291bGQgcmVhZCBzZXF1 ZW5jaWFsbHkNCj4gdGhvdWdoIGl0LCBleGNlcHQgZm9yIG9uZSBjYXNlIHdoZXJlIEkgaGFkIHRv IGxvb2sgZnVydGhlciB0bw0KPiB1bmRlcnN0YW5kIHdoYXQgd2FzIGV4YWN0bHkgbWVhbnQ6IFRo ZSB0aGlyZCBwYXJhZ3JhcGggb2Ygc2VjdGlvbiAzLjENCj4gd2FzICIuLi4gVENQIHNlcnZlciBN VVNUIHNldCBOUz0xIG9uIHRoZSBTWU4vQUNLLiIuIENoYW5naW5nIGl0IHRvICIuLi4NCj4gVENQ IHNlcnZlciBNVVNUIHNldCBhZGRpdGlvbmFsbHkgTlM9MSBvbiB0aGUgU1lOL0FDSy4iLCBhbmQg b3B0aW9uYWxseQ0KPiBhZGQgdGhhdCB0aGlzIGlzIG9ubHkgd2hlbiBpbiBhY2NFQ04gbW9kZSwg d291bGQgaGF2ZSBjbGFyaWZpZWQgYWxsDQo+IGRvdWJ0cy4NCj4gDQo+IC0gVGhlIHNpeHRoIHBh cmFncmFwaCBvZiBzZWN0aW9uIDMuMSBjb250YWlucyBhbiBJZiAod2l0aCBjYXBpdGFsIEkpIGlu DQo+IHRoZSBtaWRkbGUgb2YgdGhlIGZpc3Qgc2VudGVuY2UuDQo+IA0KPiAtIFNlY3Rpb24gMy4y LjEgaGFzICJ0aHkiIGluc3RlYWQgb2YgInRoZXkiLg0KPiANCj4gLSBTZWN0aW9uIDMuMi4yIGhh cyAiKG4pIiBpbnN0ZWFkIG9mICJuIiwgbGF0ZXIgaXQgYWxzbyBhcHBlYXJzIGFzICJuIg0KPiB3 aXRob3V0IHRoZSBicmFja2V0cy4NCj4gDQo+IC0gU2VjdGlvbiA1IGhhcyB0cmlpZ2dlcmVkIGlu c3RlYWQgb2YgdHJpZ2dlcmVkDQo+IA0KPiANCj4gUmVnYXJkcywNCj4gS29lbi4NCj4gDQo+IA0K PiANCj4gDQo+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiBGcm9tOiB0Y3BtIFtt YWlsdG86dGNwbS1ib3VuY2VzQGlldGYub3JnXSBPbiBCZWhhbGYgT2YgTWlyamENCj4gS8O8aGxl d2luZA0KPiA+IFNlbnQ6IHdvZW5zZGFnIDkgc2VwdGVtYmVyIDIwMTUgMTc6NTINCj4gPiBUbzog dGNwbUBpZXRmLm9yZyBFeHRlbnNpb25zOyBCb2IgQnJpc2NvZTsgUmljaGFyZCBTY2hlZmZlbmVn Z2VyDQo+ID4gU3ViamVjdDogW3RjcG1dIEZ3ZDogTmV3IFZlcnNpb24gTm90aWZpY2F0aW9uIGZv ciBkcmFmdC1rdWVobGV3aW5kLQ0KPiA+IHRjcG0tYWNjdXJhdGUtZWNuLTA0LnR4dA0KPiA+DQo+ ID4gSGkgYWxsLA0KPiA+DQo+ID4gd2Ugc3VibWl0dGVkIGEgbmV3IGRyYWZ0IGZvciBBY2NFQ04g KHNlZSBiZWxvdykuIFdlIHNpZ25pZmljYW50bHkNCj4gPiBzaW1wbGlmaWVkIHRoZQ0KPiA+IGRy YWZ0IGJlY2F1c2Ugd2UgYmVsaWV2ZSBpdCBpcyBtb3JlIGltcG9ydGFudCB0byBoYXZlIGEgc29s dXRpb24gdGhhdA0KPiA+IGlzIGVhc3kgdG8NCj4gPiB1bmRlcnN0YW5kIGFuZCB0aGVyZWZvcmUg bW9yZSBlYXN5IHRvIGNvcnJlY3RseSBpbXBsZW1lbnQsIHRoYW4NCj4gaGF2aW5nDQo+ID4gYQ0K PiA+IHNvbHV0aW9uIHRoYXQgdHJpZXMgdG8gZnVsZmlsbCBhbGwgcmVxdWlyZW1lbnRzIGJ1dCBp cyBvdmVybHkNCj4gY29tcGxleC4NCj4gPg0KPiA+IEluIHNob3J0LCB3ZSBub3cgb25seSBvdmVy d3JpdGUgdGhlIDMgVENQIGhlYWRlciBiaXRzIChubyBhZGRpdGlvbmFsDQo+ID4gYml0cyBpbiB0 aGUNCj4gPiBUQ1AgaGVhZGVyIGFyZSB1c2VkKSBhbmQgdXNlIHRoZW0gc2ltcGx5IGZvciBhIENF IHBhY2tldCBjb3VudGVyLA0KPiB3aGlsZQ0KPiA+IGJ5dGVzLXdpc2UgaW5mb3JtYXRpb24gb24g YWxsIG90aGVyIG1hcmtpbmdzIGlzIHByb3ZpZGVkIGJ5IGEgbmV3IFRDUA0KPiA+IG9wdGlvbi4g SW4NCj4gPiBjYXNlIHRoZSBvcHRpb24gaXMgbm90IGF2YWlsYWJsZSwgZS5nLiBiZWNhdXNlIGl0 J3MgYmxvY2tlZCwgdGhpcw0KPiB3aWxsDQo+ID4gc3RpbGwNCj4gPiBwcm92aWRlIHRoZSBuZWVk ZWQgaW5mb3JtYXRpb24gdG8gcmVhY3QgdG8gYSBDRS1iYXNlZCBjb25nZXN0aW9uDQo+ID4gZmVl ZGJhY2suDQo+ID4gSG93ZXZlciwgYW55IGFkdmFuY2VkIG1lY2hhbmlzbSB0aGF0IG5lZWRzIGZ1 cnRoZXIgaW5mb3JtYXRpb24gb24gdGhlDQo+ID4gb3RoZXINCj4gPiBtYXJraW5ncyByZWNlaXZl ZCB3aWxsIG9ubHkgd29yayBpZiB0aGUgb3B0aW9uIGlzIGF2YWlsYWJsZS4NCj4gPg0KPiA+IEZ1 cnRoZXIsIHdlIGFsc28gdHJpZWQgdG8ga2VlcCB0aGUgZHJhZnQgYXMgc2hvcnQgYXMgcG9zc2li bGUuIFRoYXQNCj4gPiBtZWFucyB0aGUNCj4gPiBhY3R1YWwgbm9ybWF0aXZlIHBhcnQgb25seSBo YXMgOSBwYWdlcy4gSG93ZXZlciwgd2Ugc3RpbGwgcHJvdmlkZSBhbg0KPiA+IG92ZXJ2aWV3DQo+ ID4gc2VjdGlvbiB3aXRoIHNvbWUgcmVhc29uaW5nIGFzIHdlbGwgYXMgc29tZSBkaXNjdXNzaW9u IG9uIGludGVyYWN0aW9uDQo+ID4gd2l0aCBvdGhlcg0KPiA+IG1lY2hhbmlzbXMgd2hpY2ggbGVh ZHMgaW4gdG90YWwgdG8gMjUgcGFnZXMgKHdpdGhvdXQgYXBwZW5kaXgpLg0KPiA+DQo+ID4gSSdt IGN1cnJlbnRseSBhbHNvIHdvcmtpbmcgb24gYW4gaW1wbGVtZW50YXRpb24gb2YgdGhpcyBwcm9w b3NlZA0KPiA+IHNvbHV0aW9uLiBJDQo+ID4gd2lsbCBhbm5vdW5jZSBpdCBhcyBzb29uIGFzIEkn bSByZWFkeSENCj4gPg0KPiA+IEluIGFueSBjYXNlIHdlIHdvdWxkIGxpa2UgdG8gZGlzY3VzcyB0 aGlzIGF0IHRoZSBuZXh0IG1lZXRpbmcuIFdlLA0KPiB0aGUNCj4gPiBhdXRob3IsDQo+ID4gdGhp bmsgdGhhdCB0aGlzIHByb3Bvc2FsIGlzIG5vdyB0aGUgcmlnaHQgd2F5IGZvcndhcmQgYW5kIGhv cGUgdGhhdA0KPiA+IHRoaXMNCj4gPiBzaW1wbGlmaWVkIHByb3Bvc2FsIHdpbGwgYWxsb3dzIHVz IHRvIHF1aWNrbHkgcHJvY2VlZCBvbiBBY2NFQ04gYXMNCj4gdGhlDQo+ID4gbmVlZCBmb3INCj4g PiBpdCBpcyBpbmNyZWFzaW5nLg0KPiA+DQo+ID4gUGxlYXNlIGxldCB1cyBrbm93IGlmIHlvdSBo YXZlIGFueSBmZWVkYmFjayBvbiB0aGUgZHJhZnQhDQo+ID4NCj4gPiBNaXJqYQ0KPiA+DQo+ID4N Cj4gPiAtLS0tLS0tLSBGb3J3YXJkZWQgTWVzc2FnZSAtLS0tLS0tLQ0KPiA+IFN1YmplY3Q6IE5l dyBWZXJzaW9uIE5vdGlmaWNhdGlvbiBmb3IgZHJhZnQta3VlaGxld2luZC10Y3BtLWFjY3VyYXRl LQ0KPiA+IGVjbi0wNC50eHQNCj4gPiBEYXRlOiBTdW4sIDA2IFNlcCAyMDE1IDE2OjE5OjQ3IC0w NzAwDQo+ID4gRnJvbTogaW50ZXJuZXQtZHJhZnRzQGlldGYub3JnDQo+ID4gVG86IFJpY2hhcmQg U2NoZWZmZW5lZ2dlciA8cnNAbmV0YXBwLmNvbT4sICJNaXJqYSBLw7xobGV3aW5kIg0KPiA+IDxt aXJqYS5rdWVobGV3aW5kQHRpay5lZS5ldGh6LmNoPiwgTWlyamEgS3VlaGxld2luZA0KPiA+IDxt aXJqYS5rdWVobGV3aW5kQHRpay5lZS5ldGh6LmNoPiwgUmljaGFyZCBTY2hlZmZlbmVnZ2VyDQo+ ID4gPHJzQG5ldGFwcC5jb20+LCBCb2INCj4gPiBCcmlzY29lIDxpZXRmQGJvYmJyaXNjb2UubmV0 PiwgQm9iIEJyaXNjb2UgPGlldGZAYm9iYnJpc2NvZS5uZXQ+DQo+ID4NCj4gPg0KPiA+IEEgbmV3 IHZlcnNpb24gb2YgSS1ELCBkcmFmdC1rdWVobGV3aW5kLXRjcG0tYWNjdXJhdGUtZWNuLTA0LnR4 dA0KPiA+IGhhcyBiZWVuIHN1Y2Nlc3NmdWxseSBzdWJtaXR0ZWQgYnkgQm9iIEJyaXNjb2UgYW5k IHBvc3RlZCB0byB0aGUNCj4gPiBJRVRGIHJlcG9zaXRvcnkuDQo+ID4NCj4gPiBOYW1lOgkJZHJh ZnQta3VlaGxld2luZC10Y3BtLWFjY3VyYXRlLWVjbg0KPiA+IFJldmlzaW9uOgkwNA0KPiA+IFRp dGxlOgkJTW9yZSBBY2N1cmF0ZSBFQ04gRmVlZGJhY2sgaW4gVENQDQo+ID4gRG9jdW1lbnQgZGF0 ZToJMjAxNS0wOS0wNg0KPiA+IEdyb3VwOgkJSW5kaXZpZHVhbCBTdWJtaXNzaW9uDQo+ID4gUGFn ZXM6CQkzNg0KPiA+IFVSTDoNCj4gPiBodHRwczovL3d3dy5pZXRmLm9yZy9pbnRlcm5ldC1kcmFm dHMvZHJhZnQta3VlaGxld2luZC10Y3BtLWFjY3VyYXRlLQ0KPiA+IGVjbi0wNC50eHQNCj4gPiBT dGF0dXM6ICAgICAgICAgaHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9kb2MvZHJhZnQta3Vl aGxld2luZC0NCj4gdGNwbS0NCj4gPiBhY2N1cmF0ZS1lY24vDQo+ID4gSHRtbGl6ZWQ6ICAgICAg IGh0dHBzOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1rdWVobGV3aW5kLXRjcG0tDQo+ID4g YWNjdXJhdGUtZWNuLTA0DQo+ID4gRGlmZjoNCj4gPiBodHRwczovL3d3dy5pZXRmLm9yZy9yZmNk aWZmP3VybDI9ZHJhZnQta3VlaGxld2luZC10Y3BtLWFjY3VyYXRlLWVjbi0NCj4gMDQNCj4gPg0K PiA+IEFic3RyYWN0Og0KPiA+ICAgICBFeHBsaWNpdCBDb25nZXN0aW9uIE5vdGlmaWNhdGlvbiAo RUNOKSBpcyBhIG1lY2hhbmlzbSB3aGVyZQ0KPiBuZXR3b3JrDQo+ID4gICAgIG5vZGVzIGNhbiBt YXJrIElQIHBhY2tldHMgaW5zdGVhZCBvZiBkcm9wcGluZyB0aGVtIHRvIGluZGljYXRlDQo+ID4g ICAgIGluY2lwaWVudCBjb25nZXN0aW9uIHRvIHRoZSBlbmQtcG9pbnRzLiAgUmVjZWl2ZXJzIHdp dGggYW4gRUNOLQ0KPiA+ICAgICBjYXBhYmxlIHRyYW5zcG9ydCBwcm90b2NvbCBmZWVkIGJhY2sg dGhpcyBpbmZvcm1hdGlvbiB0byB0aGUNCj4gPiBzZW5kZXIuDQo+ID4gICAgIEVDTiBpcyBzcGVj aWZpZWQgZm9yIFRDUCBpbiBzdWNoIGEgd2F5IHRoYXQgb25seSBvbmUgZmVlZGJhY2sNCj4gPiBz aWduYWwNCj4gPiAgICAgY2FuIGJlIHRyYW5zbWl0dGVkIHBlciBSb3VuZC1UcmlwIFRpbWUgKFJU VCkuICBSZWNlbnRseSwgbmV3IFRDUA0KPiA+ICAgICBtZWNoYW5pc21zIGxpa2UgQ29uZ2VzdGlv biBFeHBvc3VyZSAoQ29uRXgpIG9yIERhdGEgQ2VudGVyIFRDUA0KPiA+ICAgICAoRENUQ1ApIG5l ZWQgbW9yZSBhY2N1cmF0ZSBFQ04gZmVlZGJhY2sgaW5mb3JtYXRpb24gd2hlbmV2ZXIgbW9yZQ0K PiA+ICAgICB0aGFuIG9uZSBtYXJraW5nIGlzIHJlY2VpdmVkIGluIG9uZSBSVFQuICBUaGlzIGRv Y3VtZW50IHNwZWNpZmllcw0KPiA+IGFuDQo+ID4gICAgIGV4cGVyaW1lbnRhbCBzY2hlbWUgdG8g cHJvdmlkZSBtb3JlIHRoYW4gb25lIGZlZWRiYWNrIHNpZ25hbCBwZXINCj4gPiBSVFQNCj4gPiAg ICAgaW4gdGhlIFRDUCBoZWFkZXIuICBHaXZlbiBUQ1AgaGVhZGVyIHNwYWNlIGlzIHNjYXJjZSwg aXQNCj4gb3ZlcmxvYWRzDQo+ID4gICAgIHRoZSB0aHJlZSBleGlzdGluZyBFQ04tcmVsYXRlZCBm bGFncyBpbiB0aGUgVENQIGhlYWRlciBhbmQNCj4gcHJvdmlkZXMNCj4gPiAgICAgYWRkaXRpb25h bCBpbmZvcm1hdGlvbiBpbiBhIG5ldyBUQ1Agb3B0aW9uLg0KPiA+DQo+ID4NCj4gPg0KPiA+DQo+ ID4gUGxlYXNlIG5vdGUgdGhhdCBpdCBtYXkgdGFrZSBhIGNvdXBsZSBvZiBtaW51dGVzIGZyb20g dGhlIHRpbWUgb2YNCj4gPiBzdWJtaXNzaW9uDQo+ID4gdW50aWwgdGhlIGh0bWxpemVkIHZlcnNp b24gYW5kIGRpZmYgYXJlIGF2YWlsYWJsZSBhdCB0b29scy5pZXRmLm9yZy4NCj4gPg0KPiA+IFRo ZSBJRVRGIFNlY3JldGFyaWF0DQo+ID4NCj4gPg0KPiA+IF9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fDQo+ID4gdGNwbSBtYWlsaW5nIGxpc3QNCj4gPiB0Y3Bt QGlldGYub3JnDQo+ID4gaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby90Y3Bt DQo= From nobody Wed Oct 14 10:58:50 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1C9921AD0C9; Wed, 14 Oct 2015 10:58:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Tfo_GBgxoQWw; Wed, 14 Oct 2015 10:58:45 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id A65361AD0C7; Wed, 14 Oct 2015 10:58:45 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit From: "Barry Leiba" To: "The IESG" X-Test-IDTracker: no X-IETF-IDTracker: 6.5.1 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20151014175844.19378.27174.idtracker@ietfa.amsl.com> Date: Wed, 14 Oct 2015 10:58:44 -0700 Archived-At: Cc: tcpm@ietf.org Subject: [tcpm] Barry Leiba's No Objection on draft-ietf-tcpm-rtorestart-08: (with COMMENT) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 17:58:48 -0000 Barry Leiba has entered the following ballot position for draft-ietf-tcpm-rtorestart-08: No Objection When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.) Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html for more information about IESG DISCUSS and COMMENT positions. The document, along with other ballot positions, can be found here: https://datatracker.ietf.org/doc/draft-ietf-tcpm-rtorestart/ ---------------------------------------------------------------------- COMMENT: ---------------------------------------------------------------------- A comment to Benoît's comment that quotes the opsdir review saying, 'I believe that it should be "provide" singular and not plural.' No, "provides" goes with "algorithm", and is correct as written. From nobody Wed Oct 14 11:03:12 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD8F01AD23D; Wed, 14 Oct 2015 11:03:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XGmUzz8SdmHi; Wed, 14 Oct 2015 11:03:08 -0700 (PDT) Received: from mail-qk0-x233.google.com (mail-qk0-x233.google.com [IPv6:2607:f8b0:400d:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2BDA81AD210; Wed, 14 Oct 2015 11:03:08 -0700 (PDT) Received: by qkas79 with SMTP id s79so26179464qka.0; Wed, 14 Oct 2015 11:03:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=P1LUCzBAsIzw99mm+MhS7RT/i8d/PcxGSi8irLObBgU=; b=eS6u+9t5cMfyIPIRmeQwPscqypjlPBWFQae4+dEhiQdgqP/3NaI20OAPrCsP5A3PDe rIgnqokAlfGLqjzIOZ02QBVcKsBtgYwA/P5cnMtW8XSjEd9Ysz34OxIUsnqy1Md5rt+D DqiRztZBJxdoIK4CUsT9WZjt/763Ykp1MOp3YXAKujfx5ot3+xaWkm2cbQBD3y7r6JSJ JYUCwxv6xnpHgNcQujSj1a/gNgZuDvpopPuuRrZU7NOzI/rBO4StVzXu+zwv501MKsXl WauhmsNx23osov9uv4bY+iNu4K9vV1RUGHsb1hGIar2qYA4Lnm59Prqv7fvFpjbxOYde 6pSQ== X-Received: by 10.55.48.148 with SMTP id w142mr5828460qkw.87.1444845787230; Wed, 14 Oct 2015 11:03:07 -0700 (PDT) Received: from [192.168.1.77] ([184.13.114.26]) by smtp.gmail.com with ESMTPSA id u89sm3804334qkl.27.2015.10.14.11.03.06 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 14 Oct 2015 11:03:06 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) From: Tim Wicinski X-Mailer: iPhone Mail (13A452) In-Reply-To: <20151014175844.19378.27174.idtracker@ietfa.amsl.com> Date: Wed, 14 Oct 2015 14:03:05 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <831DE7D2-E6F9-4F9F-8DA0-3369926DF516@gmail.com> References: <20151014175844.19378.27174.idtracker@ietfa.amsl.com> To: Barry Leiba Archived-At: Cc: tcpm@ietf.org, The IESG Subject: Re: [tcpm] Barry Leiba's No Objection on draft-ietf-tcpm-rtorestart-08: (with COMMENT) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 18:03:11 -0000 That was my comment on the use of plural and I was unsure so I threw it in. T= hanks for that Barry =46rom my high tech gadget > On Oct 14, 2015, at 13:58, Barry Leiba wrote: >=20 > Barry Leiba has entered the following ballot position for > draft-ietf-tcpm-rtorestart-08: No Objection >=20 > When responding, please keep the subject line intact and reply to all > email addresses included in the To and CC lines. (Feel free to cut this > introductory paragraph, however.) >=20 >=20 > Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html > for more information about IESG DISCUSS and COMMENT positions. >=20 >=20 > The document, along with other ballot positions, can be found here: > https://datatracker.ietf.org/doc/draft-ietf-tcpm-rtorestart/ >=20 >=20 >=20 > ---------------------------------------------------------------------- > COMMENT: > ---------------------------------------------------------------------- >=20 > A comment to Beno=C3=AEt's comment that quotes the opsdir review saying, '= I > believe that it should be "provide" singular and not plural.' No, > "provides" goes with "algorithm", and is correct as written. >=20 >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Wed Oct 14 11:34:43 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 586C21B29B4 for ; Wed, 14 Oct 2015 11:34:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.539 X-Spam-Level: X-Spam-Status: No, score=0.539 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, STOX_REPLY_TYPE=0.439] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K1hHhD8-AGyK for ; Wed, 14 Oct 2015 11:34:40 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5DD531B29B3 for ; Wed, 14 Oct 2015 11:34:40 -0700 (PDT) Received: from srichardlxp2 ([213.143.121.76]) by mail.gmx.com (mrgmx102) with ESMTPSA (Nemesis) id 0LqW8j-1aHO2q1Q93-00e0zo; Wed, 14 Oct 2015 20:34:33 +0200 Message-ID: <6B6197EBDA6D400AAB7C6E88D1F8AC71@srichardlxp2> From: "Richard Scheffenegger" To: , "Jeremy Harris" References: <5618005A.8070303@isi.edu> <70335.1444421059@lawyers.icir.org> <56183B49.4000506@isi.edu> <56183E93.1010308@isi.edu> <5618420E.9040609@isi.edu> <5618554F.3080103@isi.edu> <56185E44.9050702@isi.edu> <561891C3.90004@isi.edu> <5618AF0A.4010101@isi.edu> <5619F00A.2040009@isi.edu> <561AD47B.9030602@isi.edu> <561BE357.4090405@wi zmail.org> Date: Wed, 14 Oct 2015 20:30:18 +0200 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-Provags-ID: V03:K0:nEqWhcCtsY3yXfiBZrU46IlOI7P/Zw3Y7OGhhS/DP/RDWtK62s0 vnGiE581S3XOhthTD4wXhaPeWfpL9f/i+e49RpUCuQSySsE3QbJ9aUeUPIDfjlojW30bXQr qIUzOMc2/t6ad/ZDnPLjnXQz9vSfP0U7aJ5ZQHIVN5ZsN+A0e0Lx5n7J+Gr12+TTr78Yjoj 0JZt3eT9uflNuBPQxSyqA== X-UI-Out-Filterresults: notjunk:1;V01:K0:xy6Dc2eJrF4=:mOqAugDSwSYVzD7QJChm+y LDCqyS1iA9XI6ozbM4tax3fkwr6tpcDy1w0dVe0JHCjUPOZD4FuJPXpp3EfCJdpNeleID5Elp Ny6VJlaQJfpEdqIl2QobSjZ9b2U88kTYC8EKy+ucWdqfJVQxmP+Ubowz/8nY+rDsN0V0XT3N0 lqyc+zulXj0BmgFjzM9ikBAjAjkh9q3Uz3W1Q4kHIXNRXP1d2EPPu1ewEVxzIdXhfIRHKsSYd lj2cX1cmPTQDGUfG0A01RssHsQOoHsZ2oHBw8eToCAB/4n60vJabiSgsApvO673tfA3sORqYW 1AVhLo/G80xuPR7DvGix5oZ0bsut1JdGlgoiAnLkP48ar6RJpJLZpNwscCOWlnFemgiD7VwCj DnzUWGr8fSVePOpq4mC0H9v2s/1mpxV21eUgiXiB8PXzx95Zg46KgKR+M1AcOE3hnx5uiYf5I 1z6Z6H02pntZCEJHTnUfWwLCsRxDj6jZcaURXJZ7Rp4MSARqPV/Lo1nsCV593UsnZeeDngZO0 VJzpdoKNP2tCBzua3489KTOLp/mQnR9X3Lli0soIrVjcNNt1zFTyD6KB4jEfpe7DgqERKmheF ssNl9SBdmsXQL7sN133hdWWV2hEvmPe+l0isIrl+a+DNXZxOrEeggM/LwMjseG5JavrS5AU2l LJqABIzjGqHcBy/D7uhH42/WlE0fgLNG6ewtT3jifqi0WS723I6/K5Jlk+UA7aaAnk5Fr4NOk KvYQAfOSTO5Ti50ZpSppYTJWP+VYxsKXG+le+o0mwgZoWp+2f+ISy0/glToqTlDwW3Se5ydjN EnwhrC7 Archived-At: Subject: Re: [tcpm] [aqm] TCP ACK Suppression X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 18:34:42 -0000 Well, for the Timestamps case, getting all of them (plus the typically broken EDWA factors) will tend to increase your RTTvar, and thus the RTO. I doubt that it would be smart to keep the TSecr of the oldest, removed ACK and replace the TSecr value of the ACK that is to be transmitted with that old value though... OTOH, TSval often has a granularity of only 10ms (or higher), so chances are that multiple ACKs do actually contain the very same TSval/TSecr values before the thinning. In general, I would think any ACK suppression should take all options into consideration, and only iff they all are equal, earlier ACKs may be discarded while in the queue... Best regards, Richard ----- Original Message ----- From: "Jeremy Harris" To: Sent: Monday, October 12, 2015 6:44 PM Subject: Re: [tcpm] [aqm] TCP ACK Suppression > On 11/10/15 23:49, David Lang wrote: >> packet loss is still pretty rare, > > Not in my day-job it isn't. > >> The ACK packets are already collapsed >> time-wise. Whatever timing they had when they were generated, the fact >> that they are all sitting in the same queue at the same time means taht >> that timing has been lost, > > Unless they carried timestamps? What are you doing with TCP options > (and ECN marks)? > -- > Cheers, > Jeremy > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Wed Oct 14 22:08:33 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B7A0A1B305A; Wed, 14 Oct 2015 22:08:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=unavailable Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t28IPwVp0y5X; Wed, 14 Oct 2015 22:08:30 -0700 (PDT) Received: from sesbmg22.ericsson.net (sesbmg22.ericsson.net [193.180.251.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E621F1B2B57; Wed, 14 Oct 2015 22:08:29 -0700 (PDT) X-AuditID: c1b4fb30-f79626d000006adf-7d-561f34cb751a Received: from ESESSHC001.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg22.ericsson.net (Symantec Mail Security) with SMTP id 05.A6.27359.BC43F165; Thu, 15 Oct 2015 07:08:27 +0200 (CEST) Received: from ESESSMB205.ericsson.se ([169.254.5.98]) by ESESSHC001.ericsson.se ([153.88.183.21]) with mapi id 14.03.0248.002; Thu, 15 Oct 2015 07:08:27 +0200 From: Ingemar Johansson S To: "iccrg@irtf.org" , "tcpm@ietf.org" , "tsvwg@ietf.org" Thread-Topic: New Version Notification for draft-johansson-cc-for-4g-5g-00.txt Thread-Index: AQHRBqyIURyzaqIT1UWeXHewI01EFJ5rS8UA Date: Thu, 15 Oct 2015 05:08:26 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA34BF4F0F@ESESSMB205.ericsson.se> References: <20151014181702.10618.83714.idtracker@ietfa.amsl.com> In-Reply-To: <20151014181702.10618.83714.idtracker@ietfa.amsl.com> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.150] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrCLMWRmVeSWpSXmKPExsUyM+Jvje5pE/kwg98X1C0OLNjJbrHt5Hwm i2Nv7rI5MHssWfKTyWPyxsNsAUxRXDYpqTmZZalF+nYJXBlzL3ezFFwSqtj25jVbA+MUoS5G Tg4JAROJAytuMELYYhIX7q1n62Lk4hASOMoocXVhA5SzmFHi5MNVbCBVbAI2EisPfQfrEBHI lpj66SFYXFjAV+Lq/pusEPEAiSOLrzJB2EYS174/BqthEVCV2PFhMpDNwcELVN/3xwgkLCTg KPF21XR2EJtTwEmiac5qZhCbUUBW4v73eywgNrOAuMStJ/OZIA4VkFiy5zwzhC0q8fLxP1YI W0lixfZLjCDjmQU0Jdbv0odoVZSY0v0QbDyvgKDEyZlPWCYwis5CMnUWQscsJB2zkHQsYGRZ xShanFqclJtuZKSXWpSZXFycn6eXl1qyiREYLQe3/DbYwfjyueMhRgEORiUe3gUc8mFCrIll xZW5hxilOViUxHmbmR6ECgmkJ5akZqemFqQWxReV5qQWH2Jk4uCUamC088w4yMQVvrrhg52g dKjo7t2vr0Qc2Z60lrXsgsTVMK/1lanmR2SfzrHtuS/9wO/9o+/HLrd9XDmnaclDZyt98xsv y6q9+a+dn7Alyv3nitPHbiaXR8RMzNO77Mvz4F0Dty7z7YISN/6YT99bhESv5m1ovmtUlPAv 32HS338nnMyldjrcPuStxFKckWioxVxUnAgAT1UxancCAAA= Archived-At: Subject: [tcpm] FW: New Version Notification for draft-johansson-cc-for-4g-5g-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 05:08:32 -0000 SGkNCg0KQSBuZXcgSUVURiBkcmFmdCBpcyBzdWJtaXR0ZWQgaW4gcmVzcG9uc2UgdG8gZ2VuZXJh bCBkaXNjdXNzaW9uIHRoYXQgZm9yIGluc3RhbmNlIFRDUCBDdWJpYyBpcyBub3QgYW4gYXBwcm9w cmlhdGUgY29uZ2VzdGlvbiBjb250cm9sIGFsZ29yaXRobSBmb3IgTFRFLiANClRoZSBkcmFmdCBi cmllZmx5IG91dGxpbmVzIHR5cGljYWwgNEcgYWNjZXNzIGJlaGF2aW9yIG9uIHRoZSBQRENQLCBS TEMgYW5kIE1BQyAgbGF5ZXJzIHRoYXQgY2FuIGhhdmUgYW4gaW1wYWN0IG9uIHRyYW5zcG9ydCBw cm90b2NvbCBwZXJmb3JtYW5jZS4gVGhlIGRyYWZ0IGFsc28gb3V0bGluZXMgdHdvIHJlbGF0aXZl bHkgc2ltcGxlIG1vZGlmaWNhdGlvbnMgdG8gdGhlIEN1YmljIGNvbmdlc3Rpb24gY29udHJvbC4N Cg0KL0luZ2VtYXIgIA0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogaW50ZXJu ZXQtZHJhZnRzQGlldGYub3JnIFttYWlsdG86aW50ZXJuZXQtZHJhZnRzQGlldGYub3JnXSANClNl bnQ6IGRlbiAxNCBva3RvYmVyIDIwMTUgMjA6MTcNClRvOiBJbmdlbWFyIEpvaGFuc3NvbiBTDQpT dWJqZWN0OiBOZXcgVmVyc2lvbiBOb3RpZmljYXRpb24gZm9yIGRyYWZ0LWpvaGFuc3Nvbi1jYy1m b3ItNGctNWctMDAudHh0DQoNCg0KQSBuZXcgdmVyc2lvbiBvZiBJLUQsIGRyYWZ0LWpvaGFuc3Nv bi1jYy1mb3ItNGctNWctMDAudHh0DQpoYXMgYmVlbiBzdWNjZXNzZnVsbHkgc3VibWl0dGVkIGJ5 IEluZ2VtYXIgSm9oYW5zc29uIGFuZCBwb3N0ZWQgdG8gdGhlIElFVEYgcmVwb3NpdG9yeS4NCg0K TmFtZToJCWRyYWZ0LWpvaGFuc3Nvbi1jYy1mb3ItNGctNWcNClJldmlzaW9uOgkwMA0KVGl0bGU6 CQlDb25nZXN0aW9uIGNvbnRyb2wgZm9yIDRHIGFuZCA1RyBhY2Nlc3MNCkRvY3VtZW50IGRhdGU6 CTIwMTUtMTAtMTQNCkdyb3VwOgkJSW5kaXZpZHVhbCBTdWJtaXNzaW9uDQpQYWdlczoJCTE0DQpV Ukw6ICAgICAgICAgICAgaHR0cHM6Ly93d3cuaWV0Zi5vcmcvaW50ZXJuZXQtZHJhZnRzL2RyYWZ0 LWpvaGFuc3Nvbi1jYy1mb3ItNGctNWctMDAudHh0DQpTdGF0dXM6ICAgICAgICAgaHR0cHM6Ly9k YXRhdHJhY2tlci5pZXRmLm9yZy9kb2MvZHJhZnQtam9oYW5zc29uLWNjLWZvci00Zy01Zy8NCkh0 bWxpemVkOiAgICAgICBodHRwczovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtam9oYW5zc29u LWNjLWZvci00Zy01Zy0wMA0KDQoNCkFic3RyYWN0Og0KICAgVGhpcyBtZW1vIG91dGxpbmVzIHRo ZSBjaGFsbGVuZ2UgdGhhdCA0RyBhbmQgNUcgYWNjZXNzIGJyaW5ncyBmb3INCiAgIHRyYW5zcG9y dCBwcm90b2NvbCBjb25nZXN0aW9uIGNvbnRyb2wgYW5kIGFsc28gb3V0bGluZXMgYSBmZXcgc2lt cGxlDQogICBleGFtcGxlcyB0aGF0IGNhbiBpbXByb3ZlIHRyYW5zcG9ydCBwcm90b2NvbCBjb25n ZXN0aW9uIGNvbnRyb2wNCiAgIHBlcmZvcm1hbmNlIGluIDRHIGFuZCA1RyBhY2Nlc3MuDQoNCg0K ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIA0KDQoNClBsZWFzZSBub3RlIHRoYXQgaXQgbWF5IHRh a2UgYSBjb3VwbGUgb2YgbWludXRlcyBmcm9tIHRoZSB0aW1lIG9mIHN1Ym1pc3Npb24gdW50aWwg dGhlIGh0bWxpemVkIHZlcnNpb24gYW5kIGRpZmYgYXJlIGF2YWlsYWJsZSBhdCB0b29scy5pZXRm Lm9yZy4NCg0KVGhlIElFVEYgU2VjcmV0YXJpYXQNCg0K From nobody Thu Oct 15 15:05:48 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 80B281A7012 for ; Thu, 15 Oct 2015 15:05:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.701 X-Spam-Level: X-Spam-Status: No, score=-1.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_21=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M4sLhQAUiP_l for ; Thu, 15 Oct 2015 15:05:44 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8EDF01A700D for ; Thu, 15 Oct 2015 15:05:43 -0700 (PDT) Received: from 64.229.200.146.dyn.plus.net ([146.200.229.64]:53326 helo=[192.168.0.2]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85) (envelope-from ) id 1Zmqeb-0002iG-Av; Thu, 15 Oct 2015 23:05:41 +0100 To: "De Schepper, Koen (Koen)" , =?UTF-8?Q?Mirja_K=c3=bchlewind?= , "tcpm@ietf.org Extensions" , Richard Scheffenegger References: <20150906231947.7501.54950.idtracker@ietfa.amsl.com> <55F055AD.3050809@tik.ee.ethz.ch> From: Bob Briscoe Message-ID: <56202334.9020807@bobbriscoe.net> Date: Thu, 15 Oct 2015 23:05:40 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net Archived-At: Subject: Re: [tcpm] Fwd: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 22:05:47 -0000 Koen, On 14/10/15 18:37, De Schepper, Koen (Koen) wrote: > Hi, > > Just finished reading/reviewing this draft. I think it is really well written, simple and it has gone clearly through a lot of thoughts already. Thx. Praise is always nice. Here's a question back to you, especially if you read the previous version (-03): If you had to try to meet all the conflicting requirements for accurate ECN feedback in RFC 7560, would you have chosen the set of compromises we chose here? This time we haven't included as many tricks to traverse middlebox meddling, to keep it simpler. And this time we've not tried so hard to minimise header overhead, again to keep it simple. > > Below some questions and remarks: > > - Is how this protocol is intended to be used by classic-ECN-TCP and DCTCP documented somewhere? I think it useful to describe (not sure if you want to add it to this doc?) That isn't how to interpret an IETF RFC. RFCs do not tell anyone that they have to do anything. An RFC describes the behaviours that an implementation MUST/SHOULD/MAY exhibit in order to be able to claim that it complies with that RFC.Then, voluntarily, implementers can choose to comply with the RFCs in order to be able to claim that they will interoperate with others in respect of the relevant scope. Unlike the ITU or ETSI, the IETF is not a standards body with any legal standing. It has a much more practical voluntary model, which means it is still populated by some engineers not just political officers. So, once AccECN is an experimental RFC, an implementer of an RFC3168-compliant stack could choose to write a variant that complies with AccECN. And similarly, for an implementer of a DCTCP stack. Have I understood the question correctly? > > - Why the quite strong restrictions in what offload engines are allowed to do. Is it useful to clearly specify what an offload engine must do? See also my response to the same point made by Praveen: any help more clearly specifying offload behaviour is welcome. > For example I can imagine is a large list of packets are received in a NIC, the engine can combine all acks and data packets as long as only one counter is incremented, and only report the last value of the counter in the option. Also as long as it can guarantee that the ACE counter is not interpreted as being wrapped, it also can offload by combining up to 7 packets or acks... Yes it will be necessary to carefully write this down, because any coalescing has to meet constraints on both half-connections - not hiding transitions in the ECN field in one direction, and not coalescing ACKs that change more than one counter in the other. It's quite tricky to find all these constraints in different parts of the doc right now, so it would be good to re-write what it means for offload hardware. I assume you are mostly referring to the sentence: Nonetheless, such hardware MUST attempt to preserve the timing of each ACK (for example, if it coalesced ACKs it would not be AccECN- compliant). Timing information is part of the "Accurate Feedback" of AccECN. Timing could be written as bits in a packet, or it can be conveyed through the actual time at which separate ACKs arrive. We decided not to write timing as bits in ACKs, and not to require the timing when every ECN marking arrives to be reflected in the timing of separate ACKs. But we did want {Note 1} the timing of /transitions/ in the marking to be reflected in the timing of an ACK. That is why that statement is there. {Note 1} I should say "I did want", because Mirja & Richard gave me an action to write-up a use-case where this timing info could be useful. My argument was based on the work of Mirja & I (and others) on faster start up, by the sender probing with different spacing between packets (chirps). This can be necessary whenever another flow leaves, not just when a new flow starts. > > - If L4S (DCTCP/TCP-Prague) should use ECT1, then it needs always 11 and sometimes 8 bytes options... Why not putting ECEB first, and allow a more relaxed requirement on signaling EE0B and EE1B? If option space is limited, at least ECEB will be able to get through enough to estimate the congestion. If L4S were to use ECT1, we would have to change the way the option is arranged. We had already had a number of different ideas on how to be efficient for different possible future patterns of ECT(0) and ECT(1), but we wrote only the simplest one for now. it is more important for now to keep this draft short and simple. Then, dependent on how the L4S identifier debate goes, and whether the WG wants something more generic, we can change this aspect. If you're interested, you can pull the XML from the IETF datatracker server. In the XML of Appendix B, there is a commented out alternative arrangement for the option - but for a different reason. > > - If I am interested in the probability of marks, just echoing the last received IP mark would perfectly fit this requirement (assuming random marking). Currently this is not supported, Surely you mean that extracting the probability of a mark /is/ currently supported, but the simpler feedback that you propose is not supported, even though it would be /sufficient/ for your need. We wanted to have a feedback protocol without any sub-negotiation of the behaviour. So that a single receiver behaviour suffices for a wide range of possible sender requirements. We could have gone down the route of negotiating different behaviours, but that would lead to both complex code (e.g. if each end implements something different) and interminable arguments about which ones are standardised, and which ones are not. > and thinking about this, your 'safe' algorithms will give potentially quite deviating results, taking all possible ACK losses/misses into account. Pls expand on this. It sounds like you're saying we're wrong when we claim: Resilience against Bias: Because feedback is based on repetition of counters, random losses do not remove any information, they only delay it. Therefore, even though some ACKs are change-triiggered, random losses will not alter the proportions of the different ECN markings in the feedback. > Are these future-use requirements worth the overhead/complexity? The TCP wire protocol can only be changed once every decade or so, so we have to get it right for multiple uses, not just one. If this draft gets adopted, it becomes WG property, then everyone (including the authors) will be entitled to argue their particular choice of compromise on this point. > > - L4S (DCTCP/Prague/...) can have often (random) marking probabilities around 50%, causing the change triggered acks to ack almost every packet (this is what the current DCTCP is doing too, but on a step-RED AQM you get large on-off series, with DualQ you get a random distribution with none or small series). Is order so important? Otherwise, the acks could just carry EE0B (or even immediately ECEB as above), the sender knows how much bytes it has sent, assuming all the rest is ECEB (EE0B/EE1B as appropriate). Agreed that there could be a lot of change-triggered ACKs for some marking patterns. But, as the marking probability increases, the strings of CE will probably get longer as well - I think it would be quite unusual (tho possible) to have constant pkt-by-pkt alternation between CE and ECT. Also, to mitigate the extra change-triggered ACKs, the receiver could increase the delayed ACK factor (above 2), so that when there is a string of constant marking (whether CE or ECT), it sends less ACKs, which could lead to the same or even less ACKs overall (at the expense of more burstiness from the sender if it is not pacing). Again, your question depends on the premise that this feedback will only be used for the one thing you want to do today. > > Some typo's and confusing text: > > - The term classic ECN confused me a few times. I suppose you mean classic ECN echoing protocol, not the response to congestion in the TCP sender. Maybe consistent use of 'classic ECN echoing' is clearer. Good point. We need to define classic ECN echoing for this doc as "the TCP feedback aspect of RFC 3168 ECN". The scope outlined in the introduction is only the layer 4 feedback protocol on the wire, not the sender behaviour in response. This was the source of a lot of robust discussion between the co-authors. [I've just realised we've made the abbreviation L4 for layer 4 a bit confusable with 'low loss low latency' now!] > > - The draft is structured very good, as I could read sequencially though it, except for one case where I had to look further to understand what was exactly meant: The third paragraph of section 3.1 was "... TCP server MUST set NS=1 on the SYN/ACK.". Changing it to "... TCP server MUST set additionally NS=1 on the SYN/ACK.", and optionally add that this is only when in accECN mode, would have clarified all doubts. > > - The sixth paragraph of section 3.1 contains an If (with capital I) in the middle of the fist sentence. > > - Section 3.2.1 has "thy" instead of "they". > > - Section 3.2.2 has "(n)" instead of "n", later it also appears as "n" without the brackets. > > - Section 5 has triiggered instead of triggered All accepted - I'll try to do an update before Monday's deadline, including those I promised Praveen, a few I've noticed myself, and Mirja has some changes to the algos from having better understood through implementation. Bob > > > Regards, > Koen. > > > > >> -----Original Message----- >> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Mirja Kühlewind >> Sent: woensdag 9 september 2015 17:52 >> To: tcpm@ietf.org Extensions; Bob Briscoe; Richard Scheffenegger >> Subject: [tcpm] Fwd: New Version Notification for draft-kuehlewind- >> tcpm-accurate-ecn-04.txt >> >> Hi all, >> >> we submitted a new draft for AccECN (see below). We significantly >> simplified the >> draft because we believe it is more important to have a solution that >> is easy to >> understand and therefore more easy to correctly implement, than having >> a >> solution that tries to fulfill all requirements but is overly complex. >> >> In short, we now only overwrite the 3 TCP header bits (no additional >> bits in the >> TCP header are used) and use them simply for a CE packet counter, while >> bytes-wise information on all other markings is provided by a new TCP >> option. In >> case the option is not available, e.g. because it's blocked, this will >> still >> provide the needed information to react to a CE-based congestion >> feedback. >> However, any advanced mechanism that needs further information on the >> other >> markings received will only work if the option is available. >> >> Further, we also tried to keep the draft as short as possible. That >> means the >> actual normative part only has 9 pages. However, we still provide an >> overview >> section with some reasoning as well as some discussion on interaction >> with other >> mechanisms which leads in total to 25 pages (without appendix). >> >> I'm currently also working on an implementation of this proposed >> solution. I >> will announce it as soon as I'm ready! >> >> In any case we would like to discuss this at the next meeting. We, the >> author, >> think that this proposal is now the right way forward and hope that >> this >> simplified proposal will allows us to quickly proceed on AccECN as the >> need for >> it is increasing. >> >> Please let us know if you have any feedback on the draft! >> >> Mirja >> >> >> -------- Forwarded Message -------- >> Subject: New Version Notification for draft-kuehlewind-tcpm-accurate- >> ecn-04.txt >> Date: Sun, 06 Sep 2015 16:19:47 -0700 >> From: internet-drafts@ietf.org >> To: Richard Scheffenegger , "Mirja Kühlewind" >> , Mirja Kuehlewind >> , Richard Scheffenegger >> , Bob >> Briscoe , Bob Briscoe >> >> >> A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-04.txt >> has been successfully submitted by Bob Briscoe and posted to the >> IETF repository. >> >> Name: draft-kuehlewind-tcpm-accurate-ecn >> Revision: 04 >> Title: More Accurate ECN Feedback in TCP >> Document date: 2015-09-06 >> Group: Individual Submission >> Pages: 36 >> URL: >> https://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate- >> ecn-04.txt >> Status: https://datatracker.ietf.org/doc/draft-kuehlewind-tcpm- >> accurate-ecn/ >> Htmlized: https://tools.ietf.org/html/draft-kuehlewind-tcpm- >> accurate-ecn-04 >> Diff: >> https://www.ietf.org/rfcdiff?url2=draft-kuehlewind-tcpm-accurate-ecn-04 >> >> Abstract: >> Explicit Congestion Notification (ECN) is a mechanism where network >> nodes can mark IP packets instead of dropping them to indicate >> incipient congestion to the end-points. Receivers with an ECN- >> capable transport protocol feed back this information to the >> sender. >> ECN is specified for TCP in such a way that only one feedback >> signal >> can be transmitted per Round-Trip Time (RTT). Recently, new TCP >> mechanisms like Congestion Exposure (ConEx) or Data Center TCP >> (DCTCP) need more accurate ECN feedback information whenever more >> than one marking is received in one RTT. This document specifies >> an >> experimental scheme to provide more than one feedback signal per >> RTT >> in the TCP header. Given TCP header space is scarce, it overloads >> the three existing ECN-related flags in the TCP header and provides >> additional information in a new TCP option. >> >> >> >> >> Please note that it may take a couple of minutes from the time of >> submission >> until the htmlized version and diff are available at tools.ietf.org. >> >> The IETF Secretariat >> >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ From nobody Fri Oct 16 01:21:58 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B5CB1B3082 for ; Fri, 16 Oct 2015 01:21:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.91 X-Spam-Level: X-Spam-Status: No, score=-3.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pT0b62Z5Mewz for ; Fri, 16 Oct 2015 01:21:44 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7D6591B3080 for ; Fri, 16 Oct 2015 01:21:43 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 19DAFD930E; Fri, 16 Oct 2015 10:21:42 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id RrVRKHDKLSvs; Fri, 16 Oct 2015 10:21:41 +0200 (MEST) Received: from [192.168.43.60] (unknown [93.51.79.63]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: mirjak) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 5F26ED930C; Fri, 16 Oct 2015 10:21:40 +0200 (MEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\)) From: =?utf-8?Q?Mirja_K=C3=BChlewind?= In-Reply-To: <561DA02D.8020001@bobbriscoe.net> Date: Fri, 16 Oct 2015 09:22:00 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <0F4F0BA2-A1AE-40BB-9E0D-0EB5F4DE79E1@tik.ee.ethz.ch> References: <55F055AD.3050809@tik.ee.ethz.ch> <55F05D54.5060708@tik.ee.ethz.ch> <55FF2910.7080908@bobbriscoe.net> <561DA02D.8020001@bobbriscoe.net> To: Bob Briscoe X-Mailer: Apple Mail (2.2102) Archived-At: Cc: tcpm IETF list , Richard Scheffenegger Subject: Re: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2015 08:21:47 -0000 Hi, > Am 14.10.2015 um 02:22 schrieb Bob Briscoe : >=20 > At present DCTCP measures the extent of congestion in packets not = bytes. But we envisaged that an algorithm like Relentless TCP might be = used where, every time new CE feedback arrives, the sender just = decrements cwnd by (CE bytes)/2. As a side note, the current DCTCP implementation in Linux uses number of = bytes... From nobody Fri Oct 16 09:32:10 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B65A1A8A3B for ; Fri, 16 Oct 2015 09:32:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.492 X-Spam-Level: X-Spam-Status: No, score=-1.492 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_DKIM_INVALID=0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mxqFYWgua23l for ; Fri, 16 Oct 2015 09:32:07 -0700 (PDT) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0129.outbound.protection.outlook.com [65.55.169.129]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A4CD61A89FC for ; Fri, 16 Oct 2015 09:32:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ocOEqwxsyKFYmYJ5v25SsuJTtigmzzHkBB4i+U0WF5E=; b=IyTBwVFLS1OR5tWqhV0r5ju5zZo0YeUY0Lu+N/LtFdzNh2Ql5eh9sFI3RzYAk1QKhH7YDggYLcZ9B6yZ5GlzCLewaNcmTNNffiR4vnhjSM9TYuxvXpFPl4iPtORE0M5I1SEa/XVlDxfeBZuTKEvQBgNiIgaP+KcUgwi94VDhUxg= Received: from BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) by BN1PR03MB005.namprd03.prod.outlook.com (10.255.224.35) with Microsoft SMTP Server (TLS) id 15.1.293.16; Fri, 16 Oct 2015 16:32:05 +0000 Received: from BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.173]) by BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.153]) with mapi id 15.01.0293.007; Fri, 16 Oct 2015 16:32:05 +0000 From: Praveen Balasubramanian To: =?iso-8859-1?Q?Mirja_K=FChlewind?= , "Bob Briscoe" Thread-Topic: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt Thread-Index: AQHRB+uxdsyPBgf4d0S7f+erzM2kv55uTxtA Date: Fri, 16 Oct 2015 16:32:04 +0000 Message-ID: References: <55F055AD.3050809@tik.ee.ethz.ch> <55F05D54.5060708@tik.ee.ethz.ch> <55FF2910.7080908@bobbriscoe.net> <561DA02D.8020001@bobbriscoe.net> <0F4F0BA2-A1AE-40BB-9E0D-0EB5F4DE79E1@tik.ee.ethz.ch> In-Reply-To: <0F4F0BA2-A1AE-40BB-9E0D-0EB5F4DE79E1@tik.ee.ethz.ch> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com; x-originating-ip: [2001:4898:80e8:2::7ba] x-microsoft-exchange-diagnostics: 1; BN1PR03MB005; 5:kAYF6mq8obWIFrPfxCxRRhVj7f1ey3boWbS4nVlwnPJwoUk9i5Fvf7RsiPAUVpEMaqESOz78o2XymFY1IGN/Xa/doVu2XNMAaAo8yZlW9P1vC2KA32jf6pNDf1Lo1ljlANmwsR2tRUHuFtJyYcMybA==; 24:IpraUp9p/4xlvkL60LA2hA89oGKSAmsYAzLBsVx+wBqy2yzYHTnlC0jQRovOjqvc+6fNRwpoBvMSKCiG26Qm5elkeMtz5sKVDNsO+oIyMBE=; 20:m28O295luz3Y6a2Bdczcsksf4JLrNx/oyw1332/V/5+xe5/pRUDnWneKdMo4U3yQokcS/m2C5k1r2tGMuJa2AQ== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN1PR03MB005; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(107488617086295)(42673675456677)(108003899814671)(83020558694031); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425024)(601004)(2401047)(5005006)(520078)(8121501046)(3002001)(61426024)(61427024); SRVR:BN1PR03MB005; BCL:0; PCL:0; RULEID:; SRVR:BN1PR03MB005; x-forefront-prvs: 0731AA2DE6 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(13464003)(199003)(189002)(377454003)(10290500002)(86362001)(86612001)(33656002)(5008740100001)(74316001)(5007970100001)(64706001)(40100003)(5003600100002)(122556002)(76576001)(76176999)(5002640100001)(189998001)(102836002)(230783001)(5001960100002)(10400500002)(5001920100001)(5001770100001)(5005710100001)(93886004)(19580395003)(87936001)(106356001)(8990500004)(5004730100002)(46102003)(106116001)(92566002)(2950100001)(11100500001)(101416001)(97736004)(99286002)(10090500001)(19580405001)(2900100001)(50986999)(54356999)(105586002)(81156007)(7059030)(3826002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR03MB005; H:BN1PR03MB008.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Oct 2015 16:32:05.0121 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR03MB005 Archived-At: Cc: tcpm IETF list , Richard Scheffenegger Subject: Re: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2015 16:32:09 -0000 Even the Windows implementation uses the number of bytes in CE marked packe= ts. Bob, I am not sure why you think extent of congestion is based on packe= ts and not bytes, please check section 3.3 in the draft.=20 -----Original Message----- From: Mirja K=FChlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch]=20 Sent: Friday, October 16, 2015 12:22 AM To: Bob Briscoe Cc: Praveen Balasubramanian ; tcpm IETF list ; Richard Scheffenegger Subject: Re: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accu= rate-ecn-04.txt Hi, > Am 14.10.2015 um 02:22 schrieb Bob Briscoe : >=20 > At present DCTCP measures the extent of congestion in packets not bytes. = But we envisaged that an algorithm like Relentless TCP might be used where,= every time new CE feedback arrives, the sender just decrements cwnd by (C= E bytes)/2. As a side note, the current DCTCP implementation in Linux uses number of by= tes... From nobody Sat Oct 17 09:48:51 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 325BC1A0275 for ; Sat, 17 Oct 2015 09:48:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.388 X-Spam-Level: X-Spam-Status: No, score=-1.388 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BsFI8R38K7dn for ; Sat, 17 Oct 2015 09:48:49 -0700 (PDT) Received: from mail-vk0-x231.google.com (mail-vk0-x231.google.com [IPv6:2607:f8b0:400c:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 209F61A01FC for ; Sat, 17 Oct 2015 09:48:49 -0700 (PDT) Received: by vkfw189 with SMTP id w189so366400vkf.2 for ; Sat, 17 Oct 2015 09:48:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=K3L/StKDsbu0oUALl5XDtWlMkoOM0r8H6pJA8T8Bxmg=; b=S8EtrPc1GuRUK69infLW+S6BtDkCzYVAZcGTn6hUuin0GUJeFun0U7ZIuF0wTH8BOg V5jnrQo3t13iLhQZ2uXGCN582Bq2gzL6HX8RQI41ju2v4VX/DHC9GdptvUrfkDPAnezT 10AEljfw6JHhchQX1YzAhCBCPuQzCmP/9ETMhM3su9mbxD2d/ByhpElO1ElqHODlTOGy wtPQmyPrlorZHyjKGf4yFPC1LQ21PqGcblRst/cfjCj6C+D2qg99C3XHIzWRbR7WJPz6 qaWI4VBq35To41HPUSbTCa6HVHu/uHCVVXOlJ7D3C88QfZzr6MURhkbIKODQNGC3Te8Y iFYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-type; bh=K3L/StKDsbu0oUALl5XDtWlMkoOM0r8H6pJA8T8Bxmg=; b=eVk3NXQ6K1ZhRM/iW1sYfaWWng29dSg+XkulX8WtPMRLiUIeLImQgs3Ed1AM5cLAJo BWxJrQ39zCabBfweWoN8mx7fJqidcud4ffLyj1fnAdQ4Pl1DLTV4vqkYirtW5/4ezh6z 51JioWDbOFmtFBUmtzXuWbpy3xLTK23yix4HUMDvJcb5COqXU/X8eiWzV9MLhe99hgBe zDKFUMmp8m63YmNKOhbqvkP1oL6CpQbK3QMDjbS8cUQkrxFLh1785v5CRpk2yfb7EdJx 0gVV34qywCeBH1/Se8P8M0shPpyQOJ6TVsO4d/q/qvZZUO0E+TvEqRpcMWcYtEpKeS/E B94g== X-Gm-Message-State: ALoCoQlwzLWi34UvX5bkpy6Qw1qH65OthemPRbOA6pHlf10o0CKXBAnLq2RkXBvlWyBsxq3N7ppe X-Received: by 10.31.5.144 with SMTP id 138mr14278273vkf.62.1445100528119; Sat, 17 Oct 2015 09:48:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.153.134 with HTTP; Sat, 17 Oct 2015 09:48:08 -0700 (PDT) From: Yuchung Cheng Date: Sat, 17 Oct 2015 09:48:08 -0700 Message-ID: To: "tcpm@ietf.org Extensions" Content-Type: multipart/alternative; boundary=001a1143ef62a502fa05224faf2f Archived-At: Subject: [tcpm] Question to TCP implementors regarding dupack threshold X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2015 16:48:50 -0000 --001a1143ef62a502fa05224faf2f Content-Type: text/plain; charset=UTF-8 Hi, The question is: When a TCP sender receives a DUPACK with SACK option(s) covering at least three packets (or three MSS bytes), will it trigger fast recovery immediately? I know Linux does (for at least +8 years). I am curious about other implementations like Windows, Mac, *BSD? Thanks. Yuchung --001a1143ef62a502fa05224faf2f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,

The question is:

When a TCP sender receives a DUPACK with SACK option(s) covering at = least three packets (or three MSS bytes), will it trigger fast recovery imm= ediately?

I know Linux does (for at least +8 years= ). I am curious about other implementations like Windows, Mac, *BSD?
<= div>
Thanks.

Yuchung
--001a1143ef62a502fa05224faf2f-- From nobody Sat Oct 17 12:10:12 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0E5D31A6F6A for ; Sat, 17 Oct 2015 12:10:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.912 X-Spam-Level: X-Spam-Status: No, score=-3.912 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id erqdwt1j9uBS for ; Sat, 17 Oct 2015 12:10:08 -0700 (PDT) Received: from script.cs.helsinki.fi (script.cs.helsinki.fi [128.214.11.1]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E65AE1A6F51 for ; Sat, 17 Oct 2015 12:10:07 -0700 (PDT) Received: from melkinpaasi.cs.helsinki.fi (melkinpaasi.cs.helsinki.fi [128.214.9.14]) (AUTH: PLAIN cs-relay, TLS: TLSv1/SSLv3,256bits,AES256-GCM-SHA384) by mail.cs.helsinki.fi with ESMTPSA; Sat, 17 Oct 2015 22:10:04 +0300 id 00000000005A002A.0000000056229D0C.00004A20 Received: by melkinpaasi.cs.helsinki.fi (Postfix, from userid 50795) id BE4EE80030; Sat, 17 Oct 2015 22:10:04 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by melkinpaasi.cs.helsinki.fi (Postfix) with ESMTP id B040A1E9C72 for ; Sat, 17 Oct 2015 22:10:04 +0300 (EEST) Date: Sat, 17 Oct 2015 22:10:04 +0300 (EEST) From: =?ISO-8859-15?Q?Ilpo_J=E4rvinen?= X-X-Sender: "ijjarvin@cs"@melkinpaasi.cs.helsinki.fi To: "tcpm@ietf.org Extensions" In-Reply-To: Message-ID: References: User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-CS-Test-DKIM: none Archived-At: Subject: Re: [tcpm] Question to TCP implementors regarding dupack threshold X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2015 19:10:10 -0000 On Sat, 17 Oct 2015, Yuchung Cheng wrote: > Hi, > The question is: > > When a TCP sender receives a DUPACK with SACK option(s) covering at least > three packets (or three MSS bytes), will it trigger fast recovery > immediately? > > I know Linux does (for at least +8 years). I am curious about other > implementations like Windows, Mac, *BSD? I don't know about implementations other than Linux, especially the current status, but RFC6675 uses less strict condition than "three MSS bytes" for this detection: DupThresh discontiguous SACKed sequences have arrived above 'SeqNum' or more than (DupThresh - 1) * SMSS with sequence numbers greater than 'SeqNum' have been SACKed Some stacks wouldn't know where packet boundaries are anyway (I think that at least BSD is/was like that) so three packets cannot be tested other than using the above RFC6675 condition (or by depending on three DUPACKs). -- i. From nobody Sat Oct 17 13:48:36 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D80D11ACE4E for ; Sat, 17 Oct 2015 13:48:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.21 X-Spam-Level: X-Spam-Status: No, score=-1.21 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id whTNCDs4MRQO for ; Sat, 17 Oct 2015 13:48:32 -0700 (PDT) Received: from mailfilter.orange.md (mailfilter.orange.md [94.243.64.204]) by ietfa.amsl.com (Postfix) with ESMTP id 5CE541ACE4B for ; Sat, 17 Oct 2015 13:48:32 -0700 (PDT) Received: from localhost.localdomain (antispam.orange.md [127.0.0.1]) by localhost (Postfix) with ESMTP id 5D34E3B2715; Sat, 17 Oct 2015 23:48:51 +0300 (EEST) Received: from antispam.orange.md by antispam.orange.md with queue id 251381-1; Sat, 17 Oct 2015 20:48:51 GMT Received: from XCHSRV04.main.orange.md (unknown [192.168.200.64]) by mailfilter.orange.md (Postfix) with ESMTP id 33E3E3B2715; Sat, 17 Oct 2015 23:48:51 +0300 (EEST) Received: from XCHSRV01.main.orange.md ([fe80::685f:aef6:93b0:dea7]) by XCHSRV04.main.orange.md ([fe80::bdbd:3bcd:a0d2:9b6%14]) with mapi id 14.02.0328.009; Sat, 17 Oct 2015 23:48:26 +0300 From: Veaceslav ROMAN To: Neal Cardwell Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8Pq87vsQ9XoBmgDq7PdWOtXZ4gsAq6EW+gA= Date: Sat, 17 Oct 2015 20:48:26 +0000 Message-ID: <7DBBB686E19D2049ADAACD210B474BB10167E64F3E@XCHSRV01.main.orange.md> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md> <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E8BDBA@XCHSRV01.main.orange.md> In-Reply-To: Accept-Language: en-US, ro-RO Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.107.165] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , Ingemar Johansson S , Eric Dumazet , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2015 20:48:35 -0000 SGksDQpSZWNvbXBpbGVkIGtlcm5lbCB0byBIWVNUQVJUX0RFTEFZX01JTiAgMjAgbXM6IA0KDQog IERvd25sb2FkIDEwTUIsIEhpZ2ggQmFuZHdpZHRocywgSFlTVEFSVF9ERUxBWV9NSU4gIDIwIG1z Og0KICAgICAgICAgICBIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS4xOCBzDQogICAg ICAgICAgIE5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDEuMDAgIHMNCg0KTm8gc2ln bmlmaWNhbnQgaW1wcm92ZW1lbnQgYWdhaW5zdCB0aGUgY2FzZSB3aXRoIEhZU1RBUlRfREVMQVlf TUlOICAxMCBtcy4NCg0KU2lsbCBIeXN0YXJ0IGVhcmx5IGV4aXQgaXMgdmlzaWJsZS4gDQoNCk90 aGVyIHRlc3RzIHdpdGggSFlTVEFSVF9ERUxBWV9NSU4gIDIwIG1zIGFuZCBmaWxlcyBvZiAzTUIu DQoNCiAgRG93bmxvYWQgM01CLCBIaWdoIEJhbmR3aWR0aHMsIEhZU1RBUlRfREVMQVlfTUlOICAy MCBtczoNCiAgICAgICAgICAgSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDAuNDcgcw0K ICAgICAgICAgICBOb0h5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAwLjM5ICBzDQoNCmNv bXBhcmUgd2l0aCA0IG1zDQoNCiAgRG93bmxvYWQgM01CLCBIaWdoIEJhbmR3aWR0aHMsIEhZU1RB UlRfREVMQVlfTUlOICA0IG1zOg0KICAgICAgICAgICBIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQg dGltZTogMC42NSBzDQogICAgICAgICAgIE5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6 IDAuMzcgIHMNCg0KRmV3IG1vcmUgY29tbWVudDogSGlnaCBCYW5kd2lkdGggaXMgdHlwaWNhbGx5 IGFib3ZlIDMwLTQwIE1icHMuDQpGb3IgTG93IEJhbmR3aWR0aCwgd2hlbiByYWRpbyBkbyBub3Qg cGVybWl0IGFib3ZlIDQwIE1icHMgSHlzdGFydCBlYXJseSBleGl0IGRvZXMgbm90IGhhdmUgYSBz aWduaWZpY2FudCBpbXBhY3QuIEkgZG8gZXhwbGFpbiBpdCB3aXRoIHRoZSBmYWN0IHRoYXQgdGhl IG1pbmltdW0gZXhpdCBDV05EIGZvciBIeXNzdGFydCBpcyAyOSB3aGljaCwgZm9yIFJUVCBvZiB+ MTVtcyAobWluaW11bSBwb3NzaWJsZSBpbiBMVEUgYW5kIHZpc2libGUgaW4gbWFqb3JpdHkgb2Yg dHJhY2VzKSBsZWFkcyB0byB+MjUgTWJwcywgc28gSHlzdGFydCBleGl0LCBhdCBsZWFzdCwgYXQg MjUgTWJwcyBhbmQgdGhlbiBpdCB0YWtlcyBub3QgdG9vIGxvbmcgdG8gY3ViaWMgdG8gcmVhY2gg MzAtNDAgTWJwcy4gIA0KDQpWZWFjZXNsYXYgUm9tYW4NCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdl LS0tLS0NCkZyb206IE5lYWwgQ2FyZHdlbGwgW21haWx0bzpuY2FyZHdlbGxAZ29vZ2xlLmNvbV0g DQpTZW50OiBUdWVzZGF5LCAwNiBPY3RvYmVyIDIwMTUgMDQ6MDYNClRvOiBWZWFjZXNsYXYgUk9N QU4NCkNjOiBFcmljIER1bWF6ZXQ7IEluZ2VtYXIgSm9oYW5zc29uIFM7IEVyaWMgRHVtYXpldDsg dGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0YWUgSGE7IGVuZDJlbmQtaW50ZXJl c3RAcG9zdGVsLm9yZw0KU3ViamVjdDogUmU6IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBwYXRj aCBkZXBsb3ltZW50DQoNCk9uIE1vbiwgT2N0IDUsIDIwMTUgYXQgNjowNyBQTSwgVmVhY2VzbGF2 IFJPTUFOIDxWZWFjZXNsYXYuUm9tYW5Ab3JhbmdlLm1kPiB3cm90ZToNCj4gV2UndmUgbWFuYWdl ZCB0byByZWNvbXBpbGUgdGhlIGtlcm5lbCA0LjEgd2l0aCB0Y3BfY3ViaWMgSFlTVEFSVF9ERUxB WV9NSU4gICgxMFU8PDMpICwgMTBtcy4NCj4gV2VsbCwgd2UgaGFkIHRpbWUgb25seSBmb3Igb25l IHRlc3QgY3ljbGUsIGJ1dCByZXN1bHRzIGFyZSB2ZXJ5IHByb21pc2luZzoNCj4NCj4gRG93bmxv YWQgMTBNQiwgSGlnaCBCYW5kd2lkdGhzLCBIWVNUQVJUX0RFTEFZX01JTiAgMTAgbXM6DQo+ICAg ICAgICAgICAgSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDEuMTYgcw0KPiAgICAgICAg ICAgIE5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDAuOTMgIHMNCi4uLg0KPiAgRG93 bmxvYWQgMTBNQiwgSGlnaCBCYW5kd2lkdGhzLCBIWVNUQVJUX0RFTEFZX01JTiAgNCBtczoNCj4g ICAgICAgICAgICBIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS40MiBzDQo+ICAgICAg ICAgICAgTm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC44OSAgcw0KDQpUaGFua3Mh IFRoYXQgaXMgdmVyeSB1c2VmdWwgYW5kIGludGVyZXN0aW5nIGRhdGEuIFdvdWxkIHlvdSBiZSBh YmxlIHRvIHRyeSBhIGZldyBvdGhlciB2YWx1ZXMsIGxpa2UgMTVtcyBhbmQgMjBtcz8NCg0KbmVh bA0K From nobody Sat Oct 17 17:13:19 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C9CFC1A6F20 for ; Sat, 17 Oct 2015 17:13:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.51 X-Spam-Level: X-Spam-Status: No, score=-0.51 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z2mfiB5BaIA6 for ; Sat, 17 Oct 2015 17:13:17 -0700 (PDT) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E54501A6F10 for ; Sat, 17 Oct 2015 17:13:17 -0700 (PDT) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id 0FF0710A7FD; Sat, 17 Oct 2015 17:13:17 -0700 (PDT) Date: Sat, 17 Oct 2015 17:13:17 -0700 From: hiren panchasara To: Yuchung Cheng Message-ID: <20151018001317.GD87252@strugglingcoder.info> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="tpXFRyzAyQ/GSAaD" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] Question to TCP implementors regarding dupack threshold X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2015 00:13:18 -0000 --tpXFRyzAyQ/GSAaD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 10/17/15 at 09:48P, Yuchung Cheng wrote: > Hi, >=20 > The question is: >=20 > When a TCP sender receives a DUPACK with SACK option(s) covering at least > three packets (or three MSS bytes), will it trigger fast recovery > immediately? >=20 > I know Linux does (for at least +8 years). I am curious about other > implementations like Windows, Mac, *BSD? FreeBSD doesn't do this currently. Looks like a sensible thing to do. Cheers, Hiren --tpXFRyzAyQ/GSAaD Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJWIuQVXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lsGIH/0VtBvawxi7oyaFij/rxXdaK upXxUraSXdUpZ2IWb4Cf2WMmmOzJpzrJ6VOAiW+ICSf6WGyFJlLtvhaqxo4Mnpfs MWjO6xXigA402kxP6N6EUiAWSgeXr1JIpZKC7cSKwl1HLk8MINTVBjYO2FmMJE3J J0vMde3wSs3OM02eTsoR+E2HubsHeB11IMR8C0bLURSeMXl60cNjEyDe9rZVChzi 9xQm5rQqOJLo/FaEgIV7wbLel07DTlqHoAwDe4aRZe1GQM2Bbv1bkK3h7FMpbRb+ fQ9JBjRqMdI6KWg0GDmHpfJFq951n7sJzem1FQ7ibVGBi3AH8nJXP819eCAVSUE= =/NKV -----END PGP SIGNATURE----- --tpXFRyzAyQ/GSAaD-- From nobody Sun Oct 18 10:02:27 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E43BC1A90B7 for ; Sun, 18 Oct 2015 10:02:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.301 X-Spam-Level: X-Spam-Status: No, score=-2.301 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U58W_O6UiR9h for ; Sun, 18 Oct 2015 10:02:23 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7C93A1A90BD for ; Sun, 18 Oct 2015 10:02:23 -0700 (PDT) Received: from 64.229.200.146.dyn.plus.net ([146.200.229.64]:58684 helo=[192.168.0.2]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85) (envelope-from ) id 1ZnrLh-00020X-6G; Sun, 18 Oct 2015 18:02:21 +0100 To: Praveen Balasubramanian , =?UTF-8?Q?Mirja_K=c3=bchlewind?= References: <55F055AD.3050809@tik.ee.ethz.ch> <55F05D54.5060708@tik.ee.ethz.ch> <55FF2910.7080908@bobbriscoe.net> <561DA02D.8020001@bobbriscoe.net> <0F4F0BA2-A1AE-40BB-9E0D-0EB5F4DE79E1@tik.ee.ethz.ch> From: Bob Briscoe Message-ID: <5623D09C.4020002@bobbriscoe.net> Date: Sun, 18 Oct 2015 18:02:20 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net Archived-At: Cc: tcpm IETF list , Richard Scheffenegger Subject: Re: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2015 17:02:26 -0000 Praveen, Sorry, there's ambiguity in these emails between the units of feedback and the units of calculation of the congestion response. Current DCTCP feedback (both Windows and Linux) is in packets, then the sender calculates/estimates what this means in bytes which it applies in the window reduction algo. This calculation/estimation is accurate for long-running flows, but hard for app-limited flows where there are a lot of sub-MSS segments. The example congestion response I gave (Relentless) uses bytes for feedback and bytes for the response calculation. This works because Relentless is designed for loss and SACK gives loss feedback in bytes (Relentless didn't include any change to ECN feedback). The AccECN draft-04 proposes essential feedback in packets using the main TCP header and supplementary feedback in bytes using a new AccECN TCP option (middleboxes permitting). If the AccECN option gets through OK, it means the sender has bytes directly, so it doesn't have to estimate or calculate bytes from packet feedback. I should add that providing byte counters wasn't the main reason for the TCP option. It was primarily to improve protocol resilience in the face of heavy ACK loss (when congestion avoidance could be most critical), because the 3-bit field available for the packet counter in the main header is rather small (it wraps every 8 packets, which is at least better than the 1-bit feedback in current DCTCP). So, once we added the TCP option for resilience, we could use it for full byte feedback instead of packet, and we could also provide enough info to detect bleaching of the ECN field and to support potential future use of ECT(1) as well as ECT(0). Hope this clarifies the confusion. Bob On 16/10/15 17:32, Praveen Balasubramanian wrote: > Even the Windows implementation uses the number of bytes in CE marked packets. Bob, I am not sure why you think extent of congestion is based on packets and not bytes, please check section 3.3 in the draft. > > -----Original Message----- > From: Mirja Khlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch] > Sent: Friday, October 16, 2015 12:22 AM > To: Bob Briscoe > Cc: Praveen Balasubramanian ; tcpm IETF list ; Richard Scheffenegger > Subject: Re: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt > > Hi, > >> Am 14.10.2015 um 02:22 schrieb Bob Briscoe : >> >> At present DCTCP measures the extent of congestion in packets not bytes. But we envisaged that an algorithm like Relentless TCP might be used where, every time new CE feedback arrives, the sender just decrements cwnd by (CE bytes)/2. > As a side note, the current DCTCP implementation in Linux uses number of bytes... > -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ From nobody Mon Oct 19 06:59:06 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B85B1A6F88 for ; Mon, 19 Oct 2015 06:59:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6xQcTxLNtfa3 for ; Mon, 19 Oct 2015 06:59:03 -0700 (PDT) Received: from Chamillionaire.breakpoint.cc (Chamillionaire.breakpoint.cc [IPv6:2001:4d88:1ffa:82:880:aa0:9009:64ae]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D5FB31A6F56 for ; Mon, 19 Oct 2015 06:59:02 -0700 (PDT) Received: from fw by Chamillionaire.breakpoint.cc with local (Exim 4.80) (envelope-from ) id 1ZoAxn-0005dg-49 for tcpm@ietf.org; Mon, 19 Oct 2015 15:58:59 +0200 Date: Mon, 19 Oct 2015 15:58:59 +0200 From: Florian Westphal To: tcpm@ietf.org Message-ID: <20151019135859.GA20906@breakpoint.cc> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Archived-At: Subject: [tcpm] dctcp: alpha may not reach 0 in current linux implementation X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2015 13:59:04 -0000 Quoting from a patch[1] from Andrew Shewmaker: ============ If alpha is strictly reduced by alpha >> dctcp_shift_g and if alpha is less than 1 << dctcp_shift_g, then alpha may never reach zero. [..] The effect isn't noticeable in this case below cwnd=137, but could gradually drive uncongested flows with leftover alpha down to cwnd=137. A larger dctcp_shift_g would have a greater effect. =========== So this doesn't appear to be a problem in practice since cwnd=137 is quite big. >From looking at current ietf draft alpha should be allowed to reach 0 if there is no congestion so it seems this patch fits what the dctcp draft recommends wrt. alpha update in section 3.3. Nevertheless, some of the other implementors might want to look into this bogus cwnd reduction too. [1] http://patchwork.ozlabs.org/patch/532144/ From nobody Mon Oct 19 11:16:38 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 16F301B2B38 for ; Mon, 19 Oct 2015 11:16:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.002 X-Spam-Level: X-Spam-Status: No, score=-2.002 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JiGlYR9f9hUC for ; Mon, 19 Oct 2015 11:16:31 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bon0736.outbound.protection.outlook.com [IPv6:2a01:111:f400:fc10::1:736]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C15651B2B2B for ; Mon, 19 Oct 2015 11:16:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=M9HGNZKBW2NMO9k+C/qubyCBpuxt8KuSDRqWi3ErPuo=; b=BAzinhm/gE4qvTlA//dQRlaBElaym4lJZeBwlmS/NxVSy5wYTvCWzuPCusXYCvDgohmEjH9tfKmcX3Nq0ubKIpw2W/zNw1fnV75nlNqxLT99oJxHsi9MMS1OV7aFX3zuyqK1MRTvWEuWa2bTuDyj+l80aLCDIhK91wwZilhKwzQ= Received: from BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) by BN1PR03MB006.namprd03.prod.outlook.com (10.255.224.36) with Microsoft SMTP Server (TLS) id 15.1.293.16; Mon, 19 Oct 2015 18:16:00 +0000 Received: from BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.173]) by BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.153]) with mapi id 15.01.0293.007; Mon, 19 Oct 2015 18:15:59 +0000 From: Praveen Balasubramanian To: "Scharf, Michael (Michael)" , "draft-ietf-tcpm-dctcp@tools.ietf.org" Thread-Topic: [tcpm] Wording on enabling DCTCP in draft-ietf-tcpm-dctcp Thread-Index: AdEEUTThiGgw+TVnST6f8Mttkqg+nwGRffUQ Date: Mon, 19 Oct 2015 18:15:59 +0000 Message-ID: References: <655C07320163294895BBADA28372AF5D484FFECB@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: <655C07320163294895BBADA28372AF5D484FFECB@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com; x-originating-ip: [2001:4898:80e8:5::584] x-microsoft-exchange-diagnostics: 1; BN1PR03MB006; 5:GNCWM7b0PEmPM2ce2Gpmf5o6B+thKMGVJHdEYQ8zdkIXxsJ0Z2NX6vn6VTa6a5zL9mFgzjVuEcZPxkyCEKzXo6PthHW2WVy8yYwKuP9Z8NgAEyJakhaWFq+9fOnFaxv2kFlFLi63gQzkuZRWTcprug==; 24:MfiHzMzfbNRljPvIlV/4cVVzMxOKaLCvyLEsqSgb1PytplUMPNj0ebge1nPDPnSLB8bDJpoYLWwuW6nfrWtHcoKYiuYwCgnL89cHGgmqWEg=; 20:ElPJ53yOehBoDI+UEt4m8ihC5uVY9xduIFSu7cLzbhs+fO273klS6hEkRbVQ3GuNFizYAXl5JTheBK/dJlaX3A== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN1PR03MB006; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(189930954265078); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425024)(601004)(2401047)(520078)(5005006)(8121501046)(3002001)(61426024)(61427024); SRVR:BN1PR03MB006; BCL:0; PCL:0; RULEID:; SRVR:BN1PR03MB006; x-forefront-prvs: 07349BFAD2 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(199003)(189002)(377454003)(13464003)(51914003)(5005710100001)(76176999)(101416001)(74316001)(5001770100001)(8990500004)(86362001)(575784001)(106356001)(102836002)(76576001)(2900100001)(87936001)(54356999)(19580405001)(92566002)(33656002)(15975445007)(64706001)(19580395003)(50986999)(230783001)(105586002)(2950100001)(2501003)(46102003)(86612001)(5002640100001)(10090500001)(10400500002)(189998001)(5008740100001)(97736004)(5001920100001)(10290500002)(81156007)(5007970100001)(5001960100002)(99286002)(5004730100002)(5003600100002)(122556002)(40100003)(3826002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR03MB006; H:BN1PR03MB008.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Oct 2015 18:15:59.7705 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR03MB006 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Wording on enabling DCTCP in draft-ietf-tcpm-dctcp X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2015 18:16:37 -0000 Thanks for the feedback Michael. Comments inline prefixed with >>. We will publish a draft version with the = updates. Thanks! -----Original Message----- From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Scharf, Michael (Mic= hael) Sent: Sunday, October 11, 2015 11:18 AM To: draft-ietf-tcpm-dctcp@tools.ietf.org Cc: tcpm@ietf.org Subject: [tcpm] Wording on enabling DCTCP in draft-ietf-tcpm-dctcp Hi, During IETF 93 we had a quick discussion on how to negotiate DCTCP (see htt= ps://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%2fwww.ietf.org= %2fproceedings%2f93%2fminutes%2fminutes-93-tcpm&data=3D01%7c01%7cpravb%40mi= crosoft.com%7c9a254a41ea8343c339bc08d2d2685e39%7c72f988bf86f141af91ab2d7cd0= 11db47%7c1&sdata=3D9OkF4LWSSiAuVwPWQz3XTLKyrwgO39Fl1Xyk4iuWZmI%3d). Given t= hat the WG aims for a very fast publication of this I-D, I'd like to trigge= r a follow-up discussion on what the document should state regarding enabli= ng DCTCP.=20 As an individual contributor to TCPM, I believe that it would be good if th= e document addresses the enablement (and perhaps also the implicit negotiat= ion) in a dedicated section; right now this issue is spread across differen= t parts of the document. Specifically, my personal view would be: * Abstract =3D> I think the abstract should better explain the focus on controlled d= omains. Also, it could mention coexistence with standard TCP as an issue. F= or the IESG processing, it could also help to stress the status as informat= ional (e.g., by "This informational memo..."). >> Moving some text from introduction to abstract to reflect this. Adding "= informational" edit. * Section 1 Introduction It is recommended that DCTCP be deployed in a datacenter environment where the endpoints and the switching fabric are under a single administrative domain. This memo also discusses deployment issues related to the coexistence of DCTCP and conventional TCP, the lack of a negotiating mechanism between sender and receiver, and presents some possible mitigations. =20 =3D> I wonder whether to add a more explicit warning sign such as "The pro= tocol is not meant for uncontrolled deployment in the global Internet", or = a wording like the statement in Section 4. >> Adding this text explicitly in the introduction section where the recomm= endation for datacenter deployment is also given. * Section 4 Implementation Issues The implementation must also decide when to use DCTCP. Datacenter servers may need to communicate with endpoints outside the datacenter, where DCTCP is unsuitable or unsupported. Thus, a global configuration setting to enable DCTCP will generally not suffice. DCTCP may be configured based on the IP address of the remote endpoint. Microsoft Windows Server 2012 (and later) also supports automatic selection of DCTCP if the estimated RTT is less than 10 msec and ECN is successfully negotiated, under the assumption that if the RTT is low, then the two endpoints are likely on the same datacenter network. =3D> To me, whether to enable DCTCP or not is not an "implementation issu= e" only. I think this warrants a dedicated section. I'd also suggest to spl= it the problem statement and *possible* solutions from the actual heuristic= in Windows Server 2012 into two paragraphs. This could probably be achieve= d by a purely editorial change only. >> I think when and how to enable DCTCP belongs to implementation issues se= ction. Agreed about splitting the problem and potential solutions into sepa= rate paragraphs, making these editorial changes. * Section 6 Known Issues DCTCP provides no mechanism for negotiating its use. Thus, there is additional management and configuration overhead required to ensure that DCTCP is not used with non-DCTCP endpoints. The affect of using DCTCP with a standard ECN endpoint has been analyzed in [ODCTCP][BSDCAN]. Furthermore, it is possible that other implementations may also modify [RFC3168] behavior without negotiation, causing further interoperability issues. =3D> The requirement to detect DCTCP endpoints seems related to the enabl= ing discussion above. At least, I think it would be valuable to have an int= ernal cross-reference between the sections discussing the two aspects. >> Moving some of this text to the implementation issues section.=20 Thanks Michael _______________________________________________ tcpm mailing list tcpm@ietf.org https://na01.safelinks.protection.outlook.com/?url=3Dhttps%3a%2f%2fwww.ietf= .org%2fmailman%2flistinfo%2ftcpm&data=3D01%7c01%7cpravb%40microsoft.com%7c9= a254a41ea8343c339bc08d2d2685e39%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdat= a=3DN0T2SP1SrK%2bgwFcWZI37MPsL6vtMgEC2N3kcnuPzApY%3d From nobody Mon Oct 19 12:14:43 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9AEA81B2BFD; Mon, 19 Oct 2015 12:14:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4XWRqez6fZOk; Mon, 19 Oct 2015 12:14:38 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9CDA91B2C13; Mon, 19 Oct 2015 12:14:38 -0700 (PDT) Received: from cm-84.208.86.220.getinternet.no ([84.208.86.220]:35251 helo=[192.168.0.116]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.86) (envelope-from ) id 1ZoFtA-0002S1-G5; Mon, 19 Oct 2015 20:14:32 +0100 References: <20151019190039.23693.64756.idtracker@ietfa.amsl.com> To: tcpm IETF list , "De Schepper, Koen (Koen)" , Praveen Balasubramanian From: Bob Briscoe X-Forwarded-Message-Id: <20151019190039.23693.64756.idtracker@ietfa.amsl.com> Message-ID: <56254117.4000700@bobbriscoe.net> Date: Mon, 19 Oct 2015 20:14:31 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151019190039.23693.64756.idtracker@ietfa.amsl.com> Content-Type: multipart/alternative; boundary="------------040901080408020304070000" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net Archived-At: Cc: "Scheffenegger, Richard" , TCP Prague IETF List Subject: [tcpm] Updated AccECN to draft-kuehlewind-tcpm-accurate-ecn-05.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2015 19:14:41 -0000 This is a multi-part message in MIME format. --------------040901080408020304070000 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Folks, We just posted an updated version, mainly fixing problems Koen and Praveen pointed out, as well as stuff we (the authors) noticed, particularly from Mira starting to implement it. We're asking the chairs of the IETF tcpm list if they can put out a call for chartering work on accurate ECN in the tcpm WG. If that succeeds, it would start on the path to becoming an (experimental) change to the TCP wire protocol. So the more people who read it, send critique to the tcpm list, attempt to implement it, or describe/implement alternative ways to solve the same problem, the better. Here's an overview of the diffs: From 04 to 05:: * Corrected ambiguity between Classic ECN and Classic ECN feedback throughout * Changed MUST to SHOULD send AccECN option on SYN/ACK last ACK of 3WHS and first data segment from client, to allow for cached knowledge of option traversal problems. * Removed duplication of normative language about sending a full- length option in the sections on "The AccECN Option" and "Usage of the AccECN Option", and mutually cross referenced. * Acknowledged Koen De Schepper and Praveen Balasubramanian * Noted in Appendix that algo to beacon a full-length option is work-in-progress * Editorial corrections and clarifications throughout Bob -------- Forwarded Message -------- Subject: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-05.txt Date: Mon, 19 Oct 2015 12:00:39 -0700 From: internet-drafts@ietf.org To: Mirja Kuehlewind , "Mirja Kühlewind" , Richard Scheffenegger , Bob Briscoe A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-05.txt has been successfully submitted by Bob Briscoe and posted to the IETF repository. Name: draft-kuehlewind-tcpm-accurate-ecn Revision: 05 Title: More Accurate ECN Feedback in TCP Document date: 2015-10-19 Group: Individual Submission Pages: 38 URL: https://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-05.txt Status: https://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn/ Htmlized: https://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-05 Diff: https://www.ietf.org/rfcdiff?url2=draft-kuehlewind-tcpm-accurate-ecn-05 Abstract: Explicit Congestion Notification (ECN) is a mechanism where network nodes can mark IP packets instead of dropping them to indicate incipient congestion to the end-points. Receivers with an ECN- capable transport protocol feed back this information to the sender. ECN is specified for TCP in such a way that only one feedback signal can be transmitted per Round-Trip Time (RTT). Recently, new TCP mechanisms like Congestion Exposure (ConEx) or Data Center TCP (DCTCP) need more accurate ECN feedback information whenever more than one marking is received in one RTT. This document specifies an experimental scheme to provide more than one feedback signal per RTT in the TCP header. Given TCP header space is scarce, it overloads the three existing ECN-related flags in the TCP header and provides additional information in a new TCP option. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat --------------040901080408020304070000 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Folks,

We just posted an updated version, mainly fixing problems Koen and Praveen pointed out, as well as stuff we (the authors) noticed, particularly from Mira starting to implement it.

We're asking the chairs of the IETF tcpm list if they can put out a call for chartering work on accurate ECN in the tcpm WG.
If that succeeds, it would start on the path to becoming an (experimental) change to the TCP wire protocol. So the more people who read it, send critique to the tcpm list, attempt to implement it, or describe/implement alternative ways to solve the same problem, the better.


Here's an overview of the diffs:

   From 04 to 05::

      *  Corrected ambiguity between Classic ECN and Classic ECN
         feedback throughout

      *  Changed MUST to SHOULD send AccECN option on SYN/ACK last ACK
         of 3WHS and first data segment from client, to allow for cached
         knowledge of option traversal problems.

      *  Removed duplication of normative language about sending a full-
         length option in the sections on "The AccECN Option" and "Usage
         of the AccECN Option", and mutually cross referenced.

      *  Acknowledged Koen De Schepper and Praveen Balasubramanian

      *  Noted in Appendix that algo to beacon a full-length option is
         work-in-progress

      *  Editorial corrections and clarifications throughout

Bob


-------- Forwarded Message --------
Subject: New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-05.txt
Date: Mon, 19 Oct 2015 12:00:39 -0700
From: internet-drafts@ietf.org
To: Mirja Kuehlewind <mirja.kuehlewind@tik.ee.ethz.ch>, "Mirja Kühlewind" <mirja.kuehlewind@tik.ee.ethz.ch>, Richard Scheffenegger <rs@netapp.com>, Bob Briscoe <ietf@bobbriscoe.net> 


A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-05.txt
has been successfully submitted by Bob Briscoe and posted to the
IETF repository.

Name:		draft-kuehlewind-tcpm-accurate-ecn
Revision:	05
Title:		More Accurate ECN Feedback in TCP
Document date:	2015-10-19
Group:		Individual Submission
Pages:		38
URL:            https://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-05.txt
Status:         https://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn/
Htmlized:       https://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-05
Diff:           https://www.ietf.org/rfcdiff?url2=draft-kuehlewind-tcpm-accurate-ecn-05

Abstract:
   Explicit Congestion Notification (ECN) is a mechanism where network
   nodes can mark IP packets instead of dropping them to indicate
   incipient congestion to the end-points.  Receivers with an ECN-
   capable transport protocol feed back this information to the sender.
   ECN is specified for TCP in such a way that only one feedback signal
   can be transmitted per Round-Trip Time (RTT).  Recently, new TCP
   mechanisms like Congestion Exposure (ConEx) or Data Center TCP
   (DCTCP) need more accurate ECN feedback information whenever more
   than one marking is received in one RTT.  This document specifies an
   experimental scheme to provide more than one feedback signal per RTT
   in the TCP header.  Given TCP header space is scarce, it overloads
   the three existing ECN-related flags in the TCP header and provides
   additional information in a new TCP option.

                                                                                  


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat


--------------040901080408020304070000-- From nobody Tue Oct 20 09:03:46 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B9A71B34FA for ; Tue, 20 Oct 2015 09:03:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.388 X-Spam-Level: X-Spam-Status: No, score=-1.388 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id znhE4XnPl35K for ; Tue, 20 Oct 2015 09:03:44 -0700 (PDT) Received: from mail-vk0-x234.google.com (mail-vk0-x234.google.com [IPv6:2607:f8b0:400c:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0B0D41B34F0 for ; Tue, 20 Oct 2015 09:03:44 -0700 (PDT) Received: by vkex70 with SMTP id x70so12664448vke.3 for ; Tue, 20 Oct 2015 09:03:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=q5QImIHu9JwsxzBf9l8hk1LqwDjYNTKNKt3b/2Sg8sQ=; b=BwFYaZR3LM+Mx248se177F8bsKX/yFRXoeFGahuAfYnRcwVWMlMswxQcSuZJIifEYK rvx+Ey5v6wXFspHaM6wzjLcjeiEgNGUYG8rQ+0NV4/Mcruvw1ajOfznUvlFvI2D3R7yw vUxXtSrD9uXAmQxxOSm1OdvvwpF9r2j5N5uZoOmQOFBTK84KvImgBDTY6pM3xectw6RT D+L72A8ehfS1d45J+B/ikNrRORIcCFM8IPiL1ZXNsDRDkMW+pwIkgnFukLcMI60hl5JD J7tVywqv8KEpSXmBuqLIEWvWcRwnLGBzAqCBRYv9NMewJhXBNszzacf/CFc/cbcrgKya gkXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=q5QImIHu9JwsxzBf9l8hk1LqwDjYNTKNKt3b/2Sg8sQ=; b=IBzAB18+W6qVUmKfv5cIwNKku1mEAia/PSv6prd+jFDLQOmx96vrFsYnNl8q5IqSCr Z9+M9y4ntEYqSIuDyJa2Y0FOvTtGtFWfP+fT0iabOCpcCZxqJm5RPsXlS8e78Wq8YtCQ LWIhbYP3OYdK22KcMGBh6Aj0jvmi4dhOnKfrz9q4gpkU4N+gTxtxUq5H+goFOKVFhgtX KkCoQI9+BgAPMqnAQylzCf0XFczsuLFoBm3VpAiCtMHDF5CpwakEPslRu2Yu1bLf5jEy 6y4il1sVBfMHchi4XIGdHunoqLBiAhapdcCDRzezI8thh5BMzE2l9Qy2kZEiAMs18zTC MdZw== X-Gm-Message-State: ALoCoQlsiNrd3GtKmTtFdzYfBkSI8rBTmtUjwxJa8ZqTBCAQh07zcOqQ5hMrPR1CzupDTQKNxRFs MIME-Version: 1.0 X-Received: by 10.31.185.6 with SMTP id j6mr156698vkf.98.1445357023108; Tue, 20 Oct 2015 09:03:43 -0700 (PDT) Received: by 10.31.153.134 with HTTP; Tue, 20 Oct 2015 09:03:42 -0700 (PDT) Received: by 10.31.153.134 with HTTP; Tue, 20 Oct 2015 09:03:42 -0700 (PDT) In-Reply-To: References: <20151018001317.GD87252@strugglingcoder.info> Date: Tue, 20 Oct 2015 09:03:42 -0700 Message-ID: From: Yuchung Cheng To: hiren panchasara , "tcpm@ietf.org Extensions" Content-Type: multipart/alternative; boundary=001a113bf89ef004b605228b67b5 Archived-At: Subject: Re: [tcpm] Question to TCP implementors regarding dupack threshold X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 16:03:45 -0000 --001a113bf89ef004b605228b67b5 Content-Type: text/plain; charset=UTF-8 +tcpm On Oct 19, 2015 4:03 PM, "Yuchung Cheng" wrote: > On Sat, Oct 17, 2015 at 5:13 PM, hiren panchasara > wrote: > > > > On 10/17/15 at 09:48P, Yuchung Cheng wrote: > > > Hi, > > > > > > The question is: > > > > > > When a TCP sender receives a DUPACK with SACK option(s) covering at > least > > > three packets (or three MSS bytes), will it trigger fast recovery > > > immediately? > > > > > > I know Linux does (for at least +8 years). I am curious about other > > > implementations like Windows, Mac, *BSD? > > > > FreeBSD doesn't do this currently. Looks like a sensible thing to do. > > yes I agree it's nice to have that. It's more robust against ACK > suppression discussed in the other threads, and ack loss in general. > > Anyone know about Windows and Mac? > > > > > > > Cheers, > > Hiren > --001a113bf89ef004b605228b67b5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

+tcpm

On Oct 19, 2015 4:03 PM, "Yuchung Cheng&quo= t; <ycheng@google.com> wrote= :
On Sat, Oct 17, 20= 15 at 5:13 PM, hiren panchasara
<hiren@strugglingcoder.inf= o> wrote:
>
> On 10/17/15 at 09:48P, Yuchung Cheng wrote:
> > Hi,
> >
> > The question is:
> >
> > When a TCP sender receives a DUPACK with SACK option(s) covering = at least
> > three packets (or three MSS bytes), will it trigger fast recovery=
> > immediately?
> >
> > I know Linux does (for at least +8 years). I am curious about oth= er
> > implementations like Windows, Mac, *BSD?
>
> FreeBSD doesn't do this currently. Looks like a sensible thing to = do.

yes I agree it's nice to have that. It's more robust against ACK suppression discussed in the other threads, and ack loss in general.

Anyone know about Windows and Mac?

>
>
> Cheers,
> Hiren
--001a113bf89ef004b605228b67b5-- From nobody Tue Oct 20 09:54:17 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B31B1A92EE for ; Tue, 20 Oct 2015 09:54:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.001 X-Spam-Level: X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ES10BSa9HVOf for ; Tue, 20 Oct 2015 09:54:10 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0797.outbound.protection.outlook.com [IPv6:2a01:111:f400:fc10::797]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B90C21A92F1 for ; Tue, 20 Oct 2015 09:53:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=aHlHVGSYar4UmdpmPnpczGJ3DIOpCwyVL13NJsjH0ww=; b=EfHYtQRa1jkeixQO5yntHnMmuySoZjRiCY9e0ijld1G6MJyIDUg97rRC8/vihcpFZ7+N0zqICcunEY50SBbLdzO8hRvRwzpdwSpKHQQn4aAGf9wkKDPI1K1AoG7MdpzqrrMt9htDBdCvkFwu1r719D6tN9fLWRPSAq5co3VtOyE= Received: from BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) by BN1PR03MB007.namprd03.prod.outlook.com (10.255.224.37) with Microsoft SMTP Server (TLS) id 15.1.306.13; Tue, 20 Oct 2015 16:53:24 +0000 Received: from BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.153]) by BN1PR03MB008.namprd03.prod.outlook.com ([169.254.7.153]) with mapi id 15.01.0293.007; Tue, 20 Oct 2015 16:53:23 +0000 From: Praveen Balasubramanian To: Yuchung Cheng , hiren panchasara , "tcpm@ietf.org Extensions" Thread-Topic: [tcpm] Question to TCP implementors regarding dupack threshold Thread-Index: AQHRCPu3MsAMRuiJC0WGw1AhrPWLAp5wYZuAgAQuXYWAAA2lgA== Date: Tue, 20 Oct 2015 16:53:23 +0000 Message-ID: References: <20151018001317.GD87252@strugglingcoder.info> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com; x-originating-ip: [2001:4898:80e8::584] x-microsoft-exchange-diagnostics: 1; BN1PR03MB007; 5:Vp0BTm3lQzfLKRSY1rFN4juOClhdJI6mwMVwXCKMtk0ib1RtST+yEHAtKxuW51C+9CF7bX7kmqg+NAxifY9qW/xhG3PAaJHK6gjohVvQ/77Dx0WKflSLLkpWp28z/2W0zTaJgEdIFwtx+6XPLtppew==; 24:affxzKvPi53FK2JPKM/QNE8uCMo4BxRjecNlKXwpq49CSByL6BmAIH4KinSBRLVv7s1BJaUkJUYAd2jF+xJPFFgwdLejltLoboSx8y3bxqw=; 20:eztP45RspggKbSyiebeZI4djHBhZETlU051NKJ3f7f/0oC2sCefStdTanDmONfzwJXVelkB4AnleZVZY09Dz3Q== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN1PR03MB007; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(108003899814671); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425024)(601004)(2401047)(520078)(5005006)(8121501046)(3002001)(61426024)(61427024); SRVR:BN1PR03MB007; BCL:0; PCL:0; RULEID:; SRVR:BN1PR03MB007; x-forefront-prvs: 073515755F x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(377454003)(189002)(479174004)(199003)(24454002)(86362001)(15975445007)(102836002)(92566002)(2900100001)(54356999)(76176999)(2950100001)(86612001)(122556002)(46102003)(16236675004)(40100003)(5008740100001)(93886004)(10290500002)(5005710100001)(10400500002)(76576001)(50986999)(8990500004)(106356001)(106116001)(105586002)(99286002)(74316001)(19609705001)(19580395003)(19580405001)(64706001)(5001920100001)(81156007)(33656002)(97736004)(5001770100001)(19625215002)(5002640100001)(5001960100002)(189998001)(107886002)(87936001)(5003600100002)(5004730100002)(10090500001)(5007970100001)(19300405004)(101416001)(3826002)(6606295002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR03MB007; H:BN1PR03MB008.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: multipart/alternative; boundary="_000_BN1PR03MB0087872AB69156D9095270CB6390BN1PR03MB008namprd_" MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Oct 2015 16:53:23.6692 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR03MB007 Archived-At: Subject: Re: [tcpm] Question to TCP implementors regarding dupack threshold X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 16:54:14 -0000 --_000_BN1PR03MB0087872AB69156D9095270CB6390BN1PR03MB008namprd_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 V2luZG93cyBkb2VzIG5vdCBkbyB0aGlzIHRvZGF5Lg0KDQpGcm9tOiB0Y3BtIFttYWlsdG86dGNw bS1ib3VuY2VzQGlldGYub3JnXSBPbiBCZWhhbGYgT2YgWXVjaHVuZyBDaGVuZw0KU2VudDogVHVl c2RheSwgT2N0b2JlciAyMCwgMjAxNSA5OjA0IEFNDQpUbzogaGlyZW4gcGFuY2hhc2FyYSA8aGly ZW5Ac3RydWdnbGluZ2NvZGVyLmluZm8+OyB0Y3BtQGlldGYub3JnIEV4dGVuc2lvbnMgPHRjcG1A aWV0Zi5vcmc+DQpTdWJqZWN0OiBSZTogW3RjcG1dIFF1ZXN0aW9uIHRvIFRDUCBpbXBsZW1lbnRv cnMgcmVnYXJkaW5nIGR1cGFjayB0aHJlc2hvbGQNCg0KDQordGNwbQ0KT24gT2N0IDE5LCAyMDE1 IDQ6MDMgUE0sICJZdWNodW5nIENoZW5nIiA8eWNoZW5nQGdvb2dsZS5jb208bWFpbHRvOnljaGVu Z0Bnb29nbGUuY29tPj4gd3JvdGU6DQpPbiBTYXQsIE9jdCAxNywgMjAxNSBhdCA1OjEzIFBNLCBo aXJlbiBwYW5jaGFzYXJhDQo8aGlyZW5Ac3RydWdnbGluZ2NvZGVyLmluZm88bWFpbHRvOmhpcmVu QHN0cnVnZ2xpbmdjb2Rlci5pbmZvPj4gd3JvdGU6DQo+DQo+IE9uIDEwLzE3LzE1IGF0IDA5OjQ4 UCwgWXVjaHVuZyBDaGVuZyB3cm90ZToNCj4gPiBIaSwNCj4gPg0KPiA+IFRoZSBxdWVzdGlvbiBp czoNCj4gPg0KPiA+IFdoZW4gYSBUQ1Agc2VuZGVyIHJlY2VpdmVzIGEgRFVQQUNLIHdpdGggU0FD SyBvcHRpb24ocykgY292ZXJpbmcgYXQgbGVhc3QNCj4gPiB0aHJlZSBwYWNrZXRzIChvciB0aHJl ZSBNU1MgYnl0ZXMpLCB3aWxsIGl0IHRyaWdnZXIgZmFzdCByZWNvdmVyeQ0KPiA+IGltbWVkaWF0 ZWx5Pw0KPiA+DQo+ID4gSSBrbm93IExpbnV4IGRvZXMgKGZvciBhdCBsZWFzdCArOCB5ZWFycyku IEkgYW0gY3VyaW91cyBhYm91dCBvdGhlcg0KPiA+IGltcGxlbWVudGF0aW9ucyBsaWtlIFdpbmRv d3MsIE1hYywgKkJTRD8NCj4NCj4gRnJlZUJTRCBkb2Vzbid0IGRvIHRoaXMgY3VycmVudGx5LiBM b29rcyBsaWtlIGEgc2Vuc2libGUgdGhpbmcgdG8gZG8uDQoNCnllcyBJIGFncmVlIGl0J3Mgbmlj ZSB0byBoYXZlIHRoYXQuIEl0J3MgbW9yZSByb2J1c3QgYWdhaW5zdCBBQ0sNCnN1cHByZXNzaW9u IGRpc2N1c3NlZCBpbiB0aGUgb3RoZXIgdGhyZWFkcywgYW5kIGFjayBsb3NzIGluIGdlbmVyYWwu DQoNCkFueW9uZSBrbm93IGFib3V0IFdpbmRvd3MgYW5kIE1hYz8NCg0KPg0KPg0KPiBDaGVlcnMs DQo+IEhpcmVuDQo= --_000_BN1PR03MB0087872AB69156D9095270CB6390BN1PR03MB008namprd_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m YWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAy IDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNvTm9ybWFsLCBsaS5Nc29Ob3JtYWws IGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBpbjsNCgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJ Zm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZjt9 DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCglj b2xvcjpibHVlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0KYTp2aXNpdGVkLCBzcGFu Lk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjpw dXJwbGU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQpwDQoJe21zby1zdHlsZS1wcmlv cml0eTo5OTsNCgltc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzsNCgltYXJnaW4tcmlnaHQ6MGluOw0K CW1zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvOw0KCW1hcmdpbi1sZWZ0OjBpbjsNCglmb250LXNp emU6MTIuMHB0Ow0KCWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmO30NCnAubXNv bm9ybWFsMCwgbGkubXNvbm9ybWFsMCwgZGl2Lm1zb25vcm1hbDANCgl7bXNvLXN0eWxlLW5hbWU6 bXNvbm9ybWFsOw0KCW1zby1tYXJnaW4tdG9wLWFsdDphdXRvOw0KCW1hcmdpbi1yaWdodDowaW47 DQoJbXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87DQoJbWFyZ2luLWxlZnQ6MGluOw0KCWZvbnQt c2l6ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2VyaWY7fQ0Kc3Bh bi5FbWFpbFN0eWxlMTkNCgl7bXNvLXN0eWxlLXR5cGU6cGVyc29uYWwtcmVwbHk7DQoJZm9udC1m YW1pbHk6IkNhbGlicmkiLHNhbnMtc2VyaWY7DQoJY29sb3I6IzFGNDk3RDt9DQouTXNvQ2hwRGVm YXVsdA0KCXttc28tc3R5bGUtdHlwZTpleHBvcnQtb25seTsNCglmb250LWZhbWlseToiQ2FsaWJy aSIsc2Fucy1zZXJpZjt9DQpAcGFnZSBXb3JkU2VjdGlvbjENCgl7c2l6ZTo4LjVpbiAxMS4waW47 DQoJbWFyZ2luOjEuMGluIDEuMGluIDEuMGluIDEuMGluO30NCmRpdi5Xb3JkU2VjdGlvbjENCgl7 cGFnZTpXb3JkU2VjdGlvbjE7fQ0KLS0+PC9zdHlsZT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4N CjxvOnNoYXBlZGVmYXVsdHMgdjpleHQ9ImVkaXQiIHNwaWRtYXg9IjEwMjYiIC8+DQo8L3htbD48 IVtlbmRpZl0tLT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlbGF5b3V0IHY6ZXh0 PSJlZGl0Ij4NCjxvOmlkbWFwIHY6ZXh0PSJlZGl0IiBkYXRhPSIxIiAvPg0KPC9vOnNoYXBlbGF5 b3V0PjwveG1sPjwhW2VuZGlmXS0tPg0KPC9oZWFkPg0KPGJvZHkgbGFuZz0iRU4tVVMiIGxpbms9 ImJsdWUiIHZsaW5rPSJwdXJwbGUiPg0KPGRpdiBjbGFzcz0iV29yZFNlY3Rpb24xIj4NCjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5 OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEIj5XaW5kb3dzIGRv ZXMgbm90IGRvIHRoaXMgdG9kYXkuDQo8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0i TXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVv dDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286 cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PGI+PHNwYW4gc3R5bGU9ImZvbnQt c2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj5G cm9tOjwvc3Bhbj48L2I+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6 JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj4gdGNwbSBbbWFpbHRvOnRjcG0tYm91bmNl c0BpZXRmLm9yZ10NCjxiPk9uIEJlaGFsZiBPZiA8L2I+WXVjaHVuZyBDaGVuZzxicj4NCjxiPlNl bnQ6PC9iPiBUdWVzZGF5LCBPY3RvYmVyIDIwLCAyMDE1IDk6MDQgQU08YnI+DQo8Yj5Ubzo8L2I+ IGhpcmVuIHBhbmNoYXNhcmEgJmx0O2hpcmVuQHN0cnVnZ2xpbmdjb2Rlci5pbmZvJmd0OzsgdGNw bUBpZXRmLm9yZyBFeHRlbnNpb25zICZsdDt0Y3BtQGlldGYub3JnJmd0Ozxicj4NCjxiPlN1Ympl Y3Q6PC9iPiBSZTogW3RjcG1dIFF1ZXN0aW9uIHRvIFRDUCBpbXBsZW1lbnRvcnMgcmVnYXJkaW5n IGR1cGFjayB0aHJlc2hvbGQ8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9y bWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwPiYjNDM7dGNwbTxvOnA+PC9vOnA+PC9wPg0K PGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPk9uIE9jdCAxOSwgMjAxNSA0OjAzIFBNLCAmcXVv dDtZdWNodW5nIENoZW5nJnF1b3Q7ICZsdDs8YSBocmVmPSJtYWlsdG86eWNoZW5nQGdvb2dsZS5j b20iPnljaGVuZ0Bnb29nbGUuY29tPC9hPiZndDsgd3JvdGU6PG86cD48L286cD48L3A+DQo8Ymxv Y2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBw dDtwYWRkaW5nOjBpbiAwaW4gMGluIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi1yaWdo dDowaW4iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+T24gU2F0LCBPY3QgMTcsIDIwMTUgYXQgNTox MyBQTSwgaGlyZW4gcGFuY2hhc2FyYTxicj4NCiZsdDs8YSBocmVmPSJtYWlsdG86aGlyZW5Ac3Ry dWdnbGluZ2NvZGVyLmluZm8iPmhpcmVuQHN0cnVnZ2xpbmdjb2Rlci5pbmZvPC9hPiZndDsgd3Jv dGU6PGJyPg0KJmd0Ozxicj4NCiZndDsgT24gMTAvMTcvMTUgYXQgMDk6NDhQLCBZdWNodW5nIENo ZW5nIHdyb3RlOjxicj4NCiZndDsgJmd0OyBIaSw8YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZn dDsgVGhlIHF1ZXN0aW9uIGlzOjxicj4NCiZndDsgJmd0Ozxicj4NCiZndDsgJmd0OyBXaGVuIGEg VENQIHNlbmRlciByZWNlaXZlcyBhIERVUEFDSyB3aXRoIFNBQ0sgb3B0aW9uKHMpIGNvdmVyaW5n IGF0IGxlYXN0PGJyPg0KJmd0OyAmZ3Q7IHRocmVlIHBhY2tldHMgKG9yIHRocmVlIE1TUyBieXRl cyksIHdpbGwgaXQgdHJpZ2dlciBmYXN0IHJlY292ZXJ5PGJyPg0KJmd0OyAmZ3Q7IGltbWVkaWF0 ZWx5Pzxicj4NCiZndDsgJmd0Ozxicj4NCiZndDsgJmd0OyBJIGtub3cgTGludXggZG9lcyAoZm9y IGF0IGxlYXN0ICYjNDM7OCB5ZWFycykuIEkgYW0gY3VyaW91cyBhYm91dCBvdGhlcjxicj4NCiZn dDsgJmd0OyBpbXBsZW1lbnRhdGlvbnMgbGlrZSBXaW5kb3dzLCBNYWMsICpCU0Q/PGJyPg0KJmd0 Ozxicj4NCiZndDsgRnJlZUJTRCBkb2Vzbid0IGRvIHRoaXMgY3VycmVudGx5LiBMb29rcyBsaWtl IGEgc2Vuc2libGUgdGhpbmcgdG8gZG8uPGJyPg0KPGJyPg0KeWVzIEkgYWdyZWUgaXQncyBuaWNl IHRvIGhhdmUgdGhhdC4gSXQncyBtb3JlIHJvYnVzdCBhZ2FpbnN0IEFDSzxicj4NCnN1cHByZXNz aW9uIGRpc2N1c3NlZCBpbiB0aGUgb3RoZXIgdGhyZWFkcywgYW5kIGFjayBsb3NzIGluIGdlbmVy YWwuPGJyPg0KPGJyPg0KQW55b25lIGtub3cgYWJvdXQgV2luZG93cyBhbmQgTWFjPzxicj4NCjxi cj4NCiZndDs8YnI+DQomZ3Q7PGJyPg0KJmd0OyBDaGVlcnMsPGJyPg0KJmd0OyBIaXJlbjxvOnA+ PC9vOnA+PC9wPg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRt bD4NCg== --_000_BN1PR03MB0087872AB69156D9095270CB6390BN1PR03MB008namprd_-- From nobody Tue Oct 20 12:39:55 2015 Return-Path: X-Original-To: tcpm@ietf.org Delivered-To: tcpm@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 20A181B29F6; Tue, 20 Oct 2015 12:39:54 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: X-Test-IDTracker: no X-IETF-IDTracker: 6.6.1 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20151020193954.12955.50870.idtracker@ietfa.amsl.com> Date: Tue, 20 Oct 2015 12:39:54 -0700 Archived-At: Cc: tcpm@ietf.org Subject: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 19:39:54 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the TCP Maintenance and Minor Extensions Working Group of the IETF. Title : TCP and SCTP RTO Restart Authors : Per Hurtig Anna Brunstrom Andreas Petlund Michael Welzl Filename : draft-ietf-tcpm-rtorestart-09.txt Pages : 17 Date : 2015-10-20 Abstract: This document describes a modified sender-side algorithm for managing the TCP and SCTP retransmission timers that provides faster loss recovery when there is a small amount of outstanding data for a connection. The modification, RTO Restart (RTOR), allows the transport to restart its retransmission timer using a smaller delay, so that the effective RTO becomes more aggressive in situations where fast retransmit cannot be used. This enables faster loss detection and recovery for connections that are short-lived or application- limited. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-tcpm-rtorestart/ There's also a htmlized version available at: https://tools.ietf.org/html/draft-ietf-tcpm-rtorestart-09 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-rtorestart-09 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Wed Oct 21 00:55:31 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5EAE41ACDA1 for ; Wed, 21 Oct 2015 00:55:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.85 X-Spam-Level: X-Spam-Status: No, score=-3.85 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AnJ7shIfnxXc for ; Wed, 21 Oct 2015 00:55:28 -0700 (PDT) Received: from nasse.dc.kau.se (smtp.kau.se [193.10.220.39]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 096FB1ACD60 for ; Wed, 21 Oct 2015 00:55:27 -0700 (PDT) X-Spam-Processed: mail.kau.se, Wed, 21 Oct 2015 09:55:19 +0200 (not processed: spam filter heuristic analysis disabled) X-MDRemoteIP: 193.157.119.161 X-MDHelo: eduroam-193-157-119-161.uio.no X-MDArrival-Date: Wed, 21 Oct 2015 09:55:19 +0200 X-Authenticated-Sender: per.hurtig@kau.se X-Return-Path: per.hurtig@kau.se X-Envelope-From: per.hurtig@kau.se Mime-Version: 1.0 (Mac OS X Mail 9.0 \(3094\)) Content-Type: multipart/signed; boundary="Apple-Mail=_3240B8A1-FF4E-4749-B49A-7574939CF5B7"; protocol="application/pgp-signature"; micalg=pgp-sha256 X-Pgp-Agent: GPGMail 2.6b2 From: Per Hurtig In-Reply-To: <20151020193954.12955.50870.idtracker@ietfa.amsl.com> Date: Wed, 21 Oct 2015 09:55:11 +0200 Message-Id: <3500BC2B-A986-4AC8-BEF0-4709DEFE5D35@kau.se> References: <20151020193954.12955.50870.idtracker@ietfa.amsl.com> To: "tcpm@ietf.org Extensions" X-Mailer: Apple Mail (2.3094) Archived-At: Cc: bclaise@cisco.com, barryleiba@computer.org Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 07:55:30 -0000 --Apple-Mail=_3240B8A1-FF4E-4749-B49A-7574939CF5B7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, the new draft on RTO restart address the following comments from the = IESG (thanks for the feedback): o Clarified, in the abstract, that the modified restart causes a = smaller retransmission delay in total. o Clarified, in the introduction, that the fast retransmit algorithm = may cause retransmissions upon receiving duplicate acknowledgments, not that it unconditionally = does so. o Changed wording from "to proposed standard" to "to the standards = track". o Changed algorithm description so that a TCP sender MUST track the = time elapsed since the transmission of the earliest outstanding segment. This was not = explicitly stated in previous versions of the draft. Cheers, Per > On 20 Oct 2015, at 21:39, internet-drafts@ietf.org wrote: >=20 >=20 > A New Internet-Draft is available from the on-line Internet-Drafts = directories. > This draft is a work item of the TCP Maintenance and Minor Extensions = Working Group of the IETF. >=20 > Title : TCP and SCTP RTO Restart > Authors : Per Hurtig > Anna Brunstrom > Andreas Petlund > Michael Welzl > Filename : draft-ietf-tcpm-rtorestart-09.txt > Pages : 17 > Date : 2015-10-20 >=20 > Abstract: > This document describes a modified sender-side algorithm for = managing > the TCP and SCTP retransmission timers that provides faster loss > recovery when there is a small amount of outstanding data for a > connection. The modification, RTO Restart (RTOR), allows the > transport to restart its retransmission timer using a smaller delay, > so that the effective RTO becomes more aggressive in situations = where > fast retransmit cannot be used. This enables faster loss detection > and recovery for connections that are short-lived or application- > limited. >=20 >=20 > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-tcpm-rtorestart/ >=20 > There's also a htmlized version available at: > https://tools.ietf.org/html/draft-ietf-tcpm-rtorestart-09 >=20 > A diff from the previous version is available at: > https://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-tcpm-rtorestart-09 >=20 >=20 > Please note that it may take a couple of minutes from the time of = submission > until the htmlized version and diff are available at tools.ietf.org. >=20 > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm --Apple-Mail=_3240B8A1-FF4E-4749-B49A-7574939CF5B7 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJWJ0TlAAoJEMzXVkqMT/z25aYH/jb1TCgxAVuQ5/bwJkgZI3gx zBBYg5fIZGhMCrRtCl+kfV1DklRhdYHTNDGq2NPfJUES7h6Krs7TaSEa8JUwa6Gd LOdvw+p5w5tXjDsndqLDp1HrNFR9OSNvnCRAegI9gxDxtgpPqqyeTJdlitO2Sp51 gYZWAT7l2VDRf4lRkym9CS0tGNLxZCQvsrVMaGAFJ/D4ZwWDEBi2iK6t6bF67ICh mXom0BAqdosp3xo3oCuFq0Fka6GtS6ggoAbnP1yoqHy4twjKta1CQWxO0OpvpEyU P8Iy3SNjQ5ot2IoUUUODVzKpCrteUkvFem9bHvv+tyvVoq7AnRT8oZ70DUNCrCA= =/N5I -----END PGP SIGNATURE----- --Apple-Mail=_3240B8A1-FF4E-4749-B49A-7574939CF5B7-- From nobody Wed Oct 21 04:43:40 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DA07D1A8BBE for ; Wed, 21 Oct 2015 04:43:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WF-0STgSgPfy for ; Wed, 21 Oct 2015 04:43:38 -0700 (PDT) Received: from fruitcake.ICSI.Berkeley.EDU (fruitcake.ICSI.Berkeley.EDU [192.150.186.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9B8541A88DA for ; Wed, 21 Oct 2015 04:43:38 -0700 (PDT) Received: from lawyers.icir.org (envoy.icir.org [192.150.187.30]) by fruitcake.ICSI.Berkeley.EDU (8.12.11.20060614/8.12.11) with ESMTP id t9LBhb9D028971 for ; Wed, 21 Oct 2015 04:43:37 -0700 (PDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id 8EA3C2F36BB8 for ; Wed, 21 Oct 2015 07:43:38 -0400 (EDT) To: tcpm@ietf.org From: Mark Allman Organization: International Computer Science Institute (ICSI) Song-of-the-Day: Hey Tonight X-URL-0: http://www.icir.org/mallman-files/Document94592.jpg X-URL-1: http://www.icir.org/mallman-files/Document44313.xls MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-------459435943823349593450"; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Wed, 21 Oct 2015 07:43:38 -0400 Message-ID: <57888.1445427818@lawyers.icir.org> Sender: mallman@icir.org Archived-At: Subject: [tcpm] rto-consider updated X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: mallman@icir.org List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 11:43:40 -0000 --=-------459435943823349593450 Content-Type: text/plain Folks- I have been cleaning up some old items on my 'todo' list. One of them is an old draft I wrote on RTO considerations. The idea is to provide a set of high-level guidelines that estimators must follow, but that leaves them free to set low-level details without being standardized. My view is this is mostly writing down what happens now. When I originally posted the draft several years ago it garnered little traction. My todo is to issue this as a tech report so that it lives somewhere (and I can reference the idea and such). But, I figured before I did that I'd throw it up as an I-D again and make sure folks were not interested. So, I tried to to do that yesterday, but we are in the "no I-D postings" period. So, the (very short) I-D is here: http://www.icir.org/mallman/pubs/All15a/ I have a bit set to submit it when submissions are again accepted (Nov 1). Comments welcome, of course. If the WG is interested, great. If not, that is fine and I won't say any more about it. Thanks! allman -- http://www.icir.org/mallman/ --=-------459435943823349593450 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlYnemoACgkQWyrrWs4yIs7lbACfQ+yZL+3nio/txXMLrifl7YFL DBsAn1AUerWJRDSOdabWhpNWRGy36wrg =gywX -----END PGP SIGNATURE----- --=-------459435943823349593450-- From nobody Wed Oct 21 05:59:08 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F14491A6FB1 for ; Wed, 21 Oct 2015 05:59:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.253 X-Spam-Level: * X-Spam-Status: No, score=1.253 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FRT_BELOW2=2.154, J_CHICKENPOX_36=0.6, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OzzstMjovN2w for ; Wed, 21 Oct 2015 05:59:01 -0700 (PDT) Received: from sesbmg23.ericsson.net (sesbmg23.ericsson.net [193.180.251.37]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0843E1A6F83 for ; Wed, 21 Oct 2015 05:58:59 -0700 (PDT) X-AuditID: c1b4fb25-f79a26d00000149a-29-56278c115f92 Received: from ESESSHC018.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg23.ericsson.net (Symantec Mail Security) with SMTP id 23.BA.05274.11C87265; Wed, 21 Oct 2015 14:58:58 +0200 (CEST) Received: from ESESSMB205.ericsson.se ([169.254.5.167]) by ESESSHC018.ericsson.se ([153.88.183.72]) with mapi id 14.03.0248.002; Wed, 21 Oct 2015 14:58:57 +0200 From: Ingemar Johansson S To: Bob Briscoe , Veaceslav ROMAN , Eric Dumazet Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8Pq87vsQ9Xnw1wDq87cGANXnMWygq7idKADXZ9ZHAA== Date: Wed, 21 Oct 2015 12:58:56 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA34C09584@ESESSMB205.ericsson.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com> <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E863AD@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD89C3@ESESSMB205.ericsson.se> <561FC4DA.3020905@bobbriscoe.net> In-Reply-To: <561FC4DA.3020905@bobbriscoe.net> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.20] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrFIsWRmVeSWpSXmKPExsUyM+Jvja5Qj3qYwcfJvBZPjz1it9h9biqL xb4ZExktHk5rYrQ4cuUYo8W2k/OZLN7/38jswO7x6v4FVo+ds+6yeyzYVOqxZMlPJo8ly54y enQc38AcwBbFZZOSmpNZllqkb5fAlbGu9wNbwep1TBU9uy+wNTD+/cDYxcjJISFgInH39Q1m CFtM4sK99WxdjFwcQgJHGSVOz9vIBOEsYZSYtWYSE0gVm4CNxMpD38G6RQRqJI4cf8oIUsQs 8JxRYsnadywgCWEBM4lbTe+YIYrMJSadbmMGKRIROMcoseDcZVaQBIuAqsTTRV/ZQGxeAV+J jgMnGCHW/WaX6H29hB0kwSmgJ/FjFsRqRgFZifvf74FtYBYQl7j1ZD4TxOECEkv2nId6QlTi 5eN/rBC2osTV6cuZIOp1JBbs/sQGYWtLLFv4mhlisaDEyZlPWCYwis1CMnYWkpZZSFpmIWlZ wMiyilG0OLU4KTfdyFgvtSgzubg4P08vL7VkEyMwRg9u+a26g/HyG8dDjAIcjEo8vAk71cKE WBPLiitzDzFKcDArifBOSVQPE+JNSaysSi3Kjy8qzUktPsQozcGiJM7bzPQgVEggPbEkNTs1 tSC1CCbLxMEp1cDI+FWj4YCI9G2T4JsurJYvzxWv9rnBNXNr2brP1748VUsvf2JsdWjndf8P pywuXmgxfb/swXrHBLWAwBntjGaXVx5P+Pr6k3FobZarxGSpmRXV0g8+Ll6hGzfXysnpNs/h k8uzX7xWe/pKYJ7l8sOpy3ifRPPfCF3WU9R5SHHvHInKleuXsKotVGIpzkg01GIuKk4EAEBc P6XNAgAA Archived-At: Cc: Ingemar Johansson S , "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , "end2end-interest@postel.org" Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 12:59:07 -0000 Hi Bob Yes, path chirp and RAPID is quite interesting, one thing that I wonder abo= ut is how this will work in cases where access technology adds jitter. =20 One such example is LTE, whose properties I tried to outline in=20 https://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-00=20 Promised to come back with simulation results on different HyStart threshol= d values,=20 Time is however a sparse commodity it seems, and besides that I feel I need= to back up possible simulation results with some real data. /Ingemar > -----Original Message----- > From: Bob Briscoe [mailto:research@bobbriscoe.net] > Sent: den 15 oktober 2015 17:23 > To: Ingemar Johansson S; Veaceslav ROMAN; Eric Dumazet > Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; end2end- > interest@postel.org > Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment >=20 > Ingemar, >=20 > On 01/10/15 18:02, Ingemar Johansson S wrote: > > Hi > > > > An increased threshold would probably solve the problem and is possibly > the quick fix. > > I will anyway need to rerun parts of my simulations because of my own > 32ms mistake, it is quite straightforward to add a few alternative thresh= old > values in the same work. > > > > Wonder though if one can do it smarter?. > > For instance, assuming that the timestamp option is enabled, it should > there be possible to infer if ACKs belong to the same HyStart round, base= d > on TSval. > > It of course adds some extra lines of code to the HyStart algorithm but= then > one can perhaps keep the current thresholds ?. > Re: doing it smarter, Mirja & I wrote up the following about how to use h= igh > precision timestamps in Linux to rapidly get info on available capacity w= ith > minimum intrusiveness using packet chirps (based on the ideas in pathchir= p > and TCP RAPID): > > The paper doesn't focus particularly on slow-start, but that's usually th= e most > useful case. >=20 > BTW, we were (mis)using TCP timestamps to infer one-way delay, so we had > to arrange for both ends to be writing high precision timestamps into the= TS > option. But if you are happy with round-trip delay, host A can put a high > precision timestamp in the TSval field, and the receiver will dumbly refl= ect it > in the TSecr field without trying to understand its precision. See S.3.1. >=20 >=20 > Bob > > > > /Ingemar > > > >> -----Original Message----- > >> From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] > >> Sent: den 1 oktober 2015 17:17 > >> To: Eric Dumazet; Ingemar Johansson S > >> Cc: Eric Dumazet; Neal Cardwell; tcpm@ietf.org; Piers O'Hanlon; > >> Sangtae Ha; end2end-interest@postel.org > >> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment > >> > >> Sure 10 ms will be better as more of the LTE technological jumps will > >> be covered and, at least, will help get rid of exit at the minimum > >> exit cwnd (29, which at 20 ms RTT is ~16Mbps) and will shift to 49 > >> (~28 Mbps) or to 89 (~ 50 Mbps). > >> We have all necessary to repeat those tests. > >> > >> Veaceslav Roman > >> > >> > >> -----Original Message----- > >> From: Eric Dumazet [mailto:edumazet@google.com] > >> Sent: Thursday, 01 October 2015 15:44 > >> To: Ingemar Johansson S > >> Cc: Veaceslav ROMAN; Eric Dumazet; Neal Cardwell; tcpm@ietf.org; > >> Piers O'Hanlon; Sangtae Ha; end2end-interest@postel.org > >> Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment > >> > >> Yes, this 4ms value problem is even more visible with TCP pacing (as > >> implemented with linux sch_fq packet scheduler) because of apparent > >> delays caused by sojourn time in packet scheduler itself. > >> TCP does not offset this sojourn time (as RTT is also used for RTO > >> determination, better have an inflated view) > >> > >> Is switching to 10 ms would be better in your LTE tests ? > >> > >> > >> > >> On Thu, Oct 1, 2015 at 4:57 AM, Ingemar Johansson S > >> wrote: > >>> Hmm > >>> You are right !. > >>> Well... That of course should explain why HyStart performs badly for > LTE... > >> Atleast it is part of the reason. > >>> /Ingemar > >>> > >>> > >>> > >>>> -----Original Message----- > >>>> From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] > >>>> Sent: den 1 oktober 2015 13:46 > >>>> To: Ingemar Johansson S; Eric Dumazet; Neal Cardwell > >>>> Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Eric Dumazet; > >>>> end2end- interest@postel.org > >>>> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment > >>>> > >>>> Well this say > >>>> If curr_rtt is higher than delay_min + (1/8 * delay_min or 4 > >>>> ms, whichever is higher) then exit slow start. > >>>> > >>>> When actual minimum delay is less than 32 ms the delay_min will be > >>>> less than 32<<3, then the delay_min>>3 will be less than 32 and > >>>> will compare with > >>>> 4<<3 which is 32 and therefor the threshold will be > >>>> (real_curr_rtt<<3) > > >>>> (real_delay<<3 + 4<<3) or (real_curr_rtt)>(real_delay +4) > >>>> > >>>> > >>>> Veaceslav Roman > >>>> > >>>> -----Original Message----- > >>>> From: Ingemar Johansson S > [mailto:ingemar.s.johansson@ericsson.com] > >>>> Sent: Thursday, 01 October 2015 13:00 > >>>> To: Veaceslav ROMAN; Eric Dumazet; Neal Cardwell > >>>> Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Eric Dumazet; > >>>> end2end- interest@postel.org; Ingemar Johansson S > >>>> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment > >>>> > >>>> Hi > >>>> > >>>> Yes, the variable delay_min is in Q3 for reasons of precision I > >>>> guess But the call to HYSTART_DELAY_THRESH is with delay_min > >>>> shifted down to Q0 > >>>> > >>>> HYSTART_DELAY_THRESH(ca->delay_min >> 3)) (see http://lxr.free- > >>>> electrons.com/source/net/ipv4/tcp_cubic.c#L403 ) I may be off here.. > >>>> > >>>> Regarding pacing. I don't have real data to show the effects of > >>>> pacing, in simulations it is however possible to see the gains in > >>>> terms of less coalescing, yes pacing does not solve the ACK > >>>> compression but it seem to solve the follow up effect that ACK > >> compression gives. > >>>> /Ingemar > >>>> > >>>>> -----Original Message----- > >>>>> From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] > >>>>> Sent: den 1 oktober 2015 10:32 > >>>>> To: Ingemar Johansson S; Eric Dumazet; Neal Cardwell > >>>>> Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Eric Dumazet; > >>>>> end2end- interest@postel.org > >>>>> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment > >>>>> > >>>>> Yes, HYSTART_DELAY_MIN (4U<<3) . > >>>>> But curr_rtt and delay_min come from bictcp_acked which is also <<3 > : > >>>>> In tcp_cubic.c, function line 433 (kernel 4.2) > >>>>> delay =3D (rtt_us << 3) / USEC_PER_MSEC; So, > >>>>> 4U<<3 is equivalent to 4 ms . > >>>>> > >>>>> Regarding pacing I am not sure will help the performance: there > >>>>> are gaps in ACK trains of the order of milliseconds and this is a > >>>>> lost time and there is no way to compensate it. If introduce ACK > >>>>> pacing or data packets pacing this would add to the overall lost ti= me. > >>>>> Pacing can help only to alleviate a potential problem of some > >>>>> downstream routers with insufficient buffers which may introduce > >>>>> unexpected packets drops when large train sent at a high speed as > >>>>> a reply to highly compressed ACK train, but, on another hands. > >>>>> But how much pacing and how to accommodate to largely changing > >>>>> radio conditions ? And with permanently changing technology and > >> speed ? > >>>>> Anyway the pacing will add to the problem of gaps and better size > >>>>> properly routers buffers. > >>>>> > >>>>> Ideally ACK train is split in 2 parts, but in practice I do see mor= e. > >>>>> But as the stream progress the less gaps. The bigger the downlink > >>>>> pressure the more ACK and less chances to have a pause in ACK > >>>>> sending, then there will be a continuous flow of Buffer Status > >>>>> Reports and less of the Scheduling Request (SR, "Hey, I have some > >>>>> data to send"). The only issue I am not sure is on the potential > >>>>> gaps in other networks. I did not put bellow all details, but you > >>>>> should know that device can not send SR in arbitrary moments of > >>>>> time but in predefined for him intervals. In my network this > >>>>> period is 5 ms, therefore the RTT of the first ACK in the train > >>>>> will have variations of 2.5 ms. But standards admit this period to > >>>>> be also 10 ms ,20 ms, 40 ms, 80 ms and I can not say what is the > >>>>> practice of others. I may only guess that most will try to use > >>>> the best and fastest, i.e. 5 ms. > >>>>> Anyway, to get a realistic modeling of LTE interaction with TCP > >>>>> the detailed MAC scheduler model is needed. Unfortunately, most of > >>>>> the literature focus on scheduling of the Physical Resource > >>>>> Blocks, Adaptive Modulation and Coding, MIMO, i.e., focusing on > >>>>> the distribution of resources between users, which is good and > >>>>> needed, but do not address very much the timing interaction with > >>>>> TCP. At least, I never met in LTE scheduling literature a term "TC= P self- > clocking". > >>>>> > >>>>> The 8 ms HYSTART_DELAY_MIN could only be a simple solution for > >>>>> the problem of counting of the round' curr_rtt from the second ACK > >>>>> of the train (as the second ACK RTT will be, typically, due to > >>>>> scheduling, > >>>>> 5-7 ms longer than the first one), but this only if SR periodicity = is 5 ms. > >>>>> The other 2 problems will remain unsolved. > >>>>> > >>>>> Veaceslav Roman > >>>>> > >>>>> -----Original Message----- > >>>>> From: Ingemar Johansson S > >> [mailto:ingemar.s.johansson@ericsson.com] > >>>>> Sent: Thursday, 01 October 2015 08:59 > >>>>> To: Veaceslav ROMAN; Eric Dumazet; Neal Cardwell > >>>>> Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Eric Dumazet; > >>>>> end2end- interest@postel.org; Ingemar Johansson S > >>>>> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment > >>>>> > >>>>> Hi > >>>>> > >>>>> Thanks for the detailed report. What seems unclear to me is the > >>>>> value of HYSTART_DELAY_MIN. You mention 8ms but when I look into > >>>>> the 4.2 kernel code I get the feeling that it is 32ms ( #define > >> HYSTART_DELAY_MIN > >>>>> (4U<<3) ). Is this a mistake from my side ? > >>>>> I have run LTE simulations and tried to wring this HyStart issue > >>>>> inside out and upside down but sofar it has been very difficult to > >>>>> make HyStart exit prematurely but then of course I have run with > >>>>> HYSTART_DELAY_MIN =3D 32ms. > >>>>> > >>>>> As regards to the ACK-compression problem, yes, ACK compression > >>>>> easily occur in LTE, even in simulations. What I have seen is that > >>>>> packet pacing is a very efficient remedy, for instance the packet > >>>>> implemented in QUIC seems to solve ACK compression issues very > well. > >>>>> > >>>>> It is interesting what you say that ACK trains are so much split > >>>>> up, I could understand that they are split up in two parts > >>>>> (initial grant + additional after received BSR). > >>>>> > >>>>> /Ingemar > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md] > >>>>>> Sent: den 1 oktober 2015 02:25 > >>>>>> To: Eric Dumazet; Neal Cardwell > >>>>>> Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Ingemar Johansson > >>>>>> S; Eric Dumazet; end2end-interest@postel.org > >>>>>> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment > >>>>>> > >>>>>> Finally can share some results of Hystart interaction with LTE. > >>>>>> Few words about the testing configuration: > >>>>>> Device Samsung Galaxy Note4LTE , up to 150 Mbps capable > >>>>>> Radio Network LTE 2600/1800 > >>>>>> Distance between the Core Network (EPC) and Base > >>>>>> Station - 10 > >>>> km > >>>>>> HTTP Server connected directly to the Core Network 1= Gbps > >>>>>> Core network switching(EPC) 10 Gbps > >>>>>> Backhaul transport 1 Gbps, full IP, MPLS > >>>>>> Last mile transport full IP, MPLS, 300 Mbps > >>>>>> > >>>>>> Server: Linux k40srv 4.1.0-040100-generic > >>>>>> #201507030940 SMP Fri Jul 3 > >>>>>> 09:41:47 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux > >>>>>> Apache/2.4.10 (Ubuntu), built: Jul 24 2= 015 17:25:18 > >>>>>> > >>>>>> Hystart_detect: 2 (HYSTART_DELAY only), as suggested. > >>>>>> net.ipv4.tcp_no_metrics_save =3D 1 to avoid tcp metric= s > >>>>>> caching interference > >>>>>> > >>>>>> Tests types: > >>>>>> 1. Download 3 MB file, interchanging Hystart ON > >>>>> and OFF, in > >>>>>> conditions of Low Bandwidth (25-35 Mbps, poor radio) and High > >>>>>> Bandwidth (100-120 Mbps, good radio). > >>>>>> 2. Download 10 MB file, interchanging Hystart > >>>>> ON and OFF, in > >>>>>> conditions of Low Bandwidth (25-35 Mbps, poor radio) and High > >>>>>> Bandwidth (100-120 Mbps, good radio). > >>>>>> > >>>>>> 100 Download tests per type. > >>>>>> Download duration and bandwidth measured at the > >>>>>> client side as shown by curl from the first received data packet > >>>>>> till the end of session (this exclude DNS, 3 way handshake, http > >>>>>> GET and its > >>>> ACK). > >>>>>> Hystart exit cwnd extracted from nstat (thank you > >>>>>> Eric to pointing out to it). > >>>>>> > >>>>>> Overall results: > >>>>>> Download 3MB, Low Bandwidths: > >>>>>> Hystart average download time: 1.41 s > >>>>>> NoHystart average download time: 1.36 s > >>>>>> > >>>>>> Download 3MB, High Bandwidths: > >>>>>> Hystart average download time: 0.65 s > >>>>>> NoHystart average download time: 0.37 s > >>>>>> > >>>>>> Download 10MB, Low Bandwidths: > >>>>>> Hystart average download time: 2.40 s > >>>>>> NoHystart average download time: 2.12 s > >>>>>> > >>>>>> Download 10MB, High Bandwidths: > >>>>>> Hystart average download time: 1.42 s > >>>>>> NoHystart average download time: 0.89 s > >>>>>> > >>>>>> These results show that Hystart has no significant impact in > >>>>>> conditions of low available bandwidth, but at high available the > >>>>>> decrease of performance can reach 60-70 %. > >>>>>> With the development of the LTE-A which uses larger spectrum to > >>>>>> reach > >>>>>> 300 Mbps and more the impact of Hystart will become even more > >> visible. > >>>>>> I do share with Eric and Neal some files described below and also > >>>>>> can provide links to anyone interested. > >>>>>> > >>>>>> Summary of these tests results are gathered in the following > >>>>>> Excel files (names, I believe, are self explanatory): > >>>>>> Test_3M_low_BW.xlsx, Test_3M_high_BW.xlsx, > >> Test_10M_low_BW.xlsx, > >>>>> Test_10M_high_BW.xlsx. > >>>>>> Similarly tcpdump traces are called Test_3M_low_BW.pcap, > >>>>>> Test_3M_high_BW. pcap, Test_10M_low_BW. pcap, > >>>> Test_10M_high_BW. > >>>>>> pcap (able to collect only server side traces, but they are self > >> sufficient). > >>>>>> For the 10M downloads there are few missing traces as there trace > >>>>>> files were recycled. In traces are also ssh sessions, please, > >>>>>> ignore them, they were used to put hystart ON and OFF on server > >>>>>> and extract nstat results before and after each download. > >>>>>> > >>>>>> Excel files have 2 tabs: curl_sum and Summary. > >>>>>> In the curl_sum tab the stat is cumulated in chronological order > >>>>>> of > >> tests. > >>>>>> In the Summary_tab the Hystart and NoHystart tests are shown > >>>>>> side by side (as said, Download 1 was with Hystart, Download 2 > >>>>>> without, Download 3 with Hystart, Download 4 without, etc.). It > >>>>>> also include graphs of Transfer time and a graph which shows the > >>>>>> distribution in % of different Hystart exit window DelayCWND. > >>>>>> > >>>>>> Fields: > >>>>>> comments - test case > >>>>>> Transf_Time - calculated from curl output the duration > >>>>> between first data > >>>>>> packet and end of transfer > >>>>>> Transf_speed - calculated from curl output the ratio of file = size > >>>>> and duration > >>>>>> between first data packet and end of transfer > >>>>>> Hystart D_Delay - hystart exit occurrence (difference between > >>>>>> TcpExtTCPHystartDelayDetect before and after download) > >>>>>> D_DelayCwnd - Hystart exit cwnd for this download > >>>>>> (difference of TcpExtTCPHystartDelayCwnd before and after the > >>>>>> download) Date_On_Server - time just before the download start, > >>>>>> to help navigate the pcap > >>>>>> > >>>>>> Additionally, in the file Test_10M_low_BW.xlsx in the tab > >>>>>> curl_sum are included for each download tcp.stream extract from > >>>>>> pcap of the first > >>>>>> 100 tcp.analysis.ack_rtt which may help in understanding of the > >>>>>> hystart behavior (fields ack_rtt_1 to ack_rtt_100). > >>>>>> In this file there are also fields URI_frame, URI_time, > >> URI_time_relative, > >>>>>> tcp_stream. > >>>>>> Then there are calculated fields: minRTT as a minimum of all 100 > >>>>>> ACK, > >>>> r1_5, > >>>>>> r2_8, r3_8, r4_8 which simulate Hystart calculations of the > >>>> round curr_rtt > >>>>>> and r1 min, r2 min, r3 min, r4 min represent minRTT of the whole > >> round. > >>>>>> Please, correct me if I am wrong, but in my understanding for > >>>>>> each ACK the hystart_update is called before the hystart_reset, > >>>>>> and as a result the computation of cur_rtt of the 8 packets of > >>>>>> the round starts with the 2-nd packet of the round, not the first = one. > >>>>>> I've use r1_5 as a min(ack_rtt_7...ack_rtt_11), due to > >>>>>> hystart_low_window=3D16, r2_8 as a min(ack_rtt_12 ... ack_rtt_19), > >>>>>> r3_8 as a > >>>>>> min(ack_rtt_32 ... ack_rtt_39) and ), r4_8 as a min(ack_rtt_72 ... > >>>>> ack_rtt_79). > >>>>>> To understand the impact of this there are also r1 HOL, r2 HOL, > >>>>>> r3 HOL, r4 HOL (Head Of Line) which compute the same with first > >>>>>> packet in the round included in the relevant round: r1_5 as a > >>>>>> min(ack_rtt_6...ack_rtt_10), r2_8 as a min(ack_rtt_11 ... > >>>>>> ack_rtt_18), > >>>>>> r3_8 as a min(ack_rtt_31 ... ack_rtt_38) and ), r4_8 as a > >>>>>> min(ack_rtt_71 > >> ... > >>>>> ack_rtt_78). > >>>>>> If my understanding of Hystart is correct and, assuming in each > >>>>>> train the first packet has the lowest RTT which, most probably is > >>>>>> valid in all type of networks, then the exit from hystart in this > >>>>>> implementation may happen at cwnd 29, 49, 89, 169, etc. (given > >>>>>> the IW of > >>>>> 10). > >>>>>> These tests show that this is the case, however there also > >>>>>> intermediate values. However the cwnd of 29 shall be the lowest > >>>>>> possible value and this is the case in these tests. > >>>>>> Unfortunately, in LTE the first packet in the train has chances > >>>>>> to always have ~6 ms lower than the following ones and, because > >>>>>> it is not counted in the 2-nd train, there is quite high > >>>>>> percentage of exit at 29 (14-36% in these tests), even though > >>>>>> later on cubic increases the cwnd without losses up to many > >>>>>> hundreds. For me this is too early exit from > >>>> slow start. > >>>>>> As it can be seen from comparison of the simulated calculation, > >>>>>> inclusion of the first packet in the round curr_rtt calculation > >>>>>> would eliminate the cwnd 29 and shift to cwnd 49 or 89 or higher. > >>>>>> > >>>>>> But there are also intermediate values, e.g. 69. Looking through > >>>>>> traces one may see that this is due to "compressed ACK" which > >>>>>> receives a tightly packed train of many ACK so that in traces > >>>>>> there are not data packets sent in turn by server. Looks to me, > >>>>>> because hysrart_reset sets the end_seq to snd_next and the server > >>>>>> did not managed to send yet packets in reply to this high speed > >>>>>> ACK train the snd_next points to much lower value than would > >>>>>> normally happen and this actually destroys the correct > >>>>>> identification of the borders of the round, the round fails > >>>>>> somewhere in the middle of the train not at its border, this > >>>>>> makes very likely increased delay due to queueing and earlier > >>>>>> exit from slow_start and intermediate values of exit cwnd where > >>>>> they are not expected. > >>>>>> And last but not least: there are gaps (4-7 ms) in ACK trains. On > >>>>>> the perfectly sent IW of 10 data packets train the typical for > >>>>>> LTE will be to receive 1 or 2 ACK, then a gap of 4-6 ms then > >>>>>> another couple of ACK, then another gap, then 5 ACK compressed > >>>>>> together in a fraction of microsecond. Of cause, the next train > >>>>>> from the server will contain the same gaps. Very quickly, e.g., > >>>>>> the Head of Line > >>>>>> (HOL) of the 3-d train will follow immediately the tail of the > >>>>>> 2-nd train which will lead to queueing in the end router > >>>>>> (actually the radio base station, much earlier than BDP reached), > >>>>>> but this will lead to increase of the train RTT and too yearly > >>>>>> exit from slow start. Why too early exit ? Because, should server > >>>>>> continue increasing the speed this would reduce quicker the gaps > >>>>>> both in downlink and uplink, the preice for this being the > >>>>>> increase of the RTT (I guess double) against what one would > >>>>>> normally see > >>>>> in the end-to-end Ethernet networks. > >>>>>> > >>>>>> I do like ideas of Hystart, but I don't know how could it be > >>>>>> possibly reconciled with my LTE problems. With the increase of > >>>>>> LTE speeds this early exit from slow start will become even more > >>>>>> of the > >>>> problem. > >>>>>> For LTE, not sure how good and generally acceptable, solution > >>>>>> would be, possibly, to increase the HYSTART_DELAY_MIN to 8 ms, > >>>>>> which, at least in my case, would diminish the earlier exit > >>>>>> significantly, if I believe in these tests results. > >>>>>> Another solution, which I don't like very much, but as I know > >>>>>> some operators use, would be to insert a kind of TCP accelerator > >>>>>> between the Internet and the mobile network which will intercept > >>>>>> all TCP and would send it to mobile without Hystart but this > >>>>>> would kill any end-to-end principle and this will not be the > >>>>>> Internet > >> anymore. > >>>>>> > >>>>>> If you have time and never look closely to LTE technology I do a > >>>>>> quick summary bellow. > >>>>>> > >>>>>> > >>>>>> LTE, TCP self-clocking and ACK compression. > >>>>>> > >>>>>> Spent some time to understand the basics of what could be the > >>>>>> reason for all those timing effects due to LTE. > >>>>>> First, unlike wire based transport technologies, LTE (but also > >>>>> UMTS > >>>>>> and future 5G) sends data, both uplink and downlink in predefined > >>>>>> timeslots called Transmission Time Interval (TTI). In UMTS this > >>>>>> is 2ms, in LTE is > >>>>>> 1 ms and it says that in 5G it will be 0.5 ms. > >>>>>> These TTI have exact borders and tightly follow each other and > >>>>>> require exact synchronization between the mobile device and Radio > >>>>>> Base Station (called eNodeB in LTE). To cope with varying radio > >>>>>> conditions the sending data bandwidth is modified in such a way > >>>>>> that to keep a constant probability of data loss of 10%. Bad > >>>>>> radio - less bandwidth, good > >>>>> radio - more bandwidth. > >>>>>> The bandwidth is modified by packing less or more bits in a TTI > >>>>>> of 1 ms in > >>>>> LTE. > >>>>>> But TTI remain unchanged of exactly 1 ms. That means that, at the > >>>>>> TCP/IP layer the precision of the timing information from device > >>>>>> to the network is 1 ms. In the case of a bandwidth of 1 Mbps > >>>>>> there will be 83 IP packets per second or one packet every 12 ms. > >>>>>> In this case, possibly the return of one ACK every 12 ms (every > >>>>>> 12 TTI) would be more or less sufficient to keep at a good level > >>>>>> the TCP self-clocking mechanism. But, when talking about 100 Mbps > >>>>>> (or 300 > >>>>>> Mbps) that means one packet (and one ACK) every 0.12 ms and > there > >>>>>> is no way to provide it with a TTI of 1 ms. This is the reason > >>>>>> for ACK compression in LTE and the future 5G which promises more > >>>>>> than 1Gbps with its 0.5 ms TTI will > >>>>> be in a big conflict with TCP self-clocking. > >>>>>> But the problem do not stops here. There is a significantly > >>>>> different > >>>>>> mechanisms of transmission in the downlink (from eNodeB to > mobile > >>>>>> device) and in uplink (from mobile device to the eNodeB). The > >>>>>> difference come from the fact that the scheduler for both > >>>>>> downlink and in uplink is located in the eNodeB. As of my > >>>>>> understanding the purpose of this is to simplify to the maximum > >>>>>> the device and save its > >>>> battery. > >>>>>> As a result of this design, the eNodeB can send in each TTI > >>>>>> toward the mobile device whenever data exist in its buffer > >>>>>> (eNodeB is a kind of IP wireless router). But for mobile device, > >>>>>> in order to send the data in uplink, after a pause, it has first > >>>>>> to ask the network to schedule it. And mobile has for this > >>>>>> request (after a pause of > >>>>>> sending) just one bit, actually is not even a data bit, is a > >>>>>> special radio signal which tells "Hey, I have some data in the > >>>>>> buffer to send". Lets look at initial train of IW of 10 packets. > >>>>>> In good radio conditions the eNodeB will fit all 10 packets in 1 > >>>>>> TTI of 1 ms and send them all to the device. Device will unpack > >>>>>> them and generate 10 ACK and then tell to the eNodeB "Hey, I have > >>>>>> some data in the buffer to send". The eNodeB has not knowledge > >>>>>> how much data the mobile has to send, therefor will allocate some > >>>>>> minimum > >>>>>> capacity: "Well, I give you a grant to send up to 200 Bytes and > >>>>>> you must send them exactly after 4 ms you get this notification". > >>>>>> These > >>>>>> 4 ms are allowance for mobile device to prepare data for sending > >>>>>> which is a very complicated process, these 4 ms is a must delay > >>>>>> between the grant > >>>>> and the transmission. > >>>>>> If mobile has in its buffer 10 ACK to send it will send only 2 > >>>>>> ACK and will complement them with, so called, buffer status > >>>>>> report: "I do have still 480 bytes to send". Now network has more > >>>>>> information and allocate as > >>>>>> requested: "Well, I give you a grant to send up to 200 Bytes and > >>>>>> you must send them exactly after 4 ms you get this notification". > >>>>>> But, in the mean time some other information of higher priority > >>>>>> may appear in the > >>>>> device (e.g. > >>>>>> signaling) and the device will send only 6 out of 8 remaining ACK > >>>>>> together with signaling and another Buffer Status Report: "Hey, I > >>>>>> do have another 120 bytes to send". And again "Well, I give you a > >>>>>> grant to send up to 120 Bytes and you must send them exactly > >>>>>> after 4 ms you get this notification". As a result the server > >>>>>> will receive 2 ACK, then after a gap of 6 ms another 6 ACK, then, > >>>>>> after another 6 ms, the remaining 2 ACK. With the scheduler in > >>>>>> the eNodeB these gaps are inevitable. And one more think: mobile > >>>>>> pack together those 6 ACK in 1 TTI of 1 ms and send them, then > >>>>>> the eNodeB unpack them and there is no way to recover the spacing > >> of 0.12 ms between them. > >>>>>> eNodeB simply send them to the server back to back at the speed > >>>>>> of its data card which is typically 1 Gbps and we face the ACK > >>>>>> compression. After the last 2 ACK were sent there is nothing else > >>>>>> to send, so there is no Buffer Status Report, therefore, after > >>>>>> packets of the next train arrive and ACK are generated everything > >>>>>> starts again with one bit of information "Hey, I have > >>>>> some data in the buffer to send". > >>>>>> That's why there are gaps and ACK compression, loss, or more > >>>>>> exactly, lack of any timing information which would be useful for > >>>>>> the TCP self- > >>>>> clocking. > >>>>>> Thank you and sorry for taking your time. > >>>>>> > >>>>>> > >>>>>> Veaceslav Roman > >>>>>> > >>>>>> > >>>>>> > >>>>>> -----Original Message----- > >>>>>> From: Veaceslav ROMAN > >>>>>> Sent: Saturday, 05 September 2015 01:10 > >>>>>> To: 'Eric Dumazet'; Neal Cardwell > >>>>>> Cc: tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; Ingemar Johansson > >>>>>> S; Eric Dumazet; end2end-interest@postel.org > >>>>>> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment > >>>>>> > >>>>>> If we look at the first train: all packets received in less than 1= ms. > >>>>>> Probably this is only an appearance as LTE transmits in, so > >>>>>> called transmission time interval (TTI) of 1 ms, and what we see > >>>>>> here is that all 10 packets of initial window fitted in 1 ms, > >>>>>> and, when decoded, were presented to the TCP/IP layer (and pcap) > >>>>>> all > >> at once. > >>>>>> BTW, 8 packets of 13022 bytes in 1 ms means instantaneous speed > >>>>>> of > >>>>>> 104 Mbps, good radio conditions. TCP generates 10 ACK train of > >>>>>> the duration of > >>>>>> 1.2 ms. Will it be a Fast Ethernet possibly we would consider > >>>>>> this normal, > >>>>> isn't it ? > >>>>>> I looked at how server reply to these 10 ACKs. There are 3 gaps > >>>>>> of 4, > >>>>>> 2 and 5 ms in the reply train and the total train of 18 (?, it > >>>>>> should be 20) packets reaches a duration of 14 ms. AFAIK LTE may > >>>>>> introduce gaps in ACK train due to uplink scheduling mechanism. > >>>>>> May be these gaps trigger Hystart early exit ? > >>>>>> > >>>>>> Veaceslav Roman > >>>>>> Technical and IT director > >>>>>> Orange Moldova S.A. > >>>>>> Fix: +37322575400 > >>>>>> Mob: +37369198400 > >>>>>> Fax: +37322575306 > >>>>>> > >>>>>> -----Original Message----- > >>>>>> From: Eric Dumazet [mailto:eric.dumazet@gmail.com] > >>>>>> Sent: Friday, 04 September 2015 19:48 > >>>>>> To: Neal Cardwell > >>>>>> Cc: Veaceslav ROMAN; tcpm@ietf.org; Piers O'Hanlon; Sangtae Ha; > >>>>>> Ingemar Johansson S; Eric Dumazet; end2end-interest@postel.org > >>>>>> Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment > >>>>>> > >>>>>> On Fri, 2015-09-04 at 11:32 -0400, Neal Cardwell wrote: > >>>>>>> Hi Veaceslav, > >>>>>>> > >>>>>>> I agree that in your LTE traces it looks like CUBIC Hystart is > >>>>>>> exiting slow start too early. > >>>>>>> > >>>>>>> SInce you seem to have a nice LTE testbed, would you be able to > >>>>>>> do some experiments to find a set of parameters for Hystart that > >>>>>>> work better for your LTE environment? For example, you might try > >>>>>>> the two variations I suggested earlier in the thread: > >>>>>>> > >>>>>>> if (ca->curr_rtt > ca->delay_min + > >>>>>>> HYSTART_DELAY_THRESH(ca->delay_min > >>>>>>> 2)) { or > >>>>>>> if (ca->curr_rtt > ca->delay_min + > >>>>>>> HYSTART_DELAY_THRESH(ca->delay_min > >>>>>>> 1)) { > >>>>>>> > >>>>>>> Do any of those give better results for your tests? > >>>>>>> > >>>>>>> neal > >>>>>> Also, hystart is fooled by too many ACK received in short period. > >>>>>> > >>>>>> This problem would be solved if GRO was used at receiver, as less > >>>>>> ACK would be sent. > >>>>>> > >>>>>> Presumably receiver is not a linux TCP stack ? > >>>>>> > >>>>>> Maybe we should add a logic in hystart_update() to take one ACK > >>>>>> per > >> ms. > >>>>>> 05:32:24.134172 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [S], seq 4294423215, win 65535, options [mss 1460,sackOK,TS > >>>>>> val > >>>>>> 1112351 ecr 0,nop,wscale 8], length 0 > >>>>>> 05:32:24.161986 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [S.], seq 1198877721, ack 4294423216, win 28960, options > >>>>>> [mss 1416,sackOK,TS val > >>>>>> 2808808634 ecr 1112351,nop,wscale 7], length 0 > >>>>>> 05:32:24.162108 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 1, win 343, options [nop,nop,TS val 1112354 ecr > >>>>>> 2808808634], length 0 > >>>>>> 05:32:24.163072 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [P.], seq 1:658, ack 1, win 343, options [nop,nop,TS val > >>>>>> 1112354 ecr 2808808634], length 657 > >>>>>> 05:32:24.203984 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], ack 658, win 237, options [nop,nop,TS val 2808808675 > >>>>>> ecr 1112354], length 0 > >>>>>> 05:32:24.210275 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [P.], seq 1:387, ack 658, win 237, options [nop,nop,TS val > >>>>>> 2808808677 ecr 1112354], length 386 > >>>>>> 05:32:24.210315 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 387:1791, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808677 ecr 1112354], length 1404 > >>>>>> 05:32:24.210332 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 1791:3195, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808677 ecr 1112354], length 1404 > >>>>>> 05:32:24.210347 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 3195:4599, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808677 ecr 1112354], length 1404 > >>>>>> 05:32:24.210350 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 387, win 347, options [nop,nop,TS val 1112359 ecr > >>>>>> 2808808677], length 0 > >>>>>> 05:32:24.210361 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 4599:6003, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808677 ecr 1112354], length 1404 > >>>>>> 05:32:24.210374 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 6003:7407, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808677 ecr 1112354], length 1404 > >>>>>> 05:32:24.210389 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 7407:8811, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808677 ecr 1112354], length 1404 > >>>>>> 05:32:24.210402 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 8811:10215, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808677 ecr 1112354], length 1404 > >>>>>> 05:32:24.210416 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 10215:11619, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808677 ecr 1112354], length 1404 > >>>>>> 05:32:24.210418 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 1791, win 358, options [nop,nop,TS val 1112359 ecr > >>>>>> 2808808677], length 0 > >>>>>> 05:32:24.210431 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 11619:13023, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808677 ecr 1112354], length 1404 > >>>>>> 05:32:24.210455 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 3195, win 369, options [nop,nop,TS val 1112359 ecr > >>>>>> 2808808677], length 0 > >>>>>> 05:32:24.210685 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 4599, win 380, options [nop,nop,TS val 1112359 ecr > >>>>>> 2808808677], length 0 > >>>>>> 05:32:24.210731 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 6003, win 391, options [nop,nop,TS val 1112359 ecr > >>>>>> 2808808677], length 0 > >>>>>> 05:32:24.210937 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 7407, win 402, options [nop,nop,TS val 1112359 ecr > >>>>>> 2808808677], length 0 > >>>>>> 05:32:24.211078 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 8811, win 413, options [nop,nop,TS val 1112359 ecr > >>>>>> 2808808677], length 0 > >>>>>> 05:32:24.211228 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 10215, win 424, options [nop,nop,TS val 1112359 > >>>>>> ecr 2808808677], length 0 > >>>>>> 05:32:24.211377 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 11619, win 435, options [nop,nop,TS val 1112359 > >>>>>> ecr 2808808677], length 0 > >>>>>> 05:32:24.211544 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 13023, win 446, options [nop,nop,TS val 1112359 > >>>>>> ecr 2808808677], length 0 > >>>>>> 05:32:24.237999 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 13023:14427, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808709 ecr 1112359], length 1404 > >>>>>> 05:32:24.238053 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 14427:15831, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808709 ecr 1112359], length 1404 > >>>>>> 05:32:24.238090 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 14427, win 457, options [nop,nop,TS val 1112362 > >>>>>> ecr 2808808709], length 0 > >>>>>> 05:32:24.238164 IP 94.243.104.156.32924 > 195.95.178.204.80: > >>>>>> Flags [.], ack 15831, win 468, options [nop,nop,TS val 1112362 > >>>>>> ecr 2808808709], length 0 > >>>>>> 05:32:24.244024 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 15831:17235, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808713 ecr 1112359], length 1404 > >>>>>> 05:32:24.244063 IP 195.95.178.204.80 > 94.243.104.156.32924: > >>>>>> Flags [.], seq 17235:18639, ack 658, win 237, options [nop,nop,TS > >>>>>> val > >>>>>> 2808808713 ecr 1112359], length 1404 > >>>>>> > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm > > >=20 > -- > __________________________________________________________ > ______ > Bob Briscoe http://bobbriscoe.net/ From nobody Wed Oct 21 08:45:43 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1553E1A90F0 for ; Wed, 21 Oct 2015 08:45:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.91 X-Spam-Level: X-Spam-Status: No, score=-3.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EtkJ_fxm0Gbn for ; Wed, 21 Oct 2015 08:45:39 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DBC411A90C3 for ; Wed, 21 Oct 2015 08:45:36 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 55DB6D930D; Wed, 21 Oct 2015 17:45:35 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Ofp5PbD+-LOO; Wed, 21 Oct 2015 17:45:35 +0200 (MEST) Received: from [82.130.103.143] (nb-10510.ethz.ch [82.130.103.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: mirjak) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 0F378D9304; Wed, 21 Oct 2015 17:45:35 +0200 (MEST) To: Bob Briscoe , Praveen Balasubramanian References: <55F055AD.3050809@tik.ee.ethz.ch> <55F05D54.5060708@tik.ee.ethz.ch> <55FF2910.7080908@bobbriscoe.net> <561DA02D.8020001@bobbriscoe.net> <0F4F0BA2-A1AE-40BB-9E0D-0EB5F4DE79E1@tik.ee.ethz.ch> <5623D09C.4020002@bobbriscoe.net> From: =?UTF-8?Q?Mirja_K=c3=bchlewind?= Message-ID: <5627B316.1090702@tik.ee.ethz.ch> Date: Wed, 21 Oct 2015 17:45:26 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <5623D09C.4020002@bobbriscoe.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Archived-At: Cc: tcpm IETF list , Richard Scheffenegger Subject: Re: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 15:45:42 -0000 Hi Bob, no, there is no estimation in the current DCTCP implementation because the feedback mechanism they use indicates that either all bytes that a certain ACK acknowledges have been marked or not; so you directly have the number of bytes. That means effectively AccECN without the Option would provide you less information than you currently have with DCTCP's feedback mechanism. However, in DCTCP there is a risk of loosing information/obtaining wrong information if ACKs get lost. This might be a rare case in data centers, however, I'd still argue for safety here. Mirja On 18.10.2015 19:02, Bob Briscoe wrote: > Praveen, > > Sorry, there's ambiguity in these emails between the units of feedback > and the units of calculation of the congestion response. Current DCTCP > feedback (both Windows and Linux) is in packets, then the sender > calculates/estimates what this means in bytes which it applies in the > window reduction algo. This calculation/estimation is accurate for > long-running flows, but hard for app-limited flows where there are a lot > of sub-MSS segments. > > The example congestion response I gave (Relentless) uses bytes for > feedback and bytes for the response calculation. This works because > Relentless is designed for loss and SACK gives loss feedback in bytes > (Relentless didn't include any change to ECN feedback). > > The AccECN draft-04 proposes essential feedback in packets using the > main TCP header and supplementary feedback in bytes using a new AccECN > TCP option (middleboxes permitting). If the AccECN option gets through > OK, it means the sender has bytes directly, so it doesn't have to > estimate or calculate bytes from packet feedback. > > I should add that providing byte counters wasn't the main reason for the > TCP option. It was primarily to improve protocol resilience in the face > of heavy ACK loss (when congestion avoidance could be most critical), > because the 3-bit field available for the packet counter in the main > header is rather small (it wraps every 8 packets, which is at least > better than the 1-bit feedback in current DCTCP). > > So, once we added the TCP option for resilience, we could use it for > full byte feedback instead of packet, and we could also provide enough > info to detect bleaching of the ECN field and to support potential > future use of ECT(1) as well as ECT(0). > > Hope this clarifies the confusion. > > > Bob > > On 16/10/15 17:32, Praveen Balasubramanian wrote: >> Even the Windows implementation uses the number of bytes in CE marked packets. Bob, I am not sure why you think extent of congestion is based on packets and not bytes, please check section 3.3 in the draft. >> >> -----Original Message----- >> From: Mirja Khlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch] >> Sent: Friday, October 16, 2015 12:22 AM >> To: Bob Briscoe >> Cc: Praveen Balasubramanian ; tcpm IETF list ; Richard Scheffenegger >> Subject: Re: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt >> >> Hi, >> >>> Am 14.10.2015 um 02:22 schrieb Bob Briscoe : >>> >>> At present DCTCP measures the extent of congestion in packets not bytes. But we envisaged that an algorithm like Relentless TCP might be used where, every time new CE feedback arrives, the sender just decrements cwnd by (CE bytes)/2. >> As a side note, the current DCTCP implementation in Linux uses number of bytes... >> > From nobody Wed Oct 21 11:39:24 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D4CE71A89B8 for ; Wed, 21 Oct 2015 11:39:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0C_2Ar4VpbjS for ; Wed, 21 Oct 2015 11:39:21 -0700 (PDT) Received: from mail-vk0-x233.google.com (mail-vk0-x233.google.com [IPv6:2607:f8b0:400c:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 70D5D1A89ED for ; Wed, 21 Oct 2015 11:39:21 -0700 (PDT) Received: by vkat63 with SMTP id t63so33976927vka.1 for ; Wed, 21 Oct 2015 11:39:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=jMb+dDAs5/bEhX6zf1tcNaY0PfmnKI3pQiLVYB5Yr4c=; b=ipgxf/+RbNeF71ulxBmo8PKx3WUEPDt+DdXMA1P7frG5axfS8NMJEtunxFLUta95ir 87gqOJRqrl0rjPsBTtewnoJ5lK0PQCaK4SEBSWp6nVY/F99X5he/Y3JJsSSyrUveMsOP SfBdCKqvi1uNmiWqggYLIicUrjoz/XlwRXVatZWzwyn9f0Rk/rRTdOo+Cjz0iSWHyliS Zjtv41S5NJ8944eb8iyGTWHASR5ElEOfmiLzA6bA/6V8l0pxFVAlw8ZPjzoh7nFghDQ+ mXML5CKZtjFrmz38bB5mm9wWObr7oWXJrg3IkKuIlc3XWNVyy8pS5Ggxhog0VcfCTqs+ kq0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=jMb+dDAs5/bEhX6zf1tcNaY0PfmnKI3pQiLVYB5Yr4c=; b=LxpEWJpJwe1rH8tHx60H8jYg8ZRwGNrhT4iysQzpayqw5K2DSw8fcjPQsBRF5Z1045 44kjoXos6zb/8PAsNEN3qKSJFR8JmKYDI0+MY5v16xPxDVaPiasmj35w8rKyorT4PHx0 xOS3elVfkCFZJQ9Qa1Ein+0D7WHH+w2ObUKgfUf7wqJIrJDE/RppYq7YllEZ1yWCuLiZ HzBnufwqUA47ciJSkk1GZADQT/GXy65P0+DN7Op7nnLjSG4raOV6AFJ5MuawSVDjq17U Ql3lfgyZZpItqnyVXjZib4Ww6XMQ1VEiKf5PjlDd+gMHqeW1aNFU54/b6E9BCsL1ZACu +Jww== X-Gm-Message-State: ALoCoQm0g841MIIDmvThOytCH3JD8c2R5gGoM39nK5YXdn1m6tsMGTwXZ7SrYx2qfb/ee8Zuf0O3 X-Received: by 10.31.185.6 with SMTP id j6mr4126262vkf.98.1445452760373; Wed, 21 Oct 2015 11:39:20 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.153.134 with HTTP; Wed, 21 Oct 2015 11:38:40 -0700 (PDT) In-Reply-To: <57888.1445427818@lawyers.icir.org> References: <57888.1445427818@lawyers.icir.org> From: Yuchung Cheng Date: Wed, 21 Oct 2015 11:38:40 -0700 Message-ID: To: "mallman@icir.org" Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] rto-consider updated X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 18:39:23 -0000 I am interested. Following the spirit of reflecting the reality, I have some random points after skimming your draft: * (3) MUST exp. backoff: I agree on expback off is needed for safety. But MUST "eventually" exp backoff reflects the reality better. AFAIK Linux has the "thin-stream" optimization that optionally use linear back for the first 6? retries. And I recall vaguely some stacks do so based on some (old) research paper. * Would be nice to talk about the well-known tricks for data-center communications to lower the delayed ACK and RTO. * Knowing the delayed ACK time of the remote matters. Some RFC (1122?) suggests 500ms. I think everyone uses lower value 40ms - 200ms. * I saw lots of (naive) attempts to lower the RTO for latency, it would be nice to talk about the implication of spurious RTO. One best example is https://insouciant.org/#Google_Kitten_Search that the "web-style" incast leads to massive spurious retransmission and buffer bloat. This would not happen if the connection supports Timestamp and the sender implements Eifel. but not all stacks support TS (e.g., Windows). On Wed, Oct 21, 2015 at 4:43 AM, Mark Allman wrote: > > Folks- > > I have been cleaning up some old items on my 'todo' list. One of > them is an old draft I wrote on RTO considerations. The idea is to > provide a set of high-level guidelines that estimators must follow, > but that leaves them free to set low-level details without being > standardized. My view is this is mostly writing down what happens > now. > > When I originally posted the draft several years ago it garnered > little traction. > > My todo is to issue this as a tech report so that it lives somewhere > (and I can reference the idea and such). But, I figured before I > did that I'd throw it up as an I-D again and make sure folks were > not interested. > > So, I tried to to do that yesterday, but we are in the "no I-D > postings" period. So, the (very short) I-D is here: > > http://www.icir.org/mallman/pubs/All15a/ > > I have a bit set to submit it when submissions are again accepted > (Nov 1). > > Comments welcome, of course. If the WG is interested, great. If > not, that is fine and I won't say any more about it. > > Thanks! > > allman > > > -- > http://www.icir.org/mallman/ > > > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Wed Oct 21 11:54:14 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8A7E81A914F; Wed, 21 Oct 2015 11:54:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.911 X-Spam-Level: X-Spam-Status: No, score=-6.911 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8LNve-Zs3DTx; Wed, 21 Oct 2015 11:54:10 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 890C31A8AAB; Wed, 21 Oct 2015 11:54:10 -0700 (PDT) Received: from us70uusmtp3.zam.alcatel-lucent.com (unknown [135.5.2.65]) by Websense Email Security Gateway with ESMTPS id 72B59E5CF7B2E; Wed, 21 Oct 2015 18:54:04 +0000 (GMT) Received: from cliff.eng.ascend.com (cliff.eng.ascend.com [192.207.23.55]) by us70uusmtp3.zam.alcatel-lucent.com (GMO) with ESMTP id t9LIs5Y1030539 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 21 Oct 2015 18:54:06 GMT Received: from igoyret-c1.alcatel-lucent.com (vpn-9-199.vpn.alcatel-lucent.com [135.224.9.199]) by cliff.eng.ascend.com (8.13.1/8.13.1) with ESMTP id t9LIs2Jr010990 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 21 Oct 2015 11:54:03 -0700 Message-Id: <201510211854.t9LIs2Jr010990@cliff.eng.ascend.com> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 21 Oct 2015 11:53:51 -0700 To: internet-drafts@ietf.org From: Ignacio Goyret In-Reply-To: <20151013125549.31353.52573.idtracker@ietfa.amsl.com> References: <20151013125549.31353.52573.idtracker@ietfa.amsl.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Archived-At: Cc: tcpm@ietf.org, i-d-announce@ietf.org Subject: [tcpm] draft-ietf-tcpm-undeployed-03.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 18:54:12 -0000 Just a small nit: on page 4, remove the duplicate "which": Was: o [RFC0889] U, "Internet Delay Experiments", which which describes ^^^^^^^^^^^ Should be: o [RFC0889] U, "Internet Delay Experiments", which describes ^ -Ignacio From nobody Wed Oct 21 11:54:54 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6C54D1A9148 for ; Wed, 21 Oct 2015 11:54:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.911 X-Spam-Level: X-Spam-Status: No, score=-6.911 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7UPlKnXYDtpV for ; Wed, 21 Oct 2015 11:54:53 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 38B391A8AAB for ; Wed, 21 Oct 2015 11:54:53 -0700 (PDT) Received: from us70uusmtp3.zam.alcatel-lucent.com (unknown [135.5.2.65]) by Websense Email Security Gateway with ESMTPS id B1CE1605CE5CE for ; Wed, 21 Oct 2015 18:54:47 +0000 (GMT) Received: from cliff.eng.ascend.com (cliff.eng.ascend.com [192.207.23.55]) by us70uusmtp3.zam.alcatel-lucent.com (GMO) with ESMTP id t9LIsnjX031492 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Wed, 21 Oct 2015 18:54:50 GMT Received: from igoyret-c1.alcatel-lucent.com (vpn-9-199.vpn.alcatel-lucent.com [135.224.9.199]) by cliff.eng.ascend.com (8.13.1/8.13.1) with ESMTP id t9LIslWR011033 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 21 Oct 2015 11:54:48 -0700 Message-Id: <201510211854.t9LIslWR011033@cliff.eng.ascend.com> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 21 Oct 2015 11:54:43 -0700 To: tcpm@ietf.org From: Ignacio Goyret Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Archived-At: Subject: [tcpm] draft-ietf-tcpm-undeployed-03.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 18:54:54 -0000 Just a small nit: on page 4, remove the duplicate "which": Was: o [RFC0889] U, "Internet Delay Experiments", which which describes ^^^^^^^^^^^ Should be: o [RFC0889] U, "Internet Delay Experiments", which describes ^ -Ignacio From nobody Wed Oct 21 14:06:16 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7075A1B308F for ; Wed, 21 Oct 2015 14:06:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.79 X-Spam-Level: X-Spam-Status: No, score=0.79 tagged_above=-999 required=5 tests=[BAYES_50=0.8, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zG6_SHA2nbiW for ; Wed, 21 Oct 2015 14:06:14 -0700 (PDT) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5F42E1B308D for ; Wed, 21 Oct 2015 14:06:14 -0700 (PDT) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id BE6CE10ABC2 for ; Wed, 21 Oct 2015 14:06:13 -0700 (PDT) Date: Wed, 21 Oct 2015 14:06:13 -0700 From: hiren panchasara To: tcpm@ietf.org Message-ID: <20151021210613.GJ28288@strugglingcoder.info> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="rwgQ89ZNnFUwFHTC" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) Archived-At: Subject: [tcpm] Regarding sack scoreboard cleanup (rfc6675) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 21:06:15 -0000 --rwgQ89ZNnFUwFHTC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline (My first ask/question on the mailing list so let me know if this should be directed somewhere else.) I am working on FreeBSD tcp stack and started looking into sack processing. Trying to understand when/how scoreboard should be cleared when we exit loss recovery. rfc6675 section '5. Algorithm Details' talks about handling each arriving ACK: (A) An incoming cumulative ACK for a sequence number greater than RecoveryPoint signals the end of loss recovery, and the loss recovery phase MUST be terminated. Any information contained in the scoreboard for sequence numbers greater than the new value of HighACK SHOULD NOT be cleared when leaving the loss recovery phase. First question: Should this say 'greater than or equal to RecoveryPoint' instead of 'greater than RecoveryPoint'? Another thing I do not understand is how could a scoreboard have information for sequence numbers grater than the new value of HighACK in this case when cumulative ACK is covering all the holes (as we are exiting recovery). I might be blindsided by just looking at FreeBSD stack or might be missing something basic in how sack works. In any case, insights would be useful. Cheers, Hiren --rwgQ89ZNnFUwFHTC Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJWJ/5BXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/l4+wIAIu7+Oh2Xl+L1ylB2BSrwoqY dMqBYf4S5LO+IZZezjJEwMuh9J+lEOku06I5yde/8EfcW8ddk0Z2nvCpBGP88KwU G6UZv9IT9tlKGSMMXTHUbV5MVY8ezyURoO3gySv8pcPrQGLFJ+7uLPZfjBlgPnrH tjuT83Nk6OnbILYWTsvg5S7O4iF1Mj2iARqUSL5fErMUw4ao+mnkjOGRD2qIHrVu 9IC9kSJZM+nXXJXIcRDJ4Yygg2UY8UdtwH5DSX34JJhwkilcHMeJLJqP7xxpsFx2 MWlCXbGAKw0yGy/DrLf6ijqTu6z0jHtfjfVSyDB+vxbmXKil3weEDz7rPpMwePs= =Fe2l -----END PGP SIGNATURE----- --rwgQ89ZNnFUwFHTC-- From nobody Wed Oct 21 14:19:04 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7A85F1B30E1 for ; Wed, 21 Oct 2015 14:19:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jRBKXtg37xPp for ; Wed, 21 Oct 2015 14:19:02 -0700 (PDT) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 668F91B30D9 for ; Wed, 21 Oct 2015 14:19:02 -0700 (PDT) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id EEE9510AD64; Wed, 21 Oct 2015 14:19:01 -0700 (PDT) Date: Wed, 21 Oct 2015 14:19:01 -0700 From: hiren panchasara To: Yuchung Cheng Message-ID: <20151021211901.GK28288@strugglingcoder.info> References: <20151018001317.GD87252@strugglingcoder.info> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="6K2R/cS9K4qvcBNq" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] Question to TCP implementors regarding dupack threshold X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 21:19:03 -0000 --6K2R/cS9K4qvcBNq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 10/20/15 at 09:03P, Yuchung Cheng wrote: > +tcpm > On Oct 19, 2015 4:03 PM, "Yuchung Cheng" wrote: >=20 > > On Sat, Oct 17, 2015 at 5:13 PM, hiren panchasara > > wrote: > > > > > > On 10/17/15 at 09:48P, Yuchung Cheng wrote: > > > > Hi, > > > > > > > > The question is: > > > > > > > > When a TCP sender receives a DUPACK with SACK option(s) covering at > > least > > > > three packets (or three MSS bytes), will it trigger fast recovery > > > > immediately? > > > > > > > > I know Linux does (for at least +8 years). I am curious about other > > > > implementations like Windows, Mac, *BSD? > > > > > > FreeBSD doesn't do this currently. Looks like a sensible thing to do. > > > > yes I agree it's nice to have that. It's more robust against ACK > > suppression discussed in the other threads, and ack loss in general. > > > > Anyone know about Windows and Mac? A _very_ quick look through apple/xnu code tells me that they are also not doing this currently. Cheers, Hiren --6K2R/cS9K4qvcBNq Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJWKAFBXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lDGcH/jKIdqOx0jqfDS4N/4d+67BS B2FWZU+utTQy1LYWljhKWmCcskNG4m7F/RVwqREq/ZsqeCNhSDgRbITVssokmH3B eeDiYpk73oYE55fdFzsJ/E7O6uqfrEAeCf3e4SsSb3t0GJ3+T6r3C8k4GopEColU TIygBOD50DiBz0q4nKm8qURgSMK7A1Ps7abVU1vfxzpApPDJjpb7IFlwQX0FgmJd bfiQCjP7zqlseP9sIQtT/4bTxDRcuuHHbG7J9zwTD9CHV6J5Ad8l5AcAUUyrx675 QNOofnKthbl4fz1LCyOUNsMxD1NDovv7iFFk3y/Z9nH8mTHig+T3xRjEqxSayus= =viA+ -----END PGP SIGNATURE----- --6K2R/cS9K4qvcBNq-- From nobody Wed Oct 21 15:35:27 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D3FBB1B32F1 for ; Wed, 21 Oct 2015 15:35:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i6f-_7gLX5Tr for ; Wed, 21 Oct 2015 15:35:24 -0700 (PDT) Received: from mail-vk0-x235.google.com (mail-vk0-x235.google.com [IPv6:2607:f8b0:400c:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 49D401B32F0 for ; Wed, 21 Oct 2015 15:35:24 -0700 (PDT) Received: by vkex70 with SMTP id x70so37083349vke.3 for ; Wed, 21 Oct 2015 15:35:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=OWNnsI3p5VxfWg/lf7jIjQXD496ht6wqCgGA2SUtwZE=; b=KX88Cb7n/1aVbqaOnQ0nAiQdNF2w44q282cySGxW4lN7FjPrOVMHk3A0c24lS8NDhT Uqjoh7HUQ8A7ZYKGNRiaHgXkZT8D5nN2MFyauVqCY2AxYexT0L8nw0hj3X2/Z3Vxd7YS mihS3BglLBnJOltAx8IoKuwflwqHQWrqQflfl986igxocOo6l8MR0XZ/civ4Q479gKmo l4QTEPiO7Fa1Vra2Zpo36AUQYKN5rXVLLjGW5ZqqEaP7F3Iggk1jLe+x++2gl2t7EMp+ ZWw2AKz4Gk0duP09N43/hcf48IkwCfay/Vi2kT/8tBbxK/otmtR7J9HMwu+71KAX10Ty rC7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=OWNnsI3p5VxfWg/lf7jIjQXD496ht6wqCgGA2SUtwZE=; b=eWvcfHS6tJBv10yoDihTgrKoX+vDWYQE7xxwXowdvn5iTEsqPnLV8r+E5/b5Jrqc2Q cP0sFXWP3QQnBMzl0RedRtdPBoMH9XXjieHXY4WAqEZEpgwXhuujFNCS+xKpAXolLNb6 xu/jkvoXe4iToWNY9IdE7LbCk238kZjLuTfnW9nlSjNLE1LFsr3yaVYUhD4Uryh95JoB gzS0NXHp+5OwarRIyyt3qNqtuzBX6kXoGl42/LHPHnnsv+IpkoBjXloSjJhyaPIS22ap nxjeklOhxDWxGXa0miig/nfFL2OM1eG8mfzd1GE7wqTfnk54WM0VbKq6+FnJKtNl7Bv3 xybA== X-Gm-Message-State: ALoCoQkpHBlo+oiQjCrq6/aCRDHvxFagmlEZL4OEhIXsAJLl+ZbxRya7/OvQzDnhVd4g4JgI8nCN X-Received: by 10.31.49.22 with SMTP id x22mr7197740vkx.60.1445466923336; Wed, 21 Oct 2015 15:35:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.153.134 with HTTP; Wed, 21 Oct 2015 15:34:43 -0700 (PDT) In-Reply-To: <20151021210613.GJ28288@strugglingcoder.info> References: <20151021210613.GJ28288@strugglingcoder.info> From: Yuchung Cheng Date: Wed, 21 Oct 2015 15:34:43 -0700 Message-ID: To: hiren panchasara Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] Regarding sack scoreboard cleanup (rfc6675) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 22:35:26 -0000 On Wed, Oct 21, 2015 at 2:06 PM, hiren panchasara wrote: > (My first ask/question on the mailing list so let me know if this should > be directed somewhere else.) > > I am working on FreeBSD tcp stack and started looking into sack processing. > Trying to understand when/how scoreboard should be cleared when we exit > loss recovery. > > rfc6675 section '5. Algorithm Details' talks about handling each > arriving ACK: > > (A) An incoming cumulative ACK for a sequence number greater than > RecoveryPoint signals the end of loss recovery, and the loss > recovery phase MUST be terminated. Any information contained in > the scoreboard for sequence numbers greater than the new value of > HighACK SHOULD NOT be cleared when leaving the loss recovery > phase. > > First question: Should this say 'greater than or equal to RecoveryPoint' > instead of 'greater than RecoveryPoint'? The RFC is a bit vague. If "An incoming cumulative ACK for a sequence number" means the ACK sequence, then current RFC is correct. If it means the highest sequence that's acked, then it should be >=. > > Another thing I do not understand is how could a scoreboard have information > for sequence numbers grater than the new value of HighACK in this case > when cumulative ACK is covering all the holes (as we are exiting > recovery). This is possible and fairly common. Since HighACK is set upon entering recovery, and you can send new data during the fast recovery (i.e., step (2) in the NextSeg() function). If some of those but not all get lost, you get ACKs with SACK blocks beyond HighACK. In Linux, when this happens, it'll start another fast recovery and reduce ssthresh again if CC is reno/cubic. > > I might be blindsided by just looking at FreeBSD stack or might be > missing something basic in how sack works. In any case, insights would > be useful. > > Cheers, > Hiren > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Wed Oct 21 19:55:56 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ECAEF1ACEE3 for ; Wed, 21 Oct 2015 19:55:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S6YHljgn1bO1 for ; Wed, 21 Oct 2015 19:55:53 -0700 (PDT) Received: from fruitcake.ICSI.Berkeley.EDU (fruitcake.ICSI.Berkeley.EDU [192.150.186.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A39801ACEB1 for ; Wed, 21 Oct 2015 19:55:53 -0700 (PDT) Received: from lawyers.icir.org (envoy.icir.org [192.150.187.30]) by fruitcake.ICSI.Berkeley.EDU (8.12.11.20060614/8.12.11) with ESMTP id t9M2tms1001259; Wed, 21 Oct 2015 19:55:48 -0700 (PDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id C1A562F64E98; Wed, 21 Oct 2015 22:55:47 -0400 (EDT) To: Yuchung Cheng From: Mark Allman In-Reply-To: Organization: International Computer Science Institute (ICSI) Song-of-the-Day: Hey Tonight X-URL-0: http://www.icir.org/mallman-files/Document99748.doc X-URL-1: http://www.icir.org/mallman-files/Document10348.pdf MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-------459435943823349593450"; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Wed, 21 Oct 2015 22:55:47 -0400 Message-ID: <86011.1445482547@lawyers.icir.org> Sender: mallman@icir.org Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] rto-consider updated X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: mallman@icir.org List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 02:55:55 -0000 --=-------459435943823349593450 Content-Type: text/plain > I am interested. :) > Following the spirit of reflecting the reality, The reality I meant to reflect was that the details of RTO implementations have diverged a bit from anything that is standardized. And, that seems to be OK. So, by standard it seems to me that perhaps we should just decide there are a few things that we'd like to see hold and not pretend that all the details are important enough to tightly specify. Because the world has shown us that this is simply not true. > * (3) MUST exp. backoff: I agree on expback off is needed for safety. > But MUST "eventually" exp backoff reflects the reality > better. AFAIK Linux has the "thin-stream" optimization that > optionally use linear back for the first 6? retries. And I recall > vaguely some stacks do so based on some (old) research paper. I dunno ... I guess I still think exponential backoff is the ultimate bit that protects us and so I am a little wary of adding too many wiggle words here (the draft does suggest that exponential doesn't have to be 2x). But, if the WG wants to take this up and the WG consensus is for some sort of eventual exponential backoff then I guess that is OK. > * Would be nice to talk about the well-known tricks for data-center > communications to lower the delayed ACK and RTO. This is what I don't really want to talk about. I don't want to codify tricks. I want to give an umbrella under which various estimators could flourish. > * Knowing the delayed ACK time of the remote matters. Some RFC (1122?) > suggests 500ms. I think everyone uses lower value 40ms - 200ms. Well, ... - I am not sure what you're proposing. Some option to let the receiver give the delayed ACK time? Or, a "delayed" / "not delayed" bit? Whatever it is I think it is beyond the scope of high level guidance. - This does bring to mind a pet peeve which I have sort of fallen into. We talk about the RTO in terms of RTT, but it really should be in terms of expected feedback time. I.e., the sender should not care about the delayed ACK timer. But, the estimator needs to understand when to expect an ACK for a given piece of data. That ACK might be delayed. Or, it might not. But, calling the time between data transmission and ACK reception the "RTT" is sort of mis-leading because it isn't just network latency we care about. > * I saw lots of (naive) attempts to lower the RTO for latency, it > would be nice to talk about the implication of spurious RTO. I try to sketch this tradeoff. I guess I could go back and see if it could be improved. Thanks! allman --=-------459435943823349593450 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlYoUDMACgkQWyrrWs4yIs4mJwCfXgtCfJ4WIGimaCST8trcldrH kCIAoIzcj9WYKMrrzdGJv9c9wJmc8+Tg =JRqe -----END PGP SIGNATURE----- --=-------459435943823349593450-- From nobody Wed Oct 21 21:03:05 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 29E341B29C6 for ; Wed, 21 Oct 2015 21:03:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.514 X-Spam-Level: * X-Spam-Status: No, score=1.514 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, RCVD_IN_DNSWL_LOW=-0.7, RELAY_IS_203=0.994, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GXvAtTIRh_nk for ; Wed, 21 Oct 2015 21:03:02 -0700 (PDT) Received: from mail.sfc.wide.ad.jp (shonan.sfc.wide.ad.jp [203.178.142.130]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9EF8A1B29C3 for ; Wed, 21 Oct 2015 21:03:02 -0700 (PDT) Received: from mail-ob0-f179.google.com (mail-ob0-f179.google.com [209.85.214.179]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 4246E27816C for ; Thu, 22 Oct 2015 13:03:00 +0900 (JST) Received: by obctp1 with SMTP id tp1so31794171obc.2 for ; Wed, 21 Oct 2015 21:02:58 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.182.33.98 with SMTP id q2mr9210957obi.20.1445486578756; Wed, 21 Oct 2015 21:02:58 -0700 (PDT) Received: by 10.202.239.138 with HTTP; Wed, 21 Oct 2015 21:02:58 -0700 (PDT) Date: Wed, 21 Oct 2015 21:02:58 -0700 Message-ID: From: Yoshifumi Nishida To: "tcpm@ietf.org" Content-Type: text/plain; charset=UTF-8 Archived-At: Subject: [tcpm] draft agenda for yokohama meeting X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 04:03:04 -0000 Hi, Sorry for the delay. I uploaded draft agenda for yokohama meeting. Please check the following URL. Please note this is still preliminary. https://www.ietf.org/proceedings/94/agenda/agenda-94-tcpm Please let us know if you have questions or suggestions or requests. Thanks, -- Yoshi Pasi Michael From nobody Thu Oct 22 00:23:33 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 49D401A0127 for ; Thu, 22 Oct 2015 00:23:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.911 X-Spam-Level: X-Spam-Status: No, score=-6.911 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E_rLQNjYX_KS for ; Thu, 22 Oct 2015 00:23:31 -0700 (PDT) Received: from mx142.netapp.com (mx142.netapp.com [216.240.21.19]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 090C61A1B2A for ; Thu, 22 Oct 2015 00:23:31 -0700 (PDT) X-IronPort-AV: E=Sophos;i="5.20,181,1444719600"; d="asc'?scan'208";a="72277786" Received: from hioexcmbx01-prd.hq.netapp.com ([10.122.105.34]) by mx142-out.netapp.com with ESMTP; 22 Oct 2015 00:22:30 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx01-prd.hq.netapp.com (10.122.105.34) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Thu, 22 Oct 2015 00:22:29 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com ([::1]) by hioexcmbx07-prd.hq.netapp.com ([fe80::e1d9:911e:3048:d510%21]) with mapi id 15.00.1104.000; Thu, 22 Oct 2015 00:22:29 -0700 From: "Eggert, Lars" To: Ignacio Goyret Thread-Topic: [tcpm] draft-ietf-tcpm-undeployed-03.txt Thread-Index: AQHRDDHjP7vcks4UvUeEfxzZR3W7xp53kcQA Date: Thu, 22 Oct 2015 07:22:29 +0000 Message-ID: <4F48C202-4DC5-4E37-AECB-3439CFAF8673@netapp.com> References: <20151013125549.31353.52573.idtracker@ietfa.amsl.com> <201510211854.t9LIs2Jr010990@cliff.eng.ascend.com> In-Reply-To: <201510211854.t9LIs2Jr010990@cliff.eng.ascend.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.3096.5) x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.122.56.79] Content-Type: multipart/signed; boundary="Apple-Mail=_83B3B8B2-F5EE-44BF-9419-954F133069C2"; protocol="application/pgp-signature"; micalg=pgp-sha256 MIME-Version: 1.0 Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] draft-ietf-tcpm-undeployed-03.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 07:23:32 -0000 --Apple-Mail=_83B3B8B2-F5EE-44BF-9419-954F133069C2 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Thanks, I fixed this in the working copy and it will appear in -04. On 2015-10-21, at 20:53, Ignacio Goyret wrote: > > Just a small nit: > > on page 4, remove the duplicate "which": > > Was: > o [RFC0889] U, "Internet Delay Experiments", which which describes > ^^^^^^^^^^^ > > Should be: > o [RFC0889] U, "Internet Delay Experiments", which describes > ^ > > -Ignacio > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm --Apple-Mail=_83B3B8B2-F5EE-44BF-9419-954F133069C2 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBViiOtNZcnpRveo1xAQgugQQAuKfgKkw+K9aKHABo+4CvGLwj7ilT2yrZ 3urLtvfVMsShgjvTAby3esDkJZUQxMH4QY4q4kpnGMmWOKalZh3JvrAAtA9Wea/9 ESh6G8ixepvwjq++kt1SqMtPwwRhGjxopB+RQbsAyuCkd4bHHdX+DRuLVZpalJaY No2sHkbCuOs= =hulK -----END PGP SIGNATURE----- --Apple-Mail=_83B3B8B2-F5EE-44BF-9419-954F133069C2-- From nobody Thu Oct 22 01:07:57 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3CC8C1B2EFB for ; Thu, 22 Oct 2015 01:07:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jelj4JY6k3rr for ; Thu, 22 Oct 2015 01:07:54 -0700 (PDT) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9A37C1B2EFA for ; Thu, 22 Oct 2015 01:07:54 -0700 (PDT) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id 24B4C10AD24; Thu, 22 Oct 2015 01:07:54 -0700 (PDT) Date: Thu, 22 Oct 2015 01:07:54 -0700 From: hiren panchasara To: Yuchung Cheng Message-ID: <20151022080754.GP28288@strugglingcoder.info> References: <20151021210613.GJ28288@strugglingcoder.info> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="U0B5otXy6WXfork9" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] Regarding sack scoreboard cleanup (rfc6675) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 08:07:56 -0000 --U0B5otXy6WXfork9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 10/21/15 at 03:34P, Yuchung Cheng wrote: > On Wed, Oct 21, 2015 at 2:06 PM, hiren panchasara > wrote: > > (My first ask/question on the mailing list so let me know if this should > > be directed somewhere else.) > > > > I am working on FreeBSD tcp stack and started looking into sack process= ing. > > Trying to understand when/how scoreboard should be cleared when we exit > > loss recovery. > > > > rfc6675 section '5. Algorithm Details' talks about handling each > > arriving ACK: > > > > (A) An incoming cumulative ACK for a sequence number greater than > > RecoveryPoint signals the end of loss recovery, and the loss > > recovery phase MUST be terminated. Any information contained in > > the scoreboard for sequence numbers greater than the new value of > > HighACK SHOULD NOT be cleared when leaving the loss recovery > > phase. > > > > First question: Should this say 'greater than or equal to RecoveryPoint' > > instead of 'greater than RecoveryPoint'? > The RFC is a bit vague. If "An incoming cumulative ACK for a sequence > number" means the ACK sequence, then current RFC is correct. If it > means the highest sequence that's acked, then it should be >=3D. Okay. >=20 > > > > Another thing I do not understand is how could a scoreboard have inform= ation > > for sequence numbers grater than the new value of HighACK in this case > > when cumulative ACK is covering all the holes (as we are exiting > > recovery). >=20 > This is possible and fairly common. Since HighACK is set upon entering > recovery, and you can send new data during the fast recovery (i.e., > step (2) in the NextSeg() function). If some of those but not all get > lost, you get ACKs with SACK blocks beyond HighACK. Okay, I missed the part where HighACK is not being reset as it enters recovery the second time. >=20 > In Linux, when this happens, it'll start another fast recovery and > reduce ssthresh again if CC is reno/cubic. Thank you! Cheers, Hiren --U0B5otXy6WXfork9 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJWKJlWXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lPV0H/A/WLeIpvvN8+zsjRL8GV9SM QA+ZLGwZMOacK6pxw6PTt55+xZxCmfVgQIQH+auOZrCXimHyOpKNWSUwMiP0KJhc shgAnEniOJTEq3pSyA+OWfc2Faf6RWvFru6DyFQt+yWJm5eD/T6TcoxfFFWT2k4D a+Wj2AY6Zspkohw95N/JATjC4EGW2pu4Qh8jueIHdCvkY11At/r6u0y27PBWzM4C Qv+eAhi9CIn+tq3MsXVtGRdjF6VNONoWR4nQPwhstrHplhpLEzsTP+EEEgci9kLc PV2Em+3ye1l/L0unZQnSulozkiUTp6kXyB4dsdCCNY6UBWtkUgb/BaWUBxHcPvQ= =Q4J3 -----END PGP SIGNATURE----- --U0B5otXy6WXfork9-- From nobody Thu Oct 22 11:53:56 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6F7CF1B3B8B for ; Thu, 22 Oct 2015 11:53:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mc2jWbIzd-2G for ; Thu, 22 Oct 2015 11:53:50 -0700 (PDT) Received: from mail-vk0-x22c.google.com (mail-vk0-x22c.google.com [IPv6:2607:f8b0:400c:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F07581B3B85 for ; Thu, 22 Oct 2015 11:53:38 -0700 (PDT) Received: by vkgy127 with SMTP id y127so52255370vkg.0 for ; Thu, 22 Oct 2015 11:53:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=C7w6lpQsS5yVgqhgtxOYmGpcSx/Vpmj41nAIfFfavkI=; b=BLdTvQO33UPHgS7hmiVf5gdImAnztsi9Hrh7Yc42ANEgWaS3Gf57/bY2LVDH7blmw/ XbBJYspE7bxn/0L/9IbdOvp1n+9FZdPD0AwySbcUfJMZBLmkkQ/Lgazwh73j5bxdJbtO p1lS8pdwTDLeTRhAEcS1HlHFhKQvndZ/BodvkdJ2kW5rh5guUXomc1p9XiU/M6w8NlYQ 5tIomNaclMYqV5ZJRw5Cz7D6ZOtz8ZwcphHqLydCxxYLP3QCebvWFkJe1vLXEiGEgrpj l/3x92jfH+jQg5GNJ5BCSg/MsDEsqHPoQjEXKaJmXHf+TNI41OgdIIL6ngxzLbBiZ+IS 0aHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=C7w6lpQsS5yVgqhgtxOYmGpcSx/Vpmj41nAIfFfavkI=; b=P3C1bJFkLfJJdH6jtP8Ar84He7Z/Urnm0bjS8siqpYL+MBI4+EfMp+j0i18AigmH9z 8VxbVG2KoWQAjvPFE59x/ln8naz5SpbCCD+akhOu/izRgZaBakWERrJ6wy28mTmnhz5h lVxCE/nTbmjGOsUgRY7k35v1quqlNjPlfDydHI70e+GGi25BYJs+ctx4meUKfNI0Q3bC Gv0vaMCWrH3fEpTVvcq2zkwsydr2uofweSF5jabpM5IiMrNzT2K4tRrXGC8B26oqXIZp rnxEKFQwsRT7tRjxD/JdV8kKv6TFIkdz1Z/wnwO9SiC4I4ALB6Y2/V3/2sLulxPfrgis CF2Q== X-Gm-Message-State: ALoCoQkncJDi7huXMDYK2X+iy+8cNA2eReVQUZsCAWWyWJ0VFsGfnjoac2GT77KDpOo0bDYLU+SA X-Received: by 10.31.185.6 with SMTP id j6mr6938951vkf.98.1445540018095; Thu, 22 Oct 2015 11:53:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.153.134 with HTTP; Thu, 22 Oct 2015 11:52:58 -0700 (PDT) In-Reply-To: <86011.1445482547@lawyers.icir.org> References: <86011.1445482547@lawyers.icir.org> From: Yuchung Cheng Date: Thu, 22 Oct 2015 11:52:58 -0700 Message-ID: To: "mallman@icir.org" Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: Jana Iyengar , Ian Swett , "tcpm@ietf.org Extensions" Subject: Re: [tcpm] rto-consider updated X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 18:53:51 -0000 On Wed, Oct 21, 2015 at 7:55 PM, Mark Allman wrote: > >> I am interested. > > :) > >> Following the spirit of reflecting the reality, > > The reality I meant to reflect was that the details of RTO > implementations have diverged a bit from anything that is > standardized. And, that seems to be OK. So, by standard it seems > to me that perhaps we should just decide there are a few things that > we'd like to see hold and not pretend that all the details are > important enough to tightly specify. Because the world has shown us > that this is simply not true. > >> * (3) MUST exp. backoff: I agree on expback off is needed for safety. >> But MUST "eventually" exp backoff reflects the reality >> better. AFAIK Linux has the "thin-stream" optimization that >> optionally use linear back for the first 6? retries. And I recall >> vaguely some stacks do so based on some (old) research paper. > > I dunno ... I guess I still think exponential backoff is the > ultimate bit that protects us and so I am a little wary of adding > too many wiggle words here (the draft does suggest that exponential > doesn't have to be 2x). But, if the WG wants to take this up and > the WG consensus is for some sort of eventual exponential backoff > then I guess that is OK. > >> * Would be nice to talk about the well-known tricks for data-center >> communications to lower the delayed ACK and RTO. > > This is what I don't really want to talk about. I don't want to > codify tricks. I want to give an umbrella under which various > estimators could flourish. OK. but worth articulate these non-goals in the draft. > >> * Knowing the delayed ACK time of the remote matters. Some RFC (1122?) >> suggests 500ms. I think everyone uses lower value 40ms - 200ms. > > Well, ... > > - I am not sure what you're proposing. Some option to let the > receiver give the delayed ACK time? Or, a "delayed" / "not > delayed" bit? Whatever it is I think it is beyond the scope of > high level guidance. > > - This does bring to mind a pet peeve which I have sort of fallen > into. We talk about the RTO in terms of RTT, but it really > should be in terms of expected feedback time. I.e., the sender > should not care about the delayed ACK timer. But, the estimator > needs to understand when to expect an ACK for a given piece of > data. That ACK might be delayed. Or, it might not. But, > calling the time between data transmission and ACK reception the > "RTT" is sort of mis-leading because it isn't just network > latency we care about. agreed. that's also worth discussing in the draft. perhaps a fallacy on designing "RTT" estimator. Another point to consider: Linux is moving toward the model of aggressive timeout to probe (TLP) then a conservative timeout to reset cwnd when entire flight is surely lost. AFAIK QUIC also takes this approach. https://tools.ietf.org/html/draft-tsvwg-quic-loss-recovery-00 I don't know where to fit in your draft, just FYI. > >> * I saw lots of (naive) attempts to lower the RTO for latency, it >> would be nice to talk about the implication of spurious RTO. > > I try to sketch this tradeoff. I guess I could go back and see if > it could be improved. > > Thanks! > > allman > > > From nobody Fri Oct 23 10:38:37 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 15BFF1A88AC for ; Fri, 23 Oct 2015 10:38:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.911 X-Spam-Level: X-Spam-Status: No, score=-6.911 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1T81LOH_hd59 for ; Fri, 23 Oct 2015 10:38:34 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D63871A8899 for ; Fri, 23 Oct 2015 10:38:33 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id A76648242D428; Fri, 23 Oct 2015 17:38:28 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t9NHcSKP032138 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 23 Oct 2015 19:38:31 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Fri, 23 Oct 2015 19:38:28 +0200 From: "Scharf, Michael (Michael)" To: Per Hurtig , Spencer Dawkins Thread-Topic: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt Thread-Index: AQHRC28dlvv2MomATk2ZCyphH35j3551czmAgAPglrA= Date: Fri, 23 Oct 2015 17:38:28 +0000 Message-ID: <655C07320163294895BBADA28372AF5D48524F8A@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <20151020193954.12955.50870.idtracker@ietfa.amsl.com> <3500BC2B-A986-4AC8-BEF0-4709DEFE5D35@kau.se> In-Reply-To: <3500BC2B-A986-4AC8-BEF0-4709DEFE5D35@kau.se> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.38] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 17:38:36 -0000 Sorry for speaking up late... Just to cross-check with others. Is the new w= ording in the abstract... The modification, RTO Restart (RTOR), allows the transport to restart its retransmission timer using a smaller delay, so that the effective RTO becomes more aggressive in situations where fast retransmit cannot be used. This enables faster loss detection and recovery for connections that are short-lived or application- limited. ... indeed clear to everybody? To me (as a non-native speaker) this use of = "delay" in the context of the RTO is a bit confusing. To me, a wording like replacing "delay" by "timeout duration", "timeout val= ue", etc. would sound more familiar. I'd rather use "delay" as a measure be= tween packet (re) transmissions, etc. For instance, to me the following wording would fit a bit better ... The modification, RTO Restart (RTOR), allows the transport to restart its retransmission timer using a smaller timeout du= ration, so that the effective RTO becomes more aggressive in situations where fast retransmit cannot be used. This smaller delay before the retransmi= ssion enables faster loss detection and recovery for connections that are short-lived or application- limited. But I am not a native speaker. Any thoughts? (I guess the RFC editor could just review that.) Michael > -----Original Message----- > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Per Hurtig > Sent: Wednesday, October 21, 2015 9:55 AM > To: tcpm@ietf.org Extensions > Cc: bclaise@cisco.com; barryleiba@computer.org > Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt >=20 > Hi, >=20 > the new draft on RTO restart address the following comments from the > IESG (thanks for the feedback): >=20 > o Clarified, in the abstract, that the modified restart causes a > smaller retransmission delay in total. >=20 > o Clarified, in the introduction, that the fast retransmit algorithm > may cause retransmissions upon > receiving duplicate acknowledgments, not that it unconditionally > does so. >=20 > o Changed wording from "to proposed standard" to "to the standards > track". >=20 > o Changed algorithm description so that a TCP sender MUST track the > time elapsed since the > transmission of the earliest outstanding segment. This was not > explicitly stated in previous > versions of the draft. >=20 >=20 > Cheers, > Per >=20 > > On 20 Oct 2015, at 21:39, internet-drafts@ietf.org wrote: > > > > > > A New Internet-Draft is available from the on-line Internet-Drafts > directories. > > This draft is a work item of the TCP Maintenance and Minor Extensions > Working Group of the IETF. > > > > Title : TCP and SCTP RTO Restart > > Authors : Per Hurtig > > Anna Brunstrom > > Andreas Petlund > > Michael Welzl > > Filename : draft-ietf-tcpm-rtorestart-09.txt > > Pages : 17 > > Date : 2015-10-20 > > > > Abstract: > > This document describes a modified sender-side algorithm for > managing > > the TCP and SCTP retransmission timers that provides faster loss > > recovery when there is a small amount of outstanding data for a > > connection. The modification, RTO Restart (RTOR), allows the > > transport to restart its retransmission timer using a smaller > delay, > > so that the effective RTO becomes more aggressive in situations > where > > fast retransmit cannot be used. This enables faster loss detection > > and recovery for connections that are short-lived or application- > > limited. > > > > > > The IETF datatracker status page for this draft is: > > https://datatracker.ietf.org/doc/draft-ietf-tcpm-rtorestart/ > > > > There's also a htmlized version available at: > > https://tools.ietf.org/html/draft-ietf-tcpm-rtorestart-09 > > > > A diff from the previous version is available at: > > https://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-tcpm-rtorestart-09 > > > > > > Please note that it may take a couple of minutes from the time of > submission > > until the htmlized version and diff are available at tools.ietf.org. > > > > Internet-Drafts are also available by anonymous FTP at: > > ftp://ftp.ietf.org/internet-drafts/ > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri Oct 23 14:49:26 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1FB4A1B2B3B for ; Fri, 23 Oct 2015 14:49:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.85 X-Spam-Level: X-Spam-Status: No, score=-3.85 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v9urXshXNeVL for ; Fri, 23 Oct 2015 14:49:22 -0700 (PDT) Received: from nasse.dc.kau.se (smtp.kau.se [193.10.220.39]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 667181B2B37 for ; Fri, 23 Oct 2015 14:49:22 -0700 (PDT) X-Spam-Processed: mail.kau.se, Fri, 23 Oct 2015 23:49:18 +0200 (not processed: spam filter heuristic analysis disabled) X-MDRemoteIP: 213.113.181.159 X-MDArrival-Date: Fri, 23 Oct 2015 23:49:18 +0200 X-Authenticated-Sender: anna.brunstrom@kau.se X-Return-Path: anna.brunstrom@kau.se X-Envelope-From: anna.brunstrom@kau.se To: "Scharf, Michael (Michael)" , spencerdawkins.ietf@gmail.com References: <20151020193954.12955.50870.idtracker@ietfa.amsl.com> <3500BC2B-A986-4AC8-BEF0-4709DEFE5D35@kau.se> <655C07320163294895BBADA28372AF5D48524F8A@FR712WXCHMBA15.zeu.alcatel-lucent.com> From: Anna Brunstrom Message-ID: <562AAB5E.40502@kau.se> Date: Fri, 23 Oct 2015 23:49:18 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <655C07320163294895BBADA28372AF5D48524F8A@FR712WXCHMBA15.zeu.alcatel-lucent.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 21:49:25 -0000 Hi Michael, I am not a native speaker either, but I agree with you that "timeout duration" or "timeout value" sounds more clear. BR, Anna On 2015-10-23 19:38, Scharf, Michael (Michael) wrote: > Sorry for speaking up late... Just to cross-check with others. Is the new wording in the abstract... > > The modification, RTO Restart (RTOR), allows the > transport to restart its retransmission timer using a smaller delay, > so that the effective RTO becomes more aggressive in situations where > fast retransmit cannot be used. This enables faster loss detection > and recovery for connections that are short-lived or application- > limited. > > ... indeed clear to everybody? To me (as a non-native speaker) this use of "delay" in the context of the RTO is a bit confusing. > > To me, a wording like replacing "delay" by "timeout duration", "timeout value", etc. would sound more familiar. I'd rather use "delay" as a measure between packet (re) transmissions, etc. > > For instance, to me the following wording would fit a bit better ... > > The modification, RTO Restart (RTOR), allows the > transport to restart its retransmission timer using a smaller timeout duration, > so that the effective RTO becomes more aggressive in situations where > fast retransmit cannot be used. This smaller delay before the retransmission enables faster loss detection > and recovery for connections that are short-lived or application- > limited. > > But I am not a native speaker. Any thoughts? > > (I guess the RFC editor could just review that.) > > Michael > > >> -----Original Message----- >> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Per Hurtig >> Sent: Wednesday, October 21, 2015 9:55 AM >> To: tcpm@ietf.org Extensions >> Cc: bclaise@cisco.com; barryleiba@computer.org >> Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt >> >> Hi, >> >> the new draft on RTO restart address the following comments from the >> IESG (thanks for the feedback): >> >> o Clarified, in the abstract, that the modified restart causes a >> smaller retransmission delay in total. >> >> o Clarified, in the introduction, that the fast retransmit algorithm >> may cause retransmissions upon >> receiving duplicate acknowledgments, not that it unconditionally >> does so. >> >> o Changed wording from "to proposed standard" to "to the standards >> track". >> >> o Changed algorithm description so that a TCP sender MUST track the >> time elapsed since the >> transmission of the earliest outstanding segment. This was not >> explicitly stated in previous >> versions of the draft. >> >> >> Cheers, >> Per >> >>> On 20 Oct 2015, at 21:39, internet-drafts@ietf.org wrote: >>> >>> >>> A New Internet-Draft is available from the on-line Internet-Drafts >> directories. >>> This draft is a work item of the TCP Maintenance and Minor Extensions >> Working Group of the IETF. >>> Title : TCP and SCTP RTO Restart >>> Authors : Per Hurtig >>> Anna Brunstrom >>> Andreas Petlund >>> Michael Welzl >>> Filename : draft-ietf-tcpm-rtorestart-09.txt >>> Pages : 17 >>> Date : 2015-10-20 >>> >>> Abstract: >>> This document describes a modified sender-side algorithm for >> managing >>> the TCP and SCTP retransmission timers that provides faster loss >>> recovery when there is a small amount of outstanding data for a >>> connection. The modification, RTO Restart (RTOR), allows the >>> transport to restart its retransmission timer using a smaller >> delay, >>> so that the effective RTO becomes more aggressive in situations >> where >>> fast retransmit cannot be used. This enables faster loss detection >>> and recovery for connections that are short-lived or application- >>> limited. >>> >>> >>> The IETF datatracker status page for this draft is: >>> https://datatracker.ietf.org/doc/draft-ietf-tcpm-rtorestart/ >>> >>> There's also a htmlized version available at: >>> https://tools.ietf.org/html/draft-ietf-tcpm-rtorestart-09 >>> >>> A diff from the previous version is available at: >>> https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-rtorestart-09 >>> >>> >>> Please note that it may take a couple of minutes from the time of >> submission >>> until the htmlized version and diff are available at tools.ietf.org. >>> >>> Internet-Drafts are also available by anonymous FTP at: >>> ftp://ftp.ietf.org/internet-drafts/ >>> >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Sat Oct 24 09:08:00 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DB2991A1BE6 for ; Sat, 24 Oct 2015 09:07:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ExgFbPTEWFJF for ; Sat, 24 Oct 2015 09:07:57 -0700 (PDT) Received: from mail-vk0-x22a.google.com (mail-vk0-x22a.google.com [IPv6:2607:f8b0:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 951981A1BAC for ; Sat, 24 Oct 2015 09:07:57 -0700 (PDT) Received: by vkgs66 with SMTP id s66so18070510vkg.1 for ; Sat, 24 Oct 2015 09:07:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=0SLTHs0WTsnRtUiFgYPg59ZpN6m4OpS1rHcweYMMRt0=; b=atlFingnY6DW0BRab03V90dmC9JnmZ+pk/wPQJ4BLFpG3wC0NL/hOcvEi1baPciU/v dO9KXE1t1DexAsFeqFWkXWN4DvFb61Sa2YaPMJ+lUs/gLBD6r0C65Zy2BYK9is9gtTDF P61Kxm05tgQdCK0wKTaVT34XCgCIr9jc1+chTbpQOKnMEdXdNUTEFZe0/Z7tfxgSjtlL aXCr4UQCK4b/8dTgV7/BLzui3XnfPHlK/RtLUN9+tJASr/GGesn1/HrkZeqNK8e7g88l RUx8x0I798Ju0uKNEwCLbYeecazEaxR+vEn5hQkUNhOxbO0zvE3DOlBqLf8SfdcfE5bI FLRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=0SLTHs0WTsnRtUiFgYPg59ZpN6m4OpS1rHcweYMMRt0=; b=RrsTAyBy6SPc7LYFtmwH/azQIG2ElLZObmO3ogHZJLa7OZ2qlV6OtACAclINA3sp/A L8TpUh7J9c+4sFeGk+l1MIouv93yLTGp9ZU2sK6J5dK08c7dYcI73+7YJUoMH4VaSdeo pLfNP1oMNG2CcZdT+kGl4leMX9L1/sqeYvQNXUTCEbDhBuC++BPzduHKA+IIYy28edwl D6edbBy8Hn6ypIVl4bf4QZpQ/Q3OzMUYpcSaQA6/LvAjMHK9k6E1Z1ER0UaVKqfOxmZc fcvOQvWI1UPxQjDHB2J/T/kfnUBZFNAX4maX9ZysF8bpzCmYfX0a9JscTylJ41+HgZyE 4Vtg== X-Gm-Message-State: ALoCoQkDaBXWo1hBZKuh/pRKpDdaZ9BbJOuFUHYPYbILo7YwebCVPlY+3XfN13icx/uO7o3EEzK8 X-Received: by 10.31.41.149 with SMTP id p143mr17492518vkp.4.1445702876662; Sat, 24 Oct 2015 09:07:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.153.134 with HTTP; Sat, 24 Oct 2015 09:07:17 -0700 (PDT) In-Reply-To: <20151019201145.14841.30865.idtracker@ietfa.amsl.com> References: <20151019201145.14841.30865.idtracker@ietfa.amsl.com> From: Yuchung Cheng Date: Sat, 24 Oct 2015 09:07:17 -0700 Message-ID: To: "tcpm@ietf.org Extensions" Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: Ingemar Johansson S Subject: [tcpm] Fwd: New Version Notification for draft-cheng-tcpm-rack-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2015 16:07:59 -0000 Hi folks, We finally put up a draft for RACK, a new loss detection. RACK has been deployed at Google for well over a year in both TCP and QUIC. The patches are also available for upstream Linux (https://www.mail-archive.com/netdev@vger.kernel.org/msg83215.html) . In the next -01 draft, we plan to further combine RACK with TLP (Tail Loss Probe) because both work together well and TLP will no longer require FACK. Comments are welcome. ---------- Forwarded message ---------- From: Date: Mon, Oct 19, 2015 at 1:11 PM Subject: New Version Notification for draft-cheng-tcpm-rack-00.txt To: Neal Cardwell , Yuchung Cheng A new version of I-D, draft-cheng-tcpm-rack-00.txt has been successfully submitted by Yuchung Cheng and posted to the IETF repository. Name: draft-cheng-tcpm-rack Revision: 00 Title: RACK: a time-based fast loss detection algorithm for TCP Document date: 2015-10-19 Group: Individual Submission Pages: 12 URL: https://www.ietf.org/internet-drafts/draft-cheng-tcpm-rack-00.txt Status: https://datatracker.ietf.org/doc/draft-cheng-tcpm-rack/ Htmlized: https://tools.ietf.org/html/draft-cheng-tcpm-rack-00 Abstract: This document presents a new TCP loss detection algorithm called RACK ("Recent ACKnowledgment"). RACK uses the notion of time, instead of packet or sequence counts, to detect losses, for modern TCP implementations that can support per-packet timestamps and the selective acknowledgment (SACK) option. It is intended to replace the conventional DUPACK threshold approach and its variants, as well as other nonstandard approaches. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat From nobody Mon Oct 26 04:54:39 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1FE0D1B2CCA for ; Mon, 26 Oct 2015 04:54:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.502 X-Spam-Level: X-Spam-Status: No, score=-1.502 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ekUaf7ikJ1Vt for ; Mon, 26 Oct 2015 04:54:38 -0700 (PDT) Received: from fruitcake.ICSI.Berkeley.EDU (fruitcake.ICSI.Berkeley.EDU [192.150.186.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 42A9F1B2CC5 for ; Mon, 26 Oct 2015 04:54:38 -0700 (PDT) Received: from lawyers.icir.org (envoy.icir.org [192.150.187.30]) by fruitcake.ICSI.Berkeley.EDU (8.12.11.20060614/8.12.11) with ESMTP id t9QBsZcm019216; Mon, 26 Oct 2015 04:54:35 -0700 (PDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id 797C02F8AD2B; Mon, 26 Oct 2015 07:54:36 -0400 (EDT) To: Yuchung Cheng From: Mark Allman In-Reply-To: Organization: International Computer Science Institute (ICSI) Song-of-the-Day: Roadhouse Blues X-URL-0: http://www.icir.org/mallman-files/Document49756.docx X-URL-1: http://www.icir.org/mallman-files/Document99036.pdf X-URL-2: http://www.icir.org/mallman-files/Document75272.xls X-URL-3: http://www.icir.org/mallman-files/Document64025.xlsx MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-------459435943823349593450"; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 26 Oct 2015 07:54:36 -0400 Message-ID: <46658.1445860476@lawyers.icir.org> Sender: mallman@icir.org Archived-At: Cc: Jana Iyengar , Ian Swett , "tcpm@ietf.org Extensions" Subject: Re: [tcpm] rto-consider updated X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: mallman@icir.org List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Oct 2015 11:54:39 -0000 --=-------459435943823349593450 Content-Type: text/plain Content-Transfer-Encoding: quoted-printable > >> * Would be nice to talk about the well-known tricks for data-center > >> communications to lower the delayed ACK and RTO. > > > > This is what I don't really want to talk about. I don't want to > > codify tricks. I want to give an umbrella under which various > > estimators could flourish. > > OK. but worth articulate these non-goals in the draft. I thought the goal was pretty clear, but I will certainly revisit it.=20 > agreed. that's also worth discussing in the draft. perhaps a fallacy > on designing "RTT" estimator. Yep. It is on my list to tease this out. (That said, these are great comments, but I am not seeing much interest. So, they may well just be reflected in my tech report.) Thanks, allman --=-------459435943823349593450 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlYuFHwACgkQWyrrWs4yIs4xxwCfRymfd+WgqCeGoVZin13rCbWm qKAAni2OHoOOVDur0tHxUoKnMKrEBR7G =d4Hk -----END PGP SIGNATURE----- --=-------459435943823349593450-- From nobody Mon Oct 26 05:32:39 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D0B521B3C95 for ; Mon, 26 Oct 2015 05:32:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.911 X-Spam-Level: X-Spam-Status: No, score=-6.911 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l8Ae8CTroil4 for ; Mon, 26 Oct 2015 05:32:35 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 47F0E1B3C92 for ; Mon, 26 Oct 2015 05:32:35 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (unknown [135.239.2.42]) by Websense Email Security Gateway with ESMTPS id 1FFF3498F00F1; Mon, 26 Oct 2015 12:32:30 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id t9QCWS0Q001052 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 26 Oct 2015 13:32:32 +0100 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Mon, 26 Oct 2015 13:32:27 +0100 From: "Scharf, Michael (Michael)" To: "mallman@icir.org" , Yuchung Cheng Thread-Topic: [tcpm] rto-consider updated Thread-Index: AQHRD+UdFKj9xbZRvUyML6ZIcHyYBJ59sP7A Date: Mon, 26 Oct 2015 12:32:26 +0000 Message-ID: <655C07320163294895BBADA28372AF5D485278BC@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <46658.1445860476@lawyers.icir.org> In-Reply-To: <46658.1445860476@lawyers.icir.org> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: Jana Iyengar , Ian Swett , "tcpm@ietf.org Extensions" Subject: Re: [tcpm] rto-consider updated X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Oct 2015 12:32:37 -0000 > -----Original Message----- > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Mark Allman > Sent: Monday, October 26, 2015 12:55 PM > To: Yuchung Cheng > Cc: Jana Iyengar; Ian Swett; tcpm@ietf.org Extensions > Subject: Re: [tcpm] rto-consider updated >=20 >=20 > > >> * Would be nice to talk about the well-known tricks for data- > center > > >> communications to lower the delayed ACK and RTO. > > > > > > This is what I don't really want to talk about. I don't want to > > > codify tricks. I want to give an umbrella under which various > > > estimators could flourish. > > > > OK. but worth articulate these non-goals in the draft. >=20 > I thought the goal was pretty clear, but I will certainly revisit > it. For what it is worth, if people (in particular in DCs) play with the RTO es= timator, I think it would be useful to have a TCPM document that provides s= ome guidelines on what is possible and what one should better avoid. This d= ocument seems like a good starting point for that. I don't think we should codify tricks. But we may have to talk a bit more s= pecifically about what people can or cannot do if they (believe to) know th= e RTT is small. To me, this could be an excellent value proposition for a T= CPM document. Michael (as random TCPM contributor) > > agreed. that's also worth discussing in the draft. perhaps a fallacy > > on designing "RTT" estimator. >=20 > Yep. It is on my list to tease this out. >=20 > (That said, these are great comments, but I am not seeing much > interest. So, they may well just be reflected in my tech report.) >=20 > Thanks, > allman >=20 >=20 From nobody Mon Oct 26 19:38:09 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8DB7A1B3369 for ; Mon, 26 Oct 2015 19:38:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D9o94l-vEYWd for ; Mon, 26 Oct 2015 19:38:06 -0700 (PDT) Received: from fruitcake.ICSI.Berkeley.EDU (fruitcake.ICSI.Berkeley.EDU [192.150.186.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 237161B3368 for ; Mon, 26 Oct 2015 19:38:06 -0700 (PDT) Received: from lawyers.icir.org (envoy.icir.org [192.150.187.30]) by fruitcake.ICSI.Berkeley.EDU (8.12.11.20060614/8.12.11) with ESMTP id t9R2c5kg009768 for ; Mon, 26 Oct 2015 19:38:05 -0700 (PDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id E9D7E2F9B9E3 for ; Mon, 26 Oct 2015 22:38:04 -0400 (EDT) To: tcpm@ietf.org From: Mark Allman Organization: International Computer Science Institute (ICSI) Song-of-the-Day: Roadhouse Blues X-URL-0: http://www.icir.org/mallman-files/Document8992.pdf X-URL-1: http://www.icir.org/mallman-files/Document51505.xls X-URL-2: http://www.icir.org/mallman-files/Document22260.xlsx MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-------459435943823349593450"; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 26 Oct 2015 22:38:04 -0400 Message-ID: <37544.1445913484@lawyers.icir.org> Sender: mallman@icir.org Archived-At: Subject: [tcpm] paper on blind attacks X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: mallman@icir.org List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Oct 2015 02:38:07 -0000 --=-------459435943823349593450 Content-Type: text/plain A while back this WG spent some time on blind attacks. I thought folks might be interested in some measurements we did recently on the topic. Matthew will present the paper at IMC this week. See below for details. allman Matthew Luckie, Robert Beverly, Tiange Wu, Mark Allman, kc claffy. Resilience of Deployed TCP to Blind Off-Path Attacks. ACM Internet Measurement Conference, October 2015. http://www.icir.org/mallman/pubs/LBW+15/ Abstract: As part of TCP's steady evolution, recent standards have recommended mechanisms to protect against weaknesses in TCP. But adoption, configuration, and deployment of TCP improvements can be slow. In this work, we consider the resilience of deployed TCP implementations to blind in-window attacks, where an off-path adversary disrupts an established connection by sending a packet that the victim believes came from its peer, causing data corruption or connection reset. We tested operating systems (and middleboxes deployed in front) of webservers in the wild in September 2015 and found 22% of connections vulnerable to in-window SYN and reset packets, 30% vulnerable to in-window data packets, and 38.4% vulnerable to at least one of three in-window attacks we tested. We also tested out-of-window packets and found that while few deployed systems were vulnerable to reset and SYN packets, 5.4% of connections accepted in-window data with an invalid acknowledgment number. In addition to evaluating commodity TCP stacks, we found vulnerabilities in 12 of 14 of the routers and switches we characterized -- critical network infrastructure where the potential impact of any TCP vulnerabilities is particularly acute. This surprisingly high level of extant vulnerabilities in the most mature Internet transport protocol in use today is a perfect illustration of the Internet's fragility. Embedded in historical context, it also provides a strong case for more systematic, scientific, and longitudinal measurement and quantitative analysis of fundamental properties of critical Internet infrastructure, as well as for the importance of better mechanisms to get best security practices deployed. --=-------459435943823349593450 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlYu44wACgkQWyrrWs4yIs63fgCeNaCaRGJ5xZ+rVvA93orqkmNj vBYAmwVaHet5q6uTd07xela8A8tkrCo9 =rhSb -----END PGP SIGNATURE----- --=-------459435943823349593450-- From nobody Tue Oct 27 08:26:12 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C9B61A905A for ; Tue, 27 Oct 2015 08:26:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.011 X-Spam-Level: X-Spam-Status: No, score=-5.011 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eKqWMaAyNBgQ for ; Tue, 27 Oct 2015 08:26:09 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CCEF21A907D for ; Tue, 27 Oct 2015 08:26:09 -0700 (PDT) Received: from praxis.lunabase.org (cpe-104-172-233-72.socal.res.rr.com [104.172.233.72]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t9RFPRIr029513 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 27 Oct 2015 08:25:36 -0700 (PDT) To: tcpm@ietf.org References: <46658.1445860476@lawyers.icir.org> From: Ted Faber Message-ID: <562F9767.5080901@isi.edu> Date: Tue, 27 Oct 2015 08:25:27 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <46658.1445860476@lawyers.icir.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: faber@isi.edu Archived-At: Cc: faber@isi.edu Subject: Re: [tcpm] rto-consider updated X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Oct 2015 15:26:11 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 10/26/2015 04:54 AM, Mark Allman wrote: > >>>> * Would be nice to talk about the well-known tricks for >>>> data-center communications to lower the delayed ACK and RTO. >>> >>> This is what I don't really want to talk about. I don't want >>> to codify tricks. I want to give an umbrella under which >>> various estimators could flourish. >> >> OK. but worth articulate these non-goals in the draft. > > I thought the goal was pretty clear, but I will certainly revisit > it. I did, too. > >> agreed. that's also worth discussing in the draft. perhaps a >> fallacy on designing "RTT" estimator. > > Yep. It is on my list to tease this out. > > (That said, these are great comments, but I am not seeing much > interest. So, they may well just be reflected in my tech report.) I'd like to see it published. IMHO, it is ready for last call. - -- Ted Faber, Computer Scientist, USC/ISI http://www.isi.edu/~faber http://www.isi.edu/~faber/contact.html -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJWL5dnAAoJELKW6dHrttp3A+MH/3BgrqseIVSScL7+HFqYy6LL GL3zEuEQYub4cF+ndvDAKRYFuHCYp/DkDkOtFYEaqZyngMJ78LgralPv1YIHi6oa 1sjAHkBp8aQAkeA2S0jZoZVgF1d8NP80OsgYFceaGeCMdYCfygFEnx3tkkiVveNB OY5yG04tFO1B2lSkMDwwbcyMomk+5SUJClUzHtM+MigcrTdasbLSSc7r9NwXM5pF 2vXgjRVzSuhzOhIw3llP9xExvemvj+ALSb/TYsKRfuCkNfDkSH0Clrrd4IUhqF0h ltjHnewNvfAAFTlngO8lORpRhbFIAMvCZ2/otUrtcDiYPSJsclootRrd3Mqg0kc= =8edX -----END PGP SIGNATURE----- From nobody Tue Oct 27 16:16:01 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4D0401B2DBC; Tue, 27 Oct 2015 16:16:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.912 X-Spam-Level: X-Spam-Status: No, score=-106.912 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QV_iEz3ca3W5; Tue, 27 Oct 2015 16:15:58 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [4.31.198.49]) by ietfa.amsl.com (Postfix) with ESMTP id AC9D71B2DA2; Tue, 27 Oct 2015 16:15:51 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id 5EA29187E36; Tue, 27 Oct 2015 16:15:15 -0700 (PDT) To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org X-PHP-Originating-Script: 1005:ams_util_lib.php From: rfc-editor@rfc-editor.org Message-Id: <20151027231515.5EA29187E36@rfc-editor.org> Date: Tue, 27 Oct 2015 16:15:15 -0700 (PDT) Archived-At: Cc: drafts-update-ref@iana.org, tcpm@ietf.org, rfc-editor@rfc-editor.org Subject: [tcpm] RFC 7661 on Updating TCP to Support Rate-Limited Traffic X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Oct 2015 23:16:00 -0000 A new Request for Comments is now available in online RFC libraries. RFC 7661 Title: Updating TCP to Support Rate-Limited Traffic Author: G. Fairhurst, A. Sathiaseelan, R. Secchi Status: Experimental Stream: IETF Date: October 2015 Mailbox: gorry@erg.abdn.ac.uk, arjuna@erg.abdn.ac.uk, raffaello@erg.abdn.ac.uk Pages: 21 Characters: 51267 Obsoletes: RFC 2861 I-D Tag: draft-ietf-tcpm-newcwv-13.txt URL: https://www.rfc-editor.org/info/rfc7661 DOI: http://dx.doi.org/10.17487/RFC7661 This document provides a mechanism to address issues that arise when TCP is used for traffic that exhibits periods where the sending rate is limited by the application rather than the congestion window. It provides an experimental update to TCP that allows a TCP sender to restart quickly following a rate-limited interval. This method is expected to benefit applications that send rate-limited traffic using TCP while also providing an appropriate response if congestion is experienced. This document also evaluates the Experimental specification of TCP Congestion Window Validation (CWV) defined in RFC 2861 and concludes that RFC 2861 sought to address important issues but failed to deliver a widely used solution. This document therefore reclassifies the status of RFC 2861 from Experimental to Historic. This document obsoletes RFC 2861. This document is a product of the TCP Maintenance and Minor Extensions Working Group of the IETF. EXPERIMENTAL: This memo defines an Experimental Protocol for the Internet community. It does not specify an Internet standard of any kind. Discussion and suggestions for improvement are requested. Distribution of this memo is unlimited. This announcement is sent to the IETF-Announce and rfc-dist lists. To subscribe or unsubscribe, see https://www.ietf.org/mailman/listinfo/ietf-announce https://mailman.rfc-editor.org/mailman/listinfo/rfc-dist For searching the RFC series, see https://www.rfc-editor.org/search For downloading RFCs, see https://www.rfc-editor.org/rfc.html Requests for special distribution should be addressed to either the author of the RFC in question, or to rfc-editor@rfc-editor.org. Unless specifically noted otherwise on the RFC itself, all RFCs are for unlimited distribution. The RFC Editor Team Association Management Solutions, LLC From nobody Wed Oct 28 06:46:59 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C189A1A88B8 for ; Wed, 28 Oct 2015 06:46:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.301 X-Spam-Level: X-Spam-Status: No, score=-2.301 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aLJNhuiFf9dF for ; Wed, 28 Oct 2015 06:46:56 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 25E261A88B2 for ; Wed, 28 Oct 2015 06:46:56 -0700 (PDT) Received: from 58.172.199.146.dyn.plus.net ([146.199.172.58]:59695 helo=[192.168.0.15]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.86) (envelope-from ) id 1ZrR40-00038h-Tb; Wed, 28 Oct 2015 13:46:54 +0000 To: =?UTF-8?Q?Mirja_K=c3=bchlewind?= , Praveen Balasubramanian References: <55F055AD.3050809@tik.ee.ethz.ch> <55F05D54.5060708@tik.ee.ethz.ch> <55FF2910.7080908@bobbriscoe.net> <561DA02D.8020001@bobbriscoe.net> <0F4F0BA2-A1AE-40BB-9E0D-0EB5F4DE79E1@tik.ee.ethz.ch> <5623D09C.4020002@bobbriscoe.net> <5627B316.1090702@tik.ee.ethz.ch> From: Bob Briscoe Message-ID: <5630D19D.80602@bobbriscoe.net> Date: Wed, 28 Oct 2015 13:46:05 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <5627B316.1090702@tik.ee.ethz.ch> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net Archived-At: Cc: tcpm IETF list , Richard Scheffenegger Subject: Re: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2015 13:46:58 -0000 Mirja, Surely, even if the TCP option is stripped, our proposed AccECN mechanism provides the same info as DCTCP currently does, with the addition of robustness against ACK loss (because we use a 3-bit counter, whereas DCTCP uses a 1-bit flag). At least I believe that's the case if the receiver complies with our change-triggered feedback requirement, which we have stated as a 'SHOULD' and emphasised that it means "you really really SHOULD unless you cannot". Just like DCTCP, the sender can then calculate the CE bytes from the number of bytes the ACK acknowledges. Bob On 21/10/15 16:45, Mirja Khlewind wrote: > Hi Bob, > > no, there is no estimation in the current DCTCP implementation because > the feedback mechanism they use indicates that either all bytes that a > certain ACK acknowledges have been marked or not; so you directly have > the number of bytes. > > That means effectively AccECN without the Option would provide you > less information than you currently have with DCTCP's feedback > mechanism. However, in DCTCP there is a risk of loosing > information/obtaining wrong information if ACKs get lost. This might > be a rare case in data centers, however, I'd still argue for safety here. > > Mirja > > > On 18.10.2015 19:02, Bob Briscoe wrote: >> Praveen, >> >> Sorry, there's ambiguity in these emails between the units of feedback >> and the units of calculation of the congestion response. Current DCTCP >> feedback (both Windows and Linux) is in packets, then the sender >> calculates/estimates what this means in bytes which it applies in the >> window reduction algo. This calculation/estimation is accurate for >> long-running flows, but hard for app-limited flows where there are a lot >> of sub-MSS segments. >> >> The example congestion response I gave (Relentless) uses bytes for >> feedback and bytes for the response calculation. This works because >> Relentless is designed for loss and SACK gives loss feedback in bytes >> (Relentless didn't include any change to ECN feedback). >> >> The AccECN draft-04 proposes essential feedback in packets using the >> main TCP header and supplementary feedback in bytes using a new AccECN >> TCP option (middleboxes permitting). If the AccECN option gets through >> OK, it means the sender has bytes directly, so it doesn't have to >> estimate or calculate bytes from packet feedback. >> >> I should add that providing byte counters wasn't the main reason for the >> TCP option. It was primarily to improve protocol resilience in the face >> of heavy ACK loss (when congestion avoidance could be most critical), >> because the 3-bit field available for the packet counter in the main >> header is rather small (it wraps every 8 packets, which is at least >> better than the 1-bit feedback in current DCTCP). >> >> So, once we added the TCP option for resilience, we could use it for >> full byte feedback instead of packet, and we could also provide enough >> info to detect bleaching of the ECN field and to support potential >> future use of ECT(1) as well as ECT(0). >> >> Hope this clarifies the confusion. >> >> >> Bob >> >> On 16/10/15 17:32, Praveen Balasubramanian wrote: >>> Even the Windows implementation uses the number of bytes in CE >>> marked packets. Bob, I am not sure why you think extent of >>> congestion is based on packets and not bytes, please check section >>> 3.3 in the draft. >>> >>> -----Original Message----- >>> From: Mirja Khlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch] >>> Sent: Friday, October 16, 2015 12:22 AM >>> To: Bob Briscoe >>> Cc: Praveen Balasubramanian ; tcpm IETF list >>> ; Richard Scheffenegger >>> Subject: Re: [tcpm] New Version Notification for >>> draft-kuehlewind-tcpm-accurate-ecn-04.txt >>> >>> Hi, >>> >>>> Am 14.10.2015 um 02:22 schrieb Bob Briscoe : >>>> >>>> At present DCTCP measures the extent of congestion in packets not >>>> bytes. But we envisaged that an algorithm like Relentless TCP might >>>> be used where, every time new CE feedback arrives, the sender just >>>> decrements cwnd by (CE bytes)/2. >>> As a side note, the current DCTCP implementation in Linux uses >>> number of bytes... >>> >> -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ From nobody Wed Oct 28 09:01:24 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CF2F81A90C6 for ; Wed, 28 Oct 2015 09:01:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uYCpS3BYWFLQ for ; Wed, 28 Oct 2015 09:01:17 -0700 (PDT) Received: from sesbmg22.ericsson.net (sesbmg22.ericsson.net [193.180.251.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 85E4B1A904A for ; Wed, 28 Oct 2015 09:01:16 -0700 (PDT) X-AuditID: c1b4fb30-f79626d000006adf-50-5630f14a443f Received: from ESESSHC014.ericsson.se (Unknown_Domain [153.88.253.125]) by sesbmg22.ericsson.net (Symantec Mail Security) with SMTP id 4C.A6.27359.A41F0365; Wed, 28 Oct 2015 17:01:14 +0100 (CET) Received: from ESESSMB205.ericsson.se ([169.254.5.167]) by ESESSHC014.ericsson.se ([153.88.183.60]) with mapi id 14.03.0248.002; Wed, 28 Oct 2015 17:01:13 +0100 From: Ingemar Johansson S To: Yuchung Cheng , "tcpm@ietf.org" Thread-Topic: New Version Notification for draft-cheng-tcpm-rack-00.txt Thread-Index: AQHRDnYmRuHtiOYliUy6k+19xhevN56BFH8g Date: Wed, 28 Oct 2015 16:01:13 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA34C0F674@ESESSMB205.ericsson.se> References: <20151019201145.14841.30865.idtracker@ietfa.amsl.com> In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.17] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupmkeLIzCtJLcpLzFFi42KZGfG3Vtfro0GYwYfXChYdd/ayWGw7OZ/J 4svjq2wOzB4LNpV6LFnykymAKYrLJiU1J7MstUjfLoErY/bRJewFN2QqPp59xNrA+EO6i5GT Q0LAROL3wb+MELaYxIV769m6GLk4hASOMErM3zWVHcJZwiix8vheJpAqNgEbiZWHvoN1iAi4 SexccIUFxGYW0JBoefWGFcQWBopPfXmRCaLGXWL22zlsELaRxO8ds8HiLAKqEsuXPAOr5xXw lbhw+yYjxLIORonnEzrBFnAKBEp82bIErIhRQFbi/vd7UMvEJW49mc8EcbaAxJI955khbFGJ l4//AdVzANmKEsv75UBMZgFNifW79CE6FSWmdD9kh1grKHFy5hOWCYxis5AMnYXQMQtJxywk HQsYWVYxihanFiflphsZ6aUWZSYXF+fn6eWllmxiBEbRwS2/DXYwvnzueIhRgINRiYf3wyOD MCHWxLLiytxDjBIczEoivPPeAIV4UxIrq1KL8uOLSnNSiw8xSnOwKInzNjM9CBUSSE8sSc1O TS1ILYLJMnFwSjUwrn3Qw9hwIk5BnSFc8MOrSiWeT/x3sx4Inp7yKW6rZHz2dLeJdy9ulK1f ujuAeWeu4W6m0IcxE67N+Xjt89Fv7wWTdnqE/ViYc0El3CLVLDtt+8G6O8ohZ9iCG47cnPFz 02/2n0Wv1zbNVIqKq2m+kSPzN1Pk4/3PBkoxb+flNi/P9Vwl/kuiUImlOCPRUIu5qDgRACzA SRyeAgAA Archived-At: Subject: Re: [tcpm] New Version Notification for draft-cheng-tcpm-rack-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2015 16:01:23 -0000 VGhhbmtzLCByZWFsbHkgaW50ZXJlc3RpbmcuIA0KDQpPbmUgcXVlc3Rpb24gLiANClF1b3RlICJX aGVuIHRoZSBzZW5kZXIgZGV0ZWN0cyBwYWNrZXQgIHJlb3JkZXJpbmcgUkFDSy5yZW9fd25kIE1B WSBiZSBjaGFuZ2VkIHRvIFJBQ0subWluX1JUVC80LiINCldoeSB0aGUgdmFsdWUgTWluUlRULzQg PyBJcyB0aGlzIHJlbGF0ZWQgdG8gVExQID8uIA0KT25lIG9wdGlvbiBtYXkgYmUgdG8gc2V0IFJB Q0sucmVvX3duZCBhY2NvcmRpbmcgdG8gdGhlIGV4cGVyaWVuY2VkIGFtb3VudCBvZiByZW9yZGVy aW5nIGJ1dCB0aGF0IGNhbiBvZiBjb3Vyc2UgZGVsYXkgcmV0cmFuc21pc3Npb24gb2YgbG9zdCBw YWNrZXQuDQoNCi9JbmdlbWFyDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJv bTogWXVjaHVuZyBDaGVuZyBbbWFpbHRvOnljaGVuZ0Bnb29nbGUuY29tXQ0KPiBTZW50OiBkZW4g MjQgb2t0b2JlciAyMDE1IDE4OjA3DQo+IFRvOiB0Y3BtQGlldGYub3JnIEV4dGVuc2lvbnMNCj4g Q2M6IE5lYWwgQ2FyZHdlbGw7IEluZ2VtYXIgSm9oYW5zc29uIFMNCj4gU3ViamVjdDogRndkOiBO ZXcgVmVyc2lvbiBOb3RpZmljYXRpb24gZm9yIGRyYWZ0LWNoZW5nLXRjcG0tcmFjay0wMC50eHQN Cj4gDQo+IEhpIGZvbGtzLA0KPiANCj4gV2UgZmluYWxseSBwdXQgdXAgYSBkcmFmdCBmb3IgUkFD SywgYSBuZXcgbG9zcyBkZXRlY3Rpb24uDQo+IA0KPiBSQUNLIGhhcyBiZWVuIGRlcGxveWVkIGF0 IEdvb2dsZSBmb3Igd2VsbCBvdmVyIGEgeWVhciBpbiBib3RoIFRDUCBhbmQNCj4gUVVJQy4gVGhl IHBhdGNoZXMgYXJlIGFsc28gYXZhaWxhYmxlIGZvciB1cHN0cmVhbSBMaW51eA0KPiAoaHR0cHM6 Ly93d3cubWFpbC1hcmNoaXZlLmNvbS9uZXRkZXZAdmdlci5rZXJuZWwub3JnL21zZzgzMjE1Lmh0 bWwpIC4NCj4gSW4gdGhlIG5leHQgLTAxIGRyYWZ0LCB3ZSBwbGFuIHRvIGZ1cnRoZXIgY29tYmlu ZSBSQUNLIHdpdGggVExQIChUYWlsIExvc3MNCj4gUHJvYmUpIGJlY2F1c2UgYm90aCB3b3JrIHRv Z2V0aGVyIHdlbGwgYW5kIFRMUCB3aWxsIG5vIGxvbmdlciByZXF1aXJlIEZBQ0suDQo+IA0KPiBD b21tZW50cyBhcmUgd2VsY29tZS4NCj4gDQo+IC0tLS0tLS0tLS0gRm9yd2FyZGVkIG1lc3NhZ2Ug LS0tLS0tLS0tLQ0KPiBGcm9tOiAgPGludGVybmV0LWRyYWZ0c0BpZXRmLm9yZz4NCj4gRGF0ZTog TW9uLCBPY3QgMTksIDIwMTUgYXQgMToxMSBQTQ0KPiBTdWJqZWN0OiBOZXcgVmVyc2lvbiBOb3Rp ZmljYXRpb24gZm9yIGRyYWZ0LWNoZW5nLXRjcG0tcmFjay0wMC50eHQNCj4gVG86IE5lYWwgQ2Fy ZHdlbGwgPG5jYXJkd2VsbEBnb29nbGUuY29tPiwgWXVjaHVuZyBDaGVuZw0KPiA8eWNoZW5nQGdv b2dsZS5jb20+DQo+IA0KPiANCj4gDQo+IEEgbmV3IHZlcnNpb24gb2YgSS1ELCBkcmFmdC1jaGVu Zy10Y3BtLXJhY2stMDAudHh0IGhhcyBiZWVuIHN1Y2Nlc3NmdWxseQ0KPiBzdWJtaXR0ZWQgYnkg WXVjaHVuZyBDaGVuZyBhbmQgcG9zdGVkIHRvIHRoZSBJRVRGIHJlcG9zaXRvcnkuDQo+IA0KPiBO YW1lOiAgICAgICAgICAgZHJhZnQtY2hlbmctdGNwbS1yYWNrDQo+IFJldmlzaW9uOiAgICAgICAw MA0KPiBUaXRsZTogICAgICAgICAgUkFDSzogYSB0aW1lLWJhc2VkIGZhc3QgbG9zcyBkZXRlY3Rp b24gYWxnb3JpdGhtIGZvciBUQ1ANCj4gRG9jdW1lbnQgZGF0ZTogIDIwMTUtMTAtMTkNCj4gR3Jv dXA6ICAgICAgICAgIEluZGl2aWR1YWwgU3VibWlzc2lvbg0KPiBQYWdlczogICAgICAgICAgMTIN Cj4gVVJMOg0KPiBodHRwczovL3d3dy5pZXRmLm9yZy9pbnRlcm5ldC1kcmFmdHMvZHJhZnQtY2hl bmctdGNwbS1yYWNrLTAwLnR4dA0KPiBTdGF0dXM6ICAgICAgICAgaHR0cHM6Ly9kYXRhdHJhY2tl ci5pZXRmLm9yZy9kb2MvZHJhZnQtY2hlbmctdGNwbS1yYWNrLw0KPiBIdG1saXplZDogICAgICAg aHR0cHM6Ly90b29scy5pZXRmLm9yZy9odG1sL2RyYWZ0LWNoZW5nLXRjcG0tcmFjay0wMA0KPiAN Cj4gDQo+IEFic3RyYWN0Og0KPiAgICBUaGlzIGRvY3VtZW50IHByZXNlbnRzIGEgbmV3IFRDUCBs b3NzIGRldGVjdGlvbiBhbGdvcml0aG0gY2FsbGVkIFJBQ0sNCj4gICAgKCJSZWNlbnQgQUNLbm93 bGVkZ21lbnQiKS4gIFJBQ0sgdXNlcyB0aGUgbm90aW9uIG9mIHRpbWUsIGluc3RlYWQgb2YNCj4g ICAgcGFja2V0IG9yIHNlcXVlbmNlIGNvdW50cywgdG8gZGV0ZWN0IGxvc3NlcywgZm9yIG1vZGVy biBUQ1ANCj4gICAgaW1wbGVtZW50YXRpb25zIHRoYXQgY2FuIHN1cHBvcnQgcGVyLXBhY2tldCB0 aW1lc3RhbXBzIGFuZCB0aGUNCj4gICAgc2VsZWN0aXZlIGFja25vd2xlZGdtZW50IChTQUNLKSBv cHRpb24uICBJdCBpcyBpbnRlbmRlZCB0byByZXBsYWNlDQo+ICAgIHRoZSBjb252ZW50aW9uYWwg RFVQQUNLIHRocmVzaG9sZCBhcHByb2FjaCBhbmQgaXRzIHZhcmlhbnRzLCBhcyB3ZWxsDQo+ICAg IGFzIG90aGVyIG5vbnN0YW5kYXJkIGFwcHJvYWNoZXMuDQo+IA0KPiANCj4gDQo+IA0KPiBQbGVh c2Ugbm90ZSB0aGF0IGl0IG1heSB0YWtlIGEgY291cGxlIG9mIG1pbnV0ZXMgZnJvbSB0aGUgdGlt ZSBvZiBzdWJtaXNzaW9uDQo+IHVudGlsIHRoZSBodG1saXplZCB2ZXJzaW9uIGFuZCBkaWZmIGFy ZSBhdmFpbGFibGUgYXQgdG9vbHMuaWV0Zi5vcmcuDQo+IA0KPiBUaGUgSUVURiBTZWNyZXRhcmlh dA0K From nobody Wed Oct 28 09:56:30 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C33781B5D6B for ; Wed, 28 Oct 2015 09:56:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CpQLyQhlSQKe for ; Wed, 28 Oct 2015 09:56:27 -0700 (PDT) Received: from mail-yk0-x233.google.com (mail-yk0-x233.google.com [IPv6:2607:f8b0:4002:c07::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A25FF1B5B3C for ; Wed, 28 Oct 2015 09:48:26 -0700 (PDT) Received: by ykft191 with SMTP id t191so14150583ykf.0 for ; Wed, 28 Oct 2015 09:48:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=j+4Kay6O6ioHL6byFfLicNelTa0jWkrS05FjDQIiYbY=; b=pdglPBg6voexUoF9lrhg3P6kJgDssvSVAOP5BPXIyDom5yFzMg+aO3dgRCCc6wsKGL UwUEpavDgQLsjD0SYOnPA2xQ+k0SMx1t3AJVK4m9KKNhdEMhMgJtJim3qoSkPuIYIhgl fDEqfzzQaHPzzHgrIL4KSpvA9TeI2JvqK5hjAQNU/MYE+MoIJyGV63E8V8NAHb9E2C+N Owx48yoV8X9okag81S6kpivZXg8UujnN9MzO5b9EYa1GVLDg/wxNbqhmtcsBVl//JRDb /G71zluU1jDICcG9bDrhnZ9f1G9VOBun9SZbAvcGLdkTatGqpxu3u8lcAMFDLgTTOa6A APcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=j+4Kay6O6ioHL6byFfLicNelTa0jWkrS05FjDQIiYbY=; b=WufPYM8sAF9S2JBjad+9MdmD1zozAR9qQFrSVWIWS+thmNZsYsl6pPixXIicSG23bd 3KihgjODN5qt4kyDt5h1tg9hI5g1PpJ9iCQEY3/2VzbsJlN54l+ygWO7KfzPFJUFg7ur rjT0PVur1PMRVDrpmCcdGy3liG6aKhQZyQegak2/U3yI09Ghuec2uIycHZBZYS3tDcFf KP4cIO9s6UEs92FOWWCOC5tndPId6WSwuP9nYNz/+N+fYjGb/CGMHt/s5rLpxM2CGode dBu4HrQWPxHk48zSqL2mBSg+GFpTzS6MUQua6gFfqFts9B1p7hiFqTcfEvn52CYhBr0b kjww== X-Gm-Message-State: ALoCoQlEcJKFEg4XRxZ2IHIDwSuSP/5fsq+DsjQ8IopjLvIIY1o0pL1Wl7ammIBTIuOdfhRuOmGj MIME-Version: 1.0 X-Received: by 10.129.94.132 with SMTP id s126mr28268644ywb.268.1446050905891; Wed, 28 Oct 2015 09:48:25 -0700 (PDT) Received: by 10.129.81.136 with HTTP; Wed, 28 Oct 2015 09:48:25 -0700 (PDT) In-Reply-To: <81564C0D7D4D2A4B9A86C8C7404A13DA34C0F674@ESESSMB205.ericsson.se> References: <20151019201145.14841.30865.idtracker@ietfa.amsl.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34C0F674@ESESSMB205.ericsson.se> Date: Wed, 28 Oct 2015 12:48:25 -0400 Message-ID: From: Neal Cardwell To: Ingemar Johansson S Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] New Version Notification for draft-cheng-tcpm-rack-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2015 16:56:28 -0000 On Wed, Oct 28, 2015 at 12:01 PM, Ingemar Johansson S wrote: > > Thanks, really interesting. > > One question . > Quote "When the sender detects packet reordering RACK.reo_wnd MAY be changed to RACK.min_RTT/4." > Why the value MinRTT/4 ? Is this related to TLP ?. Good question. The rationale is discussed in section 5.3, "Adjusting the reordering window": RACK uses a quarter of minimum RTT because Linux TCP uses the same factor in its implementation to delay Early Retransmit [RFC5827] to reduce spurious loss detections in the presence of reordering, and experience shows that this seems to work reasonably well. > One option may be to set RACK.reo_wnd according to > the experienced amount of reordering but that can of course > delay retransmission of lost packet. Yes, we agree that investigating this kind of approach is appealing, and this is also discussed in section 5.3, "Adjusting the reordering window": One potential improvement is to further adapt the reordering window by measuring the degree of reordering in time, instead of packet distances. But that requires storing the delivery timestamp of each packet. Some scoreboard implementations currently merge SACKed packets together to support TSO (TCP Segmentation Offload) for faster scoreboard indexing. Supporting per-packet delivery timestamps is difficult in such implementations. However, we acknowledge that the current metric can be improved by further research. Thanks for the feedback! neal From nobody Thu Oct 29 01:37:09 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 38DE61B2B04 for ; Thu, 29 Oct 2015 01:37:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.378 X-Spam-Level: X-Spam-Status: No, score=-1.378 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zjSBch-WLr_4 for ; Thu, 29 Oct 2015 01:37:06 -0700 (PDT) Received: from mail-ig0-x22c.google.com (mail-ig0-x22c.google.com [IPv6:2607:f8b0:4001:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 919D71B2B03 for ; Thu, 29 Oct 2015 01:37:06 -0700 (PDT) Received: by igdg1 with SMTP id g1so126357829igd.1 for ; Thu, 29 Oct 2015 01:37:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tieto.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:content-type; bh=myOifJnzqpd1pEpdO8Oqk2/n99dWqM6vzjx30AlmKRA=; b=IEcxSKIbLtN2yaZu0UyMSfvW03MdQUGmpQuzym/HwtqVFDLzaoZYZekUJPvZy/NPhH ltQPk7iSyC1g3m3K4N4CgJ+Ider3cOaK4s2YjTQC9gdfAikGeyKW/h0HZZPazryd9KeF p6ai//X37PLlMbgoZB9fIat3Xx7GpBcr2SJ+s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:content-type; bh=myOifJnzqpd1pEpdO8Oqk2/n99dWqM6vzjx30AlmKRA=; b=lG5A3TO9VDUd2YZCvgKX53WWzEskVDNSPoTkrCSFwESXRnBKz0pcxMCvfUeRRlA05O Eak0dWV4YaYL4wPz8KgVXozYIf9uGjtQ66dV2FuR3Gw34Rg+OTpObUMBIDSkNKlq7Uvq FxNYKcb6/mzTvrR3Q/HeYPLgmixTVqp4xr4lkv/N+AHRLpdQ2EqF/T3kR5ETDV0IWrL0 vMwsRjDqimqLBxHwdVQi3v/Cslk2s7U8zwbwJu7tT0ilQhIvGvjkpYxR9pWP0hnidjps 84Ao8VXWy/jiwlwK27NDqpkV5dnBqLs4HPT9sxthjdS4S+qsb6WR9osoGhWbzsm2uu+X XgzA== X-Gm-Message-State: ALoCoQnxbz/czn36grHk2DFRzNLT0cOi45QtenG6i9S9s1A1e1r15eSMRUcpKy401JkdtLLfXdgQfyhOP02rDDFnICBzndJa8VyK1GMQ+bdl0BseH8tLssA= X-Received: by 10.50.85.110 with SMTP id g14mr1540924igz.96.1446107826043; Thu, 29 Oct 2015 01:37:06 -0700 (PDT) From: Karen Elisabeth Egede Nielsen References: In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQERYzfIDv1QELMXHoYR/jqgkDakQ6ABgzkg Date: Thu, 29 Oct 2015 09:37:05 +0100 Message-ID: To: Yuchung Cheng , tcpm@ietf.org Content-Type: multipart/alternative; boundary=089e0149c092474f6605233a378f X-DomainID: tieto.com Archived-At: Subject: Re: [tcpm] Question to TCP implementors regarding dupack threshold X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Oct 2015 08:37:08 -0000 --089e0149c092474f6605233a378f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable HI, We do this in our in-house TCP implementation today. This is what RFC6675 prescribes for =E2=80=93 do you not agree ? BR, Karen *From:* tcpm [mailto:tcpm-bounces@ietf.org] *On Behalf Of *Yuchung Cheng *Sent:* 17. oktober 2015 18:48 *To:* tcpm@ietf.org Extensions *Subject:* [tcpm] Question to TCP implementors regarding dupack threshold Hi, The question is: When a TCP sender receives a DUPACK with SACK option(s) covering at least three packets (or three MSS bytes), will it trigger fast recovery immediately? I know Linux does (for at least +8 years). I am curious about other implementations like Windows, Mac, *BSD? Thanks. Yuchung --089e0149c092474f6605233a378f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

HI,

=C2=A0

We do this in our in-house TCP implementa= tion today.

= =C2=A0

This = is what RFC6675 prescribes for =E2=80=93 do you not agree ?

=C2=A0

BR, Karen

=C2=A0

<= div style=3D"border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0= cm 0cm">

From: tcpm [mailto:tcpm-bounces@= ietf.org] On Behalf Of Yuchung Cheng
Sent: 17. oktober= 2015 18:48
To: tcpm@ietf.org Extensions <tcpm@ietf.org>
= Subject: [tcpm] Question to TCP implementors regarding dupack thresh= old

=C2=A0

Hi,

=C2=A0

The question is:

=C2= =A0

When a TCP sender receives a DUPAC= K with SACK option(s) covering at least three packets (or three MSS bytes),= will it trigger fast recovery immediately?

=C2=A0

I know Linux does (for at= least +8 years). I am curious about other implementations like Windows, Ma= c, *BSD?

=C2=A0

Thanks.

=C2=A0

Yuchung

--089e0149c092474f6605233a378f-- From nobody Thu Oct 29 10:54:01 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E3C41B30AD; Thu, 29 Oct 2015 10:53:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=unavailable Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hjj-lw4vXisQ; Thu, 29 Oct 2015 10:53:53 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9AE601B30AB; Thu, 29 Oct 2015 10:53:52 -0700 (PDT) Received: from 58.172.199.146.dyn.plus.net ([146.199.172.58]:60993 helo=[192.168.0.15]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.86) (envelope-from ) id 1ZrrOY-0004ui-7u; Thu, 29 Oct 2015 17:53:50 +0000 From: Bob Briscoe To: Ingemar Johansson S , "iccrg@irtf.org" , "tcpm@ietf.org" , "tsvwg@ietf.org" References: <20151014181702.10618.83714.idtracker@ietfa.amsl.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34BF4F0F@ESESSMB205.ericsson.se> Message-ID: <56325D2D.1070107@bobbriscoe.net> Date: Thu, 29 Oct 2015 17:53:49 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <81564C0D7D4D2A4B9A86C8C7404A13DA34BF4F0F@ESESSMB205.ericsson.se> Content-Type: multipart/alternative; boundary="------------080705010809060607000906" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net Archived-At: Subject: Re: [tcpm] [tsvwg] FW: New Version Notification for draft-johansson-cc-for-4g-5g-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Oct 2015 17:53:58 -0000 This is a multi-part message in MIME format. --------------080705010809060607000906 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Ingemar, As promised, here's my comments. They are a mix of suggested improvements to the draft, suggested improvements to 5G, or requests for clarification. I also agree with all Kevin Smith's comments, so I didn't repeat any. [Sorry for delay - I had to find time to re-type after losing my review during a power outage] _*Context questions.*_* *Q1. Is this exercise treating the 5G specs solely in read-only mode? Or, if some useful design suggestions arise, is there any chance that the 3GPP might take them on board, given 5G is not fully baked yet? It's great that you have taken this initiative. Does the 3GPP ever formally ask the IETF for comments on its ideas for each new generation of radio? Given the layers above can be thought of as the 'customers' of the 3GPP, and these 'customer' layers are pretty much all maintained by the IETF, it would seem essential (and courteous) for the 3GPP to ask for customer comment, not least to co-ordinate with whatever future plans the IETF might have for the higher layers. Q2. What are your plans for this draft? Are you asking for adoption? And if so, in which WG or RG? ICCRG looks the most appropriate, given the charters of all the IETF cc WGs (TSVWG, TCPM, RMCAT, etc) are scoped on just a subset of your doc. _*Section by Section Review*_* * *All sections** * The document generally talks about the LTE protocol stack as if data is travelling down it. Are you implying that most sources of buffering delay are at the sending end, and the feedback end is pretty unencumbered by delays? I think it would be useful to explicitly say something about this (whether true or not), to explain why only one direction of travel is discussed in the doc. *2.1 PDCP layer** *** PDCP also ensures that all packets are delivered in order up to higher layers. I think this means "in the order sent into the link (PDCP) layer". This ought to be clarified, otherwise, when writing in the context of a layer 4 audience, it might be assumed this means in the order sent by the original transport layer source. keep the amount of data in flight as small as possible, without sacrificing throughput. Is this just a rather convoluted way of saying "keep the RTT as low as possible"? (which in turn implies keeping queuing delay as low as possible assuming you cannot influence the physical path.) Or were you trying to make a more subtle point? - if so, it was lost on me. Reliable delivery at handover may however be turned off or is simply not implemented, It would be useful if you could give a feel for roughly how often (in %) these cases hold. *2.2 RLC layer* role ... is to ensure that packets are delivered in order up to the higher layers Eh? The PDCP section just said it did ordering. And the rest of the RLC section talks exclusively about buffering delays necessary while filling transport blocks. If ordering is the role of both layers, surely one will have nothing to do once the other has got everything in order? varying size of the available transport Nit: Given 'transport' means e2e for the audience of this draft, pls use bearer or somehow disambiguate this word. Gap#1: In the draft, it seems like transport blocks arrive by magic, to be filled by data buffered in the RLC layer. To understand the delay compromises here, I think the draft needs to explain what determines how much transport block capacity each flow is given, and how they can influence it. In 3G, I believe that the transport layer could ask the radio resource controller to change radio resource allocation. Is this the same in 4G & 5G? Does the backlog of the buffer into the RLC layer have any automatic influence over allocation of available resources? I assume traffic class can also be used to influence allocation. Gap#2: In a 3G stack, the RLC layer multiplexes all the active application streams into MAC layer transport block by dicing up buffered packets and packing the pieces into each transport block. Then, at the other end of the link, each piece has to be held until the rest of the packet arrives. This sacrifices delay for bandwidth efficiency. Is this still done in 4G and 5G? If so, it would be good to write about mux delay in the draft too. *2.3 MAC layer** * the maximum number of retransmissions is configurable Wouldn't it be better to set a limit on the /time/ spent retransmitting, so it would automatically give up with less retransmissions for longer transmission distances? s/only checks for the presence on download data only at regular intervals/ /only checks for the presence of download data at regular intervals/ *2.3.2 Uplink scheduling** * The scheduling request does not indicate how many bytes there are in the uplink queue. Seems like a bad omission. Is there a reason? If there are less bytes queued than the base station's grant, surely it would be useful for the base station to know, so that it can grant more to something else while the previous short transmission is completing. You say that when HARQ breaks up a sequence of ACKs, it can also trigger "coalescing issues". I can understand that it could cause ACK compression, but are you implying something coalesces the ACKs or the subsequent data? What does the coalescing? Similarly, in the last para of 2.3.2 you say "Reduced ACKs can unfortunately cause coalescing." and again I'm not sure what is coalescing what here. There could be a positive side to the interaction between ACK clocking and link aggregation, at least for long-running flows. This has been observed in 802.11n WiFi where the buffering and aggregation of multiple ACKs into one transmission grant tend to lead the sender to bunch multiple data segments so that they all nicely fill one transmission grant in the other direction [Showail14]. As long as the transmission grant does not wait for more data (which unfortunately I don't think is the case in 3G/4G/5G), this could be good for bandwidth utilisation of elephants and latency of mice, including when they are mixed. [Showail14] Showail, A., Jamshaid, K. & Shihada, B., "WQM: An Aggregation-aware Queue Management Scheme for IEEE 802.11N Based Networks," In: Proceedings of the 2014 ACM SIGCOMM Workshop on Capacity Sharing Workshop CSWS '14 pp.15-20 ACM (2014) *3. 4G and 5G evolution** * s/a explicit/an explicit/ *4. Requirements for improved performance** * s/bufferbloat/buffer/ *5. Congestion control examples * Regarding Hybrid Cubic using OWD measurements, I have seen a paper (under submission) about getting delay-based congestion controls (e.g. LEDBAT or CAIA Delay Gradient) to interwork with AQMs with various low delay targets. It's not in the context of cellular networks, but I'll try to remember to point it out once it gets published. *5.1 HyStartRestart** * s/algorithm used/algorithm is used/ In the new code, it would be useful to explain why the second test before doubling ssthresh, ie, the test for how long it has been since the last hyrestart. Also, in the header definitions, if you intend there to be constraints on the relation between N_RTT_HYRESTART and N_RTT_LOW (e.g. N_RTT_HYRESTART > N_RTT_LOW) you ought to say, because people will try other values. Rather than the rather arbitrary wait before an arbitrary doubling of ssthresh, wouldn't it be better to continually increase ssthresh (not necessarily linearly) the longer the RTT has remained only just higher than the min RTT? (BTW, surely the main problem with this modification to HyStartRestart is that is requires a number of round trips to reduce delay - a sort-of oxymoron.) *5.2. Hybrid Cubic* Furthermore it is assumed than the timestamp clock frequency in sender and receiver are identical or that the sender can infer the timestamp clock frequency of the receiver and recompute timestamp values based on this information. Recently on tcpm I sent you a pointer to the chirping paper Mirja & I did, which also made this assumption. Partly prompted by that, Richard Scheffenegger tried to get people interested in standardising use of the timestamp option on a SYN to negotiate stuff like timer resolution. I don't think it went anywhere. Do you think we ought to revitalise that? You mention inference - have you tried that - are there cases where it doesn't work? That's it. Thanks again for the draft. V useful. Bob On 15/10/15 06:08, Ingemar Johansson S wrote: > Hi > > A new IETF draft is submitted in response to general discussion that for instance TCP Cubic is not an appropriate congestion control algorithm for LTE. > The draft briefly outlines typical 4G access behavior on the PDCP, RLC and MAC layers that can have an impact on transport protocol performance. The draft also outlines two relatively simple modifications to the Cubic congestion control. > > /Ingemar > > -----Original Message----- > From:internet-drafts@ietf.org [mailto:internet-drafts@ietf.org] > Sent: den 14 oktober 2015 20:17 > To: Ingemar Johansson S > Subject: New Version Notification for draft-johansson-cc-for-4g-5g-00.txt > > > A new version of I-D, draft-johansson-cc-for-4g-5g-00.txt > has been successfully submitted by Ingemar Johansson and posted to the IETF repository. > > Name: draft-johansson-cc-for-4g-5g > Revision: 00 > Title: Congestion control for 4G and 5G access > Document date: 2015-10-14 > Group: Individual Submission > Pages: 14 > URL:https://www.ietf.org/internet-drafts/draft-johansson-cc-for-4g-5g-00.txt > Status:https://datatracker.ietf.org/doc/draft-johansson-cc-for-4g-5g/ > Htmlized:https://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-00 > > > Abstract: > This memo outlines the challenge that 4G and 5G access brings for > transport protocol congestion control and also outlines a few simple > examples that can improve transport protocol congestion control > performance in 4G and 5G access. > > > > > > Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. > > The IETF Secretariat > -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ --------------080705010809060607000906 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Ingemar,

As promised, here's my comments. They are a mix of suggested improvements to the draft, suggested improvements to 5G, or requests for clarification. I also agree with all Kevin Smith's comments, so I didn't repeat any.
[Sorry for delay - I had to find time to re-type after losing my review during a power outage]

Context questions.
Q1. Is this exercise treating the 5G specs solely in read-only mode? Or, if some useful design suggestions arise, is there any chance that the 3GPP might take them on board, given 5G is not fully baked yet?

It's great that you have taken this initiative. Does the 3GPP ever formally ask the IETF for comments on its ideas for each new generation of radio? Given the layers above can be thought of as the 'customers' of the 3GPP, and these 'customer' layers are pretty much all maintained by the IETF, it would seem essential (and courteous) for the 3GPP to ask for customer comment, not least to co-ordinate with whatever future plans the IETF might have for the higher layers.

Q2. What are your plans for this draft? Are you asking for adoption? And if so, in which WG or RG? ICCRG looks the most appropriate, given the charters of all the IETF cc WGs (TSVWG, TCPM, RMCAT, etc) are scoped on just a subset of your doc.

Section by Section Review

All sections

The document generally talks about the LTE protocol stack as if data is travelling down it. Are you implying that most sources of buffering delay are at the sending end, and the feedback end is pretty unencumbered by delays? I think it would be useful to explicitly say something about this (whether true or not), to explain why only one direction of travel is discussed in the doc.

2.1 PDCP layer
    PDCP also ensures that all
   packets are delivered in order up to higher layers.
I think this means "in the order sent into the link (PDCP) layer". This ought to be clarified, otherwise, when writing in the context of a layer 4 audience, it might be assumed this means in the order sent by the original transport layer source.

   keep the amount of data in flight as small as possible, without sacrificing throughput.
Is this just a rather convoluted way of saying "keep the RTT as low as possible"? (which in turn implies keeping queuing delay as low as possible assuming you cannot influence the physical path.) Or were you trying to make a more subtle point? - if so, it was lost on me.

   Reliable delivery at handover may however be turned off or is simply not implemented,
It would be useful if you could give a feel for roughly how often (in %) these cases hold.

2.2 RLC layer

   role ... is to ensure that packets are delivered in order up to the higher layers
Eh? The PDCP section just said it did ordering. And the rest of the RLC section talks exclusively about buffering delays necessary while filling transport blocks. If ordering is the role of both layers, surely one will have nothing to do once the other has got everything in order?

   varying size of the available transport
Nit: Given 'transport' means e2e for the audience of this draft, pls use bearer or somehow disambiguate this word.


Gap#1: In the draft, it seems like transport blocks arrive by magic, to be filled by data buffered in the RLC layer. To understand the delay compromises here, I think the draft needs to explain what determines how much transport block capacity each flow is given, and how they can influence it. In 3G, I believe that the transport layer could ask the radio resource controller to change radio resource allocation. Is this the same in 4G & 5G? Does the backlog of the buffer into the RLC layer have any automatic influence over allocation of available resources? I assume traffic class can also be used to influence allocation.

Gap#2: In a 3G stack, the RLC layer multiplexes all the active application streams into MAC layer transport block by dicing up buffered packets and packing the pieces into each transport block. Then, at the other end of the link, each piece has to be held until the rest of the packet arrives. This sacrifices delay for bandwidth efficiency. Is this still done in 4G and 5G? If so, it would be good to write about mux delay in the draft too.

2.3 MAC layer

   the maximum number of retransmissions is configurable

Wouldn't it be better to set a limit on the time spent retransmitting, so it would automatically give up with less retransmissions for longer transmission distances?

s/only checks for the presence on download data only at regular intervals/
 /only checks for the presence of download data at regular intervals/

2.3.2 Uplink scheduling

   The scheduling request does not indicate how many bytes there are in the uplink queue.
Seems like a bad omission. Is there a reason? If there are less bytes queued than the base station's grant, surely it would be useful for the base station to know, so that it can grant more to something else while the previous short transmission is completing.

You say that when HARQ breaks up a sequence of ACKs, it can also trigger "coalescing issues". I can understand that it could cause ACK compression, but are you implying something coalesces the ACKs or the subsequent data? What does the coalescing? Similarly, in the last para of 2.3.2 you say "Reduced ACKs can unfortunately cause coalescing." and again I'm not sure what is coalescing what here.

There could be a positive side to the interaction between ACK clocking and link aggregation, at least for long-running flows. This has been observed in 802.11n WiFi where the buffering and aggregation of multiple ACKs into one transmission grant tend to lead the sender to bunch multiple data segments so that they all nicely fill one transmission grant in the other direction [Showail14]. As long as the transmission grant does not wait for more data (which unfortunately I don't think is the case in 3G/4G/5G), this could be good for bandwidth utilisation of elephants and latency of mice, including when they are mixed.


[Showail14] Showail, A., Jamshaid, K. & Shihada, B., "WQM: An Aggregation-aware Queue Management Scheme for IEEE 802.11N Based Networks," In: Proceedings of the 2014 ACM SIGCOMM Workshop on Capacity Sharing Workshop CSWS '14 pp.15-20 ACM (2014)
<http://dl.acm.org/citation.cfm?id=2630097>

3.  4G and 5G evolution

s/a explicit/an explicit/

4.  Requirements for improved performance

s/bufferbloat/buffer/

5.  Congestion control examples

Regarding Hybrid Cubic using OWD measurements, I have seen a paper (under submission) about getting delay-based congestion controls (e.g. LEDBAT or CAIA Delay Gradient) to interwork with AQMs with various low delay targets. It's not in the context of cellular networks, but I'll try to remember to point it out once it gets published.

5.1 HyStartRestart

s/algorithm used/algorithm is used/

In the new code, it would be useful to explain why the second test before doubling ssthresh, ie, the test for how long it has been since the last hyrestart. Also, in the header definitions, if you intend there to be constraints on the relation between N_RTT_HYRESTART and N_RTT_LOW (e.g.  N_RTT_HYRESTART > N_RTT_LOW) you ought to say, because people will try other values.

Rather than the rather arbitrary wait before an arbitrary doubling of ssthresh, wouldn't it be better to continually increase ssthresh (not necessarily linearly) the longer the RTT has remained only just higher than the min RTT?

(BTW, surely the main problem with this modification to HyStartRestart is that is requires a number of round trips to reduce delay - a sort-of oxymoron.)

5.2.  Hybrid Cubic

   Furthermore it is assumed than the timestamp clock frequency in
   sender and receiver are identical or that the sender can infer the
   timestamp clock frequency of the receiver and recompute timestamp
   values based on this information.
Recently on tcpm I sent you a pointer to the chirping paper Mirja & I did, which also made this assumption. Partly prompted by that, Richard Scheffenegger tried to get people interested in standardising use of the timestamp option on a SYN to negotiate stuff like timer resolution. I don't think it went anywhere. Do you think we ought to revitalise that? You mention inference - have you tried that - are there cases where it doesn't work?


That's it. Thanks again for the draft. V useful.



Bob



On 15/10/15 06:08, Ingemar Johansson S wrote:
Hi

A new IETF draft is submitted in response to general discussion that for instance TCP Cubic is not an appropriate congestion control algorithm for LTE. 
The draft briefly outlines typical 4G access behavior on the PDCP, RLC and MAC  layers that can have an impact on transport protocol performance. The draft also outlines two relatively simple modifications to the Cubic congestion control.

/Ingemar  

-----Original Message-----
From: internet-drafts@ietf.org [mailto:internet-drafts@ietf.org] 
Sent: den 14 oktober 2015 20:17
To: Ingemar Johansson S
Subject: New Version Notification for draft-johansson-cc-for-4g-5g-00.txt


A new version of I-D, draft-johansson-cc-for-4g-5g-00.txt
has been successfully submitted by Ingemar Johansson and posted to the IETF repository.

Name:		draft-johansson-cc-for-4g-5g
Revision:	00
Title:		Congestion control for 4G and 5G access
Document date:	2015-10-14
Group:		Individual Submission
Pages:		14
URL:            https://www.ietf.org/internet-drafts/draft-johansson-cc-for-4g-5g-00.txt
Status:         https://datatracker.ietf.org/doc/draft-johansson-cc-for-4g-5g/
Htmlized:       https://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-00


Abstract:
   This memo outlines the challenge that 4G and 5G access brings for
   transport protocol congestion control and also outlines a few simple
   examples that can improve transport protocol congestion control
   performance in 4G and 5G access.


                                                                                  


Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat




-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/
--------------080705010809060607000906-- From nobody Thu Oct 29 16:01:22 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 421B61A1A75 for ; Thu, 29 Oct 2015 16:01:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HIp-Iid2jNn0 for ; Thu, 29 Oct 2015 16:01:20 -0700 (PDT) Received: from mail-vk0-x22d.google.com (mail-vk0-x22d.google.com [IPv6:2607:f8b0:400c:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4541E1A1A79 for ; Thu, 29 Oct 2015 16:01:19 -0700 (PDT) Received: by vkgs66 with SMTP id s66so37064745vkg.1 for ; Thu, 29 Oct 2015 16:01:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=Ttfkc7k4FYnn773hGqeTKOo4i5wS9F1ElYbf5LbLKXk=; b=OwYGiBg9FpPn/4KD/1LkI2ghK5iNQquYjYWd42jCKsL3DczSMcf07i4F8l7sHzieUn iGH8zQzYAjYMv8+veyFj8Q4025/0RFRrq22mC3DCRR9PIXCr/KB8b/TwiqePDlNlo2xd 2/LKKOrMd583RR1ZE8KClsz1Ev4wlxjze7FVDojitOHRpOv1rryk7zjhCvptFl6jtUBY XqCyyGmdMvlDVZYG0Uk3gQXtISv/7u4O+/NQm9gnqdLiT8VQzPVgJ5pSVo5wiiVRCaIm EEaTZeZJcv0oHAcJ9xpUdgbyTBDWMGUL8gFvUZ0TRJDiufrENSLeuwyfoqYKLYp4ncG1 0kJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=Ttfkc7k4FYnn773hGqeTKOo4i5wS9F1ElYbf5LbLKXk=; b=aDLKsvQYMnIM7kHyMz2Zm7bfqG0id4WqNFTwDHHoMN2Q662SKXmHqqCLeWQJGlPybj QvGznYIfoGMlFeY7FDRdIzR2R2cicV5DEmF/MRR9szoKzk/5MXy50ioevQiDPlcVQ9Ro nxf2IiZluUaYcx26zxt4yLk96jbEQ/VErfsiq8kbLWybgi2nBYVO34p5XoAJ5mFYheSB kRrOCrwcleoB+aAVIvL5RukZG+Gj+IGyi2dwttXxomi98hR/Bmb5UTQtx8nZQRwnqTi2 EWLSEEhe/qFPGvpQKoUcC05g3OYpr6Qk20HyM96xbdKXPDHi0Ddw+hdMyv/FKXZDfcbE oGDQ== X-Gm-Message-State: ALoCoQnyyZnCP37lPP+eVyo9e4BO42loFW6F9kfK4GXq5uodInH9MzVnv/KJ62RiEl69osTbrRVS X-Received: by 10.31.49.22 with SMTP id x22mr2998169vkx.60.1446159678423; Thu, 29 Oct 2015 16:01:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.67.21 with HTTP; Thu, 29 Oct 2015 16:00:38 -0700 (PDT) In-Reply-To: References: From: Yuchung Cheng Date: Fri, 30 Oct 2015 08:00:38 +0900 Message-ID: To: Karen Elisabeth Egede Nielsen Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] Question to TCP implementors regarding dupack threshold X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Oct 2015 23:01:21 -0000 On Thu, Oct 29, 2015 at 5:37 PM, Karen Elisabeth Egede Nielsen wrote: > HI, > > > > We do this in our in-house TCP implementation today. > > > > This is what RFC6675 prescribes for =E2=80=93 do you not agree ? I do agree. Doing so will make Fast Recovery more robust on ACK losses (or suppression). Glad to know one more stack that does that. > > > > BR, Karen > > > > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Yuchung Cheng > Sent: 17. oktober 2015 18:48 > To: tcpm@ietf.org Extensions > Subject: [tcpm] Question to TCP implementors regarding dupack threshold > > > > Hi, > > > > The question is: > > > > When a TCP sender receives a DUPACK with SACK option(s) covering at least > three packets (or three MSS bytes), will it trigger fast recovery > immediately? > > > > I know Linux does (for at least +8 years). I am curious about other > implementations like Windows, Mac, *BSD? > > > > Thanks. > > > > Yuchung From nobody Fri Oct 30 17:03:29 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9DE281A00A0; Fri, 30 Oct 2015 17:03:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.912 X-Spam-Level: X-Spam-Status: No, score=-101.912 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ie_E_fzPQotC; Fri, 30 Oct 2015 17:03:27 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [IPv6:2001:1900:3001:11::31]) by ietfa.amsl.com (Postfix) with ESMTP id 64DE81A0049; Fri, 30 Oct 2015 17:03:27 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id C1AA5180003; Fri, 30 Oct 2015 17:02:41 -0700 (PDT) To: mjl@caida.org, ycheng@google.com, hkchu@google.com, sivasankar@cs.ucsd.edu, arvind@google.com X-PHP-Originating-Script: 30:errata_mail_lib.php From: RFC Errata System Message-Id: <20151031000241.C1AA5180003@rfc-editor.org> Date: Fri, 30 Oct 2015 17:02:41 -0700 (PDT) Archived-At: Cc: mls.ietf@gmail.com, tcpm@ietf.org, iesg@ietf.org, rfc-editor@rfc-editor.org Subject: [tcpm] [Errata Verified] RFC7413 (4239) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Oct 2015 00:03:28 -0000 The following errata report has been verified for RFC7413, "TCP Fast Open". -------------------------------------- You may review the report below and at: http://www.rfc-editor.org/errata_search.php?rfc=7413&eid=4239 -------------------------------------- Status: Verified Type: Technical Reported by: Matthew Luckie Date Reported: 2015-01-22 Verified by: Martin Stiemerling (IESG) Section: 4.2.1 Original Text ------------- 1. The client sends a SYN packet with a Fast Open option with a Length field of 0 (empty cookie field). Corrected Text -------------- 1. The client sends a SYN packet with a Fast Open option with a Length field of 2 (empty cookie field). Notes ----- A Nil fast-open option has an option length of 2. A length field of zero would mean an invalid TCP option. -------------------------------------- RFC7413 (draft-ietf-tcpm-fastopen-10) -------------------------------------- Title : TCP Fast Open Publication Date : December 2014 Author(s) : Y. Cheng, J. Chu, S. Radhakrishnan, A. Jain Category : EXPERIMENTAL Source : TCP Maintenance and Minor Extensions Area : Transport Stream : IETF Verifying Party : IESG From nobody Fri Oct 30 17:49:27 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B74471ACDA3 for ; Fri, 30 Oct 2015 17:49:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.911 X-Spam-Level: X-Spam-Status: No, score=-6.911 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g8G56xHjFqqo for ; Fri, 30 Oct 2015 17:49:24 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AAA301ACDA1 for ; Fri, 30 Oct 2015 17:49:23 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 4FF4073B475B9; Sat, 31 Oct 2015 00:49:17 +0000 (GMT) Received: from FR711WXCHHUB02.zeu.alcatel-lucent.com (fr711wxchhub02.zeu.alcatel-lucent.com [135.239.2.112]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t9V0nKwt002181 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sat, 31 Oct 2015 01:49:21 +0100 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.114]) by FR711WXCHHUB02.zeu.alcatel-lucent.com ([135.239.2.112]) with mapi id 14.03.0195.001; Sat, 31 Oct 2015 01:49:20 +0100 From: "Scharf, Michael (Michael)" To: Anna Brunstrom , "spencerdawkins.ietf@gmail.com" Thread-Topic: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt Thread-Index: AQHRC28dlvv2MomATk2ZCyphH35j3551czmAgAPglrCAAC0hAIALUsi0 Date: Sat, 31 Oct 2015 00:49:19 +0000 Message-ID: <655C07320163294895BBADA28372AF5D485360EA@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <20151020193954.12955.50870.idtracker@ietfa.amsl.com> <3500BC2B-A986-4AC8-BEF0-4709DEFE5D35@kau.se> <655C07320163294895BBADA28372AF5D48524F8A@FR712WXCHMBA15.zeu.alcatel-lucent.com>, <562AAB5E.40502@kau.se> In-Reply-To: <562AAB5E.40502@kau.se> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.202.192.14] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Oct 2015 00:49:26 -0000 Maybe the authors could propose an alternative phrasing on the mailing list= , picking one of the potential alternative wordings, and update the draft i= f there is no objection?=0A= =0A= Since this is in the abstract, this seems to me better than waiting for the= RFC editor.=0A= =0A= Michael=0A= =0A= ________________________________________=0A= Von: Anna Brunstrom [anna.brunstrom@kau.se]=0A= Gesendet: Freitag, 23. Oktober 2015 23:49=0A= An: Scharf, Michael (Michael); spencerdawkins.ietf@gmail.com=0A= Cc: tcpm@ietf.org=0A= Betreff: Re: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt=0A= =0A= Hi Michael,=0A= =0A= I am not a native speaker either, but I agree with you that "timeout=0A= duration" or "timeout value" sounds more clear.=0A= =0A= BR,=0A= Anna=0A= =0A= On 2015-10-23 19:38, Scharf, Michael (Michael) wrote:=0A= > Sorry for speaking up late... Just to cross-check with others. Is the new= wording in the abstract...=0A= >=0A= > The modification, RTO Restart (RTOR), allows the=0A= > transport to restart its retransmission timer using a smaller delay,= =0A= > so that the effective RTO becomes more aggressive in situations where= =0A= > fast retransmit cannot be used. This enables faster loss detection= =0A= > and recovery for connections that are short-lived or application-=0A= > limited.=0A= >=0A= > ... indeed clear to everybody? To me (as a non-native speaker) this use o= f "delay" in the context of the RTO is a bit confusing.=0A= >=0A= > To me, a wording like replacing "delay" by "timeout duration", "timeout v= alue", etc. would sound more familiar. I'd rather use "delay" as a measure = between packet (re) transmissions, etc.=0A= >=0A= > For instance, to me the following wording would fit a bit better ...=0A= >=0A= > The modification, RTO Restart (RTOR), allows the=0A= > transport to restart its retransmission timer using a smaller timeout= duration,=0A= > so that the effective RTO becomes more aggressive in situations where= =0A= > fast retransmit cannot be used. This smaller delay before the retran= smission enables faster loss detection=0A= > and recovery for connections that are short-lived or application-=0A= > limited.=0A= >=0A= > But I am not a native speaker. Any thoughts?=0A= >=0A= > (I guess the RFC editor could just review that.)=0A= >=0A= > Michael=0A= >=0A= >=0A= >> -----Original Message-----=0A= >> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Per Hurtig=0A= >> Sent: Wednesday, October 21, 2015 9:55 AM=0A= >> To: tcpm@ietf.org Extensions=0A= >> Cc: bclaise@cisco.com; barryleiba@computer.org=0A= >> Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rtorestart-09.txt=0A= >>=0A= >> Hi,=0A= >>=0A= >> the new draft on RTO restart address the following comments from the=0A= >> IESG (thanks for the feedback):=0A= >>=0A= >> o Clarified, in the abstract, that the modified restart causes a=0A= >> smaller retransmission delay in total.=0A= >>=0A= >> o Clarified, in the introduction, that the fast retransmit algorithm=0A= >> may cause retransmissions upon=0A= >> receiving duplicate acknowledgments, not that it unconditionally=0A= >> does so.=0A= >>=0A= >> o Changed wording from "to proposed standard" to "to the standards=0A= >> track".=0A= >>=0A= >> o Changed algorithm description so that a TCP sender MUST track the=0A= >> time elapsed since the=0A= >> transmission of the earliest outstanding segment. This was not=0A= >> explicitly stated in previous=0A= >> versions of the draft.=0A= >>=0A= >>=0A= >> Cheers,=0A= >> Per=0A= >>=0A= >>> On 20 Oct 2015, at 21:39, internet-drafts@ietf.org wrote:=0A= >>>=0A= >>>=0A= >>> A New Internet-Draft is available from the on-line Internet-Drafts=0A= >> directories.=0A= >>> This draft is a work item of the TCP Maintenance and Minor Extensions= =0A= >> Working Group of the IETF.=0A= >>> Title : TCP and SCTP RTO Restart=0A= >>> Authors : Per Hurtig=0A= >>> Anna Brunstrom=0A= >>> Andreas Petlund=0A= >>> Michael Welzl=0A= >>> Filename : draft-ietf-tcpm-rtorestart-09.txt=0A= >>> Pages : 17=0A= >>> Date : 2015-10-20=0A= >>>=0A= >>> Abstract:=0A= >>> This document describes a modified sender-side algorithm for=0A= >> managing=0A= >>> the TCP and SCTP retransmission timers that provides faster loss=0A= >>> recovery when there is a small amount of outstanding data for a=0A= >>> connection. The modification, RTO Restart (RTOR), allows the=0A= >>> transport to restart its retransmission timer using a smaller=0A= >> delay,=0A= >>> so that the effective RTO becomes more aggressive in situations=0A= >> where=0A= >>> fast retransmit cannot be used. This enables faster loss detection= =0A= >>> and recovery for connections that are short-lived or application-=0A= >>> limited.=0A= >>>=0A= >>>=0A= >>> The IETF datatracker status page for this draft is:=0A= >>> https://datatracker.ietf.org/doc/draft-ietf-tcpm-rtorestart/=0A= >>>=0A= >>> There's also a htmlized version available at:=0A= >>> https://tools.ietf.org/html/draft-ietf-tcpm-rtorestart-09=0A= >>>=0A= >>> A diff from the previous version is available at:=0A= >>> https://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-tcpm-rtorestart-09=0A= >>>=0A= >>>=0A= >>> Please note that it may take a couple of minutes from the time of=0A= >> submission=0A= >>> until the htmlized version and diff are available at tools.ietf.org.=0A= >>>=0A= >>> Internet-Drafts are also available by anonymous FTP at:=0A= >>> ftp://ftp.ietf.org/internet-drafts/=0A= >>>=0A= >>> _______________________________________________=0A= >>> tcpm mailing list=0A= >>> tcpm@ietf.org=0A= >>> https://www.ietf.org/mailman/listinfo/tcpm=0A= > _______________________________________________=0A= > tcpm mailing list=0A= > tcpm@ietf.org=0A= > https://www.ietf.org/mailman/listinfo/tcpm=0A= =0A= =0A=