From philip.eardley@bt.com Fri Mar 1 09:14:57 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1137621F8CDA for ; Fri, 1 Mar 2013 09:14:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.367 X-Spam-Level: X-Spam-Status: No, score=-103.367 tagged_above=-999 required=5 tests=[AWL=0.230, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 00-SazVKwW3M for ; Fri, 1 Mar 2013 09:14:55 -0800 (PST) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id 6EB5D21F90C8 for ; Fri, 1 Mar 2013 09:14:54 -0800 (PST) Received: from EVMHT65-UKRD.domain1.systemhost.net (10.36.3.102) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.279.1; Fri, 1 Mar 2013 17:14:41 +0000 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.59]) by EVMHT65-UKRD.domain1.systemhost.net ([10.36.3.102]) with mapi; Fri, 1 Mar 2013 17:14:44 +0000 From: To: , , , , Date: Fri, 1 Mar 2013 17:14:42 +0000 Thread-Topic: Draft Agenda [was: LMAP BoF has been approved.] Thread-Index: Ac4O0/pYJRW4w7TWRXiKv33bF479JQBKT2iwAAvWBHAAAKrgAAGbxhnA Message-ID: <9510D26531EF184D9017DF24659BB87F342ED975A8@EMV65-UKRD.domain1.systemhost.net> References: <94C682931C08B048B7A8645303FDC9F36EB3C82528@PUEXCB1B.nanterre.francetelecom.fr> <94C682931C08B048B7A8645303FDC9F36EB3C8270A@PUEXCB1B.nanterre.francetelecom.fr> In-Reply-To: <94C682931C08B048B7A8645303FDC9F36EB3C8270A@PUEXCB1B.nanterre.francetelecom.fr> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F342ED975A8EMV65UKRDdoma_" MIME-Version: 1.0 Cc: christian.jacquenet@orange.com Subject: Re: [lmap] Draft Agenda [was: LMAP BoF has been approved.] X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 17:14:57 -0000 --_000_9510D26531EF184D9017DF24659BB87F342ED975A8EMV65UKRDdoma_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 TWVkLA0KSSByZWFkIHlvdXIgZHJhZnQgYW5kIGxvb2tlZCBmb3IgZGlmZmVyZW5jZXMgYmV0d2Vl biBpdCBhbmQgaHR0cDovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtbGluc25lci1sbWFwLXVz ZS1jYXNlcw0KeW91ciBkcmFmdCBpcyBhYm91dCDigJxVbmRlcnN0YW5kaW5nIHRoZSBxdWFsaXR5 IGV4cGVyaWVuY2VkIGJ5IGN1c3RvbWVyc+KAnSAod2hpY2ggY29ycmVzcG9uZHMgdG8gUzMuMiBv ZiBkcmFmdC1saW5zbmVyKS4gIFlvdSBkb27igJl0IG1lbnRpb24gZHJhZnQtbGluc25lcuKAmXMg b3RoZXIgdXNlIGNhc2VzLiBpcyB0aGUgb21pc3Npb24ganVzdCBiZWNhdXNlIHlvdSB3ZXJlIGNv bmNlbnRyYXRpbmcgb24gdGhlIOKAmHF1YWxpdHkgZXhwZXJpZW5jZWTigJkgdXNlIGNhc2UsIG9y IGRvIHlvdSB0aGluayB0aGUgb3RoZXIgdXNlIGNhc2VzIGFyZSB1bmltcG9ydGFudD8NCmFzc3Vt aW5nIHRoZSBmb3JtZXIsIHRoZW4gSSBzdWdnZXN0IGFkZGluZyBhIGNvdXBsZSBvZiBpbXByb3Zl bWVudHMgdG8gU2VjdGlvbiAzLjIgb2YgZHJhZnQtbGluc25lciAoSSB0aGluayB0aGlzIHdvdWxk IHRoZW4gY292ZXIgYWxsIHRoZSBkZWx0YXMpOg0KKGVuZCBvZiBwYXJhIDMpICBUaGUgb3BlcmF0 b3IgbWF5IGFsc28gd2FudCB0byB1bmRlcnN0YW5kIHRoZSBleHBlcmllbmNlIGZvciBpdHMgYXBw bGljYXRpb24gYW5kIGNvbnRlbnQgc2VydmljZXMsIHN1Y2ggYXMgVFYgb3IgVlBOLiBUaGVzZSBt YXkgaW52b2x2ZSBzcGVjaWZpYyBuZXR3b3JrIGZlYXR1cmVzIHN1Y2ggYXMgUW9TIHByaW9yaXRp c2F0aW9uLCBtdWx0aWNhc3Qgb3IgcG9saWN5IGVuZm9yY2VtZW50IHdoaWNoIGEgbWV0cmljIHdv dWxkIG5lZWQgdG8gZW5jb21wYXNzLg0KKGVuZCBvZiBwYXJhIDUpIFRoZSBvcGVyYXRvciBtYXkg YWxzbyB3YW50IHRvIGNoZWNrIChhbmQgYWxsb3cgY3VzdG9tZXJzIHRvIGNoZWNrKSB0aGF0IGl0 IGlzIGZ1bGZpbGxpbmcgYSBjdXN0b21lcuKAmXMgc2VydmljZSBsZXZlbCBhZ3JlZW1lbnQuDQp3 aGF0IGRvIHlvdSByZWNrb24/DQp0aGFua3MNCnBoaWwNCg0KRnJvbTogbG1hcC1ib3VuY2VzQGll dGYub3JnIFttYWlsdG86bG1hcC1ib3VuY2VzQGlldGYub3JnXSBPbiBCZWhhbGYgT2YgbW9oYW1l ZC5ib3VjYWRhaXJAb3JhbmdlLmNvbQ0KU2VudDogMjEgRmVicnVhcnkgMjAxMyAxMjo0MA0KVG86 IE1PUlRPTiBKUi4sIEFMRlJFRCAoQUwpOyBCZW5vaXQgQ2xhaXNlOyBsbWFwQGlldGYub3JnOyBS b21hc2NhbnUsIERhbiAoRGFuKSAoZHJvbWFzY2FAYXZheWEuY29tKQ0KQ2M6IEpBQ1FVRU5FVCBD aHJpc3RpYW4gT0xOQy9PTE4NClN1YmplY3Q6IFJlOiBbbG1hcF0gRHJhZnQgQWdlbmRhIFt3YXM6 IExNQVAgQm9GIGhhcyBiZWVuIGFwcHJvdmVkLl0NCg0KUmUtLA0KDQpkcmFmdC1ib3VjYWRhaXIt KiBpbmNsdWRlcyBzb21lIHBvaW50cyBub3QgbWVudGlvbmVkIGluIG90aGVyIGRyYWZ0LSotbG1h cCBJLURzIHN1Y2ggdGhlIERTQ1AgbWFya2luZyBpbmhlcml0YW5jZTogc29tZSBvZiB0aGUgc2Vy dmljZXMgd2UgYXJlIGRlcGxveWluZyBkbyBub3QgdHJ1c3QgdGhlIG1hcmtpbmcgc2V0IGJ5IHRo ZSBhcHBsaWNhdGlvbiBidXQgdGhlcmUgYXJlIHNwZWNpZmljIG1vZHVsZXMgd2hpY2ggYXJlIGVu YWJsZWQgaW4gdGhlIENQRSBmb3IgaW5zdGFuY2UuIENvbmR1Y3RpbmcgYSBzZXQgb2YgbWVhc3Vy ZW1lbnQgZm9yIHRob3NlIHNlcnZpY2Ugd2lsbCBuZWVkIHRvIGJlIGRlc2lnbmVkIHRvIGJlIGNv bXBsaWFudCB3aXRoIHN1Y2ggY29uc3RyYWludC4NCg0KSSBjYW4gbGlzdCBhbm90aGVyIGl0ZW0g c3VjaCBhcyB0aGUgY29tcGF0aWJpbGl0eSB3aXRoIGEgbXVsdGktYWRkcmVzc2VkIHNlcnZpY2Ug ZGVzaWduIHNjaGVtZSAoaW4gd2hpY2ggIGRpc3RpbmN0IElQIGFkZHJlc3NlcyBhcmUgYXNzaWdu ZWQgZm9yIGVhY2ggc3Vic2NyaWJlZCBzZXJ2aWNlKTogcGFja2V0cyBpc3N1ZWQgZnJvbSBhIG1l YXN1cmVtZW50IGFnZW50IChsb2NhdGVkIGJlaGluZCB0aGUgQ1BFKSB3aWxsIG5lZWQgdG8gYmUg Ym91bmQgdG8gdGhlIGFwcHJvcHJpYXRlIHNlcnZpY2Utc3BlY2lmaWMgY2hhbm5lbCwgZXRjLg0K DQpDaGVlcnMsDQpNZWQNCg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCkRlIDog TU9SVE9OIEpSLiwgQUxGUkVEIChBTCkgW21haWx0bzphY21vcnRvbkBhdHQuY29tXQ0KRW52b3nD qSA6IGpldWRpIDIxIGbDqXZyaWVyIDIwMTMgMTM6MjkNCsOAIDogQk9VQ0FEQUlSIE1vaGFtZWQg T0xOQy9PTE47IEJlbm9pdCBDbGFpc2U7IGxtYXBAaWV0Zi5vcmc8bWFpbHRvOmxtYXBAaWV0Zi5v cmc+OyBSb21hc2NhbnUsIERhbiAoRGFuKSAoZHJvbWFzY2FAYXZheWEuY29tPG1haWx0bzpkcm9t YXNjYUBhdmF5YS5jb20+KQ0KQ2MgOiBKQUNRVUVORVQgQ2hyaXN0aWFuIE9MTkMvT0xODQpPYmpl dCA6IFJFOiBEcmFmdCBBZ2VuZGEgW3dhczogTE1BUCBCb0YgaGFzIGJlZW4gYXBwcm92ZWQuXQ0K SGkgTWVkLA0KDQpZZXMsIG9uZSBwb2ludCB5b3UndmUgbWFkZSB0aGF0IChJTU8pIHNob3VsZCBi ZSBhZGRyZXNzZWQNCndpdGhpbiB0aGUgY3VycmVudCBjb25zdHJhaW50cyBvZiB0aGUgRnJhbWV3 b3JrIGRyYWZ0LA0KaXMgYWJvdXQgdGVzdGluZyBjcml0aWNhbCBzZXJ2aWNlcyB3aGljaCB0aGUN CmFjY2VzcyBJU1AgY29udHJhY3RzIGZvciwgc28gZG9lcyBub3Qgb3duIGl0c2VsZi4NCllvdXIg ZXhhbXBsZSwgRE5TIHRlc3RpbmcsIGlzIHBhcnRpY3VsYXJseSByZWxldmFudA0Kc2luY2UgZXZl cnkgdXNlciBwZXJmb3JtcyBxdWVyaWVzIHRoYXQgY3Jvc3Mgb3JnLiBib3VuZGFyaWVzLA0KYW5k IHNvIHdvdWxkIHRoZSBtZWFzdXJlbWVudCBwYXRoIGZvciBETlMgcmVzcG9uc2UgdGltZS4NCkhv d2V2ZXIsIGFsbCB0aGUgbWVhc3VyZW1lbnQgaW5mcmFzdHJ1Y3R1cmUgd291bGQgcmVtYWluDQp3 aXRoaW4gYSBzaW5nbGUgb3JnYW5pemF0aW9uJ3MgZG9tYWluLCBhbmQgd2hpbGUgSSB0aGluayB0 aGlzDQptYXRjaGVzIHRoZSBjdXJyZW50IGNvbnN0cmFpbnQsIGl0J3MgY2VydGFpbmx5IHdvcnRo IGluY2x1ZGluZy4NCg0KQWZ0ZXIgcmV2aWV3aW5nIHRoZSBVc2UgQ2FzZSBEcmFmdCwgcGFydGlj dWxhcmx5IHRoZSBkZXRhaWxlZCBJU1ANClVzZSBDYXNlLCBhbmQgdGhlIEZyYW1ld29yayBEcmFm dCBpdHNlbGYsIGNhbiB5b3UgcG9pbnQgb3V0DQphZGRpdGlvbmFsIGNvbnNpZGVyYXRpb25zIG5v dC15ZXQgZGVzY3JpYmVkIChidXQgd2l0aGluIHRoZSBjdXJyZW50DQpjb25zdHJhaW50cyk/DQoN CnRoYW5rcywNCkFsDQoNCg0KRnJvbTogbW9oYW1lZC5ib3VjYWRhaXJAb3JhbmdlLmNvbTxtYWls dG86bW9oYW1lZC5ib3VjYWRhaXJAb3JhbmdlLmNvbT4gW21haWx0bzptb2hhbWVkLmJvdWNhZGFp ckBvcmFuZ2UuY29tXQ0KU2VudDogVGh1cnNkYXksIEZlYnJ1YXJ5IDIxLCAyMDEzIDE6MzUgQU0N ClRvOiBNT1JUT04gSlIuLCBBTEZSRUQgKEFMKTsgQmVub2l0IENsYWlzZTsgbG1hcEBpZXRmLm9y ZzxtYWlsdG86bG1hcEBpZXRmLm9yZz4NCkNjOiBKQUNRVUVORVQgQ2hyaXN0aWFuIE9MTkMvT0xO DQpTdWJqZWN0OiBSRTogRHJhZnQgQWdlbmRhIFt3YXM6IExNQVAgQm9GIGhhcyBiZWVuIGFwcHJv dmVkLl0NCg0KRGVhciBBbCwNCg0KV291bGQgaXQgYmUgcG9zc2libGUgdGhlIGZvbGxvd2luZyBz bG90DQoNCiI1LiBGcmFtZXdvcms6IFByb2JsZW0gU3RhdGVtZW50LCBJU1AgVXNlIGNhc2UsIEdh cHMsIFJlcXVpcmVtZW50cyAtIFBoaWwgRWFyZGxleSAtIDE1IG1pbiINCg0KdG8gaW5jbHVkZSBz b21lIG9mIHRoZSBuZXR3b3JrIHByb3ZpZGVyJyByZXF1aXJlbWVudHMgYW5kIGlzc3VlcyBkaXNj dXNzZWQgaW4gZHJhZnQtYm91Y2FkYWlyLWxtYXAtY29uc2lkZXJhdGlvbnM/DQoNClRoYW5rcy4N CkNoZWVycywNCk1lZA0KDQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQpEZSA6 IGxtYXAtYm91bmNlc0BpZXRmLm9yZzxtYWlsdG86bG1hcC1ib3VuY2VzQGlldGYub3JnPiBbbWFp bHRvOmxtYXAtYm91bmNlc0BpZXRmLm9yZ10gRGUgbGEgcGFydCBkZSBNT1JUT04gSlIuLCBBTEZS RUQgKEFMKQ0KRW52b3nDqSA6IG1hcmRpIDE5IGbDqXZyaWVyIDIwMTMgMjA6MDUNCsOAIDogQmVu b2l0IENsYWlzZTsgbG1hcEBpZXRmLm9yZzxtYWlsdG86bG1hcEBpZXRmLm9yZz4NCk9iamV0IDog W2xtYXBdIERyYWZ0IEFnZW5kYSBbd2FzOiBMTUFQIEJvRiBoYXMgYmVlbiBhcHByb3ZlZC5dDQpM TUFQIGxpc3QsDQoNCldlIGhhdmUgcHJlcGFyZWQgYSBkcmFmdCBCb0YgYWdlbmRhLA0KYW5kIG5v dyB0aGF0IG91ciBwb3N0aW5nIHJpZ2h0cyBhcmUgaW5zdGFsbGVkLA0KaXQgY2FuIGJlIGZvdW5k IHdpdGggYWxsIG1lZXRpbmcgbWF0ZXJpYWxzOg0KaHR0cDovL3d3dy5pZXRmLm9yZy9wcm9jZWVk aW5ncy84Ni9hZ2VuZGEvYWdlbmRhLTg2LWxtYXANCg0KV2UgYmVsaWV2ZSB3ZSBoYXZlIGVtYnJh Y2VkIHRoZSB0c3VuYW1pIG9mIDAwIGRyYWZ0cw0KYW5kIGZvdW5kIGFsbCB0aG9zZSB0aGF0IGFy ZSByZWxldmFudCwNCmFuZCBjb250YWN0ZWQgYWxsLTEgb2YgdGhlIHByZXNlbnRlcnMuDQoNCkNv bW1lbnRzIHRvIHRoZSBsaXN0LCBwbGVhc2UsDQpEYW4gYW5kIEFsDQpib2YgY28tY2hhaXJzDQoN CkZyb206IGxtYXAtYm91bmNlc0BpZXRmLm9yZzxtYWlsdG86bG1hcC1ib3VuY2VzQGlldGYub3Jn PiBbbWFpbHRvOmxtYXAtYm91bmNlc0BpZXRmLm9yZ10gT24gQmVoYWxmIE9mIEJlbm9pdCBDbGFp c2UNClNlbnQ6IFdlZG5lc2RheSwgRmVicnVhcnkgMDYsIDIwMTMgMTA6MDAgQU0NClRvOiBsbWFw QGlldGYub3JnPG1haWx0bzpsbWFwQGlldGYub3JnPg0KU3ViamVjdDogW2xtYXBdIExNQVAgQm9G IGhhcyBiZWVuIGFwcHJvdmVkLg0KDQpEZWFyIGFsbCwNCg0KTGV0IHVzIHNoYXJlIHRoZSBnb29k IG5ld3MuIFRoZSBJRVNHIGFwcHJvdmVkIHRoaXMgQm9GLg0KRXZlbiBpZiBpdCB3YXMgaW5pdGlh bGx5IHJlcXVlc3RlZCBpbiBUU1YsIHRoZSBCb0Ygd2lsbCBiZSBzcG9uc29yZWQgaW4gT1BTIGJ5 IEJlbm9pdCBDbGFpc2UuDQpUaGUgQm9GIGluZm9ybWF0aW9uIGhhdmUgYmVlbiB1cGRhdGVkIChT ZWUgaHR0cDovL3RyYWMudG9vbHMuaWV0Zi5vcmcvYm9mL3RyYWMvd2lraS9XaWtpU3RhcnQpDQoN CkxhcmdlLVNjYWxlIE1lYXN1cmVtZW50IG9mIEJyb2FkYmFuZCBQZXJmb3JtYW5jZSAoTE1BUCkN Cg0KICogICBTdGF0dXM6IFJlcXVlc3RlZC4NCiAqICAgVHlwZTogV0cgZm9ybWluZyAoMXN0IGF0 dGVtcHQpLg0KICogICBSZXNwb25zaWJsZSBBRDogQmVub2l0IENsYWlzZSwgT1BTDQogKiAgIEJP RiBDaGFpcnM6IERhbiBSb21hc2NhbnUgKOKAi2Ryb21hc2NhQGF2YXlhLmNvbTxtYWlsdG86ZHJv bWFzY2FAYXZheWEuY29tPiksIEFsIE1vcnRvbiAo4oCLYWNtb3J0b25AYXR0LmNvbTxtYWlsdG86 YWNtb3J0b25AYXR0LmNvbT4pKQ0KICogICBBdHRlbmRhbmNlOiA4MA0KICogICBTZXNzaW9uIGxl bmd0aCAxMjAgbWludXRlcw0KICogICBDb25mbGljdHMgdG8gYXZvaWQ6DQpGaXJzdCBsZXZlbCBw cmlvcml0eTogSVBQTSwgRUNSSVQsIEJNV0csIE9QU0FSRUEvT1BTQVdHLCBJUEZJWCwgRU1BTiwg TkVUQ09ORiwgTkVUTU9ELCBYUkJMT0NLLCBESU1FLCBSQURFWFQsIFNBQ00gQm9GLCBUU1ZBUkVB LCBBTFRPLCBDRE5JLCBORlNWNCwgUFBTUCwgVFNWV0cNClNlY29uZCBsZXZlbCBwcmlvcml0eTog Q09ORVgNCiAqICAgRG9lcyBpdCByZXF1aXJlIFdlYkVYPyBObw0KICogICBEb2VzIGl0IHJlcXVp cmUgTWVldGVjaG8/IE5vDQogKiAgIE1haWxpbmcgbGlzdDog4oCLIOKAi2xtYXBAaWVmdC5vcmc8 bWFpbHRvOmxtYXBAaWVmdC5vcmc+DQogKiAgIExpc3QgYXJjaGl2ZTog4oCL4oCLaHR0cDovL3d3 dy5pZXRmLm9yZy9tYWlsLWFyY2hpdmUvd2ViL2xtYXAvY3VycmVudC9tYWlsbGlzdC5odG1sPGh0 dHA6Ly93d3cuaWV0Zi5vcmcvbWFpbC1hcmNoaXZlL3dlYi9sbWFwL2N1cnJlbnQvbWFpbGxpc3Qu aHRtbD4NCiAqICAgRG9jdW1lbnRzOiBkcmFmdC1zY2h1bHpyaW5uZS1sbWFwLXJlcXVpcmVtZW50 czxodHRwOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1zY2h1bHpyaW5uZS1sbWFwLXJlcXVp cmVtZW50cz4sIGRyYWZ0LWxpbnNuZXItbG1hcC11c2UtY2FzZXM8aHR0cDovL3Rvb2xzLmlldGYu b3JnL2h0bWwvZHJhZnQtbGluc25lci1sbWFwLXVzZS1jYXNlcz4sIGFuZCBkcmFmdC1tb3J0b24t aXBwbS1sbWFwLXBhdGgtMDA8aHR0cDovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtbW9ydG9u LWlwcG0tbG1hcC1wYXRoLTAwPg0KVGhlIGNoYWlycyBoYXZlIGJlZW4gY2hvc2VuIGJhc2VkIG9u IHRoZWlyIGxldmVscyBvZiBleHBlcnRpc2UvZXhwZXJpZW5jZSBpbiBPUFMsIG1lYXN1cmVtZW50 LCBhbmQgSUVURi4gVGhhbmtzIERhbiBhbmQgQWwgZm9yIGFjY2VwdGluZyB0aGUgY2hhaXIgcG9z aXRpb25zLg0KDQpHb2FscyBmb3IgdGhpcyBCb0Y6DQotIGJlIGluIHN5bmNoIG9uIHRoZSBwcm9i bGVtIHN0YXRlbWVudCBhbmQgdXNlIGNhc2VzDQotIHVuZGVyc3RhbmQgd2hhdCB0aGUgZXhpc3Rp bmcgSUVURiBwcm90b2NvbHMgYW5kIGRhdGEgbW9kZWxzIChJUFBNLCBORVRDT05GL1lBTkcsIElQ RklYLCBldGMuLi4pIGNhbiBvciBjYW4ndCBkbyBmb3IgTE1BUC4gVGhpcyB3aWxsIGNlcnRhaW5s eSByZXF1aXJlIGEgbGl0dGxlIG9mIGVkdWNhdGlvbiBkaXJlY3RseSBmcm9tIHRoZSBleHBlcnRz IGR1cmluZyB0aGUgQm9GLg0KLSBjbGFyaWZ5IHdoYXQncyBpbiBzY29wZSBvciBub3QgZm9yIExN QVANCg0KRmluYWxseSwgbGV0IG1lIHN0cmVzcyB0aGF0IGEgcHJvcGVyIHByZXBhcmF0aW9uIGlz IGtleSB0byBhIEJvRiBzdWNjZXNzLg0KaHR0cDovL3Rvb2xzLmlldGYub3JnL2h0bWwvcmZjNTQz NCAgIkNvbnNpZGVyYXRpb25zIGZvciBIYXZpbmcgYSBTdWNjZXNzZnVsIEJpcmRzLW9mLWEtRmVh dGhlciAoQk9GKSBTZXNzaW9uIiBpcyBhIGZpcnN0IGdvb2Qgc3RlcC4NCg0KUmVnYXJkcywgTWFy dGluIFN0aWVtZXJsaW5nIChUU1YpIGFuZCBCZW5vaXQgQ2xhaXNlIChPUFMpDQoNCg== --_000_9510D26531EF184D9017DF24659BB87F342ED975A8EMV65UKRDdoma_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+PGhlYWQ+PG1ldGEgaHR0cC1lcXVpdj1Db250ZW50LVR5cGUgY29udGVu dD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij48bWV0YSBuYW1lPUdlbmVyYXRvciBjb250ZW50 PSJNaWNyb3NvZnQgV29yZCAxMiAoZmlsdGVyZWQgbWVkaXVtKSI+PCEtLVtpZiAhbXNvXT48c3R5 bGU+dlw6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0Kb1w6KiB7YmVoYXZpb3I6dXJs KCNkZWZhdWx0I1ZNTCk7fQ0Kd1w6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0KLnNo YXBlIHtiZWhhdmlvcjp1cmwoI2RlZmF1bHQjVk1MKTt9DQo8L3N0eWxlPjwhW2VuZGlmXS0tPjxz dHlsZT48IS0tDQovKiBGb250IERlZmluaXRpb25zICovDQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFt aWx5OiJDYW1icmlhIE1hdGgiOw0KCXBhbm9zZS0xOjIgNCA1IDMgNSA0IDYgMyAyIDQ7fQ0KQGZv bnQtZmFjZQ0KCXtmb250LWZhbWlseTpDYWxpYnJpOw0KCXBhbm9zZS0xOjIgMTUgNSAyIDIgMiA0 IDMgMiA0O30NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6VGFob21hOw0KCXBhbm9zZS0xOjIg MTEgNiA0IDMgNSA0IDQgMiA0O30NCi8qIFN0eWxlIERlZmluaXRpb25zICovDQpwLk1zb05vcm1h bCwgbGkuTXNvTm9ybWFsLCBkaXYuTXNvTm9ybWFsDQoJe21hcmdpbjowY207DQoJbWFyZ2luLWJv dHRvbTouMDAwMXB0Ow0KCWZvbnQtc2l6ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6IlRpbWVzIE5l dyBSb21hbiIsInNlcmlmIjsNCgljb2xvcjpibGFjazt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJs aW5rDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjpibHVlOw0KCXRleHQtZGVjb3Jh dGlvbjp1bmRlcmxpbmU7fQ0KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJ e21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjpwdXJwbGU7DQoJdGV4dC1kZWNvcmF0aW9u OnVuZGVybGluZTt9DQpwDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgltc28tbWFyZ2luLXRv cC1hbHQ6YXV0bzsNCgltYXJnaW4tcmlnaHQ6MGNtOw0KCW1zby1tYXJnaW4tYm90dG9tLWFsdDph dXRvOw0KCW1hcmdpbi1sZWZ0OjBjbTsNCglmb250LXNpemU6MTIuMHB0Ow0KCWZvbnQtZmFtaWx5 OiJUaW1lcyBOZXcgUm9tYW4iLCJzZXJpZiI7DQoJY29sb3I6YmxhY2s7fQ0KcC5Nc29BY2V0YXRl LCBsaS5Nc29BY2V0YXRlLCBkaXYuTXNvQWNldGF0ZQ0KCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7 DQoJbXNvLXN0eWxlLWxpbms6IkJhbGxvb24gVGV4dCBDaGFyIjsNCgltYXJnaW46MGNtOw0KCW1h cmdpbi1ib3R0b206LjAwMDFwdDsNCglmb250LXNpemU6OC4wcHQ7DQoJZm9udC1mYW1pbHk6IlRh aG9tYSIsInNhbnMtc2VyaWYiOw0KCWNvbG9yOmJsYWNrO30NCnNwYW4uQmFsbG9vblRleHRDaGFy DQoJe21zby1zdHlsZS1uYW1lOiJCYWxsb29uIFRleHQgQ2hhciI7DQoJbXNvLXN0eWxlLXByaW9y aXR5Ojk5Ow0KCW1zby1zdHlsZS1saW5rOiJCYWxsb29uIFRleHQiOw0KCWZvbnQtZmFtaWx5OiJU YWhvbWEiLCJzYW5zLXNlcmlmIjsNCgljb2xvcjpibGFjazt9DQpzcGFuLmljb24NCgl7bXNvLXN0 eWxlLW5hbWU6aWNvbjt9DQpzcGFuLkVtYWlsU3R5bGUyMQ0KCXttc28tc3R5bGUtdHlwZTpwZXJz b25hbDsNCglmb250LWZhbWlseToiQ291cmllciBOZXciOw0KCWNvbG9yOndpbmRvd3RleHQ7fQ0K c3Bhbi5FbWFpbFN0eWxlMjINCgl7bXNvLXN0eWxlLXR5cGU6cGVyc29uYWw7DQoJZm9udC1mYW1p bHk6IkNvdXJpZXIgTmV3IjsNCgljb2xvcjp3aW5kb3d0ZXh0O30NCnNwYW4uRW1haWxTdHlsZTI0 DQoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsLXJlcGx5Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJp Iiwic2Fucy1zZXJpZiI7DQoJY29sb3I6IzFGNDk3RDt9DQouTXNvQ2hwRGVmYXVsdA0KCXttc28t c3R5bGUtdHlwZTpleHBvcnQtb25seTsNCglmb250LXNpemU6MTAuMHB0O30NCkBwYWdlIFdvcmRT ZWN0aW9uMQ0KCXtzaXplOjYxMi4wcHQgNzkyLjBwdDsNCgltYXJnaW46NzIuMHB0IDcyLjBwdCA3 Mi4wcHQgNzIuMHB0O30NCmRpdi5Xb3JkU2VjdGlvbjENCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQ0K LyogTGlzdCBEZWZpbml0aW9ucyAqLw0KQGxpc3QgbDANCgl7bXNvLWxpc3QtaWQ6OTY5MDkwNzM2 Ow0KCW1zby1saXN0LXRlbXBsYXRlLWlkczotNzg4NzI4MDEyO30NCkBsaXN0IGwwOmxldmVsMQ0K CXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ674K3Ow0K CW1zby1sZXZlbC10YWItc3RvcDozNi4wcHQ7DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjps ZWZ0Ow0KCXRleHQtaW5kZW50Oi0xOC4wcHQ7DQoJbXNvLWFuc2ktZm9udC1zaXplOjEwLjBwdDsN Cglmb250LWZhbWlseTpTeW1ib2w7fQ0Kb2wNCgl7bWFyZ2luLWJvdHRvbTowY207fQ0KdWwNCgl7 bWFyZ2luLWJvdHRvbTowY207fQ0KLS0+PC9zdHlsZT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4N CjxvOnNoYXBlZGVmYXVsdHMgdjpleHQ9ImVkaXQiIHNwaWRtYXg9IjEwMjYiIC8+DQo8L3htbD48 IVtlbmRpZl0tLT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlbGF5b3V0IHY6ZXh0 PSJlZGl0Ij4NCjxvOmlkbWFwIHY6ZXh0PSJlZGl0IiBkYXRhPSIxIiAvPg0KPC9vOnNoYXBlbGF5 b3V0PjwveG1sPjwhW2VuZGlmXS0tPjwvaGVhZD48Ym9keSBiZ2NvbG9yPXdoaXRlIGxhbmc9RU4t R0IgbGluaz1ibHVlIHZsaW5rPXB1cnBsZT48ZGl2IGNsYXNzPVdvcmRTZWN0aW9uMT48ZGl2Pjxw IGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdp bi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFt aWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+TWVkLDxvOnA+PC9vOnA+ PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDph dXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEx LjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPkkg cmVhZCB5b3VyIGRyYWZ0IGFuZCBsb29rZWQgZm9yIGRpZmZlcmVuY2VzIGJldHdlZW4gaXQgYW5k IDxhIGhyZWY9Imh0dHA6Ly90b29scy5pZXRmLm9yZy9odG1sL2RyYWZ0LWxpbnNuZXItbG1hcC11 c2UtY2FzZXMiPmh0dHA6Ly90b29scy5pZXRmLm9yZy9odG1sL2RyYWZ0LWxpbnNuZXItbG1hcC11 c2UtY2FzZXM8L2E+PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHls ZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxz cGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1z ZXJpZiI7Y29sb3I6IzFGNDk3RCc+eW91ciBkcmFmdCBpcyBhYm91dCDigJxVbmRlcnN0YW5kaW5n IHRoZSBxdWFsaXR5IGV4cGVyaWVuY2VkIGJ5IGN1c3RvbWVyc+KAnSAod2hpY2ggY29ycmVzcG9u ZHMgdG8gUzMuMiBvZiBkcmFmdC1saW5zbmVyKS7CoCBZb3UgZG9u4oCZdCBtZW50aW9uIGRyYWZ0 LWxpbnNuZXLigJlzIG90aGVyIHVzZSBjYXNlcy4gaXMgdGhlIG9taXNzaW9uIGp1c3QgYmVjYXVz ZSB5b3Ugd2VyZSBjb25jZW50cmF0aW5nIG9uIHRoZSDigJhxdWFsaXR5IGV4cGVyaWVuY2Vk4oCZ IHVzZSBjYXNlLCBvciBkbyB5b3UgdGhpbmsgdGhlIG90aGVyIHVzZSBjYXNlcyBhcmUgdW5pbXBv cnRhbnQ/PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNv LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0 eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7 Y29sb3I6IzFGNDk3RCc+YXNzdW1pbmcgdGhlIGZvcm1lciwgdGhlbiBJIHN1Z2dlc3QgYWRkaW5n IGEgY291cGxlIG9mIGltcHJvdmVtZW50cyB0byBTZWN0aW9uIDMuMiBvZiBkcmFmdC1saW5zbmVy IChJIHRoaW5rIHRoaXMgd291bGQgdGhlbiBjb3ZlciBhbGwgdGhlIGRlbHRhcyk6IDxvOnA+PC9v OnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFs dDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXpl OjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0Qn PihlbmQgb2YgcGFyYSAzKcKgIFRoZSBvcGVyYXRvciBtYXkgYWxzbyB3YW50IHRvIHVuZGVyc3Rh bmQgdGhlIGV4cGVyaWVuY2UgZm9yIGl0cyBhcHBsaWNhdGlvbiBhbmQgY29udGVudCBzZXJ2aWNl cywgc3VjaCBhcyBUViBvciBWUE4uIFRoZXNlIG1heSBpbnZvbHZlIHNwZWNpZmljIG5ldHdvcmsg ZmVhdHVyZXMgc3VjaCBhcyBRb1MgcHJpb3JpdGlzYXRpb24sIG11bHRpY2FzdCBvciBwb2xpY3kg ZW5mb3JjZW1lbnQgd2hpY2ggYSBtZXRyaWMgd291bGQgbmVlZCB0byBlbmNvbXBhc3MuPG86cD48 L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3At YWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNp emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3 RCc+KGVuZCBvZiBwYXJhIDUpIFRoZSBvcGVyYXRvciBtYXkgYWxzbyB3YW50IHRvIGNoZWNrIChh bmQgYWxsb3cgY3VzdG9tZXJzIHRvIGNoZWNrKSB0aGF0IGl0IGlzIGZ1bGZpbGxpbmcgYSBjdXN0 b21lcuKAmXMgc2VydmljZSBsZXZlbCBhZ3JlZW1lbnQuPG86cD48L286cD48L3NwYW4+PC9wPjxw IGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdp bi1ib3R0b20tYWx0OmF1dG8nPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFt aWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3RCc+d2hhdCBkbyB5b3UgcmVj a29uPzxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1t YXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvJz48c3BhbiBzdHls ZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO2Nv bG9yOiMxRjQ5N0QnPnRoYW5rczxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3Jt YWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph dXRvJz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseToiQ2FsaWJyaSIs InNhbnMtc2VyaWYiO2NvbG9yOiMxRjQ5N0QnPnBoaWw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAg Y2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2lu LWJvdHRvbS1hbHQ6YXV0byc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1p bHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTdEJz48bzpwPiZuYnNwOzwvbzpw Pjwvc3Bhbj48L3A+PC9kaXY+PGRpdj48ZGl2IHN0eWxlPSdib3JkZXI6bm9uZTtib3JkZXItdG9w OnNvbGlkICNCNUM0REYgMS4wcHQ7cGFkZGluZzozLjBwdCAwY20gMGNtIDBjbSc+PHAgY2xhc3M9 TXNvTm9ybWFsPjxiPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9u dC1mYW1pbHk6IlRhaG9tYSIsInNhbnMtc2VyaWYiO2NvbG9yOndpbmRvd3RleHQnPkZyb206PC9z cGFuPjwvYj48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFt aWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlmIjtjb2xvcjp3aW5kb3d0ZXh0Jz4gbG1hcC1ib3VuY2Vz QGlldGYub3JnIFttYWlsdG86bG1hcC1ib3VuY2VzQGlldGYub3JnXSA8Yj5PbiBCZWhhbGYgT2Yg PC9iPm1vaGFtZWQuYm91Y2FkYWlyQG9yYW5nZS5jb208YnI+PGI+U2VudDo8L2I+IDIxIEZlYnJ1 YXJ5IDIwMTMgMTI6NDA8YnI+PGI+VG86PC9iPiBNT1JUT04gSlIuLCBBTEZSRUQgKEFMKTsgQmVu b2l0IENsYWlzZTsgbG1hcEBpZXRmLm9yZzsgUm9tYXNjYW51LCBEYW4gKERhbikgKGRyb21hc2Nh QGF2YXlhLmNvbSk8YnI+PGI+Q2M6PC9iPiBKQUNRVUVORVQgQ2hyaXN0aWFuIE9MTkMvT0xOPGJy PjxiPlN1YmplY3Q6PC9iPiBSZTogW2xtYXBdIERyYWZ0IEFnZW5kYSBbd2FzOiBMTUFQIEJvRiBo YXMgYmVlbiBhcHByb3ZlZC5dPG86cD48L286cD48L3NwYW4+PC9wPjwvZGl2PjwvZGl2PjxwIGNs YXNzPU1zb05vcm1hbD48bzpwPiZuYnNwOzwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNw YW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQ291cmll ciBOZXciO2NvbG9yOmJsdWUnPlJlLSw8L3NwYW4+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nY29s b3I6d2luZG93dGV4dCc+PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48 c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdjb2xvcjp3aW5kb3d0ZXh0Jz4mbmJzcDs8bzpwPjwvbzpw Pjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9J2Zv bnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xvcjpibHVlJz5kcmFm dC1ib3VjYWRhaXItKiBpbmNsdWRlcyBzb21lIHBvaW50cyBub3QgbWVudGlvbmVkIGluIG90aGVy IGRyYWZ0LSotbG1hcCZuYnNwO0ktRHMgc3VjaCB0aGUgRFNDUCBtYXJraW5nIGluaGVyaXRhbmNl OiBzb21lIG9mIHRoZSBzZXJ2aWNlcyB3ZSBhcmUgZGVwbG95aW5nIGRvIG5vdCB0cnVzdCB0aGUg bWFya2luZyBzZXQgYnkgdGhlIGFwcGxpY2F0aW9uIGJ1dCB0aGVyZSZuYnNwO2FyZSBzcGVjaWZp YyBtb2R1bGVzIHdoaWNoIGFyZSZuYnNwO2VuYWJsZWQgaW4gdGhlIENQRSBmb3IgaW5zdGFuY2Uu IENvbmR1Y3RpbmcgYSBzZXQgb2YgbWVhc3VyZW1lbnQgZm9yIHRob3NlIHNlcnZpY2Ugd2lsbCBu ZWVkIHRvIGJlIGRlc2lnbmVkIHRvIGJlIGNvbXBsaWFudCB3aXRoIHN1Y2ggY29uc3RyYWludC4m bmJzcDs8L3NwYW4+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nY29sb3I6d2luZG93dGV4dCc+PG86 cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0 eWxlPSdjb2xvcjp3aW5kb3d0ZXh0Jz4mbmJzcDs8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xh c3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9u dC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xvcjpibHVlJz5JIGNhbiBsaXN0IGFub3RoZXImbmJz cDtpdGVtIHN1Y2ggYXMgdGhlIGNvbXBhdGliaWxpdHkgd2l0aCBhJm5ic3A7bXVsdGktYWRkcmVz c2VkIHNlcnZpY2UgZGVzaWduIHNjaGVtZSZuYnNwOyhpbiB3aGljaCZuYnNwOyBkaXN0aW5jdCBJ UCBhZGRyZXNzZXMmbmJzcDthcmUgYXNzaWduZWQmbmJzcDtmb3IgZWFjaCBzdWJzY3JpYmVkIHNl cnZpY2UpOiBwYWNrZXRzIGlzc3VlZCBmcm9tJm5ic3A7YSZuYnNwO21lYXN1cmVtZW50IGFnZW50 IChsb2NhdGVkIGJlaGluZCZuYnNwO3RoZSBDUEUpIHdpbGwgbmVlZCB0byBiZSBib3VuZCB0byB0 aGUgYXBwcm9wcmlhdGUgc2VydmljZS1zcGVjaWZpYyBjaGFubmVsLCBldGMuPC9zcGFuPjxzcGFu IGxhbmc9RU4tVVMgc3R5bGU9J2NvbG9yOndpbmRvd3RleHQnPjxvOnA+PC9vOnA+PC9zcGFuPjwv cD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nY29sb3I6d2luZG93 dGV4dCc+Jm5ic3A7PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3Bh biBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVy IE5ldyI7Y29sb3I6Ymx1ZSc+Q2hlZXJzLDwvc3Bhbj48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdj b2xvcjp3aW5kb3d0ZXh0Jz48bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFs PjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNv dXJpZXIgTmV3Ijtjb2xvcjpibHVlJz5NZWQ8L3NwYW4+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0n Y29sb3I6d2luZG93dGV4dCc+PG86cD48L286cD48L3NwYW4+PC9wPjxibG9ja3F1b3RlIHN0eWxl PSdib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCBibHVlIDEuNXB0O3BhZGRpbmc6MGNtIDBj bSAwY20gNC4wcHQ7bWFyZ2luLWxlZnQ6My43NXB0O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJp Z2h0OjBjbTttYXJnaW4tYm90dG9tOjUuMHB0Jz48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFu Zz1FTi1VUyBzdHlsZT0nY29sb3I6d2luZG93dGV4dCc+PG86cD4mbmJzcDs8L286cD48L3NwYW4+ PC9wPjxkaXYgY2xhc3M9TXNvTm9ybWFsIGFsaWduPWNlbnRlciBzdHlsZT0ndGV4dC1hbGlnbjpj ZW50ZXInPjxzcGFuIGxhbmc9RlIgc3R5bGU9J2NvbG9yOndpbmRvd3RleHQnPjxociBzaXplPTIg d2lkdGg9IjEwMCUiIGFsaWduPWNlbnRlcj48L3NwYW4+PC9kaXY+PHAgY2xhc3M9TXNvTm9ybWFs IHN0eWxlPSdtYXJnaW4tYm90dG9tOjEyLjBwdCc+PGI+PHNwYW4gbGFuZz1GUiBzdHlsZT0nZm9u dC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiVGFob21hIiwic2Fucy1zZXJpZiI7Y29sb3I6d2lu ZG93dGV4dCc+RGUmbmJzcDs6PC9zcGFuPjwvYj48c3BhbiBsYW5nPUZSIHN0eWxlPSdmb250LXNp emU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlmIjtjb2xvcjp3aW5kb3d0 ZXh0Jz4gTU9SVE9OIEpSLiwgQUxGUkVEIChBTCkgWzxhIGhyZWY9Im1haWx0bzphY21vcnRvbkBh dHQuY29tIj5tYWlsdG86YWNtb3J0b25AYXR0LmNvbTwvYT5dIDxicj48Yj5FbnZvecOpJm5ic3A7 OjwvYj4gamV1ZGkgMjEgZsOpdnJpZXIgMjAxMyAxMzoyOTxicj48Yj7DgCZuYnNwOzo8L2I+IEJP VUNBREFJUiBNb2hhbWVkIE9MTkMvT0xOOyBCZW5vaXQgQ2xhaXNlOyA8YSBocmVmPSJtYWlsdG86 bG1hcEBpZXRmLm9yZyI+bG1hcEBpZXRmLm9yZzwvYT47IFJvbWFzY2FudSwgRGFuIChEYW4pICg8 YSBocmVmPSJtYWlsdG86ZHJvbWFzY2FAYXZheWEuY29tIj5kcm9tYXNjYUBhdmF5YS5jb208L2E+ KTxicj48Yj5DYyZuYnNwOzo8L2I+IEpBQ1FVRU5FVCBDaHJpc3RpYW4gT0xOQy9PTE48YnI+PGI+ T2JqZXQmbmJzcDs6PC9iPiBSRTogRHJhZnQgQWdlbmRhIFt3YXM6IExNQVAgQm9GIGhhcyBiZWVu IGFwcHJvdmVkLl08L3NwYW4+PHNwYW4gbGFuZz1GUiBzdHlsZT0nY29sb3I6d2luZG93dGV4dCc+ PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVT IHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6 d2luZG93dGV4dCc+SGkgTWVkLDxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3Jt YWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToi Q291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwv cD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEw LjBwdDtmb250LWZhbWlseToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPlllcywgb25l IHBvaW50IHlvdSd2ZSBtYWRlIHRoYXQgKElNTykgc2hvdWxkIGJlIGFkZHJlc3NlZCA8bzpwPjwv bzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9 J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xvcjp3aW5kb3d0 ZXh0Jz53aXRoaW4gdGhlIGN1cnJlbnQgY29uc3RyYWludHMgb2YgdGhlIEZyYW1ld29yayBkcmFm dCw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4t VVMgc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xv cjp3aW5kb3d0ZXh0Jz5pcyBhYm91dCB0ZXN0aW5nIGNyaXRpY2FsIHNlcnZpY2VzIHdoaWNoIHRo ZSA8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4t VVMgc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xv cjp3aW5kb3d0ZXh0Jz5hY2Nlc3MgSVNQIGNvbnRyYWN0cyBmb3IsIHNvIGRvZXMgbm90IG93biBp dHNlbGYuPG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5n PUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7 Y29sb3I6d2luZG93dGV4dCc+WW91ciBleGFtcGxlLCBETlMgdGVzdGluZywgaXMgcGFydGljdWxh cmx5IHJlbGV2YW50PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3Bh biBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVy IE5ldyI7Y29sb3I6d2luZG93dGV4dCc+c2luY2UgZXZlcnkgdXNlciBwZXJmb3JtcyBxdWVyaWVz IHRoYXQgY3Jvc3Mgb3JnLiBib3VuZGFyaWVzLDxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFz cz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250 LWZhbWlseToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPmFuZCBzbyB3b3VsZCB0aGUg bWVhc3VyZW1lbnQgcGF0aCBmb3IgRE5TIHJlc3BvbnNlIHRpbWUuPG86cD48L286cD48L3NwYW4+ PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6 MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6d2luZG93dGV4dCc+SG93ZXZl ciwgYWxsIHRoZSBtZWFzdXJlbWVudCBpbmZyYXN0cnVjdHVyZSB3b3VsZCByZW1haW48bzpwPjwv bzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9 J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xvcjp3aW5kb3d0 ZXh0Jz53aXRoaW4gYSBzaW5nbGUgb3JnYW5pemF0aW9uJ3MgZG9tYWluLCBhbmQgd2hpbGUgSSB0 aGluayB0aGlzPG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBs YW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5l dyI7Y29sb3I6d2luZG93dGV4dCc+bWF0Y2hlcyB0aGUgY3VycmVudCBjb25zdHJhaW50LCBpdCdz IGNlcnRhaW5seSB3b3J0aCBpbmNsdWRpbmcuPG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNz PU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQt ZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6d2luZG93dGV4dCc+PG86cD4mbmJzcDs8L286cD48 L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250 LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6d2luZG93dGV4dCc+ QWZ0ZXIgcmV2aWV3aW5nIHRoZSBVc2UgQ2FzZSBEcmFmdCwgcGFydGljdWxhcmx5IHRoZSBkZXRh aWxlZCBJU1A8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxh bmc9RU4tVVMgc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3 Ijtjb2xvcjp3aW5kb3d0ZXh0Jz5Vc2UgQ2FzZSwgYW5kIHRoZSBGcmFtZXdvcmsgRHJhZnQgaXRz ZWxmLCBjYW4geW91IHBvaW50IG91dDxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29O b3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWls eToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPmFkZGl0aW9uYWwgY29uc2lkZXJhdGlv bnMgbm90LXlldCBkZXNjcmliZWQgKGJ1dCB3aXRoaW4gdGhlIGN1cnJlbnQgPG86cD48L286cD48 L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250 LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6d2luZG93dGV4dCc+ Y29uc3RyYWludHMpPzxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNw YW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQ291cmll ciBOZXciO2NvbG9yOndpbmRvd3RleHQnPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD48cCBj bGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtm b250LWZhbWlseToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPnRoYW5rcyw8bzpwPjwv bzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9 J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xvcjp3aW5kb3d0 ZXh0Jz5BbDxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFu Zz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQ291cmllciBOZXci O2NvbG9yOndpbmRvd3RleHQnPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1N c29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZh bWlseToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPjxvOnA+Jm5ic3A7PC9vOnA+PC9z cGFuPjwvcD48ZGl2IHN0eWxlPSdib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCBibHVlIDEu NXB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNC4wcHQnPjxkaXY+PGRpdiBzdHlsZT0nYm9yZGVyOm5v bmU7Ym9yZGVyLXRvcDpzb2xpZCAjQjVDNERGIDEuMHB0O3BhZGRpbmc6My4wcHQgMGNtIDBjbSAw Y20nPjxwIGNsYXNzPU1zb05vcm1hbD48Yj48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNp emU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlmIjtjb2xvcjp3aW5kb3d0 ZXh0Jz5Gcm9tOjwvc3Bhbj48L2I+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEw LjBwdDtmb250LWZhbWlseToiVGFob21hIiwic2Fucy1zZXJpZiI7Y29sb3I6d2luZG93dGV4dCc+ IDxhIGhyZWY9Im1haWx0bzptb2hhbWVkLmJvdWNhZGFpckBvcmFuZ2UuY29tIj5tb2hhbWVkLmJv dWNhZGFpckBvcmFuZ2UuY29tPC9hPiBbPGEgaHJlZj0ibWFpbHRvOm1vaGFtZWQuYm91Y2FkYWly QG9yYW5nZS5jb20iPm1haWx0bzptb2hhbWVkLmJvdWNhZGFpckBvcmFuZ2UuY29tPC9hPl0gPGJy PjxiPlNlbnQ6PC9iPiBUaHVyc2RheSwgRmVicnVhcnkgMjEsIDIwMTMgMTozNSBBTTxicj48Yj5U bzo8L2I+IE1PUlRPTiBKUi4sIEFMRlJFRCAoQUwpOyBCZW5vaXQgQ2xhaXNlOyA8YSBocmVmPSJt YWlsdG86bG1hcEBpZXRmLm9yZyI+bG1hcEBpZXRmLm9yZzwvYT48YnI+PGI+Q2M6PC9iPiBKQUNR VUVORVQgQ2hyaXN0aWFuIE9MTkMvT0xOPGJyPjxiPlN1YmplY3Q6PC9iPiBSRTogRHJhZnQgQWdl bmRhIFt3YXM6IExNQVAgQm9GIGhhcyBiZWVuIGFwcHJvdmVkLl08bzpwPjwvbzpwPjwvc3Bhbj48 L3A+PC9kaXY+PC9kaXY+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4tVVM+PG86cD4m bmJzcDs8L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVT IHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6 Ymx1ZSc+RGVhciBBbCw8L3NwYW4+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nY29sb3I6d2luZG93 dGV4dCc+PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5n PUVOLVVTIHN0eWxlPSdjb2xvcjp3aW5kb3d0ZXh0Jz4mbmJzcDs8bzpwPjwvbzpwPjwvc3Bhbj48 L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9J2ZvbnQtc2l6ZTox MC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xvcjpibHVlJz5Xb3VsZCBpdCBiZSBw b3NzaWJsZSB0aGUgZm9sbG93aW5nIHNsb3QgPC9zcGFuPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9 J2NvbG9yOndpbmRvd3RleHQnPjxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3Jt YWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nY29sb3I6d2luZG93dGV4dCc+Jm5ic3A7PG86cD48 L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxl PSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6Ymx1ZSc+ JnF1b3Q7NS4gRnJhbWV3b3JrOiBQcm9ibGVtIFN0YXRlbWVudCwgSVNQIFVzZSBjYXNlLCBHYXBz LCBSZXF1aXJlbWVudHMgLSBQaGlsIEVhcmRsZXkgLSAxNSBtaW4mcXVvdDs8L3NwYW4+PHNwYW4g bGFuZz1FTi1VUyBzdHlsZT0nY29sb3I6d2luZG93dGV4dCc+PG86cD48L286cD48L3NwYW4+PC9w PjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdjb2xvcjp3aW5kb3d0 ZXh0Jz4mbmJzcDs8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFu IGxhbmc9RU4tVVMgc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIg TmV3Ijtjb2xvcjpibHVlJz50byBpbmNsdWRlJm5ic3A7c29tZSBvZiB0aGUgbmV0d29yayBwcm92 aWRlcicgcmVxdWlyZW1lbnRzIGFuZCBpc3N1ZXMgZGlzY3Vzc2VkIGluIGRyYWZ0LWJvdWNhZGFp ci1sbWFwLWNvbnNpZGVyYXRpb25zPzwvc3Bhbj48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdjb2xv cjp3aW5kb3d0ZXh0Jz48bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxz cGFuIGxhbmc9RU4tVVMgc3R5bGU9J2NvbG9yOndpbmRvd3RleHQnPiZuYnNwOzxvOnA+PC9vOnA+ PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9u dC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQ291cmllciBOZXciO2NvbG9yOmJsdWUnPlRoYW5r cy48L3NwYW4+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nY29sb3I6d2luZG93dGV4dCc+PG86cD48 L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxl PSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6Ymx1ZSc+ Q2hlZXJzLDwvc3Bhbj48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdjb2xvcjp3aW5kb3d0ZXh0Jz48 bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4tVVMg c3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xvcjpi bHVlJz5NZWQ8L3NwYW4+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nY29sb3I6d2luZG93dGV4dCc+ PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVT IHN0eWxlPSdjb2xvcjp3aW5kb3d0ZXh0Jz4mbmJzcDs8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PGJs b2NrcXVvdGUgc3R5bGU9J2JvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkIGJsdWUgMS41cHQ7 cGFkZGluZzowY20gMGNtIDBjbSA0LjBwdDttYXJnaW4tbGVmdDozLjc1cHQ7bWFyZ2luLXRvcDo1 LjBwdDttYXJnaW4tcmlnaHQ6MGNtO21hcmdpbi1ib3R0b206NS4wcHQnPjxwIGNsYXNzPU1zb05v cm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdjb2xvcjp3aW5kb3d0ZXh0Jz48bzpwPiZuYnNw OzwvbzpwPjwvc3Bhbj48L3A+PGRpdiBjbGFzcz1Nc29Ob3JtYWwgYWxpZ249Y2VudGVyIHN0eWxl PSd0ZXh0LWFsaWduOmNlbnRlcic+PHNwYW4gbGFuZz1GUiBzdHlsZT0nY29sb3I6d2luZG93dGV4 dCc+PGhyIHNpemU9MiB3aWR0aD0iMTAwJSIgYWxpZ249Y2VudGVyPjwvc3Bhbj48L2Rpdj48cCBj bGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21hcmdpbi1ib3R0b206MTIuMHB0Jz48Yj48c3BhbiBsYW5n PUZSIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNl cmlmIjtjb2xvcjp3aW5kb3d0ZXh0Jz5EZSZuYnNwOzo8L3NwYW4+PC9iPjxzcGFuIGxhbmc9RlIg c3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IlRhaG9tYSIsInNhbnMtc2VyaWYi O2NvbG9yOndpbmRvd3RleHQnPiA8YSBocmVmPSJtYWlsdG86bG1hcC1ib3VuY2VzQGlldGYub3Jn Ij5sbWFwLWJvdW5jZXNAaWV0Zi5vcmc8L2E+IFs8YSBocmVmPSJtYWlsdG86bG1hcC1ib3VuY2Vz QGlldGYub3JnIj5tYWlsdG86bG1hcC1ib3VuY2VzQGlldGYub3JnPC9hPl0gPGI+RGUgbGEgcGFy dCBkZTwvYj4gTU9SVE9OIEpSLiwgQUxGUkVEIChBTCk8YnI+PGI+RW52b3nDqSZuYnNwOzo8L2I+ IG1hcmRpIDE5IGbDqXZyaWVyIDIwMTMgMjA6MDU8YnI+PGI+w4AmbmJzcDs6PC9iPiBCZW5vaXQg Q2xhaXNlOyA8YSBocmVmPSJtYWlsdG86bG1hcEBpZXRmLm9yZyI+bG1hcEBpZXRmLm9yZzwvYT48 YnI+PGI+T2JqZXQmbmJzcDs6PC9iPiBbbG1hcF0gRHJhZnQgQWdlbmRhIFt3YXM6IExNQVAgQm9G IGhhcyBiZWVuIGFwcHJvdmVkLl08L3NwYW4+PHNwYW4gbGFuZz1GUiBzdHlsZT0nY29sb3I6d2lu ZG93dGV4dCc+PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBs YW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5l dyI7Y29sb3I6d2luZG93dGV4dCc+TE1BUCBsaXN0LDxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBj bGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtm b250LWZhbWlseToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPjxvOnA+Jm5ic3A7PC9v OnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0n Zm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3Rl eHQnPldlIGhhdmUgcHJlcGFyZWQgYSBkcmFmdCBCb0YgYWdlbmRhLDxvOnA+PC9vOnA+PC9zcGFu PjwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXpl OjEwLjBwdDtmb250LWZhbWlseToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPmFuZCBu b3cgdGhhdCBvdXIgcG9zdGluZyByaWdodHMgYXJlIGluc3RhbGxlZCwgPG86cD48L286cD48L3Nw YW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNp emU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6d2luZG93dGV4dCc+aXQg Y2FuIGJlIGZvdW5kIHdpdGggYWxsIG1lZXRpbmcgbWF0ZXJpYWxzOiA8bzpwPjwvbzpwPjwvc3Bh bj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9J2ZvbnQtc2l6 ZToxMC4wcHQ7Zm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3Ijtjb2xvcjp3aW5kb3d0ZXh0Jz48YSBo cmVmPSJodHRwOi8vd3d3LmlldGYub3JnL3Byb2NlZWRpbmdzLzg2L2FnZW5kYS9hZ2VuZGEtODYt bG1hcCI+aHR0cDovL3d3dy5pZXRmLm9yZy9wcm9jZWVkaW5ncy84Ni9hZ2VuZGEvYWdlbmRhLTg2 LWxtYXA8L2E+PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBs YW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5l dyI7Y29sb3I6d2luZG93dGV4dCc+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPjxwIGNsYXNz PU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQt ZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6d2luZG93dGV4dCc+V2UgYmVsaWV2ZSB3ZSBoYXZl IGVtYnJhY2VkIHRoZSB0c3VuYW1pIG9mIDAwIGRyYWZ0czxvOnA+PC9vOnA+PC9zcGFuPjwvcD48 cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBw dDtmb250LWZhbWlseToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPmFuZCBmb3VuZCBh bGwgdGhvc2UgdGhhdCBhcmUgcmVsZXZhbnQsIDxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFz cz1Nc29Ob3JtYWw+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250 LWZhbWlseToiQ291cmllciBOZXciO2NvbG9yOndpbmRvd3RleHQnPmFuZCBjb250YWN0ZWQgYWxs LTEgb2YgdGhlIHByZXNlbnRlcnMuPG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05v cm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5 OiJDb3VyaWVyIE5ldyI7Y29sb3I6d2luZG93dGV4dCc+PG86cD4mbmJzcDs8L286cD48L3NwYW4+ PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6 MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6d2luZG93dGV4dCc+Q29tbWVu dHMgdG8gdGhlIGxpc3QsIHBsZWFzZSw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNv Tm9ybWFsPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1p bHk6IkNvdXJpZXIgTmV3Ijtjb2xvcjp3aW5kb3d0ZXh0Jz5EYW4gYW5kIEFsPG86cD48L286cD48 L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250 LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Y29sb3I6d2luZG93dGV4dCc+ Ym9mIGNvLWNoYWlyczxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PHNw YW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiQ291cmll ciBOZXciO2NvbG9yOndpbmRvd3RleHQnPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD48ZGl2 IHN0eWxlPSdib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCBibHVlIDEuNXB0O3BhZGRpbmc6 MGNtIDBjbSAwY20gNC4wcHQnPjxkaXY+PGRpdiBzdHlsZT0nYm9yZGVyOm5vbmU7Ym9yZGVyLXRv cDpzb2xpZCAjQjVDNERGIDEuMHB0O3BhZGRpbmc6My4wcHQgMGNtIDBjbSAwY20nPjxwIGNsYXNz PU1zb05vcm1hbD48Yj48c3BhbiBsYW5nPUVOLVVTIHN0eWxlPSdmb250LXNpemU6MTAuMHB0O2Zv bnQtZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlmIjtjb2xvcjp3aW5kb3d0ZXh0Jz5Gcm9tOjwv c3Bhbj48L2I+PHNwYW4gbGFuZz1FTi1VUyBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDtmb250LWZh bWlseToiVGFob21hIiwic2Fucy1zZXJpZiI7Y29sb3I6d2luZG93dGV4dCc+IDxhIGhyZWY9Im1h aWx0bzpsbWFwLWJvdW5jZXNAaWV0Zi5vcmciPmxtYXAtYm91bmNlc0BpZXRmLm9yZzwvYT4gWzxh IGhyZWY9Im1haWx0bzpsbWFwLWJvdW5jZXNAaWV0Zi5vcmciPm1haWx0bzpsbWFwLWJvdW5jZXNA aWV0Zi5vcmc8L2E+XSA8Yj5PbiBCZWhhbGYgT2YgPC9iPkJlbm9pdCBDbGFpc2U8YnI+PGI+U2Vu dDo8L2I+IFdlZG5lc2RheSwgRmVicnVhcnkgMDYsIDIwMTMgMTA6MDAgQU08YnI+PGI+VG86PC9i PiA8YSBocmVmPSJtYWlsdG86bG1hcEBpZXRmLm9yZyI+bG1hcEBpZXRmLm9yZzwvYT48YnI+PGI+ U3ViamVjdDo8L2I+IFtsbWFwXSBMTUFQIEJvRiBoYXMgYmVlbiBhcHByb3ZlZC48bzpwPjwvbzpw Pjwvc3Bhbj48L3A+PC9kaXY+PC9kaXY+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIGxhbmc9RU4t VVM+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBs YW5nPUVOLVVTPkRlYXIgYWxsLDxvOnA+PC9vOnA+PC9zcGFuPjwvcD48ZGl2PjxwIGNsYXNzPU1z b05vcm1hbD48c3BhbiBsYW5nPUVOLVVTPjxicj5MZXQgdXMgc2hhcmUgdGhlIGdvb2QgbmV3cy4g VGhlIElFU0cgYXBwcm92ZWQgdGhpcyBCb0YuPGJyPkV2ZW4gaWYgaXQgd2FzIGluaXRpYWxseSBy ZXF1ZXN0ZWQgaW4gVFNWLCB0aGUgQm9GIHdpbGwgYmUgc3BvbnNvcmVkIGluIE9QUyBieSBCZW5v aXQgQ2xhaXNlLjxicj5UaGUgQm9GIGluZm9ybWF0aW9uIGhhdmUgYmVlbiB1cGRhdGVkIChTZWUg PGEgaHJlZj0iaHR0cDovL3RyYWMudG9vbHMuaWV0Zi5vcmcvYm9mL3RyYWMvd2lraS9XaWtpU3Rh cnQiPmh0dHA6Ly90cmFjLnRvb2xzLmlldGYub3JnL2JvZi90cmFjL3dpa2kvV2lraVN0YXJ0PC9h Pik8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHA+PHN0cm9uZz48c3BhbiBsYW5nPUVOLVVTPkxhcmdl LVNjYWxlIE1lYXN1cmVtZW50IG9mIEJyb2FkYmFuZCBQZXJmb3JtYW5jZSAoTE1BUCk8L3NwYW4+ PC9zdHJvbmc+PHNwYW4gbGFuZz1FTi1VUz48bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHVsIHR5cGU9 ZGlzYz48bGkgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttc28tbGlzdDpsMCBsZXZlbDEgbGZvMSc+PHNwYW4g bGFuZz1FTi1VUz5TdGF0dXM6IFJlcXVlc3RlZC4gPG86cD48L286cD48L3NwYW4+PC9saT48bGkg Y2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2lu LWJvdHRvbS1hbHQ6YXV0bzttc28tbGlzdDpsMCBsZXZlbDEgbGZvMSc+PHNwYW4gbGFuZz1FTi1V Uz5UeXBlOiBXRyBmb3JtaW5nICgxc3QgYXR0ZW1wdCkuIDxvOnA+PC9vOnA+PC9zcGFuPjwvbGk+ PGxpIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1h cmdpbi1ib3R0b20tYWx0OmF1dG87bXNvLWxpc3Q6bDAgbGV2ZWwxIGxmbzEnPjxzcGFuIGxhbmc9 RU4tVVM+UmVzcG9uc2libGUgQUQ6IEJlbm9pdCBDbGFpc2UsIE9QUyA8bzpwPjwvbzpwPjwvc3Bh bj48L2xpPjxsaSBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRv O21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21zby1saXN0OmwwIGxldmVsMSBsZm8xJz48c3Bh biBsYW5nPUVOLVVTPkJPRiBDaGFpcnM6IERhbiBSb21hc2NhbnUgKDxhIGhyZWY9Im1haWx0bzpk cm9tYXNjYUBhdmF5YS5jb20iPjxzcGFuIGNsYXNzPWljb24+PHNwYW4gc3R5bGU9J2ZvbnQtZmFt aWx5OiJDYW1icmlhIE1hdGgiLCJzZXJpZiInPuKAizwvc3Bhbj48L3NwYW4+ZHJvbWFzY2FAYXZh eWEuY29tPC9hPiksIEFsIE1vcnRvbiAoPGEgaHJlZj0ibWFpbHRvOmFjbW9ydG9uQGF0dC5jb20i PjxzcGFuIGNsYXNzPWljb24+PHNwYW4gc3R5bGU9J2ZvbnQtZmFtaWx5OiJDYW1icmlhIE1hdGgi LCJzZXJpZiInPuKAizwvc3Bhbj48L3NwYW4+YWNtb3J0b25AYXR0LmNvbTwvYT4pKSA8bzpwPjwv bzpwPjwvc3Bhbj48L2xpPjxsaSBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9w LWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21zby1saXN0OmwwIGxldmVsMSBs Zm8xJz48c3BhbiBsYW5nPUVOLVVTPkF0dGVuZGFuY2U6IDgwIDxvOnA+PC9vOnA+PC9zcGFuPjwv bGk+PGxpIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNv LW1hcmdpbi1ib3R0b20tYWx0OmF1dG87bXNvLWxpc3Q6bDAgbGV2ZWwxIGxmbzEnPjxzcGFuIGxh bmc9RU4tVVM+U2Vzc2lvbiBsZW5ndGggMTIwIG1pbnV0ZXMgPG86cD48L286cD48L3NwYW4+PC9s aT48bGkgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28t bWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttc28tbGlzdDpsMCBsZXZlbDEgbGZvMSc+PHNwYW4gbGFu Zz1FTi1VUz5Db25mbGljdHMgdG8gYXZvaWQ6IDxicj5GaXJzdCBsZXZlbCBwcmlvcml0eTogSVBQ TSwgRUNSSVQsIEJNV0csIE9QU0FSRUEvT1BTQVdHLCBJUEZJWCwgRU1BTiwgTkVUQ09ORiwgTkVU TU9ELCBYUkJMT0NLLCBESU1FLCBSQURFWFQsIFNBQ00gQm9GLCBUU1ZBUkVBLCBBTFRPLCBDRE5J LCBORlNWNCwgUFBTUCwgVFNWV0cgPGJyPlNlY29uZCBsZXZlbCBwcmlvcml0eTogQ09ORVggPG86 cD48L286cD48L3NwYW4+PC9saT48bGkgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2lu LXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttc28tbGlzdDpsMCBsZXZl bDEgbGZvMSc+PHNwYW4gbGFuZz1FTi1VUz5Eb2VzIGl0IHJlcXVpcmUgV2ViRVg/IE5vIDxvOnA+ PC9vOnA+PC9zcGFuPjwvbGk+PGxpIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10 b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87bXNvLWxpc3Q6bDAgbGV2ZWwx IGxmbzEnPjxzcGFuIGxhbmc9RU4tVVM+RG9lcyBpdCByZXF1aXJlIE1lZXRlY2hvPyBObyA8bzpw PjwvbzpwPjwvc3Bhbj48L2xpPjxsaSBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4t dG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21zby1saXN0OmwwIGxldmVs MSBsZm8xJz48c3BhbiBsYW5nPUVOLVVTPk1haWxpbmcgbGlzdDogPC9zcGFuPjxzcGFuIGxhbmc9 RU4tVVMgc3R5bGU9J2ZvbnQtZmFtaWx5OiJDYW1icmlhIE1hdGgiLCJzZXJpZiInPuKAizwvc3Bh bj48c3BhbiBsYW5nPUVOLVVTPiA8YSBocmVmPSJtYWlsdG86bG1hcEBpZWZ0Lm9yZyI+PHNwYW4g Y2xhc3M9aWNvbj48c3BhbiBzdHlsZT0nZm9udC1mYW1pbHk6IkNhbWJyaWEgTWF0aCIsInNlcmlm Iic+4oCLPC9zcGFuPjwvc3Bhbj5sbWFwQGllZnQub3JnPC9hPiA8bzpwPjwvbzpwPjwvc3Bhbj48 L2xpPjxsaSBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21z by1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21zby1saXN0OmwwIGxldmVsMSBsZm8xJz48c3BhbiBs YW5nPUVOLVVTPkxpc3QgYXJjaGl2ZTogPC9zcGFuPjxzcGFuIGxhbmc9RU4tVVMgc3R5bGU9J2Zv bnQtZmFtaWx5OiJDYW1icmlhIE1hdGgiLCJzZXJpZiInPuKAizwvc3Bhbj48c3BhbiBsYW5nPUVO LVVTPjxhIGhyZWY9Imh0dHA6Ly93d3cuaWV0Zi5vcmcvbWFpbC1hcmNoaXZlL3dlYi9sbWFwL2N1 cnJlbnQvbWFpbGxpc3QuaHRtbCI+PHNwYW4gY2xhc3M9aWNvbj48c3BhbiBzdHlsZT0nZm9udC1m YW1pbHk6IkNhbWJyaWEgTWF0aCIsInNlcmlmIic+4oCLPC9zcGFuPjwvc3Bhbj5odHRwOi8vd3d3 LmlldGYub3JnL21haWwtYXJjaGl2ZS93ZWIvbG1hcC9jdXJyZW50L21haWxsaXN0Lmh0bWw8L2E+ IDxvOnA+PC9vOnA+PC9zcGFuPjwvbGk+PGxpIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1h cmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87bXNvLWxpc3Q6bDAg bGV2ZWwxIGxmbzEnPjxzcGFuIGxhbmc9RU4tVVM+RG9jdW1lbnRzOiA8YSBocmVmPSJodHRwOi8v dG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1zY2h1bHpyaW5uZS1sbWFwLXJlcXVpcmVtZW50cyI+ ZHJhZnQtc2NodWx6cmlubmUtbG1hcC1yZXF1aXJlbWVudHM8L2E+LCA8YSBocmVmPSJodHRwOi8v dG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1saW5zbmVyLWxtYXAtdXNlLWNhc2VzIj5kcmFmdC1s aW5zbmVyLWxtYXAtdXNlLWNhc2VzPC9hPiwgYW5kIDxhIGhyZWY9Imh0dHA6Ly90b29scy5pZXRm Lm9yZy9odG1sL2RyYWZ0LW1vcnRvbi1pcHBtLWxtYXAtcGF0aC0wMCI+ZHJhZnQtbW9ydG9uLWlw cG0tbG1hcC1wYXRoLTAwPC9hPiA8bzpwPjwvbzpwPjwvc3Bhbj48L2xpPjwvdWw+PHAgY2xhc3M9 TXNvTm9ybWFsIHN0eWxlPSdtYXJnaW4tYm90dG9tOjEyLjBwdCc+PHNwYW4gbGFuZz1FTi1VUz5U aGUgY2hhaXJzIGhhdmUgYmVlbiBjaG9zZW4gYmFzZWQgb24gdGhlaXIgbGV2ZWxzIG9mIGV4cGVy dGlzZS9leHBlcmllbmNlIGluIE9QUywgbWVhc3VyZW1lbnQsIGFuZCBJRVRGLiBUaGFua3MgRGFu IGFuZCBBbCBmb3IgYWNjZXB0aW5nIHRoZSBjaGFpciBwb3NpdGlvbnMuPGJyPjxicj5Hb2FscyBm b3IgdGhpcyBCb0Y6PGJyPi0gYmUgaW4gc3luY2ggb24gdGhlIHByb2JsZW0gc3RhdGVtZW50IGFu ZCB1c2UgY2FzZXM8YnI+LSB1bmRlcnN0YW5kIHdoYXQgdGhlIGV4aXN0aW5nIElFVEYgcHJvdG9j b2xzIGFuZCBkYXRhIG1vZGVscyAoSVBQTSwgTkVUQ09ORi9ZQU5HLCBJUEZJWCwgZXRjLi4uKSBj YW4gb3IgY2FuJ3QgZG8gZm9yIExNQVAuIFRoaXMgd2lsbCBjZXJ0YWlubHkgcmVxdWlyZSBhIGxp dHRsZSBvZiBlZHVjYXRpb24gZGlyZWN0bHkgZnJvbSB0aGUgZXhwZXJ0cyBkdXJpbmcgdGhlIEJv Ri48YnI+LSBjbGFyaWZ5IHdoYXQncyBpbiBzY29wZSBvciBub3QgZm9yIExNQVA8YnI+PGJyPkZp bmFsbHksIGxldCBtZSBzdHJlc3MgdGhhdCBhIHByb3BlciBwcmVwYXJhdGlvbiBpcyBrZXkgdG8g YSBCb0Ygc3VjY2Vzcy4gPGJyPjxhIGhyZWY9Imh0dHA6Ly90b29scy5pZXRmLm9yZy9odG1sL3Jm YzU0MzQiPmh0dHA6Ly90b29scy5pZXRmLm9yZy9odG1sL3JmYzU0MzQ8L2E+Jm5ic3A7ICZxdW90 O0NvbnNpZGVyYXRpb25zIGZvciBIYXZpbmcgYSBTdWNjZXNzZnVsIEJpcmRzLW9mLWEtRmVhdGhl ciAoQk9GKSBTZXNzaW9uJnF1b3Q7IGlzIGEgZmlyc3QgZ29vZCBzdGVwLjxicj48YnI+UmVnYXJk cywgTWFydGluIFN0aWVtZXJsaW5nIChUU1YpIGFuZCBCZW5vaXQgQ2xhaXNlIChPUFMpPG86cD48 L286cD48L3NwYW4+PC9wPjwvZGl2PjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBsYW5nPUVOLVVT PjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD48L2Rpdj48L2Jsb2NrcXVvdGU+PC9kaXY+PC9i bG9ja3F1b3RlPjwvZGl2PjwvYm9keT48L2h0bWw+ --_000_9510D26531EF184D9017DF24659BB87F342ED975A8EMV65UKRDdoma_-- From philip.eardley@bt.com Tue Mar 5 07:20:31 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3287421F8994 for ; Tue, 5 Mar 2013 07:20:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.299 X-Spam-Level: X-Spam-Status: No, score=-101.299 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, MANGLED_AVOID=2.3, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1AV9t1afGNiD for ; Tue, 5 Mar 2013 07:20:30 -0800 (PST) Received: from smtpe1.intersmtp.com (smtp61.intersmtp.com [62.239.224.234]) by ietfa.amsl.com (Postfix) with ESMTP id 57A2921F81FE for ; Tue, 5 Mar 2013 07:20:30 -0800 (PST) Received: from EVMHT64-UKRD.domain1.systemhost.net (10.36.3.101) by RDW083A005ED61.smtp-e1.hygiene.service (10.187.98.10) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 5 Mar 2013 15:20:29 +0000 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.2.223]) by EVMHT64-UKRD.domain1.systemhost.net ([10.36.3.101]) with mapi; Tue, 5 Mar 2013 15:20:29 +0000 From: To: , Date: Tue, 5 Mar 2013 15:20:28 +0000 Thread-Topic: security considerations for LMAP Thread-Index: Ac4VGAgYeZY94tNfSW+edT4RUeQ0EwEmz7RQ Message-ID: <9510D26531EF184D9017DF24659BB87F342F5ED212@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA099307@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA099307@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] security considerations for LMAP X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 15:20:31 -0000 Dan, my take is that cases 1 & 3 below are beyond the proposed initial charter -= they're about what happens to the data after it's reached the Collector.=20 I agree with all the concerns you raise, about authorisation to access the = data etc. In principle an additional issue might be round requests from the= 3rd party to run a particular test (since this is about what happens befor= e the Controller builds the Schedule, hence again I think it's beyond the p= roposed initial charter) on case 2, I agree it's important for lmap to think about the privacy and s= ecurity implications - especially of Measurement Agents sitting in customer= premises. The current deployments are small scale and so can have an expli= cit agreement with the broadband consumers. is this viable for embedded lma= p capability in all edge & end devices? Operators are allowed to do stuff f= or normal network operation, I have no idea to what extent lmap would be co= vered by this (and how "stuff" varies by country). seems something prospect= ive wg needs to investigate. best wishes phil -----Original Message----- From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Rom= ascanu, Dan (Dan) Sent: 27 February 2013 18:27 To: lmap@ietf.org Subject: [lmap] security considerations for LMAP Hi, I would like to raise one issue which I believe needs more discussions and = clarifications as we prepare for the LMAP BOF and this is related to the se= curity aspects of the use cases and the security requirements that we shall= need to place on solutions. The use cases I-D (draft-linsner-lmap-use-case= s-02) has a void Security Considerations section, and so does draft-boucada= ir-lmap-considerations-00. This is fine at this point in time but obviously= can't stay like this. Only draft-eardley-lmap-framework-01 deals with part= of the security aspects but more from the perspective of protecting the me= asurement information and storage.=20 The use cases however seem to raise a few potential concerns related to the= information that is being accessed by different entities, who is authorize= d to collect and what in large-scale measurements. I can see three possible= interfaces that can raise such concerns:=20 1. The ISP and Multi-Provider use cases mention the need of one provider ha= ving visibility and access into the measurements of another provider - to d= ebug specific performance problems, to realize benchmarking and competitor = insight, etc. This can be problematic and needs to put in place proper mech= anisms of authorization, as different providers may be in different type of= relations, they would like to control what information is made available t= o competitors and none would accept full visibility of its own internal dat= a 2. The ISP-customer use case: Understanding the quality experienced by cu= stomers requires deployment of either software or in some scenarios hardwar= e probes at the customers sites or on the edge. This raises similar problem= s of visibility as in the previous phase, as well as issues related to priv= acy protection which may be subject to local or regional (for example Europ= ean) laws and regulations on privacy.=20 3. The regulator case: Having regulators gather, store and process broadban= d traffic information in domains owned by ISPs raises also issues of author= ization of access to such information. Hopefully development and enforcemen= t of broadband policies will become part of the regulation and legislation = and these details will be worked eventually out, but until they do and even= afterwards proper mechanisms need to be in place in order to control autho= rization and protect information of the different levels of providers.=20 Thoughts? Opinions?=20 Regards, Dan =20 _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap From philip.eardley@bt.com Tue Mar 5 08:16:54 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8AC0E21F89E9 for ; Tue, 5 Mar 2013 08:16:54 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.448 X-Spam-Level: X-Spam-Status: No, score=-102.448 tagged_above=-999 required=5 tests=[AWL=1.149, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x6DvYJKJh8W4 for ; Tue, 5 Mar 2013 08:16:52 -0800 (PST) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 9FEF621F89DC for ; Tue, 5 Mar 2013 08:16:31 -0800 (PST) Received: from EVMHT65-UKRD.domain1.systemhost.net (10.36.3.102) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.279.1; Tue, 5 Mar 2013 16:16:30 +0000 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.2.223]) by EVMHT65-UKRD.domain1.systemhost.net ([10.36.3.102]) with mapi; Tue, 5 Mar 2013 16:16:30 +0000 From: To: Date: Tue, 5 Mar 2013 16:16:29 +0000 Thread-Topic: "Solution candidates" Thread-Index: Ac4ZAWeNhLjOfHUJSBuV0fts4Y9ETQAuvQfA Message-ID: <9510D26531EF184D9017DF24659BB87F342F5ED2A5@EMV65-UKRD.domain1.systemhost.net> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F342F5ED2A5EMV65UKRDdoma_" MIME-Version: 1.0 Subject: [lmap] "Solution candidates" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 16:16:54 -0000 --_000_9510D26531EF184D9017DF24659BB87F342F5ED2A5EMV65UKRDdoma_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi the LMAP agenda talks about Solution Candidates 6a. IPFIX - Brian Trammell - 10+ min 6b. NETCONF/YANG - Juergen Schoenwaelder - 10+ min 6c. ALTO - Jan Seedorf - 5+ min I wanted to start some discussion about what people think "candidate" means= , how many candidates the IETF would standardise (assuming the IETF agrees = to do work, so the answer is not 'none'), etc. I think it's out of scope of the BOF to mandate one of the candidates, but = I think it's worth discussing whether the prospective WG would be working o= n:- (1) one and only one, (2) one but enabling others to done later or else= where, (3) several. There are 3 levels we might standardise:- * Information model: the abstract definition of the Test /Report Sc= hedule (which is how the Controller instructs the Measurement Agent); and t= he abstract definition of the Report (which is how the Measurement Agent re= ports results to the Collector) * Data model: which instantiates the information model in a particu= lar language. It could be done using an existing IETF data modeling langua= ge, for example JSON or YANG or (for the Report) IPFIX. Or it could be done= using a non-IETF standard like XML. * Protocol: how to deliver the Test /Report Schedule and the Report= . Possibilities using IETF protocols could include NETCONF [RFC6241] as di= scussed in [lmap-netconf] or a RESTful interface or (for the Report) IPFIX.= Or a non-IETF protocol, for example from the Broadband Forum some initial thoughts: * Information model: this seems important to standardise. So that t= here is a single understanding of the contents and structure of a Schedule,= and of a Report. We'd like non-IETF bodies also to refer this Information = model (to help ensure compatibility - for instance so that results are comp= arable between measurement systems from different vendors and perhaps from = different bodies). So we should be standardising 'one and only one'. * Data model: pragmatically it would seem best to choose one, to av= oid potential incompatibilites between several data models and to reduce th= e WG's effort. In theory we could choose a different data model language fo= r the Schedule and the Report, although this may involve extra effort (for = example, it's been suggested that the Report should include essentially an = 'echo' of the Test Schedule) * Protocol: it would be nice to allow measurement platforms to use = different protocols. There could be a wide diversity of Measurement Agents = - for example, home gateways, routers in the ISP's network, set top boxes, = tablets - so there probably needs to be a variety of protocols. For example= , a DSL Home Gateway device might use a Broadband Forum TR69-based transpor= t; an edge device might implement an IETF protocol. My tentative view is th= at the IETF should do one (or one for the Controller-MA and one for the MA-= Collector, if it makes sense for them to be different); more than one might= be OK if there's enough interest to do the extra work; and potentially oth= er standards bodies would work on their own protocol. thoughts? thanks phil --_000_9510D26531EF184D9017DF24659BB87F342F5ED2A5EMV65UKRDdoma_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi

the LMAP agenda talks about Solution Candidates =

       &n= bsp;        6a. IPFIX - Brian Trammell -= 10+ min

     &= nbsp;          6b. NETCONF/YAN= G - Juergen Schoenwaelder - 10+ min

&nbs= p;            &= nbsp;  6c. ALTO - Jan Seedorf - 5+ min

 

I wanted to start some discu= ssion about what people think “candidate” means, how many candi= dates the IETF would standardise (assuming the IETF agrees to do work, so t= he answer is not ‘none’), etc.

 

I think it’s out of scop= e of the BOF to mandate one of the candidates, but I think it’s worth= discussing whether the prospective WG would be working on:- (1) one and on= ly one, (2) one but enabling others to done later or elsewhere, (3) several= .

 

There are 3 levels we might standardise:-

·   = ;      Information = model: the abstract definition of the Test /Report Schedule (which is how t= he Controller instructs the Measurement Agent); and the abstract definition= of the Report (which is how the Measurement Agent reports results to the C= ollector)

·         Data model: which instantiates the information m= odel in a particular language.  It could be done using an existing IET= F data modeling language, for example JSON or YANG or (for the Report) IPFI= X. Or it could be done using a non-IETF standard like XML.

&middo= t;    &nbs= p;    Protocol: how to delive= r the Test /Report Schedule and the Report.  Possibilities using IETF = protocols could include NETCONF [RFC6241] as discussed in [lmap-netconf] or= a RESTful interface or (for the Report) IPFIX. Or a non-IETF protocol, for= example from the Broadband Forum

some initial though= ts:

·         Information model: this seems important to standardise= . So that there is a single understanding of the contents and structure of = a Schedule, and of a Report. We’d like non-IETF bodies also to refer = this Information model (to help ensure compatibility - for instance so that= results are comparable between measurement systems from different vendors = and perhaps from different bodies). So we should be standardising ‘on= e and only one’.

·   &n= bsp;     Data model: pra= gmatically it would seem best to choose one, to avoid potential incompatibi= lites between several data models and to reduce the WG’s effort. In t= heory we could choose a different data model language for the Schedule and = the Report, although this may involve extra effort (for example, it’s= been suggested that the Report should include essentially an ‘echo&#= 8217; of the Test Schedule)

·  &nb= sp;      Protocol: = it would be nice to allow measurement platforms to use different protocols.= There could be a wide diversity of Measurement Agents – for example,= home gateways, routers in the ISP’s network, set top boxes, tablets = – so there probably needs to be a variety of protocols. For example, = a DSL Home Gateway device might use a Broadband Forum TR69-based transport;= an edge device might implement an IETF protocol. My tentative view is that= the IETF should do one (or one for the Controller-MA and one for the MA-Co= llector, if it makes sense for them to be different); more than one might b= e OK if there’s enough interest to do the extra work; and potentially= other standards bodies would work on their own protocol.

 

thoughts?

thanks

phil

 

 = ;

= --_000_9510D26531EF184D9017DF24659BB87F342F5ED2A5EMV65UKRDdoma_-- From acmorton@att.com Tue Mar 5 13:45:03 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E41E21F85C6 for ; Tue, 5 Mar 2013 13:45:03 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.599 X-Spam-Level: X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7E0ihs2S3b8v for ; Tue, 5 Mar 2013 13:45:00 -0800 (PST) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 0FF1121F85EB for ; Tue, 5 Mar 2013 13:45:00 -0800 (PST) Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id 20372120302; Tue, 5 Mar 2013 16:47:46 -0500 (EST) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-blue.research.att.com (Postfix) with ESMTP id 84771F00EC; Tue, 5 Mar 2013 16:44:59 -0500 (EST) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Tue, 5 Mar 2013 16:44:59 -0500 From: "MORTON JR., ALFRED (AL)" To: "philip.eardley@bt.com" , "lmap@ietf.org" Date: Tue, 5 Mar 2013 16:44:57 -0500 Thread-Topic: "Solution candidates" Thread-Index: Ac4ZAWeNhLjOfHUJSBuV0fts4Y9ETQAuvQfAAAp3/CA= Message-ID: References: <9510D26531EF184D9017DF24659BB87F342F5ED2A5@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F342F5ED2A5@EMV65-UKRD.domain1.systemhost.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_F1312FAF1A1E624DA0972D1C9A91379A1BF8990406njfpsrvexg7re_" MIME-Version: 1.0 Subject: Re: [lmap] "Solution candidates" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 21:45:03 -0000 --_000_F1312FAF1A1E624DA0972D1C9A91379A1BF8990406njfpsrvexg7re_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Phil, Your considerations on candidates all sound strikingly realistic to me, thanks for sharing them. Allow me to add a consideration that I've been mulling for some time: When we look at candidates for any of the 3 levels below, we need to ask the most obvious question (prompted by the title of the BoF): Will this model or protocol be suitable for Large-Scale use? Given that we will define the information elements ourselves, we should be able to avoid number-range limits and similar "gotchas" from the start. There may need to be performance metrics on the models and protocols so we can easily see the trade-offs. There are other factors sometimes inferred from "complexity". Some of the scale-aspects will be organization/operation-related. Can the Controller-Collector-MA relationships easily be expanded or modifie= d or modularized as the measurement system grows? I imagine there is valuable IETF experience to draw-on here. regards, Al From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of phi= lip.eardley@bt.com Sent: Tuesday, March 05, 2013 11:16 AM To: lmap@ietf.org Subject: [lmap] "Solution candidates" Hi the LMAP agenda talks about Solution Candidates 6a. IPFIX - Brian Trammell - 10+ min 6b. NETCONF/YANG - Juergen Schoenwaelder - 10+ min 6c. ALTO - Jan Seedorf - 5+ min I wanted to start some discussion about what people think "candidate" means= , how many candidates the IETF would standardise (assuming the IETF agrees = to do work, so the answer is not 'none'), etc. I think it's out of scope of the BOF to mandate one of the candidates, but = I think it's worth discussing whether the prospective WG would be working o= n:- (1) one and only one, (2) one but enabling others to done later or else= where, (3) several. There are 3 levels we might standardise:- * Information model: the abstract definition of the Test /Report Sc= hedule (which is how the Controller instructs the Measurement Agent); and t= he abstract definition of the Report (which is how the Measurement Agent re= ports results to the Collector) * Data model: which instantiates the information model in a particu= lar language. It could be done using an existing IETF data modeling langua= ge, for example JSON or YANG or (for the Report) IPFIX. Or it could be done= using a non-IETF standard like XML. * Protocol: how to deliver the Test /Report Schedule and the Report= . Possibilities using IETF protocols could include NETCONF [RFC6241] as di= scussed in [lmap-netconf] or a RESTful interface or (for the Report) IPFIX.= Or a non-IETF protocol, for example from the Broadband Forum some initial thoughts: * Information model: this seems important to standardise. So that t= here is a single understanding of the contents and structure of a Schedule,= and of a Report. We'd like non-IETF bodies also to refer this Information = model (to help ensure compatibility - for instance so that results are comp= arable between measurement systems from different vendors and perhaps from = different bodies). So we should be standardising 'one and only one'. * Data model: pragmatically it would seem best to choose one, to av= oid potential incompatibilites between several data models and to reduce th= e WG's effort. In theory we could choose a different data model language fo= r the Schedule and the Report, although this may involve extra effort (for = example, it's been suggested that the Report should include essentially an = 'echo' of the Test Schedule) * Protocol: it would be nice to allow measurement platforms to use = different protocols. There could be a wide diversity of Measurement Agents = - for example, home gateways, routers in the ISP's network, set top boxes, = tablets - so there probably needs to be a variety of protocols. For example= , a DSL Home Gateway device might use a Broadband Forum TR69-based transpor= t; an edge device might implement an IETF protocol. My tentative view is th= at the IETF should do one (or one for the Controller-MA and one for the MA-= Collector, if it makes sense for them to be different); more than one might= be OK if there's enough interest to do the extra work; and potentially oth= er standards bodies would work on their own protocol. thoughts? thanks phil --_000_F1312FAF1A1E624DA0972D1C9A91379A1BF8990406njfpsrvexg7re_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Phil,

Your considerations on candidates all s= ound strikingly realistic to me,

= thanks for shari= ng them.  Allow me to add a consideration that I've been

mulling for some time:

 <= /o:p>

When we look at candidates for any of the 3 levels bel= ow, we need to

ask the most obvious question (pro= mpted by the title of the BoF):

<= span style=3D'font-size:10.0pt;font-family:"Courier New"'>Will this model o= r protocol be suitable for Large-Scale use?

 

Given that we will define the information ele= ments ourselves, we should

be able to avoid numbe= r-range limits and similar "gotchas" from the start.

 

There may need to be perfor= mance metrics on the models and protocols

so we c= an easily see the trade-offs. There are other factors sometimes<= /span>

inferred from "complexity".

<= p class=3DMsoNormal> 

Some of the scale-aspects will be orga= nization/operation-related.

Can the Controller-Co= llector-MA relationships easily be expanded or modified

or modularized as the measurement system grows?

 

I imagine there is valuable IETF exp= erience to draw-on here.

 =

regards,

Al

 

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ie= tf.org] On Behalf Of philip.eardley@bt.com
Sent: Tuesday, = March 05, 2013 11:16 AM
To: lmap@ietf.org
Subject: [lma= p] "Solution candidates"

 

= Hi

the LMAP ag= enda talks about Solution Candidates

        &nb= sp;       6a. IPFIX - Brian Trammell - 10+ mi= n

  =             &nb= sp; 6b. NETCONF/YANG - Juergen Schoenwaelder - 10+ min

     &nbs= p;          6c. ALTO - Jan See= dorf - 5+ min

 

I want= ed to start some discussion about what people think “candidate”= means, how many candidates the IETF would standardise (assuming the IETF a= grees to do work, so the answer is not ‘none’), etc.=

 =

I think it’s out of scope= of the BOF to mandate one of the candidates, but I think it’s worth = discussing whether the prospective WG would be working on:- (1) one and onl= y one, (2) one but enabling others to done later or elsewhere, (3) several.=

 

There are 3 levels w= e might standardise:-

= ·   &nbs= p;     Information model: the abstract definition of the Test /Report Schedule (= which is how the Controller instructs the Measurement Agent); and the abstr= act definition of the Report (which is how the Measurement Agent reports re= sults to the Collector)

·   &n= bsp;     Data model: which instantiates the information model in a particular la= nguage.  It could be done using an existing IETF data modeling languag= e, for example JSON or YANG or (for the Report) IPFIX. Or it could be done = using a non-IETF standard like XML.

·     &n= bsp;   Protocol= : how to deliver the Test /Report Schedule and the Report.  Possibilit= ies using IETF protocols could include NETCONF [RFC6241] as discussed in [l= map-netconf] or a RESTful interface or (for the Report) IPFIX. Or a non-IET= F protocol, for example from the Broadband Forum

some initial thoughts:

·= ·         <= ![endif]>Data model: pragmatically it would seem best to= choose one, to avoid potential incompatibilites between several data model= s and to reduce the WG’s effort. In theory we could choose a differen= t data model language for the Schedule and the Report, although this may in= volve extra effort (for example, it’s been suggested that the Report = should include essentially an ‘echo’ of the Test Schedule)=

&midd= ot;    &nb= sp;    Pro= tocol: it would be nice to allow measurement platforms to use different pro= tocols. There could be a wide diversity of Measurement Agents – for e= xample, home gateways, routers in the ISP’s network, set top boxes, t= ablets – so there probably needs to be a variety of protocols. For ex= ample, a DSL Home Gateway device might use a Broadband Forum TR69-based tra= nsport; an edge device might implement an IETF protocol. My tentative view = is that the IETF should do one (or one for the Controller-MA and one for th= e MA-Collector, if it makes sense for them to be different); more than one = might be OK if there’s enough interest to do the extra work; and pote= ntially other standards bodies would work on their own protocol.

&n= bsp;

= thoughts?

thanks

phil

 

 

= --_000_F1312FAF1A1E624DA0972D1C9A91379A1BF8990406njfpsrvexg7re_-- From dromasca@avaya.com Wed Mar 6 04:16:27 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 548B721F87C4 for ; Wed, 6 Mar 2013 04:16:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.432 X-Spam-Level: X-Spam-Status: No, score=-103.432 tagged_above=-999 required=5 tests=[AWL=0.166, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mp3bE2KcG1px for ; Wed, 6 Mar 2013 04:16:24 -0800 (PST) Received: from co300216-co-outbound.net.avaya.com (co300216-co-outbound.net.avaya.com [198.152.13.100]) by ietfa.amsl.com (Postfix) with ESMTP id C1EE821F87B1 for ; Wed, 6 Mar 2013 04:16:23 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgMFAHpXMFGHCzI1/2dsb2JhbABEgkMjv05/FnOCHwEBAQEDEhtFFwIBCA0EAQMBAQsWBwcyFAMGCAEBBAESCBMHh3EBpGGcQ41CC4EWJhEBgl9hA5xailGDCIFpPg X-IronPort-AV: E=Sophos;i="4.84,760,1355115600"; d="scan'208,217";a="744835" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by co300216-co-outbound.net.avaya.com with ESMTP; 06 Mar 2013 07:16:22 -0500 Received: from unknown (HELO AZ-FFEXHC02.global.avaya.com) ([135.64.58.12]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 06 Mar 2013 07:14:57 -0500 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC02.global.avaya.com ([135.64.58.12]) with mapi id 14.02.0328.009; Wed, 6 Mar 2013 07:16:21 -0500 From: "Romascanu, Dan (Dan)" To: "MORTON JR., ALFRED (AL)" , "philip.eardley@bt.com" , "lmap@ietf.org" Thread-Topic: "Solution candidates" Thread-Index: Ac4ZAWeNhLjOfHUJSBuV0fts4Y9ETQAuvQfAAAp3/CAAH2xksA== Date: Wed, 6 Mar 2013 12:16:20 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA0A0FDE@AZ-FFEXMB04.global.avaya.com> References: <9510D26531EF184D9017DF24659BB87F342F5ED2A5@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: multipart/alternative; boundary="_000_9904FB1B0159DA42B0B887B7FA8119CA0A0FDEAZFFEXMB04globala_" MIME-Version: 1.0 Subject: Re: [lmap] "Solution candidates" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 12:16:27 -0000 --_000_9904FB1B0159DA42B0B887B7FA8119CA0A0FDEAZFFEXMB04globala_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, The architecture hinted by Phil's 'levels' has the advantage of modularity.= One information model can generate several data models using different DML= s. A DML does not necessarily work with one protocol only also there are so= me pairing preferences (SNMP/SMI, NETCONF/YANG, IPFIX/IEs). We may reach sooner than later the conclusion that we need to prioritize - = i.e start from one or a limited range of use cases, define, implement, depl= oy, and learn the lessons. Then go to the next set of use cases. And so on = ... Regards, Dan From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of MOR= TON JR., ALFRED (AL) Sent: Tuesday, March 05, 2013 11:45 PM To: philip.eardley@bt.com; lmap@ietf.org Subject: Re: [lmap] "Solution candidates" Hi Phil, Your considerations on candidates all sound strikingly realistic to me, thanks for sharing them. Allow me to add a consideration that I've been mulling for some time: When we look at candidates for any of the 3 levels below, we need to ask the most obvious question (prompted by the title of the BoF): Will this model or protocol be suitable for Large-Scale use? Given that we will define the information elements ourselves, we should be able to avoid number-range limits and similar "gotchas" from the start. There may need to be performance metrics on the models and protocols so we can easily see the trade-offs. There are other factors sometimes inferred from "complexity". Some of the scale-aspects will be organization/operation-related. Can the Controller-Collector-MA relationships easily be expanded or modifie= d or modularized as the measurement system grows? I imagine there is valuable IETF experience to draw-on here. regards, Al From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of philip.eardley@bt.com Sent: Tuesday, March 05, 2013 11:16 AM To: lmap@ietf.org Subject: [lmap] "Solution candidates" Hi the LMAP agenda talks about Solution Candidates 6a. IPFIX - Brian Trammell - 10+ min 6b. NETCONF/YANG - Juergen Schoenwaelder - 10+ min 6c. ALTO - Jan Seedorf - 5+ min I wanted to start some discussion about what people think "candidate" means= , how many candidates the IETF would standardise (assuming the IETF agrees = to do work, so the answer is not 'none'), etc. I think it's out of scope of the BOF to mandate one of the candidates, but = I think it's worth discussing whether the prospective WG would be working o= n:- (1) one and only one, (2) one but enabling others to done later or else= where, (3) several. There are 3 levels we might standardise:- * Information model: the abstract definition of the Test /Report Sc= hedule (which is how the Controller instructs the Measurement Agent); and t= he abstract definition of the Report (which is how the Measurement Agent re= ports results to the Collector) * Data model: which instantiates the information model in a particu= lar language. It could be done using an existing IETF data modeling langua= ge, for example JSON or YANG or (for the Report) IPFIX. Or it could be done= using a non-IETF standard like XML. * Protocol: how to deliver the Test /Report Schedule and the Report= . Possibilities using IETF protocols could include NETCONF [RFC6241] as di= scussed in [lmap-netconf] or a RESTful interface or (for the Report) IPFIX.= Or a non-IETF protocol, for example from the Broadband Forum some initial thoughts: * Information model: this seems important to standardise. So that t= here is a single understanding of the contents and structure of a Schedule,= and of a Report. We'd like non-IETF bodies also to refer this Information = model (to help ensure compatibility - for instance so that results are comp= arable between measurement systems from different vendors and perhaps from = different bodies). So we should be standardising 'one and only one'. * Data model: pragmatically it would seem best to choose one, to av= oid potential incompatibilites between several data models and to reduce th= e WG's effort. In theory we could choose a different data model language fo= r the Schedule and the Report, although this may involve extra effort (for = example, it's been suggested that the Report should include essentially an = 'echo' of the Test Schedule) * Protocol: it would be nice to allow measurement platforms to use = different protocols. There could be a wide diversity of Measurement Agents = - for example, home gateways, routers in the ISP's network, set top boxes, = tablets - so there probably needs to be a variety of protocols. For example= , a DSL Home Gateway device might use a Broadband Forum TR69-based transpor= t; an edge device might implement an IETF protocol. My tentative view is th= at the IETF should do one (or one for the Controller-MA and one for the MA-= Collector, if it makes sense for them to be different); more than one might= be OK if there's enough interest to do the extra work; and potentially oth= er standards bodies would work on their own protocol. thoughts? thanks phil --_000_9904FB1B0159DA42B0B887B7FA8119CA0A0FDEAZFFEXMB04globala_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,<= /p>

 

The architecture hinte= d by Phil’s ‘levels’ has the advantage of modularity. One= information model can generate several data models using different DMLs. A= DML does not necessarily work with one protocol only also there are some pairing preferences (SNMP/SMI, NETCONF/YANG, IPFIX/IEs= ).

 

We may reach sooner th= an later the conclusion that we need to prioritize – i.e start from o= ne or a limited range of use cases, define, implement, deploy, and learn th= e lessons. Then go to the next set of use cases. And so on …

 

 

Regards,

 

Dan<= /p>

 

 

 

From: lmap-bou= nces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of MORTON JR., ALFRED (AL)
Sent: Tuesday, March 05, 2013 11:45 PM
To: philip.eardley@bt.com; lmap@ietf.org
Subject: Re: [lmap] "Solution candidates"

 

Hi Phil,

 

Your considerations on candidates all sound strikingly rea= listic to me,

thanks for sharing them.  Allow me to add a considera= tion that I've been

mulling for some time:

 

When we look at candidates for any of the 3 levels below, = we need to

ask the most obvious question (prompted by the title of th= e BoF):

Will this model or protocol be suitable for Large-Scale us= e?

 

Given that we will define the information elements ourselv= es, we should

be able to avoid number-range limits and similar "got= chas" from the start.

 

There may need to be performance metrics on the models and= protocols

so we can easily see the trade-offs. There are other facto= rs sometimes

inferred from "complexity".

 

Some of the scale-aspects will be organization/operation-r= elated.

Can the Controller-Collector-MA relationships easily be ex= panded or modified

or modularized as the measurement system grows?=

 

I imagine there is valuable IETF experience to draw-on her= e.

 

regards,

Al

 

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of philip.eardley= @bt.com
Sent: Tuesday, March 05, 2013 11:16 AM
To: lmap@ietf.org
Subject: [lmap] "Solution candidates"

 

Hi

the LMAP agenda talks about Sol= ution Candidates

     &= nbsp;          6a. IPFIX - Bri= an Trammell - 10+ min

     &= nbsp;          6b. NETCONF/YAN= G - Juergen Schoenwaelder - 10+ min

     &= nbsp;          6c. ALTO - Jan = Seedorf - 5+ min

 

I wanted to start some discussi= on about what people think “candidate” means, how many candidat= es the IETF would standardise (assuming the IETF agrees to do work, so the = answer is not ‘none’), etc.

 

I think it’s out of scope= of the BOF to mandate one of the candidates, but I think it’s worth = discussing whether the prospective WG would be working on:- (1) one and onl= y one, (2) one but enabling others to done later or elsewhere, (3) several.

 

There are 3 levels we might sta= ndardise:-

·         Information model: the abstract definition of the Test /Report Schedule (= which is how the Controller instructs the Measurement Agent); and the abstr= act definition of the Report (which is how the Measurement Agent reports results to the Collector)<= /span>

·         Data model: which instantiates the information model in a particular lang= uage.  It could be done using an existing IETF data modeling language,= for example JSON or YANG or (for the Report) IPFIX. Or it could be done using a non-IETF standard like XML.<= /span>

·         Protocol: how to deliver the Test /Report Schedule and the Report.  = Possibilities using IETF protocols could include NETCONF [RFC6241] as discu= ssed in [lmap-netconf] or a RESTful interface or (for the Report) IPFIX. Or a non-IETF protocol, for example from the Br= oadband Forum

some initial thoughts:

·         Information model: this seems important to standardise. So that there is = a single understanding of the contents and structure of a Schedule, and of = a Report. We’d like non-IETF bodies also to refer this Information model (to help ensure compatibility - for i= nstance so that results are comparable between measurement systems from dif= ferent vendors and perhaps from different bodies). So we should be standard= ising ‘one and only one’.

·         Data model: pragmatically it would seem best to choose one, to avoid pote= ntial incompatibilites between several data models and to reduce the WGR= 17;s effort. In theory we could choose a different data model language for the Schedule and the Report, although this may inv= olve extra effort (for example, it’s been suggested that the Report s= hould include essentially an ‘echo’ of the Test Schedule)<= /o:p>

·         Protocol: it would be nice to allow measurement platforms to use differen= t protocols. There could be a wide diversity of Measurement Agents – = for example, home gateways, routers in the ISP’s network, set top boxes, tablets – so there probably need= s to be a variety of protocols. For example, a DSL Home Gateway device migh= t use a Broadband Forum TR69-based transport; an edge device might implemen= t an IETF protocol. My tentative view is that the IETF should do one (or one for the Controller-MA and one for the MA-Co= llector, if it makes sense for them to be different); more than one might b= e OK if there’s enough interest to do the extra work; and potentially= other standards bodies would work on their own protocol.

 

thoughts?

thanks

phil

 

 

--_000_9904FB1B0159DA42B0B887B7FA8119CA0A0FDEAZFFEXMB04globala_-- From dromasca@avaya.com Thu Mar 7 02:25:20 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A825421F8AB2 for ; Thu, 7 Mar 2013 02:25:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.956 X-Spam-Level: X-Spam-Status: No, score=-102.956 tagged_above=-999 required=5 tests=[AWL=-0.357, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qiqQLzkGLRH5 for ; Thu, 7 Mar 2013 02:25:20 -0800 (PST) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id 36FC421F8AD8 for ; Thu, 7 Mar 2013 02:25:20 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgEFALh+MVGHCzI1/2dsb2JhbABEgma/XoEAFnOCIQEBAxIoUQEVFRRCHwcBBBsah3EBn32EKpxDjmyDF2EDkwKJWopSgwiCJw X-IronPort-AV: E=Sophos;i="4.84,766,1355115600"; d="scan'208";a="777249" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 07 Mar 2013 05:25:19 -0500 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 07 Mar 2013 05:23:52 -0500 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Thu, 7 Mar 2013 05:25:18 -0500 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: presentations and minutes Thread-Index: Ac4bHhAuFBRwbVvzRCWWuy0AAQNE7A== Date: Thu, 7 Mar 2013 10:25:18 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA0A2505@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] presentations and minutes X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 10:25:20 -0000 Hi, A couple of requests for the upcoming BOF session in Orlando: - Participants who own agenda items and wish to make presentations must the= ir presentations in advance in order to allow time to the chairs to review = them and for uploading and making them available for remote participants. P= lease send these until the evening prior to the meetings the latest.=20 - We need at least two note takers and one jabber scribe in the room. We ca= nnot start the meeting without them. Please volunteer for these important p= ositions in advance, the chairs will publicly recognize your contributions. Thanks and Regards, Al and Dan From bclaise@cisco.com Thu Mar 7 02:47:49 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F1E6A21F8D23 for ; Thu, 7 Mar 2013 02:47:49 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.682 X-Spam-Level: X-Spam-Status: No, score=-10.682 tagged_above=-999 required=5 tests=[AWL=-0.084, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HcThx-SZJmBZ for ; Thu, 7 Mar 2013 02:47:49 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 2FEE621F8D27 for ; Thu, 7 Mar 2013 02:47:49 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r27AljWO027026 for ; Thu, 7 Mar 2013 11:47:45 +0100 (CET) Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r27AlG8j018497 for ; Thu, 7 Mar 2013 11:47:26 +0100 (CET) Message-ID: <51387034.2040708@cisco.com> Date: Thu, 07 Mar 2013 11:47:16 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: "lmap@ietf.org" Content-Type: multipart/alternative; boundary="------------010706030604080108040601" Subject: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 10:47:50 -0000 This is a multi-part message in MIME format. --------------010706030604080108040601 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear all, I started to review the drafts, and I will start posting a few questions to the list. Open questions, clarifying questions, in order to generate some discussions. Disclaimer: I have not yet read the entire list archive. Apologize in advance if some points have been discussed already. Here is my first question. What is broadband in the LMAP context? Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology independent? And I see also "enterprise edge router", "cellular data or satellite" in draft-schulzrinne-lmap-requirements. In or out? Or do we have in mind a phase approach, starting with the "enterprise edge router" first, and then home network? Interestingly, I don't know what A stands for in LMAP, if it stands for something. According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand for anything. However, looking at the different draft titles, there is some confusion. draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measurement Use Cases draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurement of Broadband Performance draft-boucadair-lmap-considerations-00,_Large scale Measurement of__Access__network Performance (LMAP)_: Requirements and Issues from a Network Provider Perspective Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated. Regards, Benoit --------------010706030604080108040601 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear all,

I started to review the drafts, and I will start posting a few questions to the list.
Open questions, clarifying questions, in order to generate some discussions.
Disclaimer: I have not yet read the entire list archive. Apologize in advance if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology independent?
And I see also "enterprise edge router", "cellular data or satellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise edge router" first, and then home network?

Interestingly, I don't know what A stands for in LMAP, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand for anything.
However, looking at the different draft titles, there is some confusion.
   draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measurement Use Cases
   draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurement of Broadband Performance
   draft-boucadair-lmap-considerations-00, Large scale Measurement of Access network Performance (LMAP):
      Requirements and Issues from a Network Provider Perspective

Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated.

Regards, Benoit



--------------010706030604080108040601-- From bclaise@cisco.com Thu Mar 7 03:09:16 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6F2F021F8D12 for ; Thu, 7 Mar 2013 03:09:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.676 X-Spam-Level: X-Spam-Status: No, score=-10.676 tagged_above=-999 required=5 tests=[AWL=-0.077, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XxgyylRJl2U0 for ; Thu, 7 Mar 2013 03:09:16 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id B40FE21F8CD0 for ; Thu, 7 Mar 2013 03:09:15 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r27B5UnW029234 for ; Thu, 7 Mar 2013 12:05:30 +0100 (CET) Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r27B502d006013 for ; Thu, 7 Mar 2013 12:05:10 +0100 (CET) Message-ID: <5138745B.5020501@cisco.com> Date: Thu, 07 Mar 2013 12:04:59 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: "lmap@ietf.org" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [lmap] LMAP: Voluntary or compulsory? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 11:09:16 -0000 Dear all, Another clarifying question: are those LMAP measurements based on voluntary participation or not? Or maybe it depends on the use cases described in draft-linsner-lmap-use-cases-02 2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . 3 2.2 End User Network Diagnostics . . . . . . . . . . . . . . . . 4 2.3 Multi-provider Network Measurements . . . . . . . . . . . . 4 2.4 Over the Top Providers . . . . . . . . . . . . . . . . . . . 4 2.5 Regulators . . . . . . . . . . . . . . . . . . . . . . . . . 5 Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated. Regards, Benoit From bclaise@cisco.com Thu Mar 7 03:13:32 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1E43421F8D3A for ; Thu, 7 Mar 2013 03:13:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.671 X-Spam-Level: X-Spam-Status: No, score=-10.671 tagged_above=-999 required=5 tests=[AWL=-0.072, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XMeN6+v9nNZ0 for ; Thu, 7 Mar 2013 03:13:31 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 540AA21F8D30 for ; Thu, 7 Mar 2013 03:13:31 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r27BDTcF000063 for ; Thu, 7 Mar 2013 12:13:29 +0100 (CET) Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r27BCxNS012698 for ; Thu, 7 Mar 2013 12:13:10 +0100 (CET) Message-ID: <5138763B.1000100@cisco.com> Date: Thu, 07 Mar 2013 12:12:59 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: "lmap@ietf.org" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [lmap] Which devices does LMAP target? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 11:13:32 -0000 Dear all, draft-schulzrinne-lmap-requirements-00 mentions: Thus, this document proposes an overall architecture, with emphasis on the functional and security requirements for the protocols connecting the elements of the architecture, that will make it possible to build measurement capabilities into home and enterprise edge routers, personal computers, mobile devices and other edge devices. Does LMAP really target my home devices? My set top box, PC, tablet, smartphone Or is LMAP is limited to the modem/router? And if this is only the modem/router, do we want to make a difference between an ISP managed device (typically a branch office) and my own private device? Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated. Regards, Benoit From bclaise@cisco.com Thu Mar 7 03:28:07 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E5FB21F8CFB for ; Thu, 7 Mar 2013 03:28:07 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.666 X-Spam-Level: X-Spam-Status: No, score=-10.666 tagged_above=-999 required=5 tests=[AWL=-0.068, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3Kgp33G8pZ1N for ; Thu, 7 Mar 2013 03:28:06 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 62A2421F8CD8 for ; Thu, 7 Mar 2013 03:28:06 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r27BS5YC001624 for ; Thu, 7 Mar 2013 12:28:05 +0100 (CET) Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r27BRnk6025427 for ; Thu, 7 Mar 2013 12:27:59 +0100 (CET) Message-ID: <513879B5.8030609@cisco.com> Date: Thu, 07 Mar 2013 12:27:49 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: "lmap@ietf.org" Content-Type: multipart/alternative; boundary="------------040607050609070803020001" Subject: [lmap] Which performance metrics for LMAP? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 11:28:07 -0000 This is a multi-part message in MIME format. --------------040607050609070803020001 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear all, Another clarifying question: I'm not too sure which performance metric types LMAP targets. draft-linsner-lmap-use-cases-00 mentions: It is assumed that measurement tests run by OTT providers would only include the metrics associated with layer 3 and up The different drafts use different terms: service, end user experience, quality of experience. Does Quality of Experience imply performance metrics per application, per 5 tuple (IP addresses, ports, protocol), per DCSP? Or do we want to stay at layer 3 performance metrics, typically IPPM type of metrics? If the latter, then LMAP is about large scale deployment of IPPM, right? While I fully understand the business needs for "and up" in "include the metrics associated with layer 3 and up", let's not boil the ocean. Performance Metrics at Other Layer (PMOL ), a concluded WG, tried and it proved to be difficult. Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated. Regards, Benoit --------------040607050609070803020001 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear all,

Another clarifying question: I'm not too sure which performance metric types LMAP targets.
draft-linsner-lmap-use-cases-00 mentions:
   It is assumed that measurement
   tests run by OTT providers would only include the metrics associated
   with layer 3 and up

The different drafts use different terms: service, end user experience, quality of experience.
Does Quality of Experience imply performance metrics per application, per 5 tuple (IP addresses, ports, protocol), per DCSP?
Or do we want to stay at layer 3 performance metrics, typically IPPM type of metrics?
If the latter, then LMAP is about large scale deployment of IPPM, right?

While I fully understand the business needs for "and up" in "include the metrics associated with layer 3 and up", let's not boil the ocean.
Performance Metrics at Other Layer (PMOL), a concluded WG, tried and it proved to be difficult.

Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated.

Regards, Benoit
--------------040607050609070803020001-- From shane@castlepoint.net Thu Mar 7 07:41:13 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A73D121F8D27 for ; Thu, 7 Mar 2013 07:41:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.436 X-Spam-Level: X-Spam-Status: No, score=-0.436 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FH_RELAY_NODNS=1.451, HELO_MISMATCH_ORG=0.611, HTML_MESSAGE=0.001, RDNS_NONE=0.1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Js-CdPeuRGfM for ; Thu, 7 Mar 2013 07:41:13 -0800 (PST) Received: from mail.friendswithtools.org (unknown [64.78.239.70]) by ietfa.amsl.com (Postfix) with ESMTP id 08E7821F8988 for ; Thu, 7 Mar 2013 07:41:12 -0800 (PST) Received: from dspam (unknown [127.0.0.1]) by mail.friendswithtools.org (Postfix) with SMTP id 6CE30300015 for ; Thu, 7 Mar 2013 15:41:12 +0000 (UTC) Received: from [10.9.0.10] (web.hollyman.com [64.78.239.73]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.friendswithtools.org (Postfix) with ESMTPSA id D0C90300010; Thu, 7 Mar 2013 08:41:11 -0700 (MST) Content-Type: multipart/alternative; boundary="Apple-Mail=_4BE8A308-C9B0-4589-A47F-F1BB0FD92140" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: Shane Amante In-Reply-To: <51387034.2040708@cisco.com> Date: Thu, 7 Mar 2013 08:41:11 -0700 Message-Id: References: <51387034.2040708@cisco.com> To: Benoit Claise X-Mailer: Apple Mail (2.1499) X-DSPAM-Result: Innocent X-DSPAM-Processed: Thu Mar 7 08:41:12 2013 X-DSPAM-Confidence: 1.0000 X-DSPAM-Improbability: 1 in 98689409 chance of being spam X-DSPAM-Probability: 0.0023 X-DSPAM-Signature: 5138b51842071687014029 X-DSPAM-Factors: 27, 2013+at, 0.40000, 2013+at, 0.40000, Mar+#+2013, 0.40000, Mar+#+2013, 0.40000, list+or, 0.40000, list+or, 0.40000, To*Benoit+Claise, 0.40000, data+#+#+#+draft, 0.40000, data+#+#+#+draft, 0.40000, the+#+#+stand, 0.40000, the+#+#+stand, 0.40000, My+#+02, 0.40000, My+#+02, 0.40000, a+#+#+#+with, 0.40000, a+#+#+#+with, 0.40000, Fiber+#+#+home, 0.40000, Fiber+#+#+home, 0.40000, at+#+#+#+Benoit, 0.40000, at+#+#+#+Benoit, 0.40000, is+#+#+diminish, 0.40000, is+#+#+diminish, 0.40000, is+#+we, 0.40000, is+#+we, 0.40000, in+#+#+#+requirements, 0.40000, in+#+#+#+requirements, 0.40000, have+#+#+#+the, 0.40000, have+#+#+#+the, 0.40000 Cc: "lmap@ietf.org" Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 15:41:13 -0000 --Apple-Mail=_4BE8A308-C9B0-4589-A47F-F1BB0FD92140 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Benoit, On Mar 7, 2013, at 3:47 AM, Benoit Claise wrote: > Dear all, >=20 > I started to review the drafts, and I will start posting a few = questions to the list. > Open questions, clarifying questions, in order to generate some = discussions. > Disclaimer: I have not yet read the entire list archive. Apologize in = advance if some points have been discussed already.=20 >=20 > Here is my first question. What is broadband in the LMAP context? > Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology = independent? > And I see also "enterprise edge router", "cellular data or satellite" = in draft-schulzrinne-lmap-requirements. In or out? > Or do we have in mind a phase approach, starting with the "enterprise = edge router" first, and then home network? Speaking for the network I operate, I'm very much an advocate of saying = that "Enterprise Edge Router" is "in-scope". We would very much benefit = from a standards-based measurement enablement and collection regime vs. = mostly proprietary, and non-scalable, approaches that exist today. This is not to diminish the importance of similar test capabilities for = residential broadband use-cases. We absolutely need to work on those, = as well. =20 With respect to priority, my hope is that we do not have to choose to = prioritize one over the other. Rather, I would hope that both can be = developed in parallel, because both -- at least, IMO -- have a = substantially overlapping set of requirements/features.=20 My $0.02, -shane > Interestingly, I don't know what A stands for in LMAP, if it stands = for something. > According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A = doesn't stand for anything. > However, looking at the different draft titles, there is some = confusion. > draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband = Measurement Use Cases > draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale = Measurement of Broadband Performance > draft-boucadair-lmap-considerations-00, Large scale Measurement of = Access network Performance (LMAP): > Requirements and Issues from a Network Provider Perspective >=20 > Some more discussions, on the mailing list or during the BoF, on this = topic would be appreciated. >=20 > Regards, Benoit >=20 >=20 >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap --Apple-Mail=_4BE8A308-C9B0-4589-A47F-F1BB0FD92140 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 bclaise@cisco.com> = wrote:
=20 =20
Dear all,

I started to review the drafts, and I will start posting a few questions to the list.
Open questions, clarifying questions, in order to generate some discussions.
Disclaimer: I have not yet read the entire list archive. Apologize in advance if some points have been discussed already.

Here is my first question. What is broadband in the LMAP = context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology independent?
And I see also "enterprise edge router", "cellular data or satellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise edge router" first, and then home = network?

Speaking for the network I = operate, I'm very much an advocate of saying that "Enterprise Edge = Router" is "in-scope".  We would very much benefit from a = standards-based measurement enablement and collection regime vs. mostly = proprietary, and non-scalable, approaches that exist = today.

This is not to diminish the importance = of similar test capabilities for residential broadband use-cases. =  We absolutely need to work on those, as well. =  

With respect to priority, my hope is = that we do not have to choose to prioritize one over the other. =  Rather, I would hope that both can be developed in parallel, = because both -- at least, IMO -- have a substantially overlapping set of = requirements/features. 

My = $0.02,

-shane


Interestingly, I don't know what A stands for in = LMAP, if it stands for something.
According to http://trac.to= ols.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand for anything.
However, looking at the different draft titles, there is some confusion.
   draft-linsner-lmap-use-cases-0 title is Large-Scale =
Broadband Measurement Use Cases
   draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale =
Measurement of Broadband Performance
   draft-boucadair-lmap-considerations-00, Large scale Measurement of =
Access network Performance (LMAP):
      Requirements and Issues from a Network Provider Perspective

Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated.

Regards, Benoit



_______________________________________________
lmap mailing = list
lmap@ietf.org
https://www.ietf.org/ma= ilman/listinfo/lmap

= --Apple-Mail=_4BE8A308-C9B0-4589-A47F-F1BB0FD92140-- From acmorton@att.com Thu Mar 7 07:53:33 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 61D4421F8D08 for ; Thu, 7 Mar 2013 07:53:33 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.598 X-Spam-Level: X-Spam-Status: No, score=-106.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tg0dGUHy4xzk for ; Thu, 7 Mar 2013 07:53:32 -0800 (PST) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id EB69521F8D12 for ; Thu, 7 Mar 2013 07:53:31 -0800 (PST) Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id 331D41203F9; Thu, 7 Mar 2013 10:56:21 -0500 (EST) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-green.research.att.com (Postfix) with ESMTP id 0FD42E36E6; Thu, 7 Mar 2013 10:46:32 -0500 (EST) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Thu, 7 Mar 2013 10:53:30 -0500 From: "MORTON JR., ALFRED (AL)" To: Benoit Claise , "lmap@ietf.org" Date: Thu, 7 Mar 2013 10:53:28 -0500 Thread-Topic: [lmap] Which performance metrics for LMAP? Thread-Index: Ac4bJt9eZa0x1RKdTJWhqiJod+BeiQAH94Dw Message-ID: References: <513879B5.8030609@cisco.com> In-Reply-To: <513879B5.8030609@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_F1312FAF1A1E624DA0972D1C9A91379A1BF8C3B275njfpsrvexg7re_" MIME-Version: 1.0 Subject: Re: [lmap] Which performance metrics for LMAP? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 15:53:33 -0000 --_000_F1312FAF1A1E624DA0972D1C9A91379A1BF8C3B275njfpsrvexg7re_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Benoit, You've asked four good questions today, I'll take a stab at this one. IMO, the metrics chosen need to serve two purposes, at least. One set of metrics describe the packet transfer service (at the service lay= ers). Another (somewhat overlapping) set help subscribers understand how well their applications will perform on the service (inferred from measurements at the service layer). These two points of view (POV) and their implications on metric selection/parameter settings/reporting for IP packet transfer were investigated in http://tools.ietf.org/html/rfc6703 IPPM's scope has always included transport layer, so your 5-tuple with DSCP is covered there: these have all been part of the notion "packets of T= ype-P" since the beginning, see http://tools.ietf.org/html/rfc2330#section-13 and these aspects are more important now than ever. As a limit on "layer 3 and up", I would suggest that limited exceptions for metrics at +transport layers would be useful, such as the DNS response time metric constructed in http://tools.ietf.org/html/draft-bagnulo-ippm-new-registry-independent-00#s= ection-8 So IMO, LMAP infrastructure would support measurement at IP and transport l= ayers with a few notable exceptions, and look to others (IPPM WG) to select the b= est metrics (and develop new ones, as necessary) for each POV that LMAP serves. my 2cents, Al From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Ben= oit Claise Sent: Thursday, March 07, 2013 6:28 AM To: lmap@ietf.org Subject: [lmap] Which performance metrics for LMAP? Dear all, Another clarifying question: I'm not too sure which performance metric type= s LMAP targets. draft-linsner-lmap-use-cases-00 mentions: It is assumed that measurement tests run by OTT providers would only include the metrics associated with layer 3 and up The different drafts use different terms: service, end user experience, qua= lity of experience. Does Quality of Experience imply performance metrics per application, per 5= tuple (IP addresses, ports, protocol), per DCSP? Or do we want to stay at layer 3 performance metrics, typically IPPM type o= f metrics? If the latter, then LMAP is about large scale deployment of IPPM, right? While I fully understand the business needs for "and up" in "include the me= trics associated with layer 3 and up", let's not boil the ocean. Performance Metrics at Other Layer (PMOL), a concluded WG, tried and it proved to be difficult. Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated. Regards, Benoit --_000_F1312FAF1A1E624DA0972D1C9A91379A1BF8C3B275njfpsrvexg7re_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

= Hi Benoit,

 

You've asked four good questions today, I'll = take a stab at this one.

&n= bsp;

IMO, the metrics chosen need to = serve two purposes, at least.

 

One set of metrics describe= the packet transfer service (at the service layers).

=

Another (somewhat overlapping) set help subscribers u= nderstand

how well their applica= tions will perform on the service

 

The= se two points of view (POV) and their implications

were investigat= ed in http://tools.ietf.org/= html/rfc6703

 

IPPM's scope has always included transpo= rt layer, so your 5-tuple with

D= SCP is covered there: these have all been part of the notion "packets = of Type-P"

since the beginn= ing, see http://t= ools.ietf.org/html/rfc2330#section-13

and these aspects are more important now than ever.

 

As a limit on "layer 3 and up", I would suggest that limited= exceptions for

metrics at +tran= sport layers would be useful, such as the DNS response time

metric constructed in

http://tools.ietf.org/html/draft-ba= gnulo-ippm-new-registry-independent-00#section-8

<= p class=3DMsoNormal> 

So I= MO, LMAP infrastructure would support measurement at IP and transport layer= s

with a few notable exceptions,= and look to others (IPPM WG) to select the best

metrics (and develop new ones, as necessary) for each POV = that LMAP serves.

 

my 2cents,

Al

 

 

From: lmap-= bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Benoit = Claise
Sent: Thursday, March 07, 2013 6:28 AM
To: lmap@= ietf.org
Subject: [lmap] Which performance metrics for LMAP?=

 

Dear all,

Another clarifying question: I'm not too= sure which performance metric types LMAP targets.
draft-linsner-lmap-us= e-cases-00 mentions:

   It is assumed that mea=
surement
   tests run by OTT providers would=
 only include the metrics associated
   with=
 layer 3 and up


The different draf= ts use different terms: service, end user experience, quality of experience= .
Does Quality of Experience imply performance metrics per application, = per 5 tuple (IP addresses, ports, protocol), per DCSP?
Or do we want to = stay at layer 3 performance metrics, typically IPPM type of metrics?
If = the latter, then LMAP is about large scale deployment of IPPM, right?
While I fully understand the business needs for "and up" in &qu= ot;include the metrics associated with layer 3 and up", let's not boil= the ocean.
Performance Metrics at Other Layer (PMOL), a concluded WG, tried and it pr= oved to be difficult.

Some more discussions, on the mailing list or = during the BoF, on this topic would be appreciated.

Regards, Benoit=

= --_000_F1312FAF1A1E624DA0972D1C9A91379A1BF8C3B275njfpsrvexg7re_-- From Michael.K.Bugenhagen@centurylink.com Thu Mar 7 14:11:11 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7D76C21F8C98 for ; Thu, 7 Mar 2013 14:11:11 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qSYRXkMKVk1U for ; Thu, 7 Mar 2013 14:11:11 -0800 (PST) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id D229321F8C82 for ; Thu, 7 Mar 2013 14:11:07 -0800 (PST) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id r27MB62B022251 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 7 Mar 2013 16:11:06 -0600 (CST) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 3EBEE1E005A; Thu, 7 Mar 2013 16:11:01 -0600 (CST) Received: from sudnp797.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id 171D61E0065; Thu, 7 Mar 2013 16:11:01 -0600 (CST) Received: from sudnp797.qintra.com (localhost [127.0.0.1]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r27MB0eq009666; Thu, 7 Mar 2013 15:11:00 -0700 (MST) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.qintra.com [151.117.206.28]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r27MB09T009658 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 7 Mar 2013 15:11:00 -0700 (MST) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.02.0318.001; Thu, 7 Mar 2013 16:11:00 -0600 From: "Bugenhagen, Michael K" To: Benoit Claise Thread-Topic: [lmap] LMAP: Voluntary or compulsory? Thread-Index: AQHOGyQ3E39/AbzztEStXl5g9fSh1JiaywNU Date: Thu, 7 Mar 2013 22:10:59 +0000 Message-ID: <65FDABF1-5CDA-45C5-98C6-A8B907BE3DA6@centurylink.com> References: <5138745B.5020501@cisco.com> In-Reply-To: <5138745B.5020501@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "lmap@ietf.org" Subject: Re: [lmap] LMAP: Voluntary or compulsory? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 22:11:11 -0000 Benoit, I believe the answer here is yes, both. If a Service provider wants t= o obtain the performance evaluations in a smaller region, then they need to= be able to conduct testing without the additional work of obtaining sample= s via some large continual campaign to recruit volunteers. This also dire= ctly infers that any and all testing can not have an impact on the customer= s ability to use the service. =20 Given a customer themself may want to run a test, there should be a met= hod to allow them to do so even with their own equipment, or they could be = given the capability to execute a test if the provider has placed equipment= . This however directly infers that the customer has the ability to repeat = the same test as the service provider or we end up with apples and oranges = results. Mike Sent from my iPhone On Mar 7, 2013, at 5:09 AM, "Benoit Claise" wrote: > Dear all, >=20 > Another clarifying question: are those LMAP measurements based on volunta= ry participation or not? > Or maybe it depends on the use cases described in draft-linsner-lmap-use-= cases-02 >=20 > 2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 > 2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . 3 > 2.2 End User Network Diagnostics . . . . . . . . . . . . . . . . 4 > 2.3 Multi-provider Network Measurements . . . . . . . . . . . . 4 > 2.4 Over the Top Providers . . . . . . . . . . . . . . . . . . . 4 > 2.5 Regulators . . . . . . . . . . . . . . . . . . . . . . . . . 5 >=20 >=20 > Some more discussions, on the mailing list or during the BoF, on this top= ic would be appreciated. >=20 > Regards, Benoit >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From Michael.K.Bugenhagen@centurylink.com Thu Mar 7 14:16:25 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C80AD21F8C7C for ; Thu, 7 Mar 2013 14:16:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.299 X-Spam-Level: X-Spam-Status: No, score=-2.299 tagged_above=-999 required=5 tests=[AWL=-0.301, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_26=0.6] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e1CQEQQgYQrA for ; Thu, 7 Mar 2013 14:16:25 -0800 (PST) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id CB0BD21F8C55 for ; Thu, 7 Mar 2013 14:16:24 -0800 (PST) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id r27MGNHI028966 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 7 Mar 2013 16:16:23 -0600 (CST) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 6CDED1E0049; Thu, 7 Mar 2013 16:16:18 -0600 (CST) Received: from suomp61i.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id 5328F1E0060; Thu, 7 Mar 2013 16:16:18 -0600 (CST) Received: from suomp61i.qintra.com (localhost [127.0.0.1]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id r27MGH0h022746; Thu, 7 Mar 2013 16:16:18 -0600 (CST) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.qintra.com [151.117.206.28]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id r27MGH34022742 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 7 Mar 2013 16:16:17 -0600 (CST) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.02.0318.001; Thu, 7 Mar 2013 16:16:17 -0600 From: "Bugenhagen, Michael K" To: Benoit Claise Thread-Topic: [lmap] Which performance metrics for LMAP? Thread-Index: AQHOGybaVh0e7cNwrUuMBtZyDyvaL5iazHk4 Date: Thu, 7 Mar 2013 22:16:17 +0000 Message-ID: References: <513879B5.8030609@cisco.com> In-Reply-To: <513879B5.8030609@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_FB3C270400A145849717DB464555AEAFcenturylinkcom_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "lmap@ietf.org" Subject: Re: [lmap] Which performance metrics for LMAP? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 22:16:25 -0000 --_000_FB3C270400A145849717DB464555AEAFcenturylinkcom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Benoit, I think that the requirement to"future proof" the test framework in = order to ensure it will support any future test we throw at it requires tha= t the test framework be able to support both service commissioning (Through= put) type tests, as well as other SLA like tracking tests such as Twamp and= the Ethernet OAM, and even service level testing. So essentially they s= hould all fit inside the framework M Sent from my iPhone On Mar 7, 2013, at 5:28 AM, "Benoit Claise" > wrote: Dear all, Another clarifying question: I'm not too sure which performance metric type= s LMAP targets. draft-linsner-lmap-use-cases-00 mentions: It is assumed that measurement tests run by OTT providers would only include the metrics associated with layer 3 and up The different drafts use different terms: service, end user experience, qua= lity of experience. Does Quality of Experience imply performance metrics per application, per 5= tuple (IP addresses, ports, protocol), per DCSP? Or do we want to stay at layer 3 performance metrics, typically IPPM type o= f metrics? If the latter, then LMAP is about large scale deployment of IPPM, right? While I fully understand the business needs for "and up" in "include the me= trics associated with layer 3 and up", let's not boil the ocean. Performance Metrics at Other Layer (PMOL), a concluded WG, tried and it proved to be difficult. Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated. Regards, Benoit _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_FB3C270400A145849717DB464555AEAFcenturylinkcom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Benoit,

       I think that the requirement to"future= proof" the test framework in order to ensure it will support any futu= re test we throw at it requires that the test framework be able to support = both service commissioning (Throughput) type tests, as well as other SLA like tracking tests such as Twamp and the Ethernet OAM, and even= service level testing.    So essentially they should all fit ins= ide the framework

M

Sent from my iPhone

On Mar 7, 2013, at 5:28 AM, "Benoit Claise" <bclaise@cisco.com> wrote:

Dear all,

Another clarifying question: I'm not too sure which performance metric type= s LMAP targets.
draft-linsner-lmap-use-cases-00 mentions:
   It is assumed that measurement
   tests run by OTT providers would only include the metrics associated
   with layer 3 and up

The different drafts use different terms: service, end user experience, qua= lity of experience.
Does Quality of Experience imply performance metrics per application, per 5= tuple (IP addresses, ports, protocol), per DCSP?
Or do we want to stay at layer 3 performance metrics, typically IPPM type o= f metrics?
If the latter, then LMAP is about large scale deployment of IPPM, right?
While I fully understand the business needs for "and up" in "= ;include the metrics associated with layer 3 and up", let's not boil t= he ocean.
Performance Metrics at Other Layer (PMOL), a concluded WG, tried and it proved to be diff= icult.

Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated.

Regards, Benoit
_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ie= tf.org/mailman/listinfo/lmap
--_000_FB3C270400A145849717DB464555AEAFcenturylinkcom_-- From Michael.K.Bugenhagen@centurylink.com Thu Mar 7 14:20:25 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A0E521F8C55 for ; Thu, 7 Mar 2013 14:20:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.448 X-Spam-Level: X-Spam-Status: No, score=-2.448 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BNFZC7nQ68o0 for ; Thu, 7 Mar 2013 14:20:24 -0800 (PST) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id 556DD21F897F for ; Thu, 7 Mar 2013 14:20:24 -0800 (PST) Received: from lxdenvmpc030.qintra.com (lxdenvmpc030.qintra.com [10.1.51.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id r27MKLV4002014 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 7 Mar 2013 16:20:21 -0600 (CST) Received: from lxdenvmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id CC61A1E0053; Thu, 7 Mar 2013 15:20:15 -0700 (MST) Received: from sudnp796.qintra.com (unknown [151.119.91.93]) by lxdenvmpc030.qintra.com (Postfix) with ESMTP id AFE351E006B; Thu, 7 Mar 2013 15:20:15 -0700 (MST) Received: from sudnp796.qintra.com (localhost [127.0.0.1]) by sudnp796.qintra.com (8.14.4/8.14.4) with ESMTP id r27MKFIG006304; Thu, 7 Mar 2013 15:20:15 -0700 (MST) Received: from vodcwhubex501.ctl.intranet (vodcwhubex501.qintra.com [151.117.206.27]) by sudnp796.qintra.com (8.14.4/8.14.4) with ESMTP id r27MKELR006273 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 7 Mar 2013 15:20:15 -0700 (MST) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex501.ctl.intranet ([2002:9775:ce1b::9775:ce1b]) with mapi id 14.02.0318.001; Thu, 7 Mar 2013 16:20:14 -0600 From: "Bugenhagen, Michael K" To: Shane Amante Thread-Topic: [lmap] What is broadband? Thread-Index: AQHOGyE4Vk6FG+ho60WMp5+lLzlzSJiawrWAgAAK6cw= Date: Thu, 7 Mar 2013 22:20:13 +0000 Message-ID: References: <51387034.2040708@cisco.com>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_C6E1FAB2526A407DB69C359E60528A14centurylinkcom_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: Benoit Claise , "lmap@ietf.org" Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 22:20:25 -0000 --_000_C6E1FAB2526A407DB69C359E60528A14centurylinkcom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The word "access" should be key here as part of the definition provided we = are talking about an Internet service, which is the second component. I d= on't really think we are building tests that won't work on smaller pipes s= o questioning if it really 'broadband' or not is correct IMO. Sent from my iPhone On Mar 7, 2013, at 9:41 AM, "Shane Amante" > wrote: Benoit, On Mar 7, 2013, at 3:47 AM, Benoit Claise > wrote: Dear all, I started to review the drafts, and I will start posting a few questions to= the list. Open questions, clarifying questions, in order to generate some discussions= . Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already. Here is my first question. What is broadband in the LMAP context? Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology independent? And I see also "enterprise edge router", "cellular data or satellite" in dr= aft-schulzrinne-lmap-requirements. In or out? Or do we have in mind a phase approach, starting with the "enterprise edge = router" first, and then home network? Speaking for the network I operate, I'm very much an advocate of saying tha= t "Enterprise Edge Router" is "in-scope". We would very much benefit from = a standards-based measurement enablement and collection regime vs. mostly p= roprietary, and non-scalable, approaches that exist today. This is not to diminish the importance of similar test capabilities for res= idential broadband use-cases. We absolutely need to work on those, as well= . With respect to priority, my hope is that we do not have to choose to prior= itize one over the other. Rather, I would hope that both can be developed = in parallel, because both -- at least, IMO -- have a substantially overlapp= ing set of requirements/features. My $0.02, -shane Interestingly, I don't know what A stands for in LMAP, if it stands for som= ething. According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A does= n't stand for anything. However, looking at the different draft titles, there is some confusion. draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measuremen= t Use Cases draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurem= ent of Broadband Performance draft-boucadair-lmap-considerations-00, Large scale Measurement of Acces= s network Performance (LMAP): Requirements and Issues from a Network Provider Perspective Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated. Regards, Benoit _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_C6E1FAB2526A407DB69C359E60528A14centurylinkcom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
The word "access" should be key here as part of the definiti= on provided we are talking about an Internet service, which is the second c= omponent.   I don't really  think we are building tests that won'= t work on smaller pipes so questioning if it really 'broadband' or not is correct IMO.



Sent from my iPhone

On Mar 7, 2013, at 9:41 AM, "Shane Amante" <shane@castlepoint.net> wrote:

Benoit,

On Mar 7, 2013, at 3:47 AM, Benoit Claise <bclaise@cisco.com> wrote:
Dear all,

I started to review the drafts, and I will start posting a few questions to= the list.
Open questions, clarifying questions, in order to generate some discussions= .
Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology indepen= dent?
And I see also "enterprise edge router", "cellular data or s= atellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise = edge router" first, and then home network?

Speaking for the network I operate, I'm very much an advocate of saying tha= t "Enterprise Edge Router" is "in-scope".  We woul= d very much benefit from a standards-based measurement enablement and colle= ction regime vs. mostly proprietary, and non-scalable, approaches that exist today.

This is not to diminish the importance of similar test capabilities fo= r residential broadband use-cases.  We absolutely need to work on thos= e, as well.  

With respect to priority, my hope is that we do not have to choose to = prioritize one over the other.  Rather, I would hope that both can be = developed in parallel, because both -- at least, IMO -- have a substantiall= y overlapping set of requirements/features. 

My $0.02,

-shane


Interestingly, I don't know what = A stands for in LMAP, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand= for anything.
However, looking at the different draft titles, there is some confusion. 
   draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measu=
rement Use Cases
   draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurem=
ent of Broadband Performance
   draft-boucadair-lmap-considerations-00, Large scale Measurement of Access network Performance (LMAP):
      Requirements and Issues from a Network Provider Perspective

Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated.

Regards, Benoit



_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org= /mailman/listinfo/lmap

_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ie= tf.org/mailman/listinfo/lmap
--_000_C6E1FAB2526A407DB69C359E60528A14centurylinkcom_-- From Michael.K.Bugenhagen@centurylink.com Thu Mar 7 14:35:52 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E463221F8C08 for ; Thu, 7 Mar 2013 14:35:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.499 X-Spam-Level: X-Spam-Status: No, score=-2.499 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id unCTB4MhCJBd for ; Thu, 7 Mar 2013 14:35:52 -0800 (PST) Received: from sudnp799.qwest.com (sudnp799.qwest.com [155.70.32.99]) by ietfa.amsl.com (Postfix) with ESMTP id 62A2621F8A6E for ; Thu, 7 Mar 2013 14:35:52 -0800 (PST) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by sudnp799.qwest.com (8.14.4/8.14.4) with ESMTP id r27MZour028693 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 7 Mar 2013 15:35:51 -0700 (MST) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id CDDC11E0065; Thu, 7 Mar 2013 16:35:45 -0600 (CST) Received: from suomp61i.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id B58CE1E005B; Thu, 7 Mar 2013 16:35:45 -0600 (CST) Received: from suomp61i.qintra.com (localhost [127.0.0.1]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id r27MZjxH014755; Thu, 7 Mar 2013 16:35:45 -0600 (CST) Received: from vodcwhubex501.ctl.intranet (vodcwhubex501.qintra.com [151.117.206.27]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id r27MZiPI014742 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 7 Mar 2013 16:35:45 -0600 (CST) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex501.ctl.intranet ([2002:9775:ce1b::9775:ce1b]) with mapi id 14.02.0318.001; Thu, 7 Mar 2013 16:35:44 -0600 From: "Bugenhagen, Michael K" To: "Romascanu, Dan (Dan)" Thread-Topic: [lmap] presentations and minutes Thread-Index: AQHOG4Qa7WsQJqPpPk2yRx7ObD5jyA== Date: Thu, 7 Mar 2013 22:35:43 +0000 Message-ID: <94619D10-29A6-4DAD-A508-4ACC23C95CA2@centurylink.com> References: <9904FB1B0159DA42B0B887B7FA8119CA0A2505@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA0A2505@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "lmap@ietf.org" Subject: Re: [lmap] presentations and minutes X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 22:35:53 -0000 There will be a incoming liaison from the BBF that contains an ask about pr= otocols for the framework they resolved for testing in WT-304 BB service a= ttributed and PM Framework. It would probably be wise to share that one a= t this meeting. The liaison won't be sent until COB Friday so I can't giv= e a link to the document yet. However Ken Ko and I will be at the IETF to = answer any questions that may come up about that provided the group is will= ing to review it. Mike Sent from my iPhone On Mar 7, 2013, at 4:25 AM, "Romascanu, Dan (Dan)" wro= te: > Hi, >=20 > A couple of requests for the upcoming BOF session in Orlando: >=20 > - Participants who own agenda items and wish to make presentations must t= heir presentations in advance in order to allow time to the chairs to revie= w them and for uploading and making them available for remote participants.= Please send these until the evening prior to the meetings the latest.=20 > - We need at least two note takers and one jabber scribe in the room. We = cannot start the meeting without them. Please volunteer for these important= positions in advance, the chairs will publicly recognize your contribution= s. >=20 > Thanks and Regards, >=20 > Al and Dan >=20 >=20 >=20 >=20 >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From acmorton@att.com Thu Mar 7 15:36:56 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8949E21F86C3 for ; Thu, 7 Mar 2013 15:36:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.599 X-Spam-Level: X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iaMOwUSUFCbV for ; Thu, 7 Mar 2013 15:36:55 -0800 (PST) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id AAFB321F86BC for ; Thu, 7 Mar 2013 15:36:55 -0800 (PST) Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id 92BA412046C; Thu, 7 Mar 2013 18:39:46 -0500 (EST) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-green.research.att.com (Postfix) with ESMTP id EF8A9E36EE; Thu, 7 Mar 2013 18:29:55 -0500 (EST) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Thu, 7 Mar 2013 18:36:55 -0500 From: "MORTON JR., ALFRED (AL)" To: "Bugenhagen, Michael K" , "Romascanu, Dan (Dan)" Date: Thu, 7 Mar 2013 18:36:53 -0500 Thread-Topic: [lmap] presentations and minutes Thread-Index: AQHOG4Qa7WsQJqPpPk2yRx7ObD5jyJia3ZAA Message-ID: References: <9904FB1B0159DA42B0B887B7FA8119CA0A2505@AZ-FFEXMB04.global.avaya.com> <94619D10-29A6-4DAD-A508-4ACC23C95CA2@centurylink.com> In-Reply-To: <94619D10-29A6-4DAD-A508-4ACC23C95CA2@centurylink.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "lmap@ietf.org" Subject: Re: [lmap] presentations and minutes X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 23:36:56 -0000 Mike, Speaking as BoF co-chair, we already received very clear advice about discussing liaisons during a BoF, in response to your earlier mention of the future BBF liaison on the LMAP list: "definitely not" http://www.ietf.org/mail-archive/web/lmap/current/msg00283.html We still plan to heed that advice. Feel free to post a link to the list, when one is available. sorry, Al > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Bugenhagen, Michael K > Sent: Thursday, March 07, 2013 5:36 PM > To: Romascanu, Dan (Dan) > Cc: lmap@ietf.org > Subject: Re: [lmap] presentations and minutes >=20 > There will be a incoming liaison from the BBF that contains an ask about > protocols for the framework they resolved for testing in WT-304 BB > service attributed and PM Framework. It would probably be wise to share > that one at this meeting. The liaison won't be sent until COB Friday so > I can't give a link to the document yet. However Ken Ko and I will be at > the IETF to answer any questions that may come up about that provided the > group is willing to review it. >=20 > Mike >=20 > Sent from my iPhone >=20 > On Mar 7, 2013, at 4:25 AM, "Romascanu, Dan (Dan)" > wrote: >=20 > > Hi, > > > > A couple of requests for the upcoming BOF session in Orlando: > > > > - Participants who own agenda items and wish to make presentations must > their presentations in advance in order to allow time to the chairs to > review them and for uploading and making them available for remote > participants. Please send these until the evening prior to the meetings > the latest. > > - We need at least two note takers and one jabber scribe in the room. W= e > cannot start the meeting without them. Please volunteer for these > important positions in advance, the chairs will publicly recognize your > contributions. > > > > Thanks and Regards, > > > > Al and Dan > > > > > > > > > > > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From jamesmilleresquire@gmail.com Thu Mar 7 18:30:57 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D354C21F86B6 for ; Thu, 7 Mar 2013 18:30:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CiGpU2+GGH32 for ; Thu, 7 Mar 2013 18:30:56 -0800 (PST) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) by ietfa.amsl.com (Postfix) with ESMTP id 44D9E21F86A8 for ; Thu, 7 Mar 2013 18:30:56 -0800 (PST) Received: by mail-wi0-f177.google.com with SMTP id hm14so679556wib.4 for ; Thu, 07 Mar 2013 18:30:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=PKnR08YKybPfXYgSV9NfIBCiJrvhg3h2/BhSj2OFF8E=; b=c6+akOTfJ2yntRV9oLV24q2bJsWmAgueEU15tSkf8R85K5h7WxjZp8Pxv+9VnxJw2a qGHi3nZrNE9lwCGI0hnOibbVxR8ISZcuf2lK5RaJgeEZ+4FNmZnlQESW128P1bfVk2Fe HpEOHq0mCEvEY+5/+5kNrM8pCi9BsyhmNoD/DbKhP2fXRdel2hQrebLTA867FdxQikFX ekbsUlZ6sT/s/e8hgJh0ieM8KLPnqbTmdAS6JD3R/bVPQtyfdeWiRvVM2UXnROy5KJi9 W1Dy7uTTA6/HWseqF0IqvvvXN1/O9p/OlW4sbBUcnXMsyOqemRMWsYKvlsX0nKhFQ//1 CJxw== X-Received: by 10.180.87.170 with SMTP id az10mr893625wib.3.1362709855435; Thu, 07 Mar 2013 18:30:55 -0800 (PST) MIME-Version: 1.0 Received: by 10.180.24.10 with HTTP; Thu, 7 Mar 2013 18:30:25 -0800 (PST) In-Reply-To: References: <51387034.2040708@cisco.com> From: James Miller Date: Thu, 7 Mar 2013 21:30:25 -0500 Message-ID: To: "Bugenhagen, Michael K" Content-Type: multipart/alternative; boundary=f46d0444e97fdd83a004d7609cd7 Cc: Benoit Claise , Shane Amante , "lmap@ietf.org" Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 02:30:58 -0000 --f46d0444e97fdd83a004d7609cd7 Content-Type: text/plain; charset=ISO-8859-1 I believe that Henning had commented at some point that the LMAP definition he contemplated had "architecture" as the 'A' element but certainly access is an important piece. I think one of the problems that has been discussed also on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP performance measurements would implicate elements from the user's laptop, through wireless and other local LAN, carriers access networks, Tier 1 and other peering networks, the application host side and everything in between. Clearly there would be a lot of technologies included within that functional scope. For reference, in the FCC Measuring Broadband America Program we focused on measurement from the consumers' broadband modem through the carriers network to where it connects to a tier one peering point. LMAP should be able to address the broad mix of other use cases that would have a mix of other elements and motivations. Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K < Michael.K.Bugenhagen@centurylink.com> wrote: > The word "access" should be key here as part of the definition provided > we are talking about an Internet service, which is the second component. > I don't really think we are building tests that won't work on smaller > pipes so questioning if it really 'broadband' or not is correct IMO. > > > > Sent from my iPhone > > On Mar 7, 2013, at 9:41 AM, "Shane Amante" wrote: > > Benoit, > > On Mar 7, 2013, at 3:47 AM, Benoit Claise wrote: > > Dear all, > > I started to review the drafts, and I will start posting a few questions > to the list. > Open questions, clarifying questions, in order to generate some > discussions. > Disclaimer: I have not yet read the entire list archive. Apologize in > advance if some points have been discussed already. > > Here is my first question. What is broadband in the LMAP context? > Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology independent? > And I see also "enterprise edge router", "cellular data or satellite" in > draft-schulzrinne-lmap-requirements. In or out? > Or do we have in mind a phase approach, starting with the "enterprise edge > router" first, and then home network? > > > Speaking for the network I operate, I'm very much an advocate of saying > that "Enterprise Edge Router" is "in-scope". We would very much benefit > from a standards-based measurement enablement and collection regime vs. > mostly proprietary, and non-scalable, approaches that exist today. > > This is not to diminish the importance of similar test capabilities for > residential broadband use-cases. We absolutely need to work on those, as > well. > > With respect to priority, my hope is that we do not have to choose to > prioritize one over the other. Rather, I would hope that both can be > developed in parallel, because both -- at least, IMO -- have a > substantially overlapping set of requirements/features. > > My $0.02, > > -shane > > > Interestingly, I don't know what A stands for in LMAP, if it stands for > something. > According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A > doesn't stand for anything. > However, looking at the different draft titles, there is some confusion. > > draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measurement Use Cases > draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurement of Broadband Performance > draft-boucadair-lmap-considerations-00, *Large scale Measurement of **Access **network Performance (LMAP)*: > Requirements and Issues from a Network Provider Perspective > > > Some more discussions, on the mailing list or during the BoF, on this > topic would be appreciated. > > Regards, Benoit > > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > > --f46d0444e97fdd83a004d7609cd7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I believe that Henning had commented at some point that the LMAP definition= he contemplated had "architecture" as the 'A' element bu= t certainly access is an important piece. =A0I think one of the problems th= at has been discussed also on the LMAP and our FCC Next-Gen lists is that a= complete view of LMAP performance measurements would implicate elements fr= om the user's laptop, through wireless and other local LAN, carriers ac= cess networks, Tier 1 and other peering networks, the application host side= and everything in between. =A0Clearly there would be a lot of technologies= included within that=A0functional=A0scope.

For reference, in the FCC Measuring Broadband America Progra= m we focused on measurement from the consumers' broadband modem through= the carriers network to where it connects to a tier one peering point. =A0= LMAP should be able to address the broad mix of other use cases that would = have a mix of other elements and motivations.

Graphic in Report at page 9.=A0http://transiti= on.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf

On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K <= ;= Michael.K.Bugenhagen@centurylink.com> wrote:
The word "access" should be key here as part of the definiti= on provided we are talking about an Internet service, which is the second c= omponent. =A0 I don't really =A0think we are building tests that won= 9;t work on smaller pipes so questioning if it really 'broadband' or not is correct IMO.



Sent from my iPhone

On Mar 7, 2013, at 9:41 AM, "Shane Amante" <shane@castlepoint.net> wrot= e:

Benoit,

On Mar 7, 2013, at 3:47 AM, Benoit Claise <bclaise@cisco.com> wrote:
Dear all,

I started to review the drafts, and I will start posting a few questions to= the list.
Open questions, clarifying questions, in order to generate some discussions= .
Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?=A0 Is LMAP technology independen= t?
And I see also "enterprise edge router", "cellular data or s= atellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise = edge router" first, and then home network?

Speaking for the network I operate, I'm very much an advocate of saying= that "Enterprise Edge Router" is "in-scope". =A0We wou= ld very much benefit from a standards-based measurement enablement and coll= ection regime vs. mostly proprietary, and non-scalable, approaches that exist today.

This is not to diminish the importance of similar test capabilities fo= r residential broadband use-cases. =A0We absolutely need to work on those, = as well. =A0

With respect to priority, my hope is that we do not have to choose to = prioritize one over the other. =A0Rather, I would hope that both can be dev= eloped in parallel, because both -- at least, IMO -- have a substantially o= verlapping set of requirements/features.=A0

My $0.02,

-shane


Interestingly, I don't know w= hat A stands for in LMAP, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't s= tand for anything.
However, looking at the different draft titles, there is some confusion. 
   draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measu=
rement Use Cases
   draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurem=
ent of Broadband Performance
   draft-boucadair-lmap-considerations-00, Large scale Measurement of Access network Performance (LMAP):
      Requirements and Issues from a Network Provider Perspective

Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated.

Regards, Benoit



_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap

_______________________________________________
lmap mailing list
lmap@ietf.org<= /span>
https://www.ietf.org/mailman/listinfo/lmap

_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap


--f46d0444e97fdd83a004d7609cd7-- From shane@castlepoint.net Thu Mar 7 19:24:30 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BF54121F86BC for ; Thu, 7 Mar 2013 19:24:30 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.436 X-Spam-Level: X-Spam-Status: No, score=-0.436 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FH_RELAY_NODNS=1.451, HELO_MISMATCH_ORG=0.611, HTML_MESSAGE=0.001, RDNS_NONE=0.1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cLG0z0PmxMI3 for ; Thu, 7 Mar 2013 19:24:28 -0800 (PST) Received: from mail.friendswithtools.org (unknown [64.78.239.70]) by ietfa.amsl.com (Postfix) with ESMTP id E801421F852A for ; Thu, 7 Mar 2013 19:24:27 -0800 (PST) Received: from dspam (unknown [127.0.0.1]) by mail.friendswithtools.org (Postfix) with SMTP id 431B4300074 for ; Fri, 8 Mar 2013 03:24:27 +0000 (UTC) Received: from mbp.castlepoint.net (184-96-123-182.hlrn.qwest.net [184.96.123.182]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.friendswithtools.org (Postfix) with ESMTPSA id 5E96E300056; Thu, 7 Mar 2013 20:24:23 -0700 (MST) Content-Type: multipart/alternative; boundary="Apple-Mail=_AFBB4EEE-677C-485F-8798-95DA2F9EF3C4" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: Shane Amante In-Reply-To: Date: Thu, 7 Mar 2013 20:24:18 -0700 Message-Id: References: <51387034.2040708@cisco.com> To: James Miller X-Mailer: Apple Mail (2.1499) X-DSPAM-Result: Innocent X-DSPAM-Processed: Thu Mar 7 20:24:27 2013 X-DSPAM-Confidence: 1.0000 X-DSPAM-Improbability: 1 in 98689409 chance of being spam X-DSPAM-Probability: 0.0023 X-DSPAM-Signature: 513959eb42071337937656 X-DSPAM-Factors: 27, Measuring+#+America, 0.40000, Measuring+#+America, 0.40000, Mar+#+#+#+9, 0.40000, Mar+#+#+#+9, 0.40000, 2013+at, 0.40000, 2013+at, 0.40000, access+#+#+key, 0.40000, access+#+#+key, 0.40000, should+#+#+to, 0.40000, should+#+#+to, 0.40000, shane+#+#+in, 0.40000, shane+#+#+in, 0.40000, would+#+#+from, 0.40000, would+#+#+from, 0.40000, Broadband+#+#+we, 0.40000, Broadband+#+#+we, 0.40000, it+#+#+important, 0.40000, it+#+#+important, 0.40000, an+#+#+#+is, 0.40000, an+#+#+#+is, 0.40000, end+#+that, 0.40000, end+#+that, 0.40000, Mar+#+2013, 0.40000, Mar+#+2013, 0.40000, list+or, 0.40000, list+or, 0.40000, connects+#+a, 0.40000 Cc: Benoit Claise , "Bugenhagen, Michael K" , "lmap@ietf.org" Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 03:24:30 -0000 --Apple-Mail=_AFBB4EEE-677C-485F-8798-95DA2F9EF3C4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 James, All, On Mar 7, 2013, at 7:30 PM, James Miller = wrote: > I believe that Henning had commented at some point that the LMAP = definition he contemplated had "architecture" as the 'A' element but = certainly access is an important piece. I think one of the problems = that has been discussed also on the LMAP and our FCC Next-Gen lists is = that a complete view of LMAP performance measurements would implicate = elements from the user's laptop, through wireless and other local LAN, = carriers access networks, Tier 1 and other peering networks, the = application host side and everything in between. Clearly there would be = a lot of technologies included within that functional scope. I agree that access is important, but not to the exclusion of everything = else that constitutes an Internet end-user customer experience or, = alternatively, an Enterprise end-user customer experience -- which is = what I believe you're saying above? That's why it will be important to, = at some point, figure out *if*, and then how, to try to segment the = portions of the end-to-end path that you describe above so we can = attribute good or bad performance to a particular portion of the path so = that, ultimately, the correct network operator can be contacted to look = into the problem further. I do not believe that this requires us to = break down the end-to-end path on a router-hop by router-hop basis, but = rather we need to be able to identify 'sign posts' along the path that = can correlate well to end-to-end path. -shane > For reference, in the FCC Measuring Broadband America Program we = focused on measurement from the consumers' broadband modem through the = carriers network to where it connects to a tier one peering point. LMAP = should be able to address the broad mix of other use cases that would = have a mix of other elements and motivations. >=20 > Graphic in Report at page 9. = http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf >=20 > On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K = wrote: > The word "access" should be key here as part of the definition = provided we are talking about an Internet service, which is the second = component. I don't really think we are building tests that won't work = on smaller pipes so questioning if it really 'broadband' or not is = correct IMO. >=20 >=20 >=20 > Sent from my iPhone >=20 > On Mar 7, 2013, at 9:41 AM, "Shane Amante" = wrote: >=20 >> Benoit, >>=20 >> On Mar 7, 2013, at 3:47 AM, Benoit Claise wrote: >>> Dear all, >>>=20 >>> I started to review the drafts, and I will start posting a few = questions to the list. >>> Open questions, clarifying questions, in order to generate some = discussions. >>> Disclaimer: I have not yet read the entire list archive. Apologize = in advance if some points have been discussed already.=20 >>>=20 >>> Here is my first question. What is broadband in the LMAP context? >>> Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology = independent? >>> And I see also "enterprise edge router", "cellular data or = satellite" in draft-schulzrinne-lmap-requirements. In or out? >>> Or do we have in mind a phase approach, starting with the = "enterprise edge router" first, and then home network? >>=20 >> Speaking for the network I operate, I'm very much an advocate of = saying that "Enterprise Edge Router" is "in-scope". We would very much = benefit from a standards-based measurement enablement and collection = regime vs. mostly proprietary, and non-scalable, approaches that exist = today. >>=20 >> This is not to diminish the importance of similar test capabilities = for residential broadband use-cases. We absolutely need to work on = those, as well. =20 >>=20 >> With respect to priority, my hope is that we do not have to choose to = prioritize one over the other. Rather, I would hope that both can be = developed in parallel, because both -- at least, IMO -- have a = substantially overlapping set of requirements/features.=20 >>=20 >> My $0.02, >>=20 >> -shane >>=20 >>=20 >>> Interestingly, I don't know what A stands for in LMAP, if it stands = for something. >>> According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the = A doesn't stand for anything. >>> However, looking at the different draft titles, there is some = confusion. >>> draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband = Measurement Use Cases >>> draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale = Measurement of Broadband Performance >>> draft-boucadair-lmap-considerations-00, Large scale Measurement = of Access network Performance (LMAP): >>> Requirements and Issues from a Network Provider Perspective >>>=20 >>> Some more discussions, on the mailing list or during the BoF, on = this topic would be appreciated. >>>=20 >>> Regards, Benoit >>>=20 >>>=20 >>>=20 >>> _______________________________________________ >>> lmap mailing list >>> lmap@ietf.org >>> https://www.ietf.org/mailman/listinfo/lmap >>=20 >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap >=20 >=20 --Apple-Mail=_AFBB4EEE-677C-485F-8798-95DA2F9EF3C4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 jamesmilleresquire@gmail.com<= /a>> wrote:
I believe that Henning had = commented at some point that the LMAP definition he contemplated had = "architecture" as the 'A' element but certainly access is an important = piece.  I think one of the problems that has been discussed also on = the LMAP and our FCC Next-Gen lists is that a complete view of LMAP = performance measurements would implicate elements from the user's = laptop, through wireless and other local LAN, carriers access networks, = Tier 1 and other peering networks, the application host side and = everything in between.  Clearly there would be a lot of = technologies included within = that functional scope.

I agree that = access is important, but not to the exclusion of everything else that = constitutes an Internet end-user customer experience or, alternatively, = an Enterprise end-user customer experience -- which is what I believe = you're saying above?  That's why it will be important to, at some = point, figure out *if*, and then how, to try to segment the portions of = the end-to-end path that you describe above so we can attribute good or = bad performance to a particular portion of the path so that, ultimately, = the correct network operator can be contacted to look into the problem = further.  I do not believe that this requires us to break down the = end-to-end path on a router-hop by router-hop basis, but rather we need = to be able to identify 'sign posts' along the path that can correlate = well to end-to-end = path.

-shane


For reference, in the FCC Measuring Broadband = America Program we focused on measurement from the consumers' broadband = modem through the carriers network to where it connects to a tier one = peering point.  LMAP should be able to address the broad mix of = other use cases that would have a mix of other elements and = motivations.

Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology= .pdf

On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K <Michael.K.Bugenhagen@centurylink.com> = wrote:
The word "access" should be key here as part of the definition = provided we are talking about an Internet service, which is the second = component.   I don't really  think we are building tests that = won't work on smaller pipes so questioning if it really 'broadband' or not is correct IMO.



Sent from my iPhone

On Mar 7, 2013, at 9:41 AM, "Shane Amante" <shane@castlepoint.net> wrote:

Benoit,

On Mar 7, 2013, at 3:47 AM, Benoit Claise <bclaise@cisco.com> wrote:
Dear all,

I started to review the drafts, and I will start posting a few questions = to the list.
Open questions, clarifying questions, in order to generate some = discussions.
Disclaimer: I have not yet read the entire list archive. Apologize in = advance if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology = independent?
And I see also "enterprise edge router", "cellular data or satellite" in = draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise = edge router" first, and then home network?

Speaking for the network I operate, I'm very much an advocate of saying = that "Enterprise Edge Router" is "in-scope".  We would very much = benefit from a standards-based measurement enablement and collection = regime vs. mostly proprietary, and non-scalable, approaches that exist today.

This is not to diminish the importance of similar test capabilities = for residential broadband use-cases.  We absolutely need to work on = those, as well.  

With respect to priority, my hope is that we do not have to choose = to prioritize one over the other.  Rather, I would hope that both = can be developed in parallel, because both -- at least, IMO -- have a = substantially overlapping set of requirements/features. 

My $0.02,

-shane


Interestingly, I don't know = what A stands for in LMAP, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't = stand for anything.
However, looking at the different draft titles, there is some = confusion.
   draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband =
Measurement Use Cases
   draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale =
Measurement of Broadband Performance
   draft-boucadair-lmap-considerations-00, Large scale Measurement of =
Access network Performance (LMAP):
      Requirements and Issues from a Network Provider Perspective

Some more discussions, on the mailing list or during the BoF, on this = topic would be appreciated.

Regards, Benoit



_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap



= --Apple-Mail=_AFBB4EEE-677C-485F-8798-95DA2F9EF3C4-- From jamesmilleresquire@gmail.com Thu Mar 7 19:58:20 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 224DD21F86A1 for ; Thu, 7 Mar 2013 19:58:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.099 X-Spam-Level: X-Spam-Status: No, score=-3.099 tagged_above=-999 required=5 tests=[AWL=0.499, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2qSdPbXZjIjn for ; Thu, 7 Mar 2013 19:58:18 -0800 (PST) Received: from mail-wg0-f51.google.com (mail-wg0-f51.google.com [74.125.82.51]) by ietfa.amsl.com (Postfix) with ESMTP id 95C9F21F8698 for ; Thu, 7 Mar 2013 19:58:17 -0800 (PST) Received: by mail-wg0-f51.google.com with SMTP id 8so1963090wgl.6 for ; Thu, 07 Mar 2013 19:58:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=dG5Eo6DtN0LkdEQUbPLHcdxChM6iG2S1WBPSAIcWQ+w=; b=EPbAP6G8Wlrgn1meLstJT1xiCrgsY0K7630z5bxKP/tjq9BgVqxK2iv+GFz/G4g8i7 99qjgVHeqPN+5opfx1K1a/9f8Pj+/U8biCAiEpQx16ePc0Sg2eYUrnee9eRTCmKqZOku 5VZue5TVdjdObZ1jQ6JwSRgP5fu/uD25ehyaVFEPsUQ87Aynw6XiAWbLaNOyDtGA/RCh Lh9vFYLZw3PeaNAY1w9paGhMZeVGtKENn1zd2HiPv4FM1w/IhL3izn5mdMfxJFD125Ez c5dYUVHTaMjuQn8ClRBiVXcjZvL1X92dZ9DAAygX19tBMqw6dKHaozRC1wv8ofSZXt7i HPlw== X-Received: by 10.180.100.169 with SMTP id ez9mr38140408wib.3.1362715096644; Thu, 07 Mar 2013 19:58:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.180.24.10 with HTTP; Thu, 7 Mar 2013 19:57:46 -0800 (PST) In-Reply-To: References: <51387034.2040708@cisco.com> From: James Miller Date: Thu, 7 Mar 2013 22:57:46 -0500 Message-ID: To: Shane Amante Content-Type: multipart/alternative; boundary=14dae9cc9c564404c104d761d57f Cc: Benoit Claise , "Bugenhagen, Michael K" , "lmap@ietf.org" Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 03:58:20 -0000 --14dae9cc9c564404c104d761d57f Content-Type: text/plain; charset=ISO-8859-1 I agree Shane--was what I was trying to get at. :) Our early FCC focus was basically the Access network but I think we're all making the point that the mix-n-match of the network measurement pieces will be driven by a particular user's use case. For our FCC work, the focus (discussed in more detail in our reports) was on understanding the performance in the segment under the control of a carrier delivering broadband to a given consumer. Naturally many other elements affect an end user's experience but our initial interest was on understanding that segment for the purposes of providing better information to the consumer about the portion of Internet performance that the carrier provides. The FCC's focus and history is on that piece and so it makes sense that it might begin there, but a particular carrier's focus might mirror the access network focus, but perhaps with less market-trend orientation and more diagnostic in flavor. Maybe LMAP together with other protocols (TR69 and other BBForum work for example) could benefit all those use cases. In recent meetings we've discussed other more targeted mini-studies on things like CDN or in-home Wifi performance. Other potential LMAP users already focus their work on these or other elements of the network. I would envision the LMAP work could provide the glue between any of these elements and different use cases. Worth noting that much of the initial discussions in the FCC's Next-Gen group were on the potential value of standardized interfaces between such different elements of measurement infrastructure. Another element that's probably worth noting is that the use cases also may direct the actual tests that someone wants to run. I would imagine the actual tests you might want to run for evaluating in-home LAN, access network, CDN, mobile networks may vary. So I think the work on the test registry and other IPPM work on new tests will also improve our definitions and understanding of "broadband performance". As always my personal views and not the FCC's.. possible valued at less than $.02 :) On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante wrote: > James, All, > > On Mar 7, 2013, at 7:30 PM, James Miller > wrote: > > I believe that Henning had commented at some point that the LMAP > definition he contemplated had "architecture" as the 'A' element but > certainly access is an important piece. I think one of the problems that > has been discussed also on the LMAP and our FCC Next-Gen lists is that a > complete view of LMAP performance measurements would implicate elements > from the user's laptop, through wireless and other local LAN, carriers > access networks, Tier 1 and other peering networks, the application host > side and everything in between. Clearly there would be a lot of > technologies included within that functional scope. > > > I agree that access is important, but not to the exclusion of everything > else that constitutes an Internet end-user customer experience or, > alternatively, an Enterprise end-user customer experience -- which is what > I believe you're saying above? That's why it will be important to, at some > point, figure out *if*, and then how, to try to segment the portions of the > end-to-end path that you describe above so we can attribute good or bad > performance to a particular portion of the path so that, ultimately, the > correct network operator can be contacted to look into the problem further. > I do not believe that this requires us to break down the end-to-end path > on a router-hop by router-hop basis, but rather we need to be able to > identify 'sign posts' along the path that can correlate well to end-to-end > path. > > -shane > > > For reference, in the FCC Measuring Broadband America Program we focused > on measurement from the consumers' broadband modem through the carriers > network to where it connects to a tier one peering point. LMAP should be > able to address the broad mix of other use cases that would have a mix of > other elements and motivations. > > Graphic in Report at page 9. > http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf > > On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K < > Michael.K.Bugenhagen@centurylink.com> wrote: > >> The word "access" should be key here as part of the definition provided >> we are talking about an Internet service, which is the second component. >> I don't really think we are building tests that won't work on smaller >> pipes so questioning if it really 'broadband' or not is correct IMO. >> >> >> >> Sent from my iPhone >> >> On Mar 7, 2013, at 9:41 AM, "Shane Amante" wrote: >> >> Benoit, >> >> On Mar 7, 2013, at 3:47 AM, Benoit Claise wrote: >> >> Dear all, >> >> I started to review the drafts, and I will start posting a few questions >> to the list. >> Open questions, clarifying questions, in order to generate some >> discussions. >> Disclaimer: I have not yet read the entire list archive. Apologize in >> advance if some points have been discussed already. >> >> Here is my first question. What is broadband in the LMAP context? >> Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology >> independent? >> And I see also "enterprise edge router", "cellular data or satellite" in >> draft-schulzrinne-lmap-requirements. In or out? >> Or do we have in mind a phase approach, starting with the "enterprise >> edge router" first, and then home network? >> >> >> Speaking for the network I operate, I'm very much an advocate of saying >> that "Enterprise Edge Router" is "in-scope". We would very much benefit >> from a standards-based measurement enablement and collection regime vs. >> mostly proprietary, and non-scalable, approaches that exist today. >> >> This is not to diminish the importance of similar test capabilities for >> residential broadband use-cases. We absolutely need to work on those, as >> well. >> >> With respect to priority, my hope is that we do not have to choose to >> prioritize one over the other. Rather, I would hope that both can be >> developed in parallel, because both -- at least, IMO -- have a >> substantially overlapping set of requirements/features. >> >> My $0.02, >> >> -shane >> >> >> Interestingly, I don't know what A stands for in LMAP, if it stands for >> something. >> According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A >> doesn't stand for anything. >> However, looking at the different draft titles, there is some confusion. >> >> draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measurement Use Cases >> draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurement of Broadband Performance >> draft-boucadair-lmap-considerations-00, *Large scale Measurement of **Access **network Performance (LMAP)*: >> Requirements and Issues from a Network Provider Perspective >> >> >> Some more discussions, on the mailing list or during the BoF, on this >> topic would be appreciated. >> >> Regards, Benoit >> >> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> >> > > --14dae9cc9c564404c104d761d57f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I agree Shane--was what I was trying to get at. :)

Our e= arly FCC focus was basically the Access network but I think we're all m= aking the point that the mix-n-match of the network measurement pieces will= be driven by a particular user's use case. =A0

For our FCC work, the focus (discussed in more detail i= n our reports) was on understanding the performance in the segment under th= e control of a carrier delivering broadband to a given consumer. =A0Natural= ly many other elements affect an end user's experience but our initial = interest was on understanding that segment for the purposes of providing be= tter information to the consumer about the portion of Internet performance = that the carrier provides. =A0The FCC's focus and history is on that pi= ece and so it makes sense that it might begin there, but a particular carri= er's focus might mirror the access=A0network=A0focus, but perhaps with = less market-trend orientation and more diagnostic in flavor. =A0Maybe LMAP = together with other protocols (TR69 and other BBForum work for example) cou= ld benefit all those use cases.

In recent meetings we've discussed other more targe= ted mini-studies on things like CDN or in-home Wifi performance. =A0Other p= otential LMAP users already focus their work on these or other elements of = the network. =A0I would envision the LMAP work could provide the glue betwe= en any of these elements and different use cases. =A0Worth noting that much= of the initial discussions in the FCC's Next-Gen group were on the=A0p= otential value=A0of standardized interfaces between such different elements= of measurement infrastructure.

Another element that's probably worth noting is tha= t the use cases also may direct the actual tests that someone wants to run.= =A0I would imagine the actual tests you might want to run for evaluating i= n-home LAN, access network, CDN, mobile networks may vary. =A0So I think th= e work on the test registry and other IPPM work on new tests will also impr= ove our definitions and understanding of "broadband performance".=

As always my personal views and not the FCC's.. pos= sible valued at less than $.02 =A0:)

On T= hu, Mar 7, 2013 at 10:24 PM, Shane Amante <shane@castlepoint.net&g= t; wrote:
James, A= ll,

On Mar 7, 2013, at 7:30 PM, James M= iller <jamesmilleresquire@gmail.com> wrote:
I believe that Henning had commented at some poin= t that the LMAP definition he contemplated had "architecture" as = the 'A' element but certainly access is an important piece. =A0I th= ink one of the problems that has been discussed also on the LMAP and our FC= C Next-Gen lists is that a complete view of LMAP performance measurements w= ould implicate elements from the user's laptop, through wireless and ot= her local LAN, carriers access networks, Tier 1 and other peering networks,= the application host side and everything in between. =A0Clearly there woul= d be a lot of technologies included within that=A0functional=A0scope.

I agree that access is important, but not to the exclu= sion of everything else that constitutes an Internet end-user customer expe= rience or, alternatively, an Enterprise end-user customer experience -- whi= ch is what I believe you're saying above? =A0That's why it will be = important to, at some point, figure out *if*, and then how, to try to segme= nt the portions of the end-to-end path that you describe above so we can at= tribute good or bad performance to a particular portion of the path so that= , ultimately, the correct network operator can be contacted to look into th= e problem further. =A0I do not believe that this requires us to break down = the end-to-end path on a router-hop by router-hop basis, but rather we need= to be able to identify 'sign posts' along the path that can correl= ate well to end-to-end path.

-shane


For reference, in the FCC Measuring Broadband America = Program we focused on measurement from the consumers' broadband modem t= hrough the carriers network to where it connects to a tier one peering poin= t. =A0LMAP should be able to address the broad mix of other use cases that = would have a mix of other elements and motivations.

Graphic in Report at page 9.=A0http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf<= /a>

On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K <= ;= Michael.K.Bugenhagen@centurylink.com> wrote:
The word "access" should be key here as part of the definiti= on provided we are talking about an Internet service, which is the second c= omponent. =A0 I don't really =A0think we are building tests that won= 9;t work on smaller pipes so questioning if it really 'broadband' or not is correct IMO.



Sent from my iPhone

On Mar 7, 2013, at 9:41 AM, "Shane Amante" <shane@castlepoint.net> wrot= e:

Benoit,

On Mar 7, 2013, at 3:47 AM, Benoit Claise <bclaise@cisco.com> wrote:
Dear all,

I started to review the drafts, and I will start posting a few questions to= the list.
Open questions, clarifying questions, in order to generate some discussions= .
Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?=A0 Is LMAP technology independen= t?
And I see also "enterprise edge router", "cellular data or s= atellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise = edge router" first, and then home network?

Speaking for the network I operate, I'm very much an advocate of saying= that "Enterprise Edge Router" is "in-scope". =A0We wou= ld very much benefit from a standards-based measurement enablement and coll= ection regime vs. mostly proprietary, and non-scalable, approaches that exist today.

This is not to diminish the importance of similar test capabilities fo= r residential broadband use-cases. =A0We absolutely need to work on those, = as well. =A0

With respect to priority, my hope is that we do not have to choose to = prioritize one over the other. =A0Rather, I would hope that both can be dev= eloped in parallel, because both -- at least, IMO -- have a substantially o= verlapping set of requirements/features.=A0

My $0.02,

-shane


Interestingly, I don't know w= hat A stands for in LMAP, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't s= tand for anything.
However, looking at the different draft titles, there is some confusion. 
   draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measu=
rement Use Cases
   draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurem=
ent of Broadband Performance
   draft-boucadair-lmap-considerations-00, Large scale Measurement of Access network Performance (LMAP):
      Requirements and Issues from a Network Provider Perspective

Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated.

Regards, Benoit



_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap

_______________________________________________
lmap mailing list
lmap@ietf.org<= /span>
https://www.ietf.org/mailman/listinfo/lmap

_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap




--14dae9cc9c564404c104d761d57f-- From Henning.Schulzrinne@fcc.gov Fri Mar 8 15:16:17 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 06E1521F85E2 for ; Fri, 8 Mar 2013 15:16:17 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MxvGr58wmE7I for ; Fri, 8 Mar 2013 15:16:13 -0800 (PST) Received: from DC-IP-2.fcc.gov (dc-ip-2.fcc.gov [192.104.54.91]) by ietfa.amsl.com (Postfix) with ESMTP id D189521F85FD for ; Fri, 8 Mar 2013 15:16:12 -0800 (PST) Received: from gatekeeper4.fcc.gov (HELO p2pxcas03.fccnet.win.fcc.gov) ([192.104.54.21]) by DC-IP-2.fcc.gov with ESMTP; 08 Mar 2013 18:16:11 -0500 Received: from P2PXMB13.fccnet.win.fcc.gov ([fe80::6593:6526:65f8:66b7]) by p2pxcas03.fccnet.win.fcc.gov ([fe80::8db2:ac93:84fc:c36b%13]) with mapi id 14.01.0438.000; Fri, 8 Mar 2013 18:16:11 -0500 From: Henning Schulzrinne To: 'Shane Amante' , James Miller Thread-Topic: [lmap] What is broadband? Thread-Index: AQHOGyFCc9Tw+OlxRk6zqw2j7uy0dpiasfGAgABvfYCAAEXogIAADw4AgAD4fKA= Date: Fri, 8 Mar 2013 23:16:09 +0000 Message-ID: References: <51387034.2040708@cisco.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [165.135.229.64] Content-Type: multipart/alternative; boundary="_000_E6A16181E5FD2F46B962315BB05962D00DFE166Fp2pxmb13fccnetw_" MIME-Version: 1.0 Cc: Benoit Claise , "Bugenhagen, Michael K" , "lmap@ietf.org" Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 23:16:17 -0000 --_000_E6A16181E5FD2F46B962315BB05962D00DFE166Fp2pxmb13fccnetw_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I would hope that the protocol components are largely independent of the me= asurement endpoints, but there is indeed a need for somebody (not necessari= ly the IETF and not necessarily one entity) to define them if measurements = are meant to be comparable. For some private-use cases, e.g., within a lar= ge enterprise network or for internal carrier use, that's probably less imp= ortant. If you believe that protocol functionality depends on defining thes= e "mileposts", I'd be curious if you have examples in mind. Henning From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Sha= ne Amante Sent: Thursday, March 07, 2013 10:24 PM To: James Miller Cc: Benoit Claise; Bugenhagen, Michael K; lmap@ietf.org Subject: Re: [lmap] What is broadband? James, All, On Mar 7, 2013, at 7:30 PM, James Miller > wrote: I believe that Henning had commented at some point that the LMAP definition= he contemplated had "architecture" as the 'A' element but certainly access= is an important piece. I think one of the problems that has been discusse= d also on the LMAP and our FCC Next-Gen lists is that a complete view of LM= AP performance measurements would implicate elements from the user's laptop= , through wireless and other local LAN, carriers access networks, Tier 1 an= d other peering networks, the application host side and everything in betwe= en. Clearly there would be a lot of technologies included within that func= tional scope. I agree that access is important, but not to the exclusion of everything el= se that constitutes an Internet end-user customer experience or, alternativ= ely, an Enterprise end-user customer experience -- which is what I believe = you're saying above? That's why it will be important to, at some point, fi= gure out *if*, and then how, to try to segment the portions of the end-to-e= nd path that you describe above so we can attribute good or bad performance= to a particular portion of the path so that, ultimately, the correct netwo= rk operator can be contacted to look into the problem further. I do not be= lieve that this requires us to break down the end-to-end path on a router-h= op by router-hop basis, but rather we need to be able to identify 'sign pos= ts' along the path that can correlate well to end-to-end path. -shane For reference, in the FCC Measuring Broadband America Program we focused on= measurement from the consumers' broadband modem through the carriers netwo= rk to where it connects to a tier one peering point. LMAP should be able t= o address the broad mix of other use cases that would have a mix of other e= lements and motivations. Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadba= ndreport/2Methodology.pdf On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K > wrote: The word "access" should be key here as part of the definition provided we = are talking about an Internet service, which is the second component. I d= on't really think we are building tests that won't work on smaller pipes s= o questioning if it really 'broadband' or not is correct IMO. Sent from my iPhone On Mar 7, 2013, at 9:41 AM, "Shane Amante" > wrote: Benoit, On Mar 7, 2013, at 3:47 AM, Benoit Claise > wrote: Dear all, I started to review the drafts, and I will start posting a few questions to= the list. Open questions, clarifying questions, in order to generate some discussions= . Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already. Here is my first question. What is broadband in the LMAP context? Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology independent? And I see also "enterprise edge router", "cellular data or satellite" in dr= aft-schulzrinne-lmap-requirements. In or out? Or do we have in mind a phase approach, starting with the "enterprise edge = router" first, and then home network? Speaking for the network I operate, I'm very much an advocate of saying tha= t "Enterprise Edge Router" is "in-scope". We would very much benefit from = a standards-based measurement enablement and collection regime vs. mostly p= roprietary, and non-scalable, approaches that exist today. This is not to diminish the importance of similar test capabilities for res= idential broadband use-cases. We absolutely need to work on those, as well= . With respect to priority, my hope is that we do not have to choose to prior= itize one over the other. Rather, I would hope that both can be developed = in parallel, because both -- at least, IMO -- have a substantially overlapp= ing set of requirements/features. My $0.02, -shane Interestingly, I don't know what A stands for in LMAP, if it stands for som= ething. According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A does= n't stand for anything. However, looking at the different draft titles, there is some confusion. draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measuremen= t Use Cases draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurem= ent of Broadband Performance draft-boucadair-lmap-considerations-00, Large scale Measurement of Acces= s network Performance (LMAP): Requirements and Issues from a Network Provider Perspective Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated. Regards, Benoit _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_E6A16181E5FD2F46B962315BB05962D00DFE166Fp2pxmb13fccnetw_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I would hope that the pro= tocol components are largely independent of the measurement endpoints, but = there is indeed a need for somebody (not necessarily the IETF and not necessarily one entity) to define them if measurements are me= ant  to be comparable. For some private-use cases, e.g., within a larg= e enterprise network or for internal carrier use, that’s probably les= s important. If you believe that protocol functionality depends on defining these “mileposts”, I’d be curious if= you have examples in mind.

 <= /p>

Henning=

 <= /p>

From: lmap-bou= nces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Shane Amante
Sent: Thursday, March 07, 2013 10:24 PM
To: James Miller
Cc: Benoit Claise; Bugenhagen, Michael K; lmap@ietf.org
Subject: Re: [lmap] What is broadband?

 

James, All,

 

On Mar 7, 2013, at 7:30 PM, James Miller <jamesmilleresquire@gmail.com&g= t; wrote:

I believe that Henning had commented at some point t= hat the LMAP definition he contemplated had "architecture" as the= 'A' element but certainly access is an important piece.  I think one = of the problems that has been discussed also on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP performanc= e measurements would implicate elements from the user's laptop, through wir= eless and other local LAN, carriers access networks, Tier 1 and other peeri= ng networks, the application host side and everything in between.  Clearly there would be a lot of tech= nologies included within that functional scope.

 

I agree that access is important, but not to the exc= lusion of everything else that constitutes an Internet end-user customer ex= perience or, alternatively, an Enterprise end-user customer experience -- w= hich is what I believe you're saying above?  That's why it will be important to, at some point, figure out= *if*, and then how, to try to segment the portions of the end-to-end path = that you describe above so we can attribute good or bad performance to a pa= rticular portion of the path so that, ultimately, the correct network operator can be contacted to look into the= problem further.  I do not believe that this requires us to break dow= n the end-to-end path on a router-hop by router-hop basis, but rather we ne= ed to be able to identify 'sign posts' along the path that can correlate well to end-to-end path.

 

-shane

 



For reference, in the FCC Measuring Broadband Americ= a Program we focused on measurement from the consumers' broadband modem thr= ough the carriers network to where it connects to a tier one peering point.=  LMAP should be able to address the broad mix of other use cases that would have a mix of other elements and m= otivations.

 

Graphic in Report at = page 9. http://transition.fcc.gov/cgb/measuringbroadbandrepor= t/2Methodology.pdf

On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael = K <Michael.K.Bugenhagen@centurylink.com> wrote:

The word "access" should be key here as pa= rt of the definition provided we are talking about an Internet service, whi= ch is the second component.   I don't really  think we are buildi= ng tests that won't work on smaller pipes so questioning if it really 'broadband' or not is correct IMO.

 



Sent from my iPhone


On Mar 7, 2013, at 9:41 AM, "Shane Amante" <shane@castlepoint.net> wrot= e:

Benoit,

 

On Mar 7, 2013, at 3:47 AM, Benoit Claise <bclaise@cisco.com> = wrote:

Dear all,

I started to review the drafts, and I will start posting a few questions to= the list.
Open questions, clarifying questions, in order to generate some discussions= .
Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology indepen= dent?
And I see also "enterprise edge router", "cellular data or s= atellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise = edge router" first, and then home network?

 

Speaking for the network I operate, I'm very much an= advocate of saying that "Enterprise Edge Router" is "in-sco= pe".  We would very much benefit from a standards-based measureme= nt enablement and collection regime vs. mostly proprietary, and non-scalable, approaches that exist today.

 

This is not to diminish the importance of similar te= st capabilities for residential broadband use-cases.  We absolutely ne= ed to work on those, as well.  

 

With respect to priority, my hope is that we do not = have to choose to prioritize one over the other.  Rather, I would hope= that both can be developed in parallel, because both -- at least, IMO -- h= ave a substantially overlapping set of requirements/features. 

 

My $0.02,

 

-shane

 



Interestingly, I don't know what A stands for in LMA= P, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand= for anything.
However, looking at the different draft titles, there is some confusion.

   draft-linsner-lmap-use-cases-0 title is Large-Scale Broad=
band Measurement Use Cases
   draft-schulzrinne-lmap-requirements-00.txt title is Large=
-Scale Measurement of Broadband Performance
   draft-boucadair-lmap-considerations-00, Large scale Me=
asurement of Access network Performance (LMAP):
      Requirements and Issues from a Network =
Provider Perspective


Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated.

Regards, Benoit


_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap

 

_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap


_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap

 

 

--_000_E6A16181E5FD2F46B962315BB05962D00DFE166Fp2pxmb13fccnetw_-- From acmorton@att.com Sat Mar 9 11:47:42 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 391C621F8714 for ; Sat, 9 Mar 2013 11:47:42 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.599 X-Spam-Level: X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ppyeFoj7hs90 for ; Sat, 9 Mar 2013 11:47:41 -0800 (PST) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 0764A21F8702 for ; Sat, 9 Mar 2013 11:47:33 -0800 (PST) Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id 2974512038D for ; Sat, 9 Mar 2013 14:50:28 -0500 (EST) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-blue.research.att.com (Postfix) with ESMTP id 9C582F0106 for ; Sat, 9 Mar 2013 14:47:32 -0500 (EST) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Sat, 9 Mar 2013 14:47:32 -0500 From: "MORTON JR., ALFRED (AL)" To: "lmap@ietf.org" Date: Sat, 9 Mar 2013 14:47:28 -0500 Thread-Topic: New Liaison Statement, "Performance Measurements Architecture" Thread-Index: Ac4c/sipQPpjShq7RrCxFFoTeD8KdgAABTbA Message-ID: References: <20130309194612.24263.46325.idtracker@ietfa.amsl.com> In-Reply-To: <20130309194612.24263.46325.idtracker@ietfa.amsl.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: [lmap] FW: New Liaison Statement, "Performance Measurements Architecture" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 19:47:42 -0000 SGVyZSdzIHRoZSBCQkYgTGlhaXNvbiBmb3IgZm9sa3MgdG8gcmVhZC4NCkFsDQoNCj4gLS0tLS1P cmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogTGlhaXNvbiBTdGF0ZW1lbnQgTWFuYWdlbWVu dCBUb29sIFttYWlsdG86bHNtdEBpZXRmLm9yZ10NCj4gU2VudDogU2F0dXJkYXksIE1hcmNoIDA5 LCAyMDEzIDI6NDYgUE0NCj4gVG86IFRoZSBJRVRGIENoYWlyDQo+IENjOiBUaGUgSUVTRzsgRGF2 aWQgU2luaWNyb3BlOyBSb2JpbiBNZXJzaDsgR2FicmllbGxlIEJpbmdoYW07DQo+IGphc29uQHFh Y2FmZS5jb207IGpvaG4uYmxhY2tmb3JkQHBhY2UuY29tOyBkYXZpZC5qLnRob3JuZUBidC5jb207 DQo+IGRhdmlkLmkuYWxsYW5AZXJpY3Nzb24uY29tOyBzdmVuLm9vZ2hlQGFsY2F0ZWwtbHVjZW50 LmNvbTsgUGV0ZXIgQWRhbXM7DQo+IHRyYW1tZWxsQHRpay5lZS5ldGh6LmNoOyBiaWxsQHdqY2Vy dmVueS5jb207IGRyb21hc2NhQGF2YXlhLmNvbTsgTU9SVE9ODQo+IEpSLiwgQUxGUkVEIChBTCk7 IENoYXJsZXMgQ29vazsgTWljaGFlbC5LLkJ1Z2VuaGFnZW5AY2VudHVyeWxpbmsuY29tOyBLRU4N Cj4gS08NCj4gU3ViamVjdDogTmV3IExpYWlzb24gU3RhdGVtZW50LCAiUGVyZm9ybWFuY2UgTWVh c3VyZW1lbnRzIEFyY2hpdGVjdHVyZSINCj4gDQo+IFRpdGxlOiBQZXJmb3JtYW5jZSBNZWFzdXJl bWVudHMgQXJjaGl0ZWN0dXJlDQo+IFN1Ym1pc3Npb24gRGF0ZTogMjAxMy0wMy0wOA0KPiBVUkwg b2YgdGhlIElFVEYgV2ViIHBhZ2U6IGh0dHA6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9saWFpc29u LzEyNDMvDQo+IA0KPiBGcm9tOiBCcm9hZGJhbmQgRm9ydW0gKENocmlzdG9waGUgQWx0ZXIgPGNo cmlzdG9waGUuYWx0ZXJAb3JhbmdlLQ0KPiBmdGdyb3VwLmNvbT4pDQo+IFRvOiBUaGUgSUVURiAo VGhlIElFVEYgQ2hhaXIgPGNoYWlyQGlldGYub3JnPikNCj4gQ2M6IFRoZSBJRVNHIDxpZXNnQGll dGYub3JnPixEYXZpZCBTaW5pY3JvcGUNCj4gPGRhdmlkLnNpbmljcm9wZUBlcmljc3Nvbi5jb20+ LFJvYmluIE1lcnNoIDxybWVyc2hAYnJvYWRiYW5kLQ0KPiBmb3J1bS5vcmc+LEdhYnJpZWxsZSBC aW5naGFtIDxnYmluZ2hhbUBicm9hZGJhbmQtDQo+IGZvcnVtLm9yZz4sPGphc29uQHFhY2FmZS5j b20+LDxqb2huLmJsYWNrZm9yZEBwYWNlLmNvbT4sPGRhdmlkLmoudGhvcm5lQGJ0DQo+IC5jb20+ LDxkYXZpZC5pLmFsbGFuQGVyaWNzc29uLmNvbT4sPHN2ZW4ub29naGVAYWxjYXRlbC1sdWNlbnQu Y29tPixQZXRlcg0KPiBBZGFtcw0KPiA8UEVURVIuQURBTVNAYWR0cmFuLmNvbT4sPHRyYW1tZWxs QHRpay5lZS5ldGh6LmNoPiw8YmlsbEB3amNlcnZlbnkuY29tPiw8ZA0KPiByb21hc2NhQGF2YXlh LmNvbT4sPGFjbW9ydG9uQGF0dC5jb20+LENoYXJsZXMgQ29vaw0KPiA8Y2hhcmxlcy5jb29rMkBj ZW50dXJ5bGluay5jb20+LDxNaWNoYWVsLksuQnVnZW5oYWdlbkBDZW50dXJ5TGluay5jb20+LEtF Tg0KPiBLTyA8S0VOLktPQGFkdHJhbi5jb20+DQo+IFJlcG9uc2UgQ29udGFjdDoNCj4gVGVjaG5p Y2FsIENvbnRhY3Q6DQo+IFB1cnBvc2U6IEZvciBpbmZvcm1hdGlvbg0KPiANCj4gQm9keToNCj4g QXR0YWNobWVudHM6DQo+IA0KPiAgICAgUGVyZm9ybWFuY2UgTWVhc3VyZW1lbnRzIEFyY2hpdGVj dHVyZQ0KPiAgICAgaHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9kb2N1bWVudHMvTElBSVNP Ti9saWFpc29uLTIwMTMtMDMtMDgtDQo+IGJyb2FkYmFuZC1mb3J1bS10aGUtaWV0Zi1wZXJmb3Jt YW5jZS1tZWFzdXJlbWVudHMtYXJjaGl0ZWN0dXJlLWF0dGFjaG1lbnQtDQo+IDEuZG9jDQoNCg== From philip.eardley@bt.com Sun Mar 10 12:56:42 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 321BB11E80F0 for ; Sun, 10 Mar 2013 12:56:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.215 X-Spam-Level: X-Spam-Status: No, score=-103.215 tagged_above=-999 required=5 tests=[AWL=0.383, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GiNC0gfVyrdg for ; Sun, 10 Mar 2013 12:56:40 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp62.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id 3A0A111E80EF for ; Sun, 10 Mar 2013 12:56:30 -0700 (PDT) Received: from EVMHT61-UKRD.domain1.systemhost.net (10.36.3.127) by RDW083A006ED62.smtp-e2.hygiene.service (10.187.98.11) with Microsoft SMTP Server (TLS) id 8.3.297.1; Sun, 10 Mar 2013 19:56:28 +0000 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.43]) by EVMHT61-UKRD.domain1.systemhost.net ([10.36.3.127]) with mapi; Sun, 10 Mar 2013 19:56:28 +0000 From: To: , Date: Sun, 10 Mar 2013 19:56:28 +0000 Thread-Topic: [lmap] What is broadband? Thread-Index: Ac4bpPleAa9K0F5yQlaNHYoyH8oECQCIiOlF Message-ID: <9510D26531EF184D9017DF24659BB87F343774C51B@EMV65-UKRD.domain1.systemhost.net> References: <51387034.2040708@cisco.com> , In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F343774C51BEMV65UKRDdoma_" MIME-Version: 1.0 Cc: bclaise@cisco.com, shane@castlepoint.net, lmap@ietf.org Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 19:56:42 -0000 --_000_9510D26531EF184D9017DF24659BB87F343774C51BEMV65UKRDdoma_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable the 'A' in lmap doesnt stand for anything (somewhere lost in the mists of h= istory... it didnt seem sensible to change the abbreviation before the BoF,= since people have got used to the term) consumer and enterprise wired broadband certainly of interest to me. that i= ncludes the network beyond the access (BRAS, peering point etc). so a "reg= istry" of reference points in the network seems like work that needs to be = done [tools.ietf.org/html/draft-morton-ippm-lmap-path as first attempt] phil ________________________________ From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of James Mill= er [jamesmilleresquire@gmail.com] Sent: 08 March 2013 02:30 To: Bugenhagen, Michael K Cc: Benoit Claise; Shane Amante; lmap@ietf.org Subject: Re: [lmap] What is broadband? I believe that Henning had commented at some point that the LMAP definition= he contemplated had "architecture" as the 'A' element but certainly access= is an important piece. I think one of the problems that has been discusse= d also on the LMAP and our FCC Next-Gen lists is that a complete view of LM= AP performance measurements would implicate elements from the user's laptop= , through wireless and other local LAN, carriers access networks, Tier 1 an= d other peering networks, the application host side and everything in betwe= en. Clearly there would be a lot of technologies included within that func= tional scope. For reference, in the FCC Measuring Broadband America Program we focused on= measurement from the consumers' broadband modem through the carriers netwo= rk to where it connects to a tier one peering point. LMAP should be able t= o address the broad mix of other use cases that would have a mix of other e= lements and motivations. Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadba= ndreport/2Methodology.pdf On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K > wrote: The word "access" should be key here as part of the definition provided we = are talking about an Internet service, which is the second component. I d= on't really think we are building tests that won't work on smaller pipes s= o questioning if it really 'broadband' or not is correct IMO. Sent from my iPhone On Mar 7, 2013, at 9:41 AM, "Shane Amante" > wrote: Benoit, On Mar 7, 2013, at 3:47 AM, Benoit Claise > wrote: Dear all, I started to review the drafts, and I will start posting a few questions to= the list. Open questions, clarifying questions, in order to generate some discussions= . Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already. Here is my first question. What is broadband in the LMAP context? Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology independent? And I see also "enterprise edge router", "cellular data or satellite" in dr= aft-schulzrinne-lmap-requirements. In or out? Or do we have in mind a phase approach, starting with the "enterprise edge = router" first, and then home network? Speaking for the network I operate, I'm very much an advocate of saying tha= t "Enterprise Edge Router" is "in-scope". We would very much benefit from = a standards-based measurement enablement and collection regime vs. mostly p= roprietary, and non-scalable, approaches that exist today. This is not to diminish the importance of similar test capabilities for res= idential broadband use-cases. We absolutely need to work on those, as well= . With respect to priority, my hope is that we do not have to choose to prior= itize one over the other. Rather, I would hope that both can be developed = in parallel, because both -- at least, IMO -- have a substantially overlapp= ing set of requirements/features. My $0.02, -shane Interestingly, I don't know what A stands for in LMAP, if it stands for som= ething. According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A does= n't stand for anything. However, looking at the different draft titles, there is some confusion. draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measuremen= t Use Cases draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurem= ent of Broadband Performance draft-boucadair-lmap-considerations-00, Large scale Measurement of Acces= s network Performance (LMAP): Requirements and Issues from a Network Provider Perspective Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated. Regards, Benoit _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_9510D26531EF184D9017DF24659BB87F343774C51BEMV65UKRDdoma_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
the 'A' in lmap doesnt stand for anything (somewhere lost in the mists= of history... it didnt seem sensible to change the abbreviation before the= BoF, since people have got used to the term)
 
consumer and enterprise wired broadband certainl= y of interest to me. that includes the network beyond the access (BRAS, pee= ring point etc).  so a "registry" of reference points in the= network seems like work that needs to be done [tools.ietf.org/html/draft-m= orton-ippm-lmap-path as first attempt]
 
phil
 
From: lmap-bounces@ietf.org [lmap-b= ounces@ietf.org] On Behalf Of James Miller [jamesmilleresquire@gmail.com] Sent: 08 March 2013 02:30
To: Bugenhagen, Michael K
Cc: Benoit Claise; Shane Amante; lmap@ietf.org
Subject: Re: [lmap] What is broadband?

I believe that Henning had commented at some point that the LMAP defin= ition he contemplated had "architecture" as the 'A' element but c= ertainly access is an important piece.  I think one of the problems th= at has been discussed also on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP performance measurement= s would implicate elements from the user's laptop, through wireless and oth= er local LAN, carriers access networks, Tier 1 and other peering networks, = the application host side and everything in between.  Clearly there would be a lot of technologies included wi= thin that functional scope.

For reference, in the FCC Measuring Broadband America Program we focus= ed on measurement from the consumers' broadband modem through the carriers = network to where it connects to a tier one peering point.  LMAP should= be able to address the broad mix of other use cases that would have a mix of other elements and motivations.

Graphic in Report at page 9. http://tr= ansition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf

On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Mich= ael K <Michael.K.Bugen= hagen@centurylink.com> wrote:
The word "access" should be key here as part of the definiti= on provided we are talking about an Internet service, which is the second c= omponent.   I don't really  think we are building tests that won'= t work on smaller pipes so questioning if it really 'broadband' or not is correct IMO.



Sent from my iPhone

On Mar 7, 2013, at 9:41 AM, "Shane Amante" <shane@castlepoint.net> wrote:

Benoit,

On Mar 7, 2013, at 3:47 AM, Benoit Claise <bclaise@cisco.com> wrote:
Dear all,

I started to review the drafts, and I will start posting a few questions to= the list.
Open questions, clarifying questions, in order to generate some discussions= .
Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology indepen= dent?
And I see also "enterprise edge router", "cellular data or s= atellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise = edge router" first, and then home network?

Speaking for the network I operate, I'm very much an advocate of saying tha= t "Enterprise Edge Router" is "in-scope".  We woul= d very much benefit from a standards-based measurement enablement and colle= ction regime vs. mostly proprietary, and non-scalable, approaches that exist today.

This is not to diminish the importance of similar test capabilities fo= r residential broadband use-cases.  We absolutely need to work on thos= e, as well.  

With respect to priority, my hope is that we do not have to choose to = prioritize one over the other.  Rather, I would hope that both can be = developed in parallel, because both -- at least, IMO -- have a substantiall= y overlapping set of requirements/features. 

My $0.02,

-shane


Interestingly, I don't know what A stands for in L= MAP, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand= for anything.
However, looking at the different draft titles, there is some confusion. 
   draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measu=
rement Use Cases
   draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurem=
ent of Broadband Performance
   draft-boucadair-lmap-considerations-00, Large scale Measurement of Access network Performance (LMAP):
      Requirements and Issues from a Network Provider Perspective

Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated.

Regards, Benoit



_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap

_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap


--_000_9510D26531EF184D9017DF24659BB87F343774C51BEMV65UKRDdoma_-- From acooper@cdt.org Mon Mar 11 08:31:57 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A623E21F8B8D for ; Mon, 11 Mar 2013 08:31:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.895 X-Spam-Level: X-Spam-Status: No, score=-102.895 tagged_above=-999 required=5 tests=[AWL=-0.296, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ghqIVrbD0FFA for ; Mon, 11 Mar 2013 08:31:54 -0700 (PDT) Received: from mail.maclaboratory.net (mail.maclaboratory.net [209.190.215.232]) by ietfa.amsl.com (Postfix) with ESMTP id 854CD21F8B84 for ; Mon, 11 Mar 2013 08:31:53 -0700 (PDT) X-Footer: Y2R0Lm9yZw== Received: from localhost ([127.0.0.1]) by mail.maclaboratory.net (using TLSv1/SSLv3 with cipher AES128-SHA (128 bits)); Mon, 11 Mar 2013 11:31:48 -0400 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: Alissa Cooper In-Reply-To: <65FDABF1-5CDA-45C5-98C6-A8B907BE3DA6@centurylink.com> Date: Mon, 11 Mar 2013 11:31:45 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <0BB0FE7C-9C86-44D2-AA71-ACBD396967A6@cdt.org> References: <5138745B.5020501@cisco.com> <65FDABF1-5CDA-45C5-98C6-A8B907BE3DA6@centurylink.com> To: "Bugenhagen, Michael K" X-Mailer: Apple Mail (2.1499) Cc: Benoit Claise , "lmap@ietf.org" Subject: Re: [lmap] LMAP: Voluntary or compulsory? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 15:31:58 -0000 +1 IMO the LMAP framework should be able to accommodate all of the modes of = operation that Mike describes: (1) customers voluntary signing up to = join a panel that incurs ongoing measurements over time, (2) operators = conducting measurements across entire customer segments or all = customers, and (3) customers triggering their own measurement tests at = will. Depending on the data collected, out-of-band mechanisms for = informing customers about the measurements taking place and for = obtaining customer sign-up (case 1) or consent (case 2) will likely be = required/desired. Alissa On Mar 7, 2013, at 5:10 PM, "Bugenhagen, Michael K" = wrote: > Benoit, >=20 > I believe the answer here is yes, both. If a Service provider = wants to obtain the performance evaluations in a smaller region, then = they need to be able to conduct testing without the additional work of = obtaining samples via some large continual campaign to recruit = volunteers. This also directly infers that any and all testing can not = have an impact on the customers ability to use the service. =20 >=20 > Given a customer themself may want to run a test, there should be a = method to allow them to do so even with their own equipment, or they = could be given the capability to execute a test if the provider has = placed equipment. This however directly infers that the customer has the = ability to repeat the same test as the service provider or we end up = with apples and oranges results. >=20 > Mike >=20 >=20 >=20 > Sent from my iPhone >=20 > On Mar 7, 2013, at 5:09 AM, "Benoit Claise" wrote: >=20 >> Dear all, >>=20 >> Another clarifying question: are those LMAP measurements based on = voluntary participation or not? >> Or maybe it depends on the use cases described in = draft-linsner-lmap-use-cases-02 >>=20 >> 2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 >> 2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . = 3 >> 2.2 End User Network Diagnostics . . . . . . . . . . . . . . . . = 4 >> 2.3 Multi-provider Network Measurements . . . . . . . . . . . . = 4 >> 2.4 Over the Top Providers . . . . . . . . . . . . . . . . . . . = 4 >> 2.5 Regulators . . . . . . . . . . . . . . . . . . . . . . . . . = 5 >>=20 >>=20 >> Some more discussions, on the mailing list or during the BoF, on this = topic would be appreciated. >>=20 >> Regards, Benoit >>=20 >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap >=20 From acooper@cdt.org Mon Mar 11 08:32:08 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 775BE21F8BA7 for ; Mon, 11 Mar 2013 08:32:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.87 X-Spam-Level: X-Spam-Status: No, score=-102.87 tagged_above=-999 required=5 tests=[AWL=-0.271, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 39VTZ9KXUm2a for ; Mon, 11 Mar 2013 08:32:08 -0700 (PDT) Received: from mail.maclaboratory.net (mail.maclaboratory.net [209.190.215.232]) by ietfa.amsl.com (Postfix) with ESMTP id A633821F8B84 for ; Mon, 11 Mar 2013 08:32:07 -0700 (PDT) X-Footer: Y2R0Lm9yZw== Received: from localhost ([127.0.0.1]) by mail.maclaboratory.net (using TLSv1/SSLv3 with cipher AES128-SHA (128 bits)); Mon, 11 Mar 2013 11:32:05 -0400 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: Alissa Cooper In-Reply-To: <5138763B.1000100@cisco.com> Date: Mon, 11 Mar 2013 11:32:04 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <22981BAB-FED3-4E41-B8D7-C5AFD0934D38@cdt.org> References: <5138763B.1000100@cisco.com> To: Benoit Claise X-Mailer: Apple Mail (2.1499) Cc: "lmap@ietf.org" Subject: Re: [lmap] Which devices does LMAP target? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 15:32:08 -0000 FWIW, this strikes me as a fairly important question for understanding = the extent to which existing protocols/building blocks can be reused for = LMAP. If we expect a measurement agent to be embedded or embeddable in = every cheap piece of CPE out there, that may constrain the complexity of = the protocol logic for both receiving test schedules and reporting test = results. Whether we want all measurement agent implementations to be so = constrained and/or whether an extensibility mechanism is possible that = allows for more sophisticated logic in more sophisticated devices (PC, = tablet, smartphone, etc.) would be good to figure out. Alissa On Mar 7, 2013, at 6:12 AM, Benoit Claise wrote: > Dear all, >=20 > draft-schulzrinne-lmap-requirements-00 mentions: >=20 > Thus, this document proposes > an overall architecture, with emphasis on the functional and = security > requirements for the protocols connecting the elements of the > architecture, that will make it possible to build measurement > capabilities into home and enterprise edge routers, personal > computers, mobile devices and other edge devices. >=20 > Does LMAP really target my home devices? My set top box, PC, tablet, = smartphone > Or is LMAP is limited to the modem/router? > And if this is only the modem/router, do we want to make a difference = between an ISP managed device (typically a branch office) and my own = private device? >=20 > Some more discussions, on the mailing list or during the BoF, on this = topic would be appreciated. >=20 > Regards, Benoit >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap >=20 From sharam.hakimi@exfo.com Mon Mar 11 09:42:08 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DC64711E816B for ; Mon, 11 Mar 2013 09:42:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.358 X-Spam-Level: X-Spam-Status: No, score=-1.358 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_LWSHORTT=1.24] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n3bEwLLVi6Hy for ; Mon, 11 Mar 2013 09:42:07 -0700 (PDT) Received: from smtpinqc.exfo.com (smtpinqc.exfo.com [206.162.164.97]) by ietfa.amsl.com (Postfix) with ESMTP id EB2A811E8169 for ; Mon, 11 Mar 2013 09:42:06 -0700 (PDT) Received: from spqcexc04.exfo.com ([172.16.48.171]) by smtpinqc.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 11 Mar 2013 12:42:05 -0400 Received: from spboexc01.exfo.com ([10.10.10.16]) by spqcexc04.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 11 Mar 2013 12:42:04 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CE1E77.5BF7A42F" Date: Mon, 11 Mar 2013 12:42:55 -0400 Message-ID: <084CDC75FEC1E640B60338273BEACDFA023C47EE@spboexc01.exfo.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [lmap] What is broadband? Thread-Index: Ac4bsS4FSJ4U1D3/SmS2NxVRJV0KNACwnOAw References: <51387034.2040708@cisco.com> From: "Sharam Hakimi" To: "James Miller" , "Shane Amante" X-OriginalArrivalTime: 11 Mar 2013 16:42:04.0111 (UTC) FILETIME=[5C7005F0:01CE1E77] Cc: Benoit Claise , "Bugenhagen, Michael K" , lmap@ietf.org Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 16:42:09 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01CE1E77.5BF7A42F Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable James, I think mixing testing of wireless and wire technologies confuses the issues. One wireless is WiFi which is the extension of LAN and the other is the cellular network. They both have different requirements and two major conditions in both Signal Strength Roaming =20 have great effect on the achieved performance which does not exist in the wire condition . Some of these performance issues might also be better resolved in layer 2 rather than layer 3 and above. For example, before I run any throughput test I want to make sure that I have the highest strength signal possible be it WiFi or Cellular wireless connection ( and stay stationary during the test). On the other hand if I am trying to find the best location for signal strength a throughput test might not be the best tool for the job. =20 I would suggest to keep the two separate at least for the short term. =20 Sharam =20 =20 From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of James Miller Sent: Thursday, March 07, 2013 10:58 PM To: Shane Amante Cc: Benoit Claise; Bugenhagen,Michael K; lmap@ietf.org Subject: Re: [lmap] What is broadband? =20 I agree Shane--was what I was trying to get at. :) =20 Our early FCC focus was basically the Access network but I think we're all making the point that the mix-n-match of the network measurement pieces will be driven by a particular user's use case. =20 =20 For our FCC work, the focus (discussed in more detail in our reports) was on understanding the performance in the segment under the control of a carrier delivering broadband to a given consumer. Naturally many other elements affect an end user's experience but our initial interest was on understanding that segment for the purposes of providing better information to the consumer about the portion of Internet performance that the carrier provides. The FCC's focus and history is on that piece and so it makes sense that it might begin there, but a particular carrier's focus might mirror the access network focus, but perhaps with less market-trend orientation and more diagnostic in flavor. Maybe LMAP together with other protocols (TR69 and other BBForum work for example) could benefit all those use cases. =20 In recent meetings we've discussed other more targeted mini-studies on things like CDN or in-home Wifi performance. Other potential LMAP users already focus their work on these or other elements of the network. I would envision the LMAP work could provide the glue between any of these elements and different use cases. Worth noting that much of the initial discussions in the FCC's Next-Gen group were on the potential value of standardized interfaces between such different elements of measurement infrastructure. =20 Another element that's probably worth noting is that the use cases also may direct the actual tests that someone wants to run. I would imagine the actual tests you might want to run for evaluating in-home LAN, access network, CDN, mobile networks may vary. So I think the work on the test registry and other IPPM work on new tests will also improve our definitions and understanding of "broadband performance". =20 As always my personal views and not the FCC's.. possible valued at less than $.02 :) On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante wrote: James, All, =20 On Mar 7, 2013, at 7:30 PM, James Miller wrote: I believe that Henning had commented at some point that the LMAP definition he contemplated had "architecture" as the 'A' element but certainly access is an important piece. I think one of the problems that has been discussed also on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP performance measurements would implicate elements from the user's laptop, through wireless and other local LAN, carriers access networks, Tier 1 and other peering networks, the application host side and everything in between. Clearly there would be a lot of technologies included within that functional scope. =20 I agree that access is important, but not to the exclusion of everything else that constitutes an Internet end-user customer experience or, alternatively, an Enterprise end-user customer experience -- which is what I believe you're saying above? That's why it will be important to, at some point, figure out *if*, and then how, to try to segment the portions of the end-to-end path that you describe above so we can attribute good or bad performance to a particular portion of the path so that, ultimately, the correct network operator can be contacted to look into the problem further. I do not believe that this requires us to break down the end-to-end path on a router-hop by router-hop basis, but rather we need to be able to identify 'sign posts' along the path that can correlate well to end-to-end path. =20 -shane =20 For reference, in the FCC Measuring Broadband America Program we focused on measurement from the consumers' broadband modem through the carriers network to where it connects to a tier one peering point. LMAP should be able to address the broad mix of other use cases that would have a mix of other elements and motivations. =20 Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K wrote: The word "access" should be key here as part of the definition provided we are talking about an Internet service, which is the second component. I don't really think we are building tests that won't work on smaller pipes so questioning if it really 'broadband' or not is correct IMO. =20 Sent from my iPhone On Mar 7, 2013, at 9:41 AM, "Shane Amante" wrote: Benoit,=20 =20 On Mar 7, 2013, at 3:47 AM, Benoit Claise wrote: Dear all, =09 I started to review the drafts, and I will start posting a few questions to the list. Open questions, clarifying questions, in order to generate some discussions. Disclaimer: I have not yet read the entire list archive. Apologize in advance if some points have been discussed already.=20 =09 Here is my first question. What is broadband in the LMAP context? Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology independent? And I see also "enterprise edge router", "cellular data or satellite" in draft-schulzrinne-lmap-requirements. In or out? Or do we have in mind a phase approach, starting with the "enterprise edge router" first, and then home network? =20 Speaking for the network I operate, I'm very much an advocate of saying that "Enterprise Edge Router" is "in-scope". We would very much benefit from a standards-based measurement enablement and collection regime vs. mostly proprietary, and non-scalable, approaches that exist today. =20 This is not to diminish the importance of similar test capabilities for residential broadband use-cases. We absolutely need to work on those, as well. =20 =20 With respect to priority, my hope is that we do not have to choose to prioritize one over the other. Rather, I would hope that both can be developed in parallel, because both -- at least, IMO -- have a substantially overlapping set of requirements/features.=20 =20 My $0.02, =20 -shane =20 =09 =09 =09 Interestingly, I don't know what A stands for in LMAP, if it stands for something. According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand for anything. However, looking at the different draft titles, there is some confusion. draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measurement Use Cases draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurement of Broadband Performance draft-boucadair-lmap-considerations-00, Large scale Measurement of Access network Performance (LMAP): Requirements and Issues from a Network Provider Perspective =09 Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated. =09 Regards, Benoit =09 =09 =09 _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap =20 _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap =20 =20 =20 ------_=_NextPart_001_01CE1E77.5BF7A42F Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

James,

I think mixing testing of wireless and wire technologies confuses the issues. One wireless is WiFi which is the extension of LAN = and the other is the cellular network. They both have different requirements and = two major conditions in both

         &nbs= p;      Signal Strength

         &nbs= p;      Roaming

 

have great effect on the achieved performance which does = not exist in the wire condition . Some of these performance issues might = also be better resolved in layer 2 rather than layer 3 and above. For example, = before I run any throughput test I want to make sure that I have the highest = strength signal possible be it WiFi or Cellular wireless connection ( and stay = stationary during the test). On the other hand if I am trying to find the best = location for signal strength a throughput test might not be the best tool for the = job.

 

I would suggest to keep the two separate at least for the = short term.

 

Sharam

  

 

From:= lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of = James Miller
Sent: Thursday, March 07, 2013 10:58 PM
To: Shane Amante
Cc: Benoit Claise; Bugenhagen,Michael K; lmap@ietf.org
Subject: Re: [lmap] What is broadband?

 

I agree Shane--was what I was trying to get at. = :)

 

Our early FCC focus was basically the Access = network but I think we're all making the point that the mix-n-match of the network measurement pieces will be driven by a particular user's use case. =  

 

For our FCC work, the focus (discussed in more = detail in our reports) was on understanding the performance in the segment under the = control of a carrier delivering broadband to a given consumer.  Naturally = many other elements affect an end user's experience but our initial interest = was on understanding that segment for the purposes of providing better = information to the consumer about the portion of Internet performance that the carrier provides.  The FCC's focus and history is on that piece and so it = makes sense that it might begin there, but a particular carrier's focus might = mirror the access network focus, but perhaps with less market-trend orientation and more diagnostic in flavor.  Maybe LMAP together = with other protocols (TR69 and other BBForum work for example) could benefit all = those use cases.

 

In recent meetings we've discussed other more = targeted mini-studies on things like CDN or in-home Wifi performance.  Other potential LMAP users already focus their work on these or other elements = of the network.  I would envision the LMAP work could provide the glue = between any of these elements and different use cases.  Worth noting that = much of the initial discussions in the FCC's Next-Gen group were on = the potential value of standardized interfaces between such different elements of measurement infrastructure.

 

Another element that's probably worth noting is = that the use cases also may direct the actual tests that someone wants to run. =  I would imagine the actual tests you might want to run for evaluating in-home = LAN, access network, CDN, mobile networks may vary.  So I think the work = on the test registry and other IPPM work on new tests will also improve our definitions and understanding of "broadband = performance".

 

As always my = personal views and not the FCC's.. possible valued at less than $.02 =  :)

On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante = <shane@castlepoint.net> wrote:

James, All,

 

On Mar 7, 2013, at 7:30 PM, James Miller <jamesmilleresquire@gmail.com> wrote:

I believe that Henning had commented at some point = that the LMAP definition he contemplated had "architecture" as the 'A' = element but certainly access is an important piece.  I think one of the = problems that has been discussed also on the LMAP and our FCC Next-Gen lists is = that a complete view of LMAP performance measurements would implicate elements = from the user's laptop, through wireless and other local LAN, carriers access networks, Tier 1 and other peering networks, the application host side = and everything in between.  Clearly there would be a lot of = technologies included within that functional scope.

 

I agree that access is important, but not to the = exclusion of everything else that constitutes an Internet end-user customer = experience or, alternatively, an Enterprise end-user customer experience -- which = is what I believe you're saying above?  That's why it will be important to, = at some point, figure out *if*, and then how, to try to segment the = portions of the end-to-end path that you describe above so we can attribute good or = bad performance to a particular portion of the path so that, ultimately, the correct network operator can be contacted to look into the problem = further.  I do not believe that this requires us to break down the = end-to-end path on a router-hop by router-hop basis, but rather we need to be able to = identify 'sign posts' along the path that can correlate well to end-to-end = path.

 

-shane

 



For reference, in the FCC Measuring Broadband = America Program we focused on measurement from the consumers' broadband modem = through the carriers network to where it connects to a tier one peering point.  LMAP should be able to address the broad mix of other use cases = that would have a mix of other elements and motivations.

 

Graphic in Report at = page 9. http://transition.fcc.gov/cgb/measuringbroadbandreport/= 2Methodology.pdf

On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael = K <Michael.K.Bugenhagen@centurylink.com> wrote:

The word "access" should be key here as = part of the definition provided we are talking about an Internet service, which = is the second component.   I don't really  think we are building = tests that won't work on smaller pipes so questioning if it really 'broadband' or = not is correct IMO.

 



Sent from my iPhone


On Mar 7, 2013, at 9:41 AM, "Shane Amante" <shane@castlepoint.net> wrote:

Benoit,

 

On Mar 7, 2013, at 3:47 AM, Benoit Claise <bclaise@cisco.com> wrote:

Dear all,

I started to review the drafts, and I will start posting a few questions = to the list.
Open questions, clarifying questions, in order to generate some = discussions.
Disclaimer: I have not yet read the entire list archive. Apologize in = advance if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology independent?
And I see also "enterprise edge router", "cellular data = or satellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the = "enterprise edge router" first, and then home network?

 

Speaking for the network I operate, I'm very much = an advocate of saying that "Enterprise Edge Router" is "in-scope".  We would very much benefit from a = standards-based measurement enablement and collection regime vs. mostly proprietary, and non-scalable, approaches that exist today.

 

This is not to diminish the importance of similar = test capabilities for residential broadband use-cases.  We absolutely = need to work on those, as well.  

 

With respect to priority, my hope is that we do not = have to choose to prioritize one over the other.  Rather, I would hope that = both can be developed in parallel, because both -- at least, IMO -- have a substantially overlapping set of = requirements/features. 

 

My $0.02,

 

-shane

 



Interestingly, I don't know what A stands for in = LMAP, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart,= the A doesn't stand for anything.
However, looking at the different draft titles, there is some = confusion.

   draft-linsner-lmap-use-cases-0 title is Large-Scale =
Broadband Measurement Use Cases
   =
draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale =
Measurement of Broadband Performance
   =
draft-boucadair-lmap-considerations-00, Large scale Measurement of =
Access network Performance =
(LMAP):
      =
Requirements and Issues from a Network Provider =
Perspective


Some more discussions, on the mailing list or during the BoF, on this = topic would be appreciated.

Regards, Benoit


_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

 

_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap


_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

 

 

 

------_=_NextPart_001_01CE1E77.5BF7A42F-- From philip.eardley@bt.com Mon Mar 11 10:46:54 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B81611E80CC for ; Mon, 11 Mar 2013 10:46:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.311 X-Spam-Level: X-Spam-Status: No, score=-103.311 tagged_above=-999 required=5 tests=[AWL=0.288, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zGvgtjRVS3Jp for ; Mon, 11 Mar 2013 10:46:53 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 9689C11E80DE for ; Mon, 11 Mar 2013 10:46:51 -0700 (PDT) Received: from EVMHT67-UKRD.domain1.systemhost.net (10.36.3.104) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.279.1; Mon, 11 Mar 2013 17:46:50 +0000 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.43]) by EVMHT67-UKRD.domain1.systemhost.net ([10.36.3.104]) with mapi; Mon, 11 Mar 2013 17:46:50 +0000 From: To: , Date: Mon, 11 Mar 2013 17:46:49 +0000 Thread-Topic: [lmap] Which devices does LMAP target? Thread-Index: Ac4ebZn9zYsHOLyYQ8idqbasPkc0TgAES6nz Message-ID: <9510D26531EF184D9017DF24659BB87F343774C51F@EMV65-UKRD.domain1.systemhost.net> References: <5138763B.1000100@cisco.com>, <22981BAB-FED3-4E41-B8D7-C5AFD0934D38@cdt.org> In-Reply-To: <22981BAB-FED3-4E41-B8D7-C5AFD0934D38@cdt.org> Accept-Language: en-US, en-GB Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: lmap@ietf.org Subject: Re: [lmap] Which devices does LMAP target? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 17:46:54 -0000 personally I hope LMAP functionality can be embedded in lots of edge and en= d devices. this drives the need for large-scale and simplicity of the standardised int= erfaces (between Controller & Measurement Agent and MA-Collector). I guess that the complexity can be beyond LMAP - the 'sophisticated' or 'ex= tensions' are in the Controller (how it decides what tests should be done, = when) and in the tools that interpret the Collected results. But this guess= certainly needs checking /figuring out, as Alissa says, during the prospec= tive WG phil ________________________________________ From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of Alissa Coo= per [acooper@cdt.org] Sent: 11 March 2013 15:32 To: Benoit Claise Cc: lmap@ietf.org Subject: Re: [lmap] Which devices does LMAP target? FWIW, this strikes me as a fairly important question for understanding the = extent to which existing protocols/building blocks can be reused for LMAP. = If we expect a measurement agent to be embedded or embeddable in every chea= p piece of CPE out there, that may constrain the complexity of the protocol= logic for both receiving test schedules and reporting test results. Whethe= r we want all measurement agent implementations to be so constrained and/o= r whether an extensibility mechanism is possible that allows for more sophi= sticated logic in more sophisticated devices (PC, tablet, smartphone, etc.)= would be good to figure out. Alissa On Mar 7, 2013, at 6:12 AM, Benoit Claise wrote: > Dear all, > > draft-schulzrinne-lmap-requirements-00 mentions: > > Thus, this document proposes > an overall architecture, with emphasis on the functional and security > requirements for the protocols connecting the elements of the > architecture, that will make it possible to build measurement > capabilities into home and enterprise edge routers, personal > computers, mobile devices and other edge devices. > > Does LMAP really target my home devices? My set top box, PC, tablet, smar= tphone > Or is LMAP is limited to the modem/router? > And if this is only the modem/router, do we want to make a difference bet= ween an ISP managed device (typically a branch office) and my own private d= evice? > > Some more discussions, on the mailing list or during the BoF, on this top= ic would be appreciated. > > Regards, Benoit > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap= From j.schoenwaelder@jacobs-university.de Mon Mar 11 10:56:05 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1300521F8BCF for ; Mon, 11 Mar 2013 10:56:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.132 X-Spam-Level: X-Spam-Status: No, score=-103.132 tagged_above=-999 required=5 tests=[AWL=0.117, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CUvf4Kw+uTY1 for ; Mon, 11 Mar 2013 10:56:04 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 4714821F8BD0 for ; Mon, 11 Mar 2013 10:56:04 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 9DBBE20C15; Mon, 11 Mar 2013 18:56:03 +0100 (CET) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id SZyAkfG2mFzh; Mon, 11 Mar 2013 18:56:03 +0100 (CET) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 31ABE20BD7; Mon, 11 Mar 2013 18:56:03 +0100 (CET) Received: by elstar.local (Postfix, from userid 501) id F181024E6D7A; Mon, 11 Mar 2013 18:56:14 +0100 (CET) Date: Mon, 11 Mar 2013 18:56:14 +0100 From: Juergen Schoenwaelder To: Alissa Cooper Message-ID: <20130311175614.GA62652@elstar.local> Mail-Followup-To: Alissa Cooper , Benoit Claise , "lmap@ietf.org" References: <5138763B.1000100@cisco.com> <22981BAB-FED3-4E41-B8D7-C5AFD0934D38@cdt.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <22981BAB-FED3-4E41-B8D7-C5AFD0934D38@cdt.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Benoit Claise , "lmap@ietf.org" Subject: Re: [lmap] Which devices does LMAP target? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 17:56:05 -0000 On Mon, Mar 11, 2013 at 11:32:04AM -0400, Alissa Cooper wrote: > FWIW, this strikes me as a fairly important question for understanding the extent to which existing protocols/building blocks can be reused for LMAP. If we expect a measurement agent to be embedded or embeddable in every cheap piece of CPE out there, that may constrain the complexity of the protocol logic for both receiving test schedules and reporting test results. Whether we want all measurement agent implementations to be so constrained and/or whether an extensibility mechanism is possible that allows for more sophisticated logic in more sophisticated devices (PC, tablet, smartphone, etc.) would be good to figure out. > Alissa, can you give an example where measurement agents running on more sophisticated devices may be doing something substantially different compared to measurement agents running on embedded hardware? I am looking for a motivational example to understand why additional features on the control and reporting protocol would be needed for different classes of devices. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From jamesmilleresquire@gmail.com Mon Mar 11 11:01:21 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5442111E81A1 for ; Mon, 11 Mar 2013 11:01:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.348 X-Spam-Level: X-Spam-Status: No, score=-3.348 tagged_above=-999 required=5 tests=[AWL=0.250, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4Y-i4pN5rfGN for ; Mon, 11 Mar 2013 11:01:20 -0700 (PDT) Received: from mail-wg0-f45.google.com (mail-wg0-f45.google.com [74.125.82.45]) by ietfa.amsl.com (Postfix) with ESMTP id C952411E8196 for ; Mon, 11 Mar 2013 11:01:19 -0700 (PDT) Received: by mail-wg0-f45.google.com with SMTP id dq12so5402038wgb.24 for ; Mon, 11 Mar 2013 11:01:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=fP0UFM04+2F0hA0LoO+9t1tCd8hWOp1J8H3ZrvZOry4=; b=fmdgA4LN/yKhA9Oel04R4MjBWgX3P4U7pfB9KA4/h/VyMJRxYi7EusJSQgSK+QiJtL OGOC95D+kziNjqvYIIMDgNZPSpAH3FjEdOZ6uP6EC63kbPRkAqyQdqF6lmGEGiuEzHCK 1/JaOaSQ7KgsduJHh/s1Y/LHfAlO8aHxOj4Aw0u7hJ1lyl/uaxtzq07oCy6zBArUiUDU fiWEjMELi1q/SV4YKKGZ+KOGz5lvpXw3Rv68De9twPuMk0C/iA2L5BTYFKYL+nDwNVny SvaAt0hofitjbFHk42D93ii8DGjIBAvFQXtyjPSb2XfTC04LGPWbmgitHMbN+ts702P3 QVGw== X-Received: by 10.194.123.130 with SMTP id ma2mr4891375wjb.46.1363024876436; Mon, 11 Mar 2013 11:01:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.180.24.10 with HTTP; Mon, 11 Mar 2013 11:00:46 -0700 (PDT) In-Reply-To: <22981BAB-FED3-4E41-B8D7-C5AFD0934D38@cdt.org> References: <5138763B.1000100@cisco.com> <22981BAB-FED3-4E41-B8D7-C5AFD0934D38@cdt.org> From: James Miller Date: Mon, 11 Mar 2013 14:00:46 -0400 Message-ID: To: Alissa Cooper Content-Type: multipart/alternative; boundary=089e012287de94853a04d7a9f508 Cc: Benoit Claise , lmap@ietf.org Subject: Re: [lmap] Which devices does LMAP target? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 18:01:21 -0000 --089e012287de94853a04d7a9f508 Content-Type: text/plain; charset=ISO-8859-1 I mention for reference that SamKnows has been doing Android Smartphone based mobile testing in other jurisdictions and they are also supporting our FCC work to deploy a mobile program based on a client application running on the handset to do upload, download, latency (UDP) and packet loss running against measurement servers also used with fixed testing. The testing schedule and payloads were (among other things) elements that evolved from the fixed program, as Alissa suggests. We are moving towards modified approaches to pushing data up from the client to the data stores because of changes we made in our privacy review. We will be likely moving from the CSV data format with a whitebox specific host id to a JSON flat file with active test metric and other context information saved without any unique identifier. So as Alissa suggests, from my experience we did end up moving toward modifying both test scheduling, payload size or other changes to the test metric, and data packaging and saving out to the network. I would think there are IPPM and LMAP issues that would be implicated by the scope extending edge to edge. Outside of that we've discussed in-home as an area of interest and I think there's expectations among some in the space that in-home elements might interact with standards based measurement. Again my personal views, but hope that's helpful.. On Mon, Mar 11, 2013 at 11:32 AM, Alissa Cooper wrote: > FWIW, this strikes me as a fairly important question for understanding the > extent to which existing protocols/building blocks can be reused for LMAP. > If we expect a measurement agent to be embedded or embeddable in every > cheap piece of CPE out there, that may constrain the complexity of the > protocol logic for both receiving test schedules and reporting test > results. Whether we want all measurement agent implementations to be so > constrained and/or whether an extensibility mechanism is possible that > allows for more sophisticated logic in more sophisticated devices (PC, > tablet, smartphone, etc.) would be good to figure out. > > Alissa > > On Mar 7, 2013, at 6:12 AM, Benoit Claise wrote: > > > Dear all, > > > > draft-schulzrinne-lmap-requirements-00 mentions: > > > > Thus, this document proposes > > an overall architecture, with emphasis on the functional and security > > requirements for the protocols connecting the elements of the > > architecture, that will make it possible to build measurement > > capabilities into home and enterprise edge routers, personal > > computers, mobile devices and other edge devices. > > > > Does LMAP really target my home devices? My set top box, PC, tablet, > smartphone > > Or is LMAP is limited to the modem/router? > > And if this is only the modem/router, do we want to make a difference > between an ISP managed device (typically a branch office) and my own > private device? > > > > Some more discussions, on the mailing list or during the BoF, on this > topic would be appreciated. > > > > Regards, Benoit > > > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap > > > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > --089e012287de94853a04d7a9f508 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I mention for refere= nce that SamKnows has been doing Android Smartphone based mobile testing in= other jurisdictions and they are also supporting our FCC work to deploy a = mobile program based on a client application running on the handset to do u= pload, download, latency (UDP) and packet loss running against measurement = servers also used with fixed testing. =A0

The testing= schedule and payloads were (among other things) elements that evolved from= the fixed program, as Alissa suggests.

We are moving towards modified approaches to pushing data up from the c= lient to the data stores because of changes we made in our privacy review. = =A0We will be likely moving from the CSV data format with a whitebox specif= ic host id to a JSON flat file with active test metric and other context in= formation saved without any unique identifier.=A0

So as Alissa suggests, from my experience we did end up movi= ng toward modifying both test scheduling, payload size or other changes to = the test metric, and data packaging and saving out to the network. =A0I wou= ld think there are IPPM and LMAP issues that would be implicated by the sco= pe extending edge to edge.

Outside of that we've discussed in-home as an area = of interest and I think there's expectations among some in the space th= at in-home elements might interact with standards based measurement.

Again my personal views, but hope that's helpful..<= /div>

On Mon, Mar 11, 2013 at 11:32 AM,= Alissa Cooper <acooper@cdt.org> wrote:
FWIW, this strikes me as a fairly important = question for understanding the extent to which existing protocols/building = blocks can be reused for LMAP. If we expect a measurement agent to be embed= ded or embeddable in every cheap piece of CPE out there, that may constrain= the complexity of the protocol logic for both receiving test schedules and= reporting test results. Whether we want =A0all measurement agent implement= ations to be so constrained and/or whether an extensibility mechanism is po= ssible that allows for more sophisticated logic in more sophisticated devic= es (PC, tablet, smartphone, etc.) would be good to figure out.

Alissa

On Mar 7, 2013, at 6:12 AM, Benoit Claise <bclaise@cisco.com> wrote:

> Dear all,
>
> draft-schulzrinne-lmap-requirements-00 mentions:
>
> =A0 Thus, this document proposes
> =A0 an overall architecture, with emphasis on the functional and secur= ity
> =A0 requirements for the protocols connecting the elements of the
> =A0 architecture, that will make it possible to build measurement
> =A0 capabilities into home and enterprise edge routers, personal
> =A0 computers, mobile devices and other edge devices.
>
> Does LMAP really target my home devices? My set top box, PC, tablet, s= martphone
> Or is LMAP is limited to the modem/router?
> And if this is only the modem/router, do we want to make a difference = between an ISP managed device (typically a branch office) and my own privat= e device?
>
> Some more discussions, on the mailing list or during the BoF, on this = topic would be appreciated.
>
> Regards, Benoit
>
> _______________________________________________
> lmap mailing list
> lmap@ietf.org
> https://www.ietf.org/mailman/listinfo/lmap
>


_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap

--089e012287de94853a04d7a9f508-- From philip.eardley@bt.com Mon Mar 11 11:03:54 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 48DA511E81B7 for ; Mon, 11 Mar 2013 11:03:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.068 X-Spam-Level: X-Spam-Status: No, score=-103.068 tagged_above=-999 required=5 tests=[AWL=-0.070, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_26=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HxJtIfC-PItp for ; Mon, 11 Mar 2013 11:03:53 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 95C1D11E819D for ; Mon, 11 Mar 2013 11:03:52 -0700 (PDT) Received: from EVMHT69-UKRD.domain1.systemhost.net (10.36.3.129) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.279.1; Mon, 11 Mar 2013 18:03:51 +0000 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.43]) by EVMHT69-UKRD.domain1.systemhost.net ([10.36.3.129]) with mapi; Mon, 11 Mar 2013 18:03:51 +0000 From: To: , Date: Mon, 11 Mar 2013 18:03:51 +0000 Thread-Topic: [lmap] Which performance metrics for LMAP? Thread-Index: AQHOGybaVh0e7cNwrUuMBtZyDyvaL5iazHk4gAYACVs= Message-ID: <9510D26531EF184D9017DF24659BB87F343774C520@EMV65-UKRD.domain1.systemhost.net> References: <513879B5.8030609@cisco.com>, In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F343774C520EMV65UKRDdoma_" MIME-Version: 1.0 Cc: lmap@ietf.org Subject: Re: [lmap] Which performance metrics for LMAP? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 18:03:54 -0000 --_000_9510D26531EF184D9017DF24659BB87F343774C520EMV65UKRDdoma_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable agree with this, also, i think the work of lmap is on the Controller-MA and MA-Collector int= erface (MA =3D MEasurement Agent) for instance, Controller tells MA "Run test X to MA-2 every hour and report= results to Collector-5 once per day" "test X" is defined in a registry. IPPM will hopefully be re-chartered to d= efine a registry of IPPM tests. but I can see sometimes you might want "test X" to refer to some non-IPPM t= est - possibly another stds organisation could define such a registry? phil ________________________________ From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of Bugenhagen= , Michael K [Michael.K.Bugenhagen@centurylink.com] Sent: 07 March 2013 22:16 To: Benoit Claise Cc: lmap@ietf.org Subject: Re: [lmap] Which performance metrics for LMAP? Benoit, I think that the requirement to"future proof" the test framework in = order to ensure it will support any future test we throw at it requires tha= t the test framework be able to support both service commissioning (Through= put) type tests, as well as other SLA like tracking tests such as Twamp and= the Ethernet OAM, and even service level testing. So essentially they s= hould all fit inside the framework M Sent from my iPhone On Mar 7, 2013, at 5:28 AM, "Benoit Claise" > wrote: Dear all, Another clarifying question: I'm not too sure which performance metric type= s LMAP targets. draft-linsner-lmap-use-cases-00 mentions: It is assumed that measurement tests run by OTT providers would only include the metrics associated with layer 3 and up The different drafts use different terms: service, end user experience, qua= lity of experience. Does Quality of Experience imply performance metrics per application, per 5= tuple (IP addresses, ports, protocol), per DCSP? Or do we want to stay at layer 3 performance metrics, typically IPPM type o= f metrics? If the latter, then LMAP is about large scale deployment of IPPM, right? While I fully understand the business needs for "and up" in "include the me= trics associated with layer 3 and up", let's not boil the ocean. Performance Metrics at Other Layer (PMOL), a concluded WG, tried and it proved to be difficult. Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated. Regards, Benoit _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_9510D26531EF184D9017DF24659BB87F343774C520EMV65UKRDdoma_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
agree with this,
 
also, i think the work of lmap is on the = Controller-MA and MA-Collector interface (MA =3D MEasurement Agent)
for instance, Controller tells MA "Run test= X to MA-2 every hour and report results to Collector-5 once per = day"
"test X" is defined in a registry. IPP= M will hopefully be re-chartered to define a registry of IPPM tests.=
but I can see sometimes you might want "tes= t X" to refer to some non-IPPM test - possibly another stds organisati= on could define such a registry?
 
phil
From: lmap-bounce= s@ietf.org [lmap-bounces@ietf.org] On Behalf Of Bugenhagen, Michael K [Mich= ael.K.Bugenhagen@centurylink.com]
Sent: 07 March 2013 22:16
To: Benoit Claise
Cc: lmap@ietf.org
Subject: Re: [lmap] Which performance metrics for LMAP?

Benoit,

       I think that the requirement to"future= proof" the test framework in order to ensure it will support any futu= re test we throw at it requires that the test framework be able to support = both service commissioning (Throughput) type tests, as well as other SLA like tracking tests such as Twamp and the Ethernet OAM, and even= service level testing.    So essentially they should all fit ins= ide the framework

M

Sent from my iPhone

On Mar 7, 2013, at 5:28 AM, "Benoit Claise" <bclaise@cisco.com> wrote:

Dear all,

Another clarifying question: I'm not too sure which performance metric type= s LMAP targets.
draft-linsner-lmap-use-cases-00 mentions:
   It is assumed that measurement
   tests run by OTT providers would only include the metrics associated
   with layer 3 and up

The different drafts use different terms: service, end user experience, qua= lity of experience.
Does Quality of Experience imply performance metrics per application, per 5= tuple (IP addresses, ports, protocol), per DCSP?
Or do we want to stay at layer 3 performance metrics, typically IPPM type o= f metrics?
If the latter, then LMAP is about large scale deployment of IPPM, right?
While I fully understand the business needs for "and up" in "= ;include the metrics associated with layer 3 and up", let's not boil t= he ocean.
Performance Metrics at Other Layer (PMOL), a concluded WG, tried and it= proved to be difficult.

Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated.

Regards, Benoit
_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap
--_000_9510D26531EF184D9017DF24659BB87F343774C520EMV65UKRDdoma_-- From mlinsner@cisco.com Mon Mar 11 11:37:22 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 663BF21F8E2C for ; Mon, 11 Mar 2013 11:37:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -7.962 X-Spam-Level: X-Spam-Status: No, score=-7.962 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, RCVD_IN_DNSWL_HI=-8, SARE_LWSHORTT=1.24] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c0-znj+PcM8j for ; Mon, 11 Mar 2013 11:37:17 -0700 (PDT) Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) by ietfa.amsl.com (Postfix) with ESMTP id E381F21F8E22 for ; Mon, 11 Mar 2013 11:37:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=29406; q=dns/txt; s=iport; t=1363027037; x=1364236637; h=date:subject:from:to:cc:message-id:in-reply-to: mime-version; bh=Gl0E3tM3ILWuYIKPdHef+7mBhSNxUFQQJdrDdetYDTA=; b=Y8IOYiK01xlheYHMN/+5zUToFv/FMS/FNQ7QQfhRQGORhwcgVhrUugWe 0l/rp188AoWNkKeQCGDEKcB4Flpm6wFQO6LwE0BqwDi7e2Y5rbH7oQcff ONNBjv555G/c7GQvb6ZrmfdSY6apsxVwQrCaF3ajMSi07iUbdyg/XmvxT Y=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhoFACUjPlGtJV2a/2dsb2JhbAA5CoQbdq19iSeILXNsFnSCKQEBAQIBAQEBASoQKgcLBQ4IEQEDAQEBIAcoBh8DBggGDgWIAQMJBgy1EA2JW4xGgQ2BMAcEBgECgz4DkxCBZYFggR6EXIVihRmDJiA X-IronPort-AV: E=Sophos;i="4.84,825,1355097600"; d="scan'208,217";a="186209447" Received: from rcdn-core-3.cisco.com ([173.37.93.154]) by rcdn-iport-7.cisco.com with ESMTP; 11 Mar 2013 18:37:15 +0000 Received: from [10.82.217.152] (rtp-vpn3-406.cisco.com [10.82.217.152]) by rcdn-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r2BIbDqV012474; Mon, 11 Mar 2013 18:37:14 GMT User-Agent: Microsoft-MacOutlook/14.2.5.121010 Date: Mon, 11 Mar 2013 14:37:14 -0400 From: Marc Linsner To: Sharam Hakimi Message-ID: Thread-Topic: [lmap] What is broadband? In-Reply-To: <084CDC75FEC1E640B60338273BEACDFA023C47EE@spboexc01.exfo.com> Mime-version: 1.0 Content-type: multipart/alternative; boundary="B_3445857436_298655" Cc: lmap@ietf.org Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 18:37:22 -0000 > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3445857436_298655 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sharam, IMO, the only difference between wireless and wired are the test metrics that can be performed. LMAP is concerned with the management of the measurement clients, controllers, and collectors. Until further thought proves otherwise, I don't see any difference in the management of devices on any of the layer 1/2s. The specifics you mention, Signal Strength and Roaming, should be dealt with in the definition of the test metric. -Marc- > James, > I think mixing testing of wireless and wire technologies confuses the issues. > One wireless is WiFi which is the extension of LAN and the other is the > cellular network. They both have different requirements and two major > conditions in both > Signal Strength > Roaming > > have great effect on the achieved performance which does not exist in the wire > condition . Some of these performance issues might also be better resolved in > layer 2 rather than layer 3 and above. For example, before I run any > throughput test I want to make sure that I have the highest strength signal > possible be it WiFi or Cellular wireless connection ( and stay stationary > during the test). On the other hand if I am trying to find the best location > for signal strength a throughput test might not be the best tool for the job. > > I would suggest to keep the two separate at least for the short term. > > Sharam > > > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of James > Miller > Sent: Thursday, March 07, 2013 10:58 PM > To: Shane Amante > Cc: Benoit Claise; Bugenhagen,Michael K; lmap@ietf.org > Subject: Re: [lmap] What is broadband? > > I agree Shane--was what I was trying to get at. :) > > > > Our early FCC focus was basically the Access network but I think we're all > making the point that the mix-n-match of the network measurement pieces will > be driven by a particular user's use case. > > > > For our FCC work, the focus (discussed in more detail in our reports) was on > understanding the performance in the segment under the control of a carrier > delivering broadband to a given consumer. Naturally many other elements > affect an end user's experience but our initial interest was on understanding > that segment for the purposes of providing better information to the consumer > about the portion of Internet performance that the carrier provides. The > FCC's focus and history is on that piece and so it makes sense that it might > begin there, but a particular carrier's focus might mirror the access network > focus, but perhaps with less market-trend orientation and more diagnostic in > flavor. Maybe LMAP together with other protocols (TR69 and other BBForum work > for example) could benefit all those use cases. > > > > In recent meetings we've discussed other more targeted mini-studies on things > like CDN or in-home Wifi performance. Other potential LMAP users already > focus their work on these or other elements of the network. I would envision > the LMAP work could provide the glue between any of these elements and > different use cases. Worth noting that much of the initial discussions in the > FCC's Next-Gen group were on the potential value of standardized interfaces > between such different elements of measurement infrastructure. > > > > Another element that's probably worth noting is that the use cases also may > direct the actual tests that someone wants to run. I would imagine the actual > tests you might want to run for evaluating in-home LAN, access network, CDN, > mobile networks may vary. So I think the work on the test registry and other > IPPM work on new tests will also improve our definitions and understanding of > "broadband performance". > > > > As always my personal views and not the FCC's.. possible valued at less than > $.02 :) > > On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante wrote: > > James, All, > > > > On Mar 7, 2013, at 7:30 PM, James Miller wrote: >> I believe that Henning had commented at some point that the LMAP definition >> he contemplated had "architecture" as the 'A' element but certainly access is >> an important piece. I think one of the problems that has been discussed also >> on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP >> performance measurements would implicate elements from the user's laptop, >> through wireless and other local LAN, carriers access networks, Tier 1 and >> other peering networks, the application host side and everything in between. >> Clearly there would be a lot of technologies included within that functional >> scope. > > > I agree that access is important, but not to the exclusion of everything else > that constitutes an Internet end-user customer experience or, alternatively, > an Enterprise end-user customer experience -- which is what I believe you're > saying above? That's why it will be important to, at some point, figure out > *if*, and then how, to try to segment the portions of the end-to-end path that > you describe above so we can attribute good or bad performance to a particular > portion of the path so that, ultimately, the correct network operator can be > contacted to look into the problem further. I do not believe that this > requires us to break down the end-to-end path on a router-hop by router-hop > basis, but rather we need to be able to identify 'sign posts' along the path > that can correlate well to end-to-end path. > > > > -shane > > > > > > For reference, in the FCC Measuring Broadband America Program we focused on > measurement from the consumers' broadband modem through the carriers network > to where it connects to a tier one peering point. LMAP should be able to > address the broad mix of other use cases that would have a mix of other > elements and motivations. > > > > Graphic in Report at page 9. > http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf > > On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K > wrote: > > The word "access" should be key here as part of the definition provided we are > talking about an Internet service, which is the second component. I don't > really think we are building tests that won't work on smaller pipes so > questioning if it really 'broadband' or not is correct IMO. > > > > > > Sent from my iPhone > > > On Mar 7, 2013, at 9:41 AM, "Shane Amante" wrote: >> >> Benoit, >> >> >> >> On Mar 7, 2013, at 3:47 AM, Benoit Claise wrote: >>> >>> Dear all, >>> >>> I started to review the drafts, and I will start posting a few questions to >>> the list. >>> Open questions, clarifying questions, in order to generate some discussions. >>> Disclaimer: I have not yet read the entire list archive. Apologize in >>> advance if some points have been discussed already. >>> >>> Here is my first question. What is broadband in the LMAP context? >>> Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology independent? >>> And I see also "enterprise edge router", "cellular data or satellite" in >>> draft-schulzrinne-lmap-requirements. In or out? >>> Or do we have in mind a phase approach, starting with the "enterprise edge >>> router" first, and then home network? >> >> >> Speaking for the network I operate, I'm very much an advocate of saying that >> "Enterprise Edge Router" is "in-scope". We would very much benefit from a >> standards-based measurement enablement and collection regime vs. mostly >> proprietary, and non-scalable, approaches that exist today. >> >> >> >> This is not to diminish the importance of similar test capabilities for >> residential broadband use-cases. We absolutely need to work on those, as >> well. >> >> >> >> With respect to priority, my hope is that we do not have to choose to >> prioritize one over the other. Rather, I would hope that both can be >> developed in parallel, because both -- at least, IMO -- have a substantially >> overlapping set of requirements/features. >> >> >> >> My $0.02, >> >> >> >> -shane >> >> >> >> >> >> Interestingly, I don't know what A stands for in LMAP, if it stands for >> something. >> According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A >> doesn't stand for anything. >> However, looking at the different draft titles, there is some confusion. >> draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measurement >> Use Cases >> draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale >> Measurement of Broadband Performance >> draft-boucadair-lmap-considerations-00, Large scale Measurement of Access >> network Performance (LMAP): >> Requirements and Issues from a Network Provider Perspective >> >> Some more discussions, on the mailing list or during the BoF, on this topic >> would be appreciated. >> >> Regards, Benoit >> >> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > > > > _______________________________________________ lmap mailing list > lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --B_3445857436_298655 Content-type: text/html; charset="US-ASCII" Content-transfer-encoding: quoted-printable
Sharam,

=
IMO, the only difference between wireless and wired are the test metric= s that can be performed.  LMAP is concerned with the management of the = measurement clients, controllers, and collectors.  Until further though= t proves otherwise, I don't see any difference in the management of devices = on any of the layer 1/2s.

The specifics you mention= , Signal Strength and Roaming, should be dealt with in the definition of the= test metric.

-Marc-


<= /div>

James,

I think mixing testing of= wireless and wire technologies confuses the issues. One wireless is WiFi which is the extension of LAN and= the other is the cellular network. They both have different requirements and tw= o major conditions in both

           &nb= sp;    Signal Strength

 &nb= sp;            &= nbsp; Roaming

=  

have great effect= on the achieved performance which does not exist in the wire condition . Some of these performance issues might also b= e better resolved in layer 2 rather than layer 3 and above. For example, befo= re I run any throughput test I want to make sure that I have the highest strengt= h signal possible be it WiFi or Cellular wireless connection ( and stay stationary during the test). On the other hand if I am trying to find the best locatio= n for signal strength a throughput test might not be the best tool for the jo= b.

 

I would suggest to keep the = two separate at least for the short term.

 <= /o:p>

Sharam=

  

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf= Of James Miller
Sent: Thursday, March 07, 2013 10:58 PM
To: Shan= e Amante
Cc: Benoit Claise; Bugenhagen,Michael K; lmap@ietf.org
Subject: Re: [lmap] What is broad= band?

 

<= p class=3D"MsoNormal">I agree Shane--was what I was trying to get at. :)<= /o:p>

 

Our early FCC focus was basically the Access network but I think we're all making the point that the mix-n-match of the network measurement pieces will be driven by a particular user's use case.  

 

For our FCC work, the focus (discussed in more detail = in our reports) was on understanding the performance in the segment under the cont= rol of a carrier delivering broadband to a given consumer.  Naturally many= other elements affect an end user's experience but our initial interest was= on understanding that segment for the purposes of providing better information= to the consumer about the portion of Internet performance that the carrier provides.  The FCC's focus and history is on that piece and so it make= s sense that it might begin there, but a particular carrier's focus might mir= ror the access network focus, but perhaps with less market-trend orientation and more diagnostic in flavor.  Maybe LMAP together with o= ther protocols (TR69 and other BBForum work for example) could benefit all those= use cases.

 

<= /div>

In recent meetings we've discussed other more= targeted mini-studies on things like CDN or in-home Wifi performance.  Other potential LMAP users already focus their work on these or other elements of= the network.  I would envision the LMAP work could provide the glue betwee= n any of these elements and different use cases.  Worth noting that much= of the initial discussions in the FCC's Next-Gen group were on the potent= ial value of standardized interfaces between such different elements of measurement infrastructure.

<= o:p> 

Another element that's p= robably worth noting is that the use cases also may direct the actual tests that someone wants to run.  I w= ould imagine the actual tests you might want to run for evaluating in-home LAN, access network, CDN, mobile networks may vary.  So I think the work on= the test registry and other IPPM work on new tests will also improve our definitions and understanding of "broadband performance".

 

As always my personal views and not the FCC's.. possible valued at less than $.02  :)

On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante <shane@castlepoint.net> wrote:

James, All,

 

On Mar 7, 2013, at 7:30 PM, James Miller <jamesmilleresquire@gmail.com> wrote:

I believe that Henning had commented at some p= oint that the LMAP definition he contemplated had "architecture" as the 'A' element but certainly access is an important piece.  I think one of the proble= ms that has been discussed also on the LMAP and our FCC Next-Gen lists is that= a complete view of LMAP performance measurements would implicate elements fro= m the user's laptop, through wireless and other local LAN, carriers access networks, Tier 1 and other peering networks, the application host side and everything in between.  Clearly there would be a lot of technologies included within that functional scope.

 

I agree that access is important, but not to the exclusion of everything else that constitutes an Internet end-user customer experienc= e or, alternatively, an Enterprise end-user customer experience -- which is w= hat I believe you're saying above?  That's why it will be important to, at= some point, figure out *if*, and then how, to try to segment the portions o= f the end-to-end path that you describe above so we can attribute good or bad= performance to a particular portion of the path so that, ultimately, the correct network operator can be contacted to look into the problem further.=  I do not believe that this requires us to break down the end-to-end p= ath on a router-hop by router-hop basis, but rather we need to be able to ident= ify 'sign posts' along the path that can correlate well to end-to-end path.

=  

-shane

 



For reference, in the FCC Measuring Broadband Ame= rica Program we focused on measurement from the consumers' broadband modem throu= gh the carriers network to where it connects to a tier one peering point.  LMAP should be able to address the broad mix of other use cases that would have a mix of other elements and motivations.

 

Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadb= andreport/2Methodology.pdf

On Th= u, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K <Michael.K.Bugenhagen@centuryl= ink.com> wrote:

The word "access" shoul= d be key here as part of the definition provided we are talking about an Internet service, which is = the second component.   I don't really  think we are building tests t= hat won't work on smaller pipes so questioning if it really 'broadband' or not = is correct IMO.

 



Sent from my iPhone


On Mar 7, 2013, at 9:41 AM, "Shane Amante" <shane@castlepoint.net> wrote:

Benoit,

 

On Mar 7, 2013, = at 3:47 AM, Benoit Claise <bclaise@cisco.com> wrote:

Dear all,

I started to review the drafts, and I will start posting a few questions to= the list.
Open questions, clarifying questions, in order to generate some discussions= .
Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology independent?
And I see also "enterprise edge router", "cellular data or satellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise edge router" first, and then home network?

 

Speaki= ng for the network I operate, I'm very much an advocate of saying that "Enterprise Edge Router" is "in-scope".  We would very much benefit from a standards-based measurement enablement and collection regime vs. mostly proprietary, and non-scalable, approaches that exist today.

 

This is n= ot to diminish the importance of similar test capabilities for residential broadband use-cases.  We absolutely need = to work on those, as well.  

 

With respect to prior= ity, my hope is that we do not have to choose to prioritize one over the other.  Rather, I would hope that bo= th can be developed in parallel, because both -- at least, IMO -- have a substantially overlapping set of requirements/features. 

 

My $0.02,

 = ;

-shane

<= p class=3D"MsoNormal"> 


Interestingly, I don't know wh= at A stands for in LMAP, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A= doesn't stand for anything.
However, looking at the different draft titles, there is some confusion.

   draft-linsner-lmap-use-cases-0 title is Large-=
Scale Broadband Measurement Use Cases
   draf=
t-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurement of B=
roadband Performance
   draft-boucadair-lmap-=
considerations-00, Large scale Measurement of Access network Performance =
(LMAP):
      Requirements=
 and Issues from a Network Provider Perspective


Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated.

Regards, Benoit


______= _________________________________________
lmap mailing list
lmap@ie= tf.org
https://www.ietf.org/mailman/listinfo/lmap

 

____= ___________________________________________
lmap mailing list
lmap@ie= tf.org
https://www.ietf.org/mailman/listinfo/lmap


_______________________________________________
lmap mailing list
lmap@ie= tf.org
https://www.ietf.org/mailman/listinfo/lmap

 

 

&n= bsp;

_______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/m= ailman/listinfo/lmap --B_3445857436_298655-- From sharam.hakimi@exfo.com Mon Mar 11 13:28:03 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5B20B21F902C for ; Mon, 11 Mar 2013 13:28:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.358 X-Spam-Level: X-Spam-Status: No, score=-1.358 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_LWSHORTT=1.24] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HUAX22TuVf+A for ; Mon, 11 Mar 2013 13:28:01 -0700 (PDT) Received: from smtpinqc.exfo.com (smtpinqc.exfo.com [206.162.164.97]) by ietfa.amsl.com (Postfix) with ESMTP id 7999D21F9030 for ; Mon, 11 Mar 2013 13:28:00 -0700 (PDT) Received: from spqcexc04.exfo.com ([172.16.48.171]) by smtpinqc.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 11 Mar 2013 16:27:59 -0400 Received: from spboexc01.exfo.com ([10.10.10.16]) by spqcexc04.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 11 Mar 2013 16:27:59 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CE1E96.E50B0637" Date: Mon, 11 Mar 2013 16:28:39 -0400 Message-ID: <084CDC75FEC1E640B60338273BEACDFA023C4894@spboexc01.exfo.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [lmap] What is broadband? Thread-Index: Ac4eh3qhjz8d/pn1RDqvK//lPDgFAQAD2xrA References: From: "Sharam Hakimi" To: "Marc Linsner" X-OriginalArrivalTime: 11 Mar 2013 20:27:59.0041 (UTC) FILETIME=[EBCE3310:01CE1E96] Cc: lmap@ietf.org Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 20:28:03 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01CE1E96.E50B0637 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Marc, I think the management agent that a wireless device might use would be somewhat different than a wire device as there are metrics that a wireless device has, that do not exist in a wire device. My opinion. =20 Sharam =20 =20 From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Marc Linsner Sent: Monday, March 11, 2013 2:37 PM To: Sharam Hakimi Cc: lmap@ietf.org Subject: Re: [lmap] What is broadband? =20 Sharam, =20 IMO, the only difference between wireless and wired are the test metrics that can be performed. LMAP is concerned with the management of the measurement clients, controllers, and collectors. Until further thought proves otherwise, I don't see any difference in the management of devices on any of the layer 1/2s. =20 The specifics you mention, Signal Strength and Roaming, should be dealt with in the definition of the test metric. =20 -Marc- =20 =20 =20 James, I think mixing testing of wireless and wire technologies confuses the issues. One wireless is WiFi which is the extension of LAN and the other is the cellular network. They both have different requirements and two major conditions in both Signal Strength Roaming =20 have great effect on the achieved performance which does not exist in the wire condition . Some of these performance issues might also be better resolved in layer 2 rather than layer 3 and above. For example, before I run any throughput test I want to make sure that I have the highest strength signal possible be it WiFi or Cellular wireless connection ( and stay stationary during the test). On the other hand if I am trying to find the best location for signal strength a throughput test might not be the best tool for the job. =20 I would suggest to keep the two separate at least for the short term. =20 Sharam =20 =20 From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of James Miller Sent: Thursday, March 07, 2013 10:58 PM To: Shane Amante Cc: Benoit Claise; Bugenhagen,Michael K; lmap@ietf.org Subject: Re: [lmap] What is broadband? =20 I agree Shane--was what I was trying to get at. :) =20 Our early FCC focus was basically the Access network but I think we're all making the point that the mix-n-match of the network measurement pieces will be driven by a particular user's use case. =20 =20 For our FCC work, the focus (discussed in more detail in our reports) was on understanding the performance in the segment under the control of a carrier delivering broadband to a given consumer. Naturally many other elements affect an end user's experience but our initial interest was on understanding that segment for the purposes of providing better information to the consumer about the portion of Internet performance that the carrier provides. The FCC's focus and history is on that piece and so it makes sense that it might begin there, but a particular carrier's focus might mirror the access network focus, but perhaps with less market-trend orientation and more diagnostic in flavor. Maybe LMAP together with other protocols (TR69 and other BBForum work for example) could benefit all those use cases. =20 In recent meetings we've discussed other more targeted mini-studies on things like CDN or in-home Wifi performance. Other potential LMAP users already focus their work on these or other elements of the network. I would envision the LMAP work could provide the glue between any of these elements and different use cases. Worth noting that much of the initial discussions in the FCC's Next-Gen group were on the potential value of standardized interfaces between such different elements of measurement infrastructure. =20 Another element that's probably worth noting is that the use cases also may direct the actual tests that someone wants to run. I would imagine the actual tests you might want to run for evaluating in-home LAN, access network, CDN, mobile networks may vary. So I think the work on the test registry and other IPPM work on new tests will also improve our definitions and understanding of "broadband performance". =20 As always my personal views and not the FCC's.. possible valued at less than $.02 :) On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante wrote: James, All, =20 On Mar 7, 2013, at 7:30 PM, James Miller wrote: I believe that Henning had commented at some point that the LMAP definition he contemplated had "architecture" as the 'A' element but certainly access is an important piece. I think one of the problems that has been discussed also on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP performance measurements would implicate elements from the user's laptop, through wireless and other local LAN, carriers access networks, Tier 1 and other peering networks, the application host side and everything in between. Clearly there would be a lot of technologies included within that functional scope. =20 I agree that access is important, but not to the exclusion of everything else that constitutes an Internet end-user customer experience or, alternatively, an Enterprise end-user customer experience -- which is what I believe you're saying above? That's why it will be important to, at some point, figure out *if*, and then how, to try to segment the portions of the end-to-end path that you describe above so we can attribute good or bad performance to a particular portion of the path so that, ultimately, the correct network operator can be contacted to look into the problem further. I do not believe that this requires us to break down the end-to-end path on a router-hop by router-hop basis, but rather we need to be able to identify 'sign posts' along the path that can correlate well to end-to-end path. =20 -shane =20 =09 =09 =09 =09 For reference, in the FCC Measuring Broadband America Program we focused on measurement from the consumers' broadband modem through the carriers network to where it connects to a tier one peering point. LMAP should be able to address the broad mix of other use cases that would have a mix of other elements and motivations. =20 Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K wrote: The word "access" should be key here as part of the definition provided we are talking about an Internet service, which is the second component. I don't really think we are building tests that won't work on smaller pipes so questioning if it really 'broadband' or not is correct IMO. =20 =09 =09 Sent from my iPhone =09 On Mar 7, 2013, at 9:41 AM, "Shane Amante" wrote: Benoit,=20 =20 On Mar 7, 2013, at 3:47 AM, Benoit Claise wrote: Dear all, =09 I started to review the drafts, and I will start posting a few questions to the list. Open questions, clarifying questions, in order to generate some discussions. Disclaimer: I have not yet read the entire list archive. Apologize in advance if some points have been discussed already.=20 =09 Here is my first question. What is broadband in the LMAP context? Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology independent? And I see also "enterprise edge router", "cellular data or satellite" in draft-schulzrinne-lmap-requirements. In or out? Or do we have in mind a phase approach, starting with the "enterprise edge router" first, and then home network? =20 Speaking for the network I operate, I'm very much an advocate of saying that "Enterprise Edge Router" is "in-scope". We would very much benefit from a standards-based measurement enablement and collection regime vs. mostly proprietary, and non-scalable, approaches that exist today. =20 This is not to diminish the importance of similar test capabilities for residential broadband use-cases. We absolutely need to work on those, as well. =20 =20 With respect to priority, my hope is that we do not have to choose to prioritize one over the other. Rather, I would hope that both can be developed in parallel, because both -- at least, IMO -- have a substantially overlapping set of requirements/features.=20 =20 My $0.02, =20 -shane =20 =09 =09 =09 =09 Interestingly, I don't know what A stands for in LMAP, if it stands for something. According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand for anything. However, looking at the different draft titles, there is some confusion. draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measurement Use Cases draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurement of Broadband Performance draft-boucadair-lmap-considerations-00, Large scale Measurement of Access network Performance (LMAP): Requirements and Issues from a Network Provider Perspective =09 Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated. =09 Regards, Benoit =09 =09 =09 =09 _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap =20 _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap =09 _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap =20 =20 =20 _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap=20 ------_=_NextPart_001_01CE1E96.E50B0637 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Marc,

I think the management agent that a wireless device might = use would be somewhat different than a wire device as there are metrics that = a wireless device has, that do not exist in a wire device. My = opinion.

 

Sharam

 

 

From:= lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of = Marc Linsner
Sent: Monday, March 11, 2013 2:37 PM
To: Sharam Hakimi
Cc: lmap@ietf.org
Subject: Re: [lmap] What is broadband?

 

Sharam,

 

IMO, the only difference between wireless and wired are the = test metrics that can be performed.  LMAP is concerned with the = management of the measurement clients, controllers, and collectors.  Until = further thought proves otherwise, I don't see any difference in the management = of devices on any of the layer 1/2s.

 

The specifics you mention, Signal Strength and Roaming, = should be dealt with in the definition of the test metric.

 

-Marc-

 

 

 

James,

I think mixing testing of wireless and wire technologies confuses the issues. One wireless is WiFi which is the extension of LAN = and the other is the cellular network. They both have different requirements and = two major conditions in both

         &nbs= p;      Signal Strength

         &nbs= p;      Roaming

 

have great effect on the achieved performance which does = not exist in the wire condition . Some of these performance issues might = also be better resolved in layer 2 rather than layer 3 and above. For example, = before I run any throughput test I want to make sure that I have the highest = strength signal possible be it WiFi or Cellular wireless connection ( and stay stationary during the test). On the other hand if I am trying to find = the best location for signal strength a throughput test might not be the best = tool for the job.

 

I would suggest to keep the two separate at least for the = short term.

 

Sharam

  

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] = On Behalf Of James Miller
Sent: Thursday, March 07, 2013 10:58 PM
To: Shane Amante
Cc: Benoit Claise; Bugenhagen,Michael K; lmap@ietf.org
Subject: Re: [lmap] What is broadband?

 

I agree Shane--was what = I was trying to get at. :)

 

Our early FCC focus was = basically the Access network but I think we're all making the point that the = mix-n-match of the network measurement pieces will be driven by a particular user's = use case.  

 

For our FCC work, the = focus (discussed in more detail in our reports) was on understanding the = performance in the segment under the control of a carrier delivering broadband to a = given consumer.  Naturally many other elements affect an end user's = experience but our initial interest was on understanding that segment for the = purposes of providing better information to the consumer about the portion of = Internet performance that the carrier provides.  The FCC's focus and history = is on that piece and so it makes sense that it might begin there, but a = particular carrier's focus might mirror the access network focus, but = perhaps with less market-trend orientation and more diagnostic in flavor. =  Maybe LMAP together with other protocols (TR69 and other BBForum work for = example) could benefit all those use cases.

 

In recent meetings = we've discussed other more targeted mini-studies on things like CDN or in-home Wifi performance.  Other potential LMAP users already focus their work = on these or other elements of the network.  I would envision the LMAP work = could provide the glue between any of these elements and different use cases.  Worth noting that much of the initial discussions in the FCC's = Next-Gen group were on the potential value of standardized interfaces = between such different elements of measurement = infrastructure.

 

Another element that's = probably worth noting is that the use cases also may direct the actual tests that someone wants to run.  I would imagine the actual tests you might = want to run for evaluating in-home LAN, access network, CDN, mobile networks may = vary.  So I think the work on the test registry and other IPPM work on = new tests will also improve our definitions and understanding of "broadband performance".

 

As always my personal views and not the FCC's.. possible valued at less = than $.02  :)

On Thu, Mar 7, 2013 at = 10:24 PM, Shane Amante <shane@castlepoint.net> wrote:

James, = All,

 

On Mar 7, 2013, at 7:30 = PM, James Miller <jamesmilleresquire@gmail.com> wrote:

I believe that Henning = had commented at some point that the LMAP definition he contemplated had "architecture" as the 'A' element but certainly access is an important piece.  I think one of the problems that has been = discussed also on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP performance measurements would implicate elements from the user's = laptop, through wireless and other local LAN, carriers access networks, Tier 1 = and other peering networks, the application host side and everything in = between.  Clearly there would be a lot of technologies included within that functional scope.

 

I agree that access is = important, but not to the exclusion of everything else that constitutes an Internet end-user customer experience or, alternatively, an Enterprise end-user = customer experience -- which is what I believe you're saying above?  That's = why it will be important to, at some point, figure out *if*, and then how, to = try to segment the portions of the end-to-end path that you describe above so = we can attribute good or bad performance to a particular portion of the path so = that, ultimately, the correct network operator can be contacted to look into = the problem further.  I do not believe that this requires us to break = down the end-to-end path on a router-hop by router-hop basis, but rather we need = to be able to identify 'sign posts' along the path that can correlate well to end-to-end path.

 

-shane

 




For reference, in the = FCC Measuring Broadband America Program we focused on measurement from the consumers' broadband modem through the carriers network to where it = connects to a tier one peering point.  LMAP should be able to address the broad = mix of other use cases that would have a mix of other elements and = motivations.

 

Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadbandreport/= 2Methodology.pdf

On Thu, Mar 7, 2013 at = 5:20 PM, Bugenhagen, Michael K <Michael.K.Bugenhagen@centurylink.com> = wrote:

The word = "access" should be key here as part of the definition provided we are talking about an = Internet service, which is the second component.   I don't really =  think we are building tests that won't work on smaller pipes so questioning if it = really 'broadband' or not is correct IMO.

 



Sent from my iPhone


On Mar 7, 2013, at 9:41 AM, "Shane Amante" <shane@castlepoint.net> wrote:

Benoit, =

 

On Mar 7, 2013, at 3:47 = AM, Benoit Claise <bclaise@cisco.com> wrote:

Dear all,

I started to review the drafts, and I will start posting a few questions = to the list.
Open questions, clarifying questions, in order to generate some = discussions.
Disclaimer: I have not yet read the entire list archive. Apologize in = advance if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology independent?
And I see also "enterprise edge router", "cellular data = or satellite" in draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the = "enterprise edge router" first, and then home network?

 

Speaking for the = network I operate, I'm very much an advocate of saying that "Enterprise Edge Router" is "in-scope".  We would very much benefit = from a standards-based measurement enablement and collection regime vs. mostly proprietary, and non-scalable, approaches that exist = today.

 

This is not to diminish = the importance of similar test capabilities for residential broadband = use-cases.  We absolutely need to work on those, as well. =  

 

With respect to = priority, my hope is that we do not have to choose to prioritize one over the other.  Rather, I would hope that both can be developed in parallel, = because both -- at least, IMO -- have a substantially overlapping set of requirements/features. 

 

My = $0.02,

 

-shane

 




Interestingly, I don't = know what A stands for in LMAP, if it stands for something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart,= the A doesn't stand for anything.
However, looking at the different draft titles, there is some = confusion.

   =
draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband =
Measurement Use Cases
   =
draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale =
Measurement of Broadband Performance
   =
draft-boucadair-lmap-considerations-00, Large scale Measurement of =
Access network Performance (LMAP):
      Requirements and =
Issues from a Network Provider Perspective


Some more discussions, on the mailing list or during the BoF, on this = topic would be appreciated.

Regards, Benoit



_______________________________________________
= lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

 

_______________________________________________
= lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap


_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

 

 

 

_______________________________________________ lmap = mailing list lmap@ietf.org https://www.ietf.org/= mailman/listinfo/lmap

------_=_NextPart_001_01CE1E96.E50B0637-- From Henning.Schulzrinne@fcc.gov Mon Mar 11 13:43:58 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F3FA621F90DA for ; Mon, 11 Mar 2013 13:43:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.359 X-Spam-Level: X-Spam-Status: No, score=-1.359 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, SARE_LWSHORTT=1.24] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sSznesW752Rt for ; Mon, 11 Mar 2013 13:43:57 -0700 (PDT) Received: from GB-IP-2.fcc.gov (mail4.fcc.gov [65.207.43.167]) by ietfa.amsl.com (Postfix) with ESMTP id C057C21F90BD for ; Mon, 11 Mar 2013 13:43:55 -0700 (PDT) Received: from unknown (HELO GBPXCAS01V.fccnet.win.fcc.gov) ([65.207.43.140]) by GB-IP-2.fcc.gov with ESMTP; 11 Mar 2013 16:43:54 -0400 Received: from GBPXMB13V.fccnet.win.fcc.gov ([fe80::f1b8:d812:9a31:e47b]) by GBPXCAS01V.fccnet.win.fcc.gov ([fe80::ad35:7922:3623:4f10%12]) with mapi id 14.01.0438.000; Mon, 11 Mar 2013 16:43:52 -0400 From: Henning Schulzrinne To: Sharam Hakimi , Marc Linsner Thread-Topic: [lmap] What is broadband? Thread-Index: Ac4bsS4Fc9Tw+OlxRk6zqw2j7uy0dgAD2xrAALYUZ08= Date: Mon, 11 Mar 2013 20:43:52 +0000 Message-ID: References: , <084CDC75FEC1E640B60338273BEACDFA023C4894@spboexc01.exfo.com> In-Reply-To: <084CDC75FEC1E640B60338273BEACDFA023C4894@spboexc01.exfo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [2002:a587:f015::a587:f015] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "lmap@ietf.org" Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 20:43:58 -0000 The goal would be that the control and distribution components should be in= dependent of the metrics, so that metrics can be added, both standardized o= nes and proprietary ones specific to a commercial provider or research netw= ork. If the control mechanism doesn't work for wireless metrics, for exampl= e, that might be an indication that it is not generic enough. ________________________________ From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] on behalf of Sharam Hak= imi [sharam.hakimi@exfo.com] Sent: Monday, March 11, 2013 4:28 PM To: Marc Linsner Cc: lmap@ietf.org Subject: Re: [lmap] What is broadband? Marc, I think the management agent that a wireless device might use would be some= what different than a wire device as there are metrics that a wireless devi= ce has, that do not exist in a wire device. My opinion. Sharam From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Mar= c Linsner Sent: Monday, March 11, 2013 2:37 PM To: Sharam Hakimi Cc: lmap@ietf.org Subject: Re: [lmap] What is broadband? Sharam, IMO, the only difference between wireless and wired are the test metrics th= at can be performed. LMAP is concerned with the management of the measurem= ent clients, controllers, and collectors. Until further thought proves oth= erwise, I don't see any difference in the management of devices on any of t= he layer 1/2s. The specifics you mention, Signal Strength and Roaming, should be dealt wit= h in the definition of the test metric. -Marc- James, I think mixing testing of wireless and wire technologies confuses the issue= s. One wireless is WiFi which is the extension of LAN and the other is the = cellular network. They both have different requirements and two major condi= tions in both Signal Strength Roaming have great effect on the achieved performance which does not exist in the w= ire condition . Some of these performance issues might also be better resol= ved in layer 2 rather than layer 3 and above. For example, before I run any= throughput test I want to make sure that I have the highest strength signa= l possible be it WiFi or Cellular wireless connection ( and stay stationary= during the test). On the other hand if I am trying to find the best locati= on for signal strength a throughput test might not be the best tool for the= job. I would suggest to keep the two separate at least for the short term. Sharam From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of James Miller Sent: Thursday, March 07, 2013 10:58 PM To: Shane Amante Cc: Benoit Claise; Bugenhagen,Michael K; lmap@ietf.org Subject: Re: [lmap] What is broadband? I agree Shane--was what I was trying to get at. :) Our early FCC focus was basically the Access network but I think we're all = making the point that the mix-n-match of the network measurement pieces wil= l be driven by a particular user's use case. For our FCC work, the focus (discussed in more detail in our reports) was o= n understanding the performance in the segment under the control of a carri= er delivering broadband to a given consumer. Naturally many other elements= affect an end user's experience but our initial interest was on understand= ing that segment for the purposes of providing better information to the co= nsumer about the portion of Internet performance that the carrier provides.= The FCC's focus and history is on that piece and so it makes sense that i= t might begin there, but a particular carrier's focus might mirror the acce= ss network focus, but perhaps with less market-trend orientation and more d= iagnostic in flavor. Maybe LMAP together with other protocols (TR69 and ot= her BBForum work for example) could benefit all those use cases. In recent meetings we've discussed other more targeted mini-studies on thin= gs like CDN or in-home Wifi performance. Other potential LMAP users alread= y focus their work on these or other elements of the network. I would envi= sion the LMAP work could provide the glue between any of these elements and= different use cases. Worth noting that much of the initial discussions in= the FCC's Next-Gen group were on the potential value of standardized inter= faces between such different elements of measurement infrastructure. Another element that's probably worth noting is that the use cases also may= direct the actual tests that someone wants to run. I would imagine the ac= tual tests you might want to run for evaluating in-home LAN, access network= , CDN, mobile networks may vary. So I think the work on the test registry = and other IPPM work on new tests will also improve our definitions and unde= rstanding of "broadband performance". As always my personal views and not the FCC's.. possible valued at less tha= n $.02 :) On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante > wrote: James, All, On Mar 7, 2013, at 7:30 PM, James Miller > wrote: I believe that Henning had commented at some point that the LMAP definition= he contemplated had "architecture" as the 'A' element but certainly access= is an important piece. I think one of the problems that has been discusse= d also on the LMAP and our FCC Next-Gen lists is that a complete view of LM= AP performance measurements would implicate elements from the user's laptop= , through wireless and other local LAN, carriers access networks, Tier 1 an= d other peering networks, the application host side and everything in betwe= en. Clearly there would be a lot of technologies included within that func= tional scope. I agree that access is important, but not to the exclusion of everything el= se that constitutes an Internet end-user customer experience or, alternativ= ely, an Enterprise end-user customer experience -- which is what I believe = you're saying above? That's why it will be important to, at some point, fi= gure out *if*, and then how, to try to segment the portions of the end-to-e= nd path that you describe above so we can attribute good or bad performance= to a particular portion of the path so that, ultimately, the correct netwo= rk operator can be contacted to look into the problem further. I do not be= lieve that this requires us to break down the end-to-end path on a router-h= op by router-hop basis, but rather we need to be able to identify 'sign pos= ts' along the path that can correlate well to end-to-end path. -shane For reference, in the FCC Measuring Broadband America Program we focused on= measurement from the consumers' broadband modem through the carriers netwo= rk to where it connects to a tier one peering point. LMAP should be able t= o address the broad mix of other use cases that would have a mix of other e= lements and motivations. Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadba= ndreport/2Methodology.pdf On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K > wrote: The word "access" should be key here as part of the definition provided we = are talking about an Internet service, which is the second component. I d= on't really think we are building tests that won't work on smaller pipes s= o questioning if it really 'broadband' or not is correct IMO. Sent from my iPhone On Mar 7, 2013, at 9:41 AM, "Shane Amante" > wrote: Benoit, On Mar 7, 2013, at 3:47 AM, Benoit Claise > wrote: Dear all, I started to review the drafts, and I will start posting a few questions to= the list. Open questions, clarifying questions, in order to generate some discussions= . Disclaimer: I have not yet read the entire list archive. Apologize in advan= ce if some points have been discussed already. Here is my first question. What is broadband in the LMAP context? Is it DSL, cable, ETTH, Fiber to the home? Is LMAP technology independent? And I see also "enterprise edge router", "cellular data or satellite" in dr= aft-schulzrinne-lmap-requirements. In or out? Or do we have in mind a phase approach, starting with the "enterprise edge = router" first, and then home network? Speaking for the network I operate, I'm very much an advocate of saying tha= t "Enterprise Edge Router" is "in-scope". We would very much benefit from = a standards-based measurement enablement and collection regime vs. mostly p= roprietary, and non-scalable, approaches that exist today. This is not to diminish the importance of similar test capabilities for res= idential broadband use-cases. We absolutely need to work on those, as well= . With respect to priority, my hope is that we do not have to choose to prior= itize one over the other. Rather, I would hope that both can be developed = in parallel, because both -- at least, IMO -- have a substantially overlapp= ing set of requirements/features. My $0.02, -shane Interestingly, I don't know what A stands for in LMAP, if it stands for som= ething. According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A does= n't stand for anything. However, looking at the different draft titles, there is some confusion. draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measuremen= t Use Cases draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurem= ent of Broadband Performance draft-boucadair-lmap-considerations-00, Large scale Measurement of Acces= s network Performance (LMAP): Requirements and Issues from a Network Provider Perspective Some more discussions, on the mailing list or during the BoF, on this topic= would be appreciated. Regards, Benoit _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf= .org https://www.ietf.org/mailman/listinfo/lmap From bs7652@att.com Mon Mar 11 14:48:02 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 587CB21F9070 for ; Mon, 11 Mar 2013 14:48:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.449 X-Spam-Level: X-Spam-Status: No, score=-106.449 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lD1jngICE0kR for ; Mon, 11 Mar 2013 14:48:01 -0700 (PDT) Received: from nbfkord-smmo06.seg.att.com (nbfkord-smmo06.seg.att.com [209.65.160.94]) by ietfa.amsl.com (Postfix) with ESMTP id 0F12421F8F71 for ; Mon, 11 Mar 2013 14:48:01 -0700 (PDT) Received: from unknown [144.160.20.146] (EHLO nbfkord-smmo06.seg.att.com) by nbfkord-smmo06.seg.att.com(mxl_mta-6.15.0-1) with ESMTP id 1115e315.2aaac3404940.604941.00-552.1635414.nbfkord-smmo06.seg.att.com (envelope-from ); Mon, 11 Mar 2013 21:48:01 +0000 (UTC) X-MXL-Hash: 513e51114aa56008-9d55cb412f7c329b084c2453cf1822cf19e51f2f Received: from unknown [144.160.20.146] (EHLO mlpd194.enaf.sfdc.sbc.com) by nbfkord-smmo06.seg.att.com(mxl_mta-6.15.0-1) over TLS secured channel with ESMTP id f015e315.0.604932.00-137.1635384.nbfkord-smmo06.seg.att.com (envelope-from ); Mon, 11 Mar 2013 21:48:00 +0000 (UTC) X-MXL-Hash: 513e511031022736-c8f3a6bd3ffc6030ee263bd1ad335ab8ece849d2 Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlpd194.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r2BLlxko016218; Mon, 11 Mar 2013 17:47:59 -0400 Received: from mlpi408.sfdc.sbc.com (mlpi408.sfdc.sbc.com [130.9.128.240]) by mlpd194.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r2BLlh5Y016066 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 11 Mar 2013 17:47:52 -0400 Received: from GAALPA1MSGHUB9E.ITServices.sbc.com (gaalpa1msghub9e.itservices.sbc.com [130.8.36.91]) by mlpi408.sfdc.sbc.com (RSA Interceptor); Mon, 11 Mar 2013 21:47:36 GMT Received: from GAALPA1MSGUSR9L.ITServices.sbc.com ([130.8.36.69]) by GAALPA1MSGHUB9E.ITServices.sbc.com ([130.8.36.91]) with mapi id 14.02.0342.003; Mon, 11 Mar 2013 17:47:36 -0400 From: "STARK, BARBARA H" To: Henning Schulzrinne , Sharam Hakimi , Marc Linsner Thread-Topic: [lmap] What is broadband? Thread-Index: Ac4bsS4F227VHJtyhkuQwA1QFB5jwQAD2xrAAL6EHQAABlvRMA== Date: Mon, 11 Mar 2013 21:47:35 +0000 Message-ID: <2D09D61DDFA73D4C884805CC7865E611302698AF@GAALPA1MSGUSR9L.ITServices.sbc.com> References: , <084CDC75FEC1E640B60338273BEACDFA023C4894@spboexc01.exfo.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.70.59.244] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-RSA-Inspected: yes X-RSA-Classifications: public X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [144.160.20.146] X-AnalysisOut: [v=2.0 cv=ZLJgbwHb c=1 sm=0 a=Qs8R1XBwmid1qBFB/a8mmA==:17 a] X-AnalysisOut: [=wJxhdZA4IUwA:10 a=Y3V_VNiTAnQA:10 a=ofMgfj31e3cA:10 a=BLc] X-AnalysisOut: [eEmwcHowA:10 a=kj9zAlcOel0A:10 a=zQP7CpKOAAAA:8 a=XIqpo32R] X-AnalysisOut: [AAAA:8 a=YRvHT3Y1vWQA:10 a=DyQo4BgHBUh_poFKKUMA:9 a=CjuIK1] X-AnalysisOut: [q_8ugA:10 a=TyeRuIlZcggoAZf3:21 a=S1UflcAkXW9DYm2h:21] Cc: "lmap@ietf.org" Subject: Re: [lmap] What is broadband? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 21:48:02 -0000 > Marc, > I think the management agent that a wireless device might use would be > somewhat different than a wire device as there are metrics that a wireles= s > device has, that do not exist in a wire device. My opinion. >=20 > Sharam I agree with Marc. Most wireless providers' cellular devices are managed wi= th OMA-DM. Many consumer broadband devices are managed with TR-069. Others= with SNMP. OMA-DM and TR-069 both use xml data models. SNMP uses MIBs. The= proposed registry would be equally mappable on any of these protocols. FWI= W BBF and OMA liaise a lot to harmonize various data models. So we know it = can be done. Barbara From acooper@cdt.org Tue Mar 12 06:25:25 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 49D8321F8AB7 for ; Tue, 12 Mar 2013 06:25:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.832 X-Spam-Level: X-Spam-Status: No, score=-102.832 tagged_above=-999 required=5 tests=[AWL=-0.233, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9TQcaPx4Cq8J for ; Tue, 12 Mar 2013 06:25:24 -0700 (PDT) Received: from mail.maclaboratory.net (mail.maclaboratory.net [209.190.215.232]) by ietfa.amsl.com (Postfix) with ESMTP id C3AED21F8AE3 for ; Tue, 12 Mar 2013 06:25:07 -0700 (PDT) X-Footer: Y2R0Lm9yZw== Received: from localhost ([127.0.0.1]) by mail.maclaboratory.net (using TLSv1/SSLv3 with cipher AES128-SHA (128 bits)); Tue, 12 Mar 2013 09:25:04 -0400 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: Alissa Cooper In-Reply-To: <20130311175614.GA62652@elstar.local> Date: Tue, 12 Mar 2013 09:25:04 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5138763B.1000100@cisco.com> <22981BAB-FED3-4E41-B8D7-C5AFD0934D38@cdt.org> <20130311175614.GA62652@elstar.local> To: Juergen Schoenwaelder X-Mailer: Apple Mail (2.1499) Cc: Benoit Claise , "lmap@ietf.org" Subject: Re: [lmap] Which devices does LMAP target? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 13:25:25 -0000 Hi Juergen, I was actually thinking of it the other way around -- that it may be = more feasible to add, for example, a full-featured NETCONF server to a = PC or tablet environment than to cheap CPE. So designing for the widest = range of possible devices may have implications for the selection of = protocols to use for controller<-->MA (and MA<-->collector). Or it may = not, but it's worth a bit of discussion. Alissa=20 On Mar 11, 2013, at 1:56 PM, Juergen Schoenwaelder = wrote: > On Mon, Mar 11, 2013 at 11:32:04AM -0400, Alissa Cooper wrote: >> FWIW, this strikes me as a fairly important question for = understanding the extent to which existing protocols/building blocks can = be reused for LMAP. If we expect a measurement agent to be embedded or = embeddable in every cheap piece of CPE out there, that may constrain the = complexity of the protocol logic for both receiving test schedules and = reporting test results. Whether we want all measurement agent = implementations to be so constrained and/or whether an extensibility = mechanism is possible that allows for more sophisticated logic in more = sophisticated devices (PC, tablet, smartphone, etc.) would be good to = figure out. >>=20 >=20 > Alissa, >=20 > can you give an example where measurement agents running on more > sophisticated devices may be doing something substantially different > compared to measurement agents running on embedded hardware? I am > looking for a motivational example to understand why additional > features on the control and reporting protocol would be needed for > different classes of devices. >=20 > /js >=20 > --=20 > Juergen Schoenwaelder Jacobs University Bremen gGmbH > Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany > Fax: +49 421 200 3103 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap >=20 From hannes.tschofenig@gmx.net Tue Mar 12 07:10:45 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 38F1B21F89EE for ; Tue, 12 Mar 2013 07:10:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.021 X-Spam-Level: X-Spam-Status: No, score=-103.021 tagged_above=-999 required=5 tests=[AWL=0.578, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EqK0MxIMI3YA for ; Tue, 12 Mar 2013 07:10:44 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) by ietfa.amsl.com (Postfix) with ESMTP id 1938021F85ED for ; Tue, 12 Mar 2013 07:10:44 -0700 (PDT) Received: from mailout-de.gmx.net ([10.1.76.34]) by mrigmx.server.lan (mrigmx001) with ESMTP (Nemesis) id 0MFOSw-1U13u315wS-00ENaJ for ; Tue, 12 Mar 2013 15:10:43 +0100 Received: (qmail invoked by alias); 12 Mar 2013 14:10:42 -0000 Received: from dhcp-1077.meeting.ietf.org (EHLO dhcp-1077.meeting.ietf.org) [130.129.16.119] by mail.gmx.net (mp034) with SMTP; 12 Mar 2013 15:10:42 +0100 X-Authenticated: #29516787 X-Provags-ID: V01U2FsdGVkX188eGgpKUg8kH79GIG2SyU+V4N94QSNU38IiWGQSs XkykEbOxh9py5c Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Hannes Tschofenig In-Reply-To: Date: Tue, 12 Mar 2013 10:10:38 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5138763B.1000100@cisco.com> <22981BAB-FED3-4E41-B8D7-C5AFD0934D38@cdt.org> <20130311175614.GA62652@elstar.local> To: Alissa Cooper X-Mailer: Apple Mail (2.1085) X-Y-GMX-Trusted: 0 Cc: Benoit Claise , Hannes Tschofenig , Juergen Schoenwaelder , "lmap@ietf.org" Subject: Re: [lmap] Which devices does LMAP target? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2013 14:10:45 -0000 +1 I believe that the cases where the these tests are initiated from the = end points is far more interesting in the mid/long term. They also = provide more insightful results.=20 Netconf, for example, may be a great protocol for network devices but I = am not sure how common it is on end devices like tablets, mobile phones, = and laptops.=20 Ciao Hannes On Mar 12, 2013, at 9:25 AM, Alissa Cooper wrote: > Hi Juergen, >=20 > I was actually thinking of it the other way around -- that it may be = more feasible to add, for example, a full-featured NETCONF server to a = PC or tablet environment than to cheap CPE. So designing for the widest = range of possible devices may have implications for the selection of = protocols to use for controller<-->MA (and MA<-->collector). Or it may = not, but it's worth a bit of discussion. >=20 > Alissa=20 >=20 > On Mar 11, 2013, at 1:56 PM, Juergen Schoenwaelder = wrote: >=20 >> On Mon, Mar 11, 2013 at 11:32:04AM -0400, Alissa Cooper wrote: >>> FWIW, this strikes me as a fairly important question for = understanding the extent to which existing protocols/building blocks can = be reused for LMAP. If we expect a measurement agent to be embedded or = embeddable in every cheap piece of CPE out there, that may constrain the = complexity of the protocol logic for both receiving test schedules and = reporting test results. Whether we want all measurement agent = implementations to be so constrained and/or whether an extensibility = mechanism is possible that allows for more sophisticated logic in more = sophisticated devices (PC, tablet, smartphone, etc.) would be good to = figure out. >>>=20 >>=20 >> Alissa, >>=20 >> can you give an example where measurement agents running on more >> sophisticated devices may be doing something substantially different >> compared to measurement agents running on embedded hardware? I am >> looking for a motivational example to understand why additional >> features on the control and reporting protocol would be needed for >> different classes of devices. >>=20 >> /js >>=20 >> --=20 >> Juergen Schoenwaelder Jacobs University Bremen gGmbH >> Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany >> Fax: +49 421 200 3103 >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >>=20 >=20 >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From surfer@mauigateway.com Tue Mar 12 17:30:03 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 82F2011E80E9 for ; Tue, 12 Mar 2013 17:30:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QK3Sdark7htv for ; Tue, 12 Mar 2013 17:30:02 -0700 (PDT) Received: from imta-38.everyone.net (imta-35.everyone.net [216.200.145.35]) by ietfa.amsl.com (Postfix) with ESMTP id B22D211E8105 for ; Tue, 12 Mar 2013 17:30:00 -0700 (PDT) Received: from pps.filterd (m0004961.ppops.net [127.0.0.1]) by imta-38.everyone.net (8.14.5/8.14.5) with SMTP id r2D0RU33006100 for ; Tue, 12 Mar 2013 17:30:03 -0700 X-Eon-Dm: m0005314.ppops.net Received: by resin03.mta.everyone.net (EON-PICKUP) id resin03.51259e7a.31b16; Tue, 12 Mar 2013 17:29:54 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Message-Id: <20130312172953.FAF57AD7@resin03.mta.everyone.net> Date: Tue, 12 Mar 2013 17:29:53 -0700 From: "Scott Weeks" To: X-Eon-Sig: AQLAznNRP8iCN/655wEAAAAB,d78403355963345909792e55447f2b4f X-Originating-Ip: 72.234.26.100 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8626, 1.0.431, 0.0.0000 definitions=2013-03-12_07:2013-03-12, 2013-03-12, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=2 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1211240000 definitions=main-1303120213 Subject: Re: [lmap] Which devices does LMAP target? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: surfer@mauigateway.com List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 00:30:03 -0000 --- acooper@cdt.org wrote: From: Alissa Cooper I was actually thinking of it the other way around -- that it may be more feasible to add, for example, a full-featured NETCONF server to a PC or tablet environment than to cheap CPE. So designing for the widest range of possible devices may have implications for the selection of protocols to use for controller<-->MA (and MA<-->collector). Or it may not, but it's worth a bit of discussion. ------------------------------------------------- Operator here; not a researcher... :) Would you expect that the owners of these devices will need to give permission to install something like this? If it's going to be on an ISP's network device it's one thing. One request allows it to go on 1000s of devices, such as cable modems. If it's going on a device a consumer owns, it's a totally different thing. Am I off in left field with this question? scott _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap From dromasca@avaya.com Wed Mar 13 03:54:33 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AAE9621F8540 for ; Wed, 13 Mar 2013 03:54:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.291 X-Spam-Level: X-Spam-Status: No, score=-103.291 tagged_above=-999 required=5 tests=[AWL=0.308, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wn6kE-tUmHfk for ; Wed, 13 Mar 2013 03:54:33 -0700 (PDT) Received: from co300216-co-outbound.net.avaya.com (co300216-co-outbound.net.avaya.com [198.152.13.100]) by ietfa.amsl.com (Postfix) with ESMTP id 2FB7221F8530 for ; Wed, 13 Mar 2013 03:54:32 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgEFAHpXMFGHCzI1/2dsb2JhbABEgma/Tn8Wc4IhAQEDEihRARUVFEIfBwEEGxqHcQGgN4QqnEONUxpiFIMXYQOcWopRgwiBcjU X-IronPort-AV: E=Sophos;i="4.84,760,1355115600"; d="scan'208";a="1827346" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by co300216-co-outbound.net.avaya.com with ESMTP; 13 Mar 2013 06:54:32 -0400 Received: from unknown (HELO AZ-FFEXHC01.global.avaya.com) ([135.64.58.11]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 13 Mar 2013 06:52:52 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC01.global.avaya.com ([135.64.58.11]) with mapi id 14.02.0328.009; Wed, 13 Mar 2013 06:54:31 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: notes takers and jabber scribes Thread-Index: Ac4f2SK59kKyBPQiT2anuQnFrCDmQg== Date: Wed, 13 Mar 2013 10:54:30 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA0AA414@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] notes takers and jabber scribes X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 10:54:33 -0000 We are still short of volunteers for taking notes and taking questions from= remote participants on jabber. Please send mail or come to the chairs ahea= d of the meeting, so that we do not lose precious BOF time.=20 Thanks and Regards, Dan From Henning.Schulzrinne@fcc.gov Wed Mar 13 07:19:01 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 18F3321F8E04 for ; Wed, 13 Mar 2013 07:19:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2aKUUWLFX2mL for ; Wed, 13 Mar 2013 07:19:00 -0700 (PDT) Received: from DC-IP-1.fcc.gov (dc-ip-1.fcc.gov [192.104.54.97]) by ietfa.amsl.com (Postfix) with ESMTP id 643C221F8DFC for ; Wed, 13 Mar 2013 07:18:59 -0700 (PDT) Received: from gatekeeper4.fcc.gov (HELO p2pxcas01.fccnet.win.fcc.gov) ([192.104.54.21]) by DC-IP-1.fcc.gov with ESMTP; 13 Mar 2013 10:18:58 -0400 Received: from P2PXMB13.fccnet.win.fcc.gov ([fe80::6593:6526:65f8:66b7]) by p2pxcas01.fccnet.win.fcc.gov ([fe80::1d27:de47:ad2e:9062%13]) with mapi id 14.01.0438.000; Wed, 13 Mar 2013 10:18:59 -0400 From: Henning Schulzrinne To: "surfer@mauigateway.com" , "lmap@ietf.org" Thread-Topic: [lmap] Which devices does LMAP target? Thread-Index: AQHOH4Hh4Tz9iEU31EiQekgw7++u5Zijqyu8 Date: Wed, 13 Mar 2013 14:18:58 +0000 Message-ID: References: <20130312172953.FAF57AD7@resin03.mta.everyone.net> In-Reply-To: <20130312172953.FAF57AD7@resin03.mta.everyone.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.104.54.64] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] Which devices does LMAP target? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 14:19:01 -0000 Operators already install, in some cases, proprietary measurement applicati= ons, e.g., made by companies such as CarrierIQ. The permission needed will = probably differ based on jurisdiction and the nature of the information col= lected. For the FCC MBA projects, all participants are volunteers who are f= ully informed of the nature of the measurements and can end their participa= tion at any time.=0A= =0A= If collected by carriers, information may be subject to Customer Proprietar= y Network Information rules in the US or similar rules elsewhere, but this = is reasonably well-understood, at least by operator legal teams.=0A= =0A= Henning=0A= =0A= ________________________________________=0A= From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] on behalf of Scott Week= s [surfer@mauigateway.com]=0A= Sent: Tuesday, March 12, 2013 8:29 PM=0A= To: lmap@ietf.org=0A= Subject: Re: [lmap] Which devices does LMAP target?=0A= =0A= =0A= -------------------------------------------------=0A= =0A= =0A= Operator here; not a researcher... :)=0A= =0A= Would you expect that the owners of these devices will need to=0A= give permission to install something like this? If it's going=0A= to be on an ISP's network device it's one thing. One request=0A= allows it to go on 1000s of devices, such as cable modems. If=0A= it's going on a device a consumer owns, it's a totally different=0A= thing. Am I off in left field with this question?=0A= =0A= scott=0A= =0A= =0A= =0A= =0A= =0A= _______________________________________________=0A= lmap mailing list=0A= lmap@ietf.org=0A= https://www.ietf.org/mailman/listinfo/lmap=0A= =0A= =0A= _______________________________________________=0A= lmap mailing list=0A= lmap@ietf.org=0A= https://www.ietf.org/mailman/listinfo/lmap=0A= From dromasca@avaya.com Mon Mar 18 19:24:27 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C45F021F8DF0 for ; Mon, 18 Mar 2013 19:24:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.991 X-Spam-Level: X-Spam-Status: No, score=-102.991 tagged_above=-999 required=5 tests=[AWL=-0.392, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BOdTt9VO7iSI for ; Mon, 18 Mar 2013 19:24:27 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id 4487F21F8DEF for ; Mon, 18 Mar 2013 19:24:27 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgEFALh+MVGHCzI1/2dsb2JhbABEgma/WAaBABZzgiEBAQMSKFEBFRUUQiYBBBsBGYdxAQufcoQqjGuPWI12YhSDF2EDnFyKUoJ7DYIn X-IronPort-AV: E=Sophos;i="4.84,766,1355115600"; d="scan'208";a="2603320" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 18 Mar 2013 22:24:26 -0400 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 18 Mar 2013 22:22:35 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Mon, 18 Mar 2013 22:24:25 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: LMAP BOF draft minutes now available Thread-Index: Ac4kSN9GsGxacXnMSROn6Pd76tbyoA== Date: Tue, 19 Mar 2013 02:24:25 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA0B2A2D@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] LMAP BOF draft minutes now available X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Mar 2013 02:24:27 -0000 The draft minutes of the LMAP BOF are now available at http://www.ietf.org/= proceedings/86/minutes/minutes-86-lmap. Thanks to Mike De Leo and Andrea So= ppera for taking notes. Please send to the list all questions, comments, co= rrections.=20 Thanks and Regards, Dan From dromasca@avaya.com Thu Mar 28 01:17:56 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B4C5E21F9063; Thu, 28 Mar 2013 01:17:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.099 X-Spam-Level: X-Spam-Status: No, score=-103.099 tagged_above=-999 required=5 tests=[AWL=-0.500, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GtlzAOxzIYQz; Thu, 28 Mar 2013 01:17:54 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id 2324121F90F1; Thu, 28 Mar 2013 01:17:49 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgEFALh+MVGHCzI1/2dsb2JhbABEgma/XoEAFnOCIQEBAxIoPxIBFRUUQiYBBAENDRqHcQELpBycPwSNdmIUMYJmYQOcXINPBIZ/gwiCJw X-IronPort-AV: E=Sophos;i="4.84,766,1355115600"; d="scan'208";a="4041313" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 28 Mar 2013 04:17:49 -0400 Received: from unknown (HELO AZ-FFEXHC01.global.avaya.com) ([135.64.58.11]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 28 Mar 2013 04:15:38 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC01.global.avaya.com ([135.64.58.11]) with mapi id 14.02.0328.009; Thu, 28 Mar 2013 04:17:46 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" , "ippm@ietf.org" Thread-Topic: incoming liaison statement from IEEE Project P802.16.3 Thread-Index: Ac4rjLnfWpFw/p7tS3i28o1yTrXHQw== Date: Thu, 28 Mar 2013 08:17:46 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA0BB87E@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Roger Marks , "ops-ads@tools.ietf.org" Subject: [lmap] incoming liaison statement from IEEE Project P802.16.3 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 08:17:57 -0000 Hi, Please note the following incoming liaison statement from IEEE Project P802= .16.3 concerning the LMAP activity.=20 https://datatracker.ietf.org/liaison/1244/ Comments and discussions are welcome.=20 Regards, Dan