Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaW14-0004tc-1B; Wed, 26 Sep 2007 08:29:22 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IaW13-0004rH-8y for pmol-confirm+ok@megatron.ietf.org; Wed, 26 Sep 2007 08:29:21 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaW12-0004n0-Ov for pmol@ietf.org; Wed, 26 Sep 2007 08:29:20 -0400 Received: from mail146.messagelabs.com ([216.82.245.131]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IaW0w-0000ZV-Ji for pmol@ietf.org; Wed, 26 Sep 2007 08:29:20 -0400 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-10.tower-146.messagelabs.com!1190809743!9031302!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.20.53] Received: (qmail 5869 invoked from network); 26 Sep 2007 12:29:03 -0000 Received: from sbcsmtp6.sbc.com (HELO mlph073.enaf.sfdc.sbc.com) (144.160.20.53) by server-10.tower-146.messagelabs.com with AES256-SHA encrypted SMTP; 26 Sep 2007 12:29:03 -0000 Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlph073.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8QCT3Cl006980 for ; Wed, 26 Sep 2007 08:29:03 -0400 Received: from mlph070.sfdc.sbc.com (mlph070.sfdc.sbc.com [144.155.224.139]) by mlph073.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8QCSwHD006948 for ; Wed, 26 Sep 2007 08:28:58 -0400 Received: from sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlph070.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8QCSwuV025450 for ; Wed, 26 Sep 2007 08:28:58 -0400 Received: from maillennium.att.com (mailgw1.maillennium.att.com [135.25.114.99]) by mlph070.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8QCSpw7025370 for ; Wed, 26 Sep 2007 08:28:52 -0400 Message-Id: <200709261228.l8QCSpw7025370@mlph070.sfdc.sbc.com> Received: from acmt.att.com (acmt.mt.att.com[135.16.251.73](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20070926122851gw10010gcle>; Wed, 26 Sep 2007 12:28:51 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 26 Sep 2007 08:28:13 -0400 To: Henk Uijterwaal , "Romascanu, Dan (Dan)" From: Al Morton Subject: Re: [PMOL] Suggested updates to proposed charter wording In-Reply-To: <46FA13A2.20101@ripe.net> References: <46F9ED85.9060704@ripe.net> <46FA13A2.20101@ripe.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: -2.5 (--) X-Scan-Signature: 769a46790fb42fbb0b0cc700c82f7081 Cc: pmol@ietf.org, "Scott O. Bradner" X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Henk, Following-up here to close the loop on this, because this also addresses the issue Lars raised. I agree that further discussion could be on the IPPM list... At 04:09 AM 9/26/2007, Henk Uijterwaal wrote: >Romascanu, Dan (Dan) wrote: >>Actually Scott resubmitted his draft in June and then submitted a >>revised version in August. Henk's observation is correct however to the >>extent that this is still an Internet-Draft, so referring to it as >>anything but as 'work-in-progress' is not appropriate. > >Somebody correct me if I'm wrong but the June version was a resubmit of >an old draft, and the August version says near the end in difference >between the 02 and 03 versions: "correct author list". There was a small group who discussed the draft with the authors before the June release of 02. One of the changes responded to my suggestion for a test plan (where the accuracy/equivalency issue would be dealt with up front, on a case-by-case basis), and the new text in section 5 is partly based on Vern's reply, too: "...The implementation report must include a specific plan to test the specific metrics in the RFC in lab or real-world networks and reports of the tests performed with two or more implementations of the software. The test plan should cover key parts of the specification, speak to the accuracy required for each aspect measured and thus define "statistically equivalent" for the specific metrics being tested. Ideally, the test plan would co- evolve with the development of the metric, since that's when people have the most context in their thinking regarding the different subtleties that can arise." IOW, it made more sense to me to defer this question of equivalency to a specific metric, and allow the metricstest memo to move on and define the process (which can be accomplished for a general case). Lars - This means that the open issue you identified would be addressed in the test plan memo, and context of each metric. There may be other differences, but this is one. Al _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaSuE-0000uV-Ht; Wed, 26 Sep 2007 05:10:06 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IaSuD-0000tr-4I for pmol-confirm+ok@megatron.ietf.org; Wed, 26 Sep 2007 05:10:05 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaSuC-0000tY-Hv for pmol@ietf.org; Wed, 26 Sep 2007 05:10:04 -0400 Received: from smtp.nokia.com ([131.228.20.172] helo=mgw-ext13.nokia.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IaSu6-0003wX-1a for pmol@ietf.org; Wed, 26 Sep 2007 05:10:04 -0400 Received: from esebh105.NOE.Nokia.com (esebh105.ntc.nokia.com [172.21.138.211]) by mgw-ext13.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id l8Q99VJY014563; Wed, 26 Sep 2007 12:09:50 +0300 Received: from esebh103.NOE.Nokia.com ([172.21.143.33]) by esebh105.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 26 Sep 2007 12:09:40 +0300 Received: from esebh101.NOE.Nokia.com ([172.21.138.177]) by esebh103.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 26 Sep 2007 12:09:40 +0300 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh101.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Wed, 26 Sep 2007 12:09:40 +0300 Received: from [172.21.34.212] (esdhcp034212.research.nokia.com [172.21.34.212]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id l8Q99c6l009731; Wed, 26 Sep 2007 12:09:38 +0300 In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v752.3) Message-Id: From: Lars Eggert Subject: Re: [PMOL] Suggested updates to proposed charter wording Date: Wed, 26 Sep 2007 12:09:24 +0300 To: ext Alan Clark X-Mailer: Apple Mail (2.752.3) X-OriginalArrivalTime: 26 Sep 2007 09:09:40.0283 (UTC) FILETIME=[F82CD0B0:01C8001C] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: a2c12dacc0736f14d6b540e805505a86 Cc: pmol@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0666142055==" Errors-To: pmol-bounces@ietf.org --===============0666142055== Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-56--110316069; protocol="application/pkcs7-signature" --Apple-Mail-56--110316069 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed On 2007-9-25, at 21:22, ext Alan Clark wrote: > 1. A PMOL framework and guidelines memo that outlines the motivation > for work to define performance metrics for applications > transported > over IETF-specified protocols, how that work fills a need and > any gap > in IETF-chartered work. The framework will describe the necessary > elements of performance metric drafts and the various types of > metrics > that may be prepared in this work. The framework will also > address the > need to specify the intended audience and the motivation for the > performance metrics. There will also be guidelines for a > performance > metric development process that includes entry criteria for > new proposals (how a proposal might be evaluated for possible > endorsement by a protocol development working group), and how a > successful proposal will be developed by PMOL WG in cooperation > with a > protocol development WG. Some of the content of this proposed WG deliverable is what we typically want to see already during the BOF proposal phase: Specifically, the "motivation for work", "how that work fills a need and any gap in IETF-chartered work", "motivation for the performance metrics" seem pretty relevant already when asking for a BOF. Likewise, information on "the various types of metrics that may be prepared in this work" seem important to define the scope for the planned WG. "Entry criteria for new proposals" is also something that should be in the proposed charter already. Lars --Apple-Mail-56--110316069 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGQDCCAvkw ggJioAMCAQICEGtxkLFHQu1g67hlmYfwPuIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDEwNDE2MTE1NFoXDTA4MDEwNDE2MTE1 NFowXDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2Vy dDEkMCIGCSqGSIb3DQEJARYVbGFycy5lZ2dlcnRAbm9raWEuY29tMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEA26hjntVPZMiwh4d8tyuk9KiucvG92BVUGArk5zO1jhmq7tLFgU1mqcIg VGumfQqjkAzIuh83kxIiqBrB05Wvxp85apwt4sUCvzMe8mQWzZZZYp0rBzwpOj+VC5pxsMRYtYW+ BaTFBEmvFr7D7C7roZUk0lDppS5SZFs4SQbEe9ykB9aeUf1iTiw2+ikyP2+JEYto14WYCoxFWbms FbQ1uaaK+yn1WH2YHhyMfi0IOxuT0jtbsngjHJMpWIqq334zBhj+rPSUug47YBHr41FCDMHbbbjv WbyTyDYneOW3WY8LY8bGWVlAknQGB7lgduShe6e0RI/7SL/VE6X27lM9LwIDAQABozIwMDAgBgNV HREEGTAXgRVsYXJzLmVnZ2VydEBub2tpYS5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUF AAOBgQCx3nxxTg/WowobVTRXBiXUEEZj4LBiunWbuAnR0UbJIgijvq3JbiJvZo2gUGiW9LPaHSBz 4oxT1VR/S/6O0a4e87oV5cOQm6ReB68o9rDfUzxHTwoCfv2wwMwBrXyzQMjyQK4Lt8siV41gmZpm rSXMhjTJdd9uYYNoZKQLq+VM+TCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgG A1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMg RGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3 DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3 MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9 fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+ uxg+B79AgAJk16emu59l0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMB Af8ECDAGAQH/AgEAMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3 dGVQZXJzb25hbEZyZWVtYWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgG A1UEAxMRUHJpdmF0ZUxhYmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcP f6+svsIXoUOWlJ1/TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH 2ydxVyWN3amcOY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8x ggMQMIIDDAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ a3GQsUdC7WDruGWZh/A+4jAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcB MBwGCSqGSIb3DQEJBTEPFw0wNzA5MjYwOTA5MjVaMCMGCSqGSIb3DQEJBDEWBBR5kgYuP7bhUPvg AmbSKWSMmGGTHDCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIwgYcGCyqGSIb3DQEJEAILMXigdjBiMQsw CQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIw DQYJKoZIhvcNAQEBBQAEggEAPoM233j9muCVONgt6M7AN/6DVRTjH12QQHq6I5GJLzpR2Czx+Ip7 bGpORUQqvtLBWxPTHQFBv9iaSCzqyONvZSdijqJqGIaeKj9H3MPpKIZQRy1HsIXiXD/i31HhLG6J gms5gFAsULZIkHrIp9QYB0plLVaPvCqNlXhQ5eNsZoRUvA6MO3MJXt0f2ToULzPiOW60L08RbJmt sMrE+zfxUBAjCaIq/+ngMLIp2OtpmExTRb0cRo1rGDlcftYsBrehRvun54kDxKP2OOe14eh4772/ epjH2vRjkuI+TCdCv1R8Z5vDxsS64lHaf4IsBiREVz0pFP/JqvPAkV2ZlNt3JQAAAAAAAA== --Apple-Mail-56--110316069-- --===============0666142055== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol --===============0666142055==-- Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaSm9-0004Cr-Pk; Wed, 26 Sep 2007 05:01:45 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IaSm9-0004Ce-26 for pmol-confirm+ok@megatron.ietf.org; Wed, 26 Sep 2007 05:01:45 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaSm8-0004CW-My for pmol@ietf.org; Wed, 26 Sep 2007 05:01:44 -0400 Received: from smtp.nokia.com ([131.228.20.170] helo=mgw-ext11.nokia.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IaSm0-0003lW-Iq for pmol@ietf.org; Wed, 26 Sep 2007 05:01:44 -0400 Received: from esebh106.NOE.Nokia.com (esebh106.ntc.nokia.com [172.21.138.213]) by mgw-ext11.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id l8Q919i1015854; Wed, 26 Sep 2007 12:01:13 +0300 Received: from esebh103.NOE.Nokia.com ([172.21.143.33]) by esebh106.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 26 Sep 2007 12:00:50 +0300 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh103.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Wed, 26 Sep 2007 12:00:45 +0300 Received: from [172.21.34.212] (esdhcp034212.research.nokia.com [172.21.34.212]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id l8Q90hNZ001024; Wed, 26 Sep 2007 12:00:43 +0300 In-Reply-To: <46FA13A2.20101@ripe.net> References: <46F9ED85.9060704@ripe.net> <46FA13A2.20101@ripe.net> Mime-Version: 1.0 (Apple Message framework v752.3) Message-Id: From: Lars Eggert Subject: Re: [PMOL] Suggested updates to proposed charter wording Date: Wed, 26 Sep 2007 12:00:30 +0300 To: ext Henk Uijterwaal X-Mailer: Apple Mail (2.752.3) X-OriginalArrivalTime: 26 Sep 2007 09:00:45.0143 (UTC) FILETIME=[B934EE70:01C8001B] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: 25620135586de10c627e3628c432b04a Cc: pmol@ietf.org, "Scott O. Bradner" X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0073789384==" Errors-To: pmol-bounces@ietf.org --===============0073789384== Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-55--110850528; protocol="application/pkcs7-signature" --Apple-Mail-55--110850528 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hi, a quick comment about draft-bradner-metrics-test: I had mentioned during the IPPM meeting that the draft is pretty complete, with the exception of a generally applicable definition what it would mean for two implementations of the same metric to generate "statistically equivalent" results. Help from metrics experts is surely welcome. (This list is probably not the right place to discuss this - I suggest the IPPM list.) Lars --Apple-Mail-55--110850528 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGQDCCAvkw ggJioAMCAQICEGtxkLFHQu1g67hlmYfwPuIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDEwNDE2MTE1NFoXDTA4MDEwNDE2MTE1 NFowXDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2Vy dDEkMCIGCSqGSIb3DQEJARYVbGFycy5lZ2dlcnRAbm9raWEuY29tMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEA26hjntVPZMiwh4d8tyuk9KiucvG92BVUGArk5zO1jhmq7tLFgU1mqcIg VGumfQqjkAzIuh83kxIiqBrB05Wvxp85apwt4sUCvzMe8mQWzZZZYp0rBzwpOj+VC5pxsMRYtYW+ BaTFBEmvFr7D7C7roZUk0lDppS5SZFs4SQbEe9ykB9aeUf1iTiw2+ikyP2+JEYto14WYCoxFWbms FbQ1uaaK+yn1WH2YHhyMfi0IOxuT0jtbsngjHJMpWIqq334zBhj+rPSUug47YBHr41FCDMHbbbjv WbyTyDYneOW3WY8LY8bGWVlAknQGB7lgduShe6e0RI/7SL/VE6X27lM9LwIDAQABozIwMDAgBgNV HREEGTAXgRVsYXJzLmVnZ2VydEBub2tpYS5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUF AAOBgQCx3nxxTg/WowobVTRXBiXUEEZj4LBiunWbuAnR0UbJIgijvq3JbiJvZo2gUGiW9LPaHSBz 4oxT1VR/S/6O0a4e87oV5cOQm6ReB68o9rDfUzxHTwoCfv2wwMwBrXyzQMjyQK4Lt8siV41gmZpm rSXMhjTJdd9uYYNoZKQLq+VM+TCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgG A1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMg RGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3 DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3 MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9 fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+ uxg+B79AgAJk16emu59l0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMB Af8ECDAGAQH/AgEAMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3 dGVQZXJzb25hbEZyZWVtYWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgG A1UEAxMRUHJpdmF0ZUxhYmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcP f6+svsIXoUOWlJ1/TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH 2ydxVyWN3amcOY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8x ggMQMIIDDAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ a3GQsUdC7WDruGWZh/A+4jAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcB MBwGCSqGSIb3DQEJBTEPFw0wNzA5MjYwOTAwMzBaMCMGCSqGSIb3DQEJBDEWBBQnpRjNQQIMnddw SBucPDK97ff0gzCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIwgYcGCyqGSIb3DQEJEAILMXigdjBiMQsw CQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIw DQYJKoZIhvcNAQEBBQAEggEAesuVxE0RGo1zwUURpTbTWlzWjLc44SihU5gvqfxm6ZIIfyijmLbb ymvICA1SNjg9f9luKL6YLRUUCybOpt6NWJcNm3IjKBO8mce6GyknIgLuwgG/3iz9h7pFlXL83J3K o9jZdwFaPYTaN3v6CKT0tmN0gAPfmAhr+BlE26fME8SkB3GI4JK2GkLwEnQBg8vLiy5pw3bEghCY oWhG6qx3t+thyHjONzXxZVEp38VW8g2p0sYdqAK0OhaNFd9eejDU4CjO15QV7gRgS6QZtHf4oytd 4IgHp8LsGG5GN9COLcLbUogyX+ZrdDw9u9yB9qbhGtxNIGnc1SpNayPhuI+/CAAAAAAAAA== --Apple-Mail-55--110850528-- --===============0073789384== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol --===============0073789384==-- Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaRxU-0001ub-JK; Wed, 26 Sep 2007 04:09:24 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IaRxS-0001uV-OW for pmol-confirm+ok@megatron.ietf.org; Wed, 26 Sep 2007 04:09:22 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaRxS-0001uJ-6E for pmol@ietf.org; Wed, 26 Sep 2007 04:09:22 -0400 Received: from postboy.ripe.net ([193.0.19.3]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IaRxQ-0002gX-4W for pmol@ietf.org; Wed, 26 Sep 2007 04:09:22 -0400 Received: by postboy.ripe.net (Postfix, from userid 4008) id A87416A17D; Wed, 26 Sep 2007 10:09:14 +0200 (CEST) Received: from herring.ripe.net (herring.ripe.net [193.0.1.203]) by postboy.ripe.net (Postfix) with ESMTP id 6CF8F6A15E; Wed, 26 Sep 2007 10:09:12 +0200 (CEST) Received: from guest-wv-41.ripe.net (gw.office.nsrp.ripe.net [193.0.1.126]) by herring.ripe.net (Postfix) with ESMTP id 3C3A32F583; Wed, 26 Sep 2007 10:09:12 +0200 (CEST) Message-ID: <46FA13A2.20101@ripe.net> Date: Wed, 26 Sep 2007 10:09:06 +0200 From: Henk Uijterwaal User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: "Romascanu, Dan (Dan)" Subject: Re: [PMOL] Suggested updates to proposed charter wording References: <46F9ED85.9060704@ripe.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-RIPE-Spam-Level: X-RIPE-Spam-Tests: ALL_TRUSTED,BAYES_00 X-RIPE-Spam-Status: N 0.000020 / -4.4 X-RIPE-Signature: e1f6fdf618b85fb0223bcd1b614de8ad X-Spam-Score: -4.0 (----) X-Scan-Signature: ffa9dfbbe7cc58b3fa6b8ae3e57b0aa3 Cc: pmol@ietf.org, "Scott O. Bradner" X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Romascanu, Dan (Dan) wrote: > Actually Scott resubmitted his draft in June and then submitted a > revised version in August. Henk's observation is correct however to the > extent that this is still an Internet-Draft, so referring to it as > anything but as 'work-in-progress' is not appropriate. Somebody correct me if I'm wrong but the June version was a resubmit of an old draft, and the August version says near the end in difference between the 02 and 03 versions: "correct author list". Henk -- ------------------------------------------------------------------------------ Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The Netherlands The Netherlands Mobile: +31.6.55861746 ------------------------------------------------------------------------------ # Lawyer: "Now sir, I'm sure you are an intelligent and honest man--" # Witness: "Thank you. If I weren't under oath, I'd return the compliment." _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaRZn-0006z3-UK; Wed, 26 Sep 2007 03:44:56 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IaRZk-0006jb-Df for pmol-confirm+ok@megatron.ietf.org; Wed, 26 Sep 2007 03:44:52 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaRZj-0006gV-PQ for pmol@ietf.org; Wed, 26 Sep 2007 03:44:51 -0400 Received: from nj300815-nj-outbound.net.avaya.com ([198.152.12.100] helo=nj300815-nj-outbound.avaya.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IaRZd-0001xF-Ka for pmol@ietf.org; Wed, 26 Sep 2007 03:44:51 -0400 X-IronPort-AV: E=Sophos;i="4.20,299,1186372800"; d="scan'208";a="65379830" Received: from 16.140.8.135.in-addr.arpa (HELO 307622ANEX5.global.avaya.com) ([135.64.140.16]) by nj300815-nj-outbound.avaya.com with ESMTP; 26 Sep 2007 03:44:30 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PMOL] Suggested updates to proposed charter wording Date: Wed, 26 Sep 2007 09:44:27 +0200 Message-ID: In-Reply-To: <46F9ED85.9060704@ripe.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PMOL] Suggested updates to proposed charter wording Thread-Index: AcgACoHKo4yr8FQQRACx5uBBrsUJ4gABdSHQ References: <46F9ED85.9060704@ripe.net> From: "Romascanu, Dan (Dan)" To: "Henk Uijterwaal" , X-Spam-Score: 0.0 (/) X-Scan-Signature: 5a9a1bd6c2d06a21d748b7d0070ddcb8 Cc: "Scott O. Bradner" X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Actually Scott resubmitted his draft in June and then submitted a revised version in August. Henk's observation is correct however to the extent that this is still an Internet-Draft, so referring to it as anything but as 'work-in-progress' is not appropriate.=20 Dan =20 =20 > -----Original Message----- > From: Henk Uijterwaal [mailto:henk@ripe.net]=20 > Sent: Wednesday, September 26, 2007 7:26 AM > To: pmol@ietf.org > Subject: Re: [PMOL] Suggested updates to proposed charter wording >=20 > Al, others, >=20 > > The PMOL WG will also be guided by a document describing how memos=20 > > defining performance metrics are intended to advance along the IETF=20 > > Standards track (draft-bradner-metrics-test). >=20 > How can the PMOL group be guided by a draft that hasn't seen any work > for the last 5+ years and expired a long time ago? I do agree > the metrics-test draft is useful, but it should be picked up=20 > by some WG and finished before it serves as guidelines for PMOL. >=20 > Henk >=20 >=20 > -- > -------------------------------------------------------------- > ---------------- > Henk Uijterwaal Email:=20 > henk.uijterwaal(at)ripe.net > RIPE Network Coordination Centre =20 > http://www.amsterdamned.org/~henk > P.O.Box 10096 Singel 258 Phone: +31.20.5354414 > 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 > The Netherlands The Netherlands Mobile: +31.6.55861746 > -------------------------------------------------------------- > ---------------- >=20 > # Lawyer: "Now sir, I'm sure you are an intelligent and honest man--" > # Witness: "Thank you. If I weren't under oath, I'd return=20 > the compliment." >=20 >=20 >=20 > _______________________________________________ > PMOL mailing list > PMOL@ietf.org > https://www1.ietf.org/mailman/listinfo/pmol >=20 _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaQn9-0005Qf-F4; Wed, 26 Sep 2007 02:54:39 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IaQn7-0005Ga-W6 for pmol-confirm+ok@megatron.ietf.org; Wed, 26 Sep 2007 02:54:38 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaQn3-00057S-7J for pmol@ietf.org; Wed, 26 Sep 2007 02:54:33 -0400 Received: from postboy.ripe.net ([193.0.19.3]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IaQmx-0000ks-06 for pmol@ietf.org; Wed, 26 Sep 2007 02:54:33 -0400 Received: by postboy.ripe.net (Postfix, from userid 4008) id 64FBF6A0BE; Wed, 26 Sep 2007 08:54:16 +0200 (CEST) Received: from herring.ripe.net (herring.ripe.net [193.0.1.203]) by postboy.ripe.net (Postfix) with ESMTP id 039E36A079 for ; Wed, 26 Sep 2007 08:54:15 +0200 (CEST) Received: from guest-wv-41.ripe.net (gw.office.nsrp.ripe.net [193.0.1.126]) by herring.ripe.net (Postfix) with ESMTP id F15E92F583 for ; Wed, 26 Sep 2007 08:54:14 +0200 (CEST) Message-ID: <46F9ED85.9060704@ripe.net> Date: Wed, 26 Sep 2007 07:26:29 +0200 From: Henk Uijterwaal User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: pmol@ietf.org Subject: Re: [PMOL] Suggested updates to proposed charter wording References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-RIPE-Spam-Level: X-RIPE-Spam-Tests: ALL_TRUSTED,BAYES_00 X-RIPE-Spam-Status: N 0.118282 / -4.4 X-RIPE-Signature: f787f14a004418e55facd4f76c40c93f X-Spam-Score: -4.0 (----) X-Scan-Signature: ffa9dfbbe7cc58b3fa6b8ae3e57b0aa3 X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Al, others, > The PMOL WG will also be guided by a document describing how memos > defining performance metrics are intended to advance along the IETF > Standards track (draft-bradner-metrics-test). How can the PMOL group be guided by a draft that hasn't seen any work for the last 5+ years and expired a long time ago? I do agree the metrics-test draft is useful, but it should be picked up by some WG and finished before it serves as guidelines for PMOL. Henk -- ------------------------------------------------------------------------------ Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The Netherlands The Netherlands Mobile: +31.6.55861746 ------------------------------------------------------------------------------ # Lawyer: "Now sir, I'm sure you are an intelligent and honest man--" # Witness: "Thank you. If I weren't under oath, I'd return the compliment." _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaF3s-00017Y-KF; Tue, 25 Sep 2007 14:23:08 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IaF3q-00013W-T2 for pmol-confirm+ok@megatron.ietf.org; Tue, 25 Sep 2007 14:23:06 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IaF3q-0000ob-EW for pmol@ietf.org; Tue, 25 Sep 2007 14:23:06 -0400 Received: from mx.cbeyond.com ([66.180.96.58]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IaF3b-0000A4-8U for pmol@ietf.org; Tue, 25 Sep 2007 14:22:52 -0400 Received: from [72.54.75.1] (port=1059 helo=TELWS143) by mx.cbeyond.com with asmtp (Exim 4.34) id 1IaF3a-0001X7-Cy; Tue, 25 Sep 2007 14:22:50 -0400 From: "Alan Clark" To: "'Al Morton'" Date: Tue, 25 Sep 2007 14:22:48 -0400 MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: Acf/oQ60aWIgL79KSFO62H6N84klCw== X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 0.0 (/) X-Scan-Signature: 07d4bcb4600b627a0786c2557bc62e06 Message-Id: Cc: pmol@ietf.org Subject: [PMOL] Suggested updates to proposed charter wording X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1499824634==" Errors-To: pmol-bounces@ietf.org This is a multi-part message in MIME format. --===============1499824634== Content-Type: multipart/alternative; boundary="----=_NextPart_000_010F_01C7FF7F.8D07B310" This is a multi-part message in MIME format. ------=_NextPart_000_010F_01C7FF7F.8D07B310 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Al A proposed rewording of the initial paragraph and some minor text cleanup Regards Alan Performance Metrics at Other Layers WG (PMOL) The successful implementation and operation of IP based applications often depends on some underlying performance measurement infrastructure that helps service operators or network managers to recognize when performance is unsatisfactory and identify problems affecting service quality. Standardized performance metrics add the desirable features of consistent implementation, interpretation, and comparison. The IETF has two Working Groups dedicated to the development of performance metrics however each has strict limitations in their charters: - The Benchmarking Methodology WG has addressed a range of networking technologies and protocols in their long history (such as IEEE 802.3, ATM, Frame Relay, and Routing Protocols), but the charter strictly limits their performance characterizations to the laboratory environment. - The IP Performance Metrics WG has the mandate to develop metrics applicable to the performance of Internet data delivery, but it is specifically prohibited from developing metrics that characterize traffic (such as a VoIP stream). The IETF also has current and completed activities related to the reporting of application performance metrics (e.g. RAQMON and RTCP XR) and is also actively involved in the development of reliable transport protocols which would affect the relationship between IP performance and application performance. Thus there is a gap in the currently chartered coverage of IETF WGs: development of performance metrics for IP-based applications that operate over UDP, TCP, SCTP, DCCP, Forward Error Correction (FEC) and other robust transport protocols, and that can be used to characterize traffic on live networks. The working group will focus on the completion of two RFCs: 1. A PMOL framework and guidelines memo that outlines the motivation for work to define performance metrics for applications transported over IETF-specified protocols, how that work fills a need and any gap in IETF-chartered work. The framework will describe the necessary elements of performance metric drafts and the various types of metrics that may be prepared in this work. The framework will also address the need to specify the intended audience and the motivation for the performance metrics. There will also be guidelines for a performance metric development process that includes entry criteria for new proposals (how a proposal might be evaluated for possible endorsement by a protocol development working group), and how a successful proposal will be developed by PMOL WG in cooperation with a protocol development WG. 2. A proof-of-concept RFC defining performance metrics for SIP, based on draft-malas-performance-metrics. This memo would serve as an example of the framework and the PMOL development process in the IETF. Discussion of new work proposals is strongly discouraged in the PMOL WG, except to advise a protocol development WG when they are evaluating a new work proposal for related performance metrics. The PMOL WG will also be guided by a document describing how memos defining performance metrics are intended to advance along the IETF Standards track (draft-bradner-metrics-test). Milestones June 08 SIP Performance Metrics Draft to IESG Review Sept 08 PMOL Framework and Guidelines Draft to IESG Review ------=_NextPart_000_010F_01C7FF7F.8D07B310 Content-Type: text/html; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable

Al

 

A proposed = rewording of the initial paragraph and some minor text = cleanup

 

Regards

 

Alan

 

 

 

 

Performance Metrics = at Other Layers WG (PMOL)

 

The successful implementation and operation of IP based applications often depends on = some underlying performance measurement infrastructure that helps service = operators or network managers to recognize when performance is unsatisfactory and = identify problems affecting service quality.  Standardized performance = metrics add the desirable features of consistent implementation, interpretation, and = comparison.

 

The IETF has two = Working Groups dedicated to the development of performance metrics however each = has strict limitations in their charters:

 

  - The = Benchmarking Methodology WG has addressed a range of networking technologies and = protocols in their long history (such as IEEE 802.3, ATM, Frame Relay, and Routing Protocols), but the charter strictly limits their performance = characterizations to the laboratory environment.

 

  - The IP = Performance Metrics WG has the mandate to develop metrics applicable to the = performance of Internet data delivery, but it is specifically prohibited from developing metrics = that characterize traffic (such as a VoIP = stream).

 

The IETF also has = current and completed activities related to the reporting of application = performance metrics (e.g. RAQMON and RTCP XR) and is also actively involved in the development of reliable transport protocols which would affect the = relationship between IP performance and application = performance.

 

Thus there is a gap = in the currently chartered coverage of IETF WGs:

development of = performance metrics for IP-based applications that operate over UDP, TCP, SCTP, = DCCP, Forward Error Correction (FEC) and other robust transport protocols, and = that can be used to characterize traffic on live = networks.

 

The working group = will focus on the completion of two RFCs:

 

1. A PMOL framework = and guidelines memo that outlines the = motivation

    = for work to define performance metrics for applications = transported

    = over IETF-specified protocols, how that work fills a need and any = gap

   =  in IETF-chartered work. The framework will describe the = necessary

    = elements of performance metric drafts and the various types of = metrics

    = that may be prepared in this work. The framework will also address = the

    = need to specify the intended audience and the motivation for = the

    performance metrics. There will also be guidelines for a = performance

    = metric development process that includes entry criteria = for

    = new proposals (how a proposal might be evaluated for = possible

    endorsement by a protocol development working group), and how = a

    successful proposal will be developed by PMOL WG in cooperation with = a

    = protocol development WG.

 

2. A = proof-of-concept RFC defining performance metrics for SIP, based = on

    draft-malas-performance-metrics.  This memo would serve as an = example of

    = the framework and the PMOL development process in the = IETF.

 

Discussion of new = work proposals is strongly discouraged in the PMOL WG, except to advise a = protocol development WG when they are evaluating a new work proposal for related performance metrics.

 

The PMOL WG will = also be guided by a document describing how memos defining performance metrics = are intended to advance along the IETF Standards track = (draft-bradner-metrics-test).

 

Milestones

June 08  SIP Performance Metrics Draft to IESG Review

Sept 08  PMOL = Framework and Guidelines Draft to IESG Review

 

 

------=_NextPart_000_010F_01C7FF7F.8D07B310-- --===============1499824634== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol --===============1499824634==-- Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IZiBi-0000Ow-2q; Mon, 24 Sep 2007 03:17:02 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IZiBh-0000Lv-1Z for pmol-confirm+ok@megatron.ietf.org; Mon, 24 Sep 2007 03:17:01 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IZiBc-0000Hy-9T for pmol@ietf.org; Mon, 24 Sep 2007 03:16:56 -0400 Received: from de307622-de-outbound.net.avaya.com ([198.152.71.100]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IZiBb-0007ps-P3 for pmol@ietf.org; Mon, 24 Sep 2007 03:16:56 -0400 X-IronPort-AV: E=Sophos;i="4.20,290,1186372800"; d="scan'208";a="60474322" Received: from unknown (HELO 307622ANEX5.global.avaya.com) ([135.64.140.16]) by de307622-de-outbound.net.avaya.com with ESMTP; 24 Sep 2007 03:16:52 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PMOL] Draft Charter for discussion Date: Mon, 24 Sep 2007 09:16:07 +0200 Message-ID: In-Reply-To: <200709240341.l8O3f7mq031659@mlph070.sfdc.sbc.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PMOL] Draft Charter for discussion Thread-Index: Acf+XMjiWe2H10u3SuaevIlGbBpokAAHbmLw References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> <200709240341.l8O3f7mq031659@mlph070.sfdc.sbc.com> From: "Romascanu, Dan (Dan)" To: "Al Morton" , X-Spam-Score: 0.0 (/) X-Scan-Signature: 7655788c23eb79e336f5f8ba8bce7906 Cc: X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org It's the same but better name the WG action. AD review is already out of the WG responsibility :-) Dan =20 =20 > -----Original Message----- > From: Al Morton [mailto:acmorton@att.com]=20 > Sent: Monday, September 24, 2007 5:41 AM > To: Romascanu, Dan (Dan); pmol@ietf.org > Subject: RE: [PMOL] Draft Charter for discussion >=20 > At 10:20 AM 9/23/2007, Romascanu, Dan (Dan) wrote: > >- Guidelines should mention submission to the IESG rather than 'AD=20 > >review' >=20 > OK. I was thinking of "Publication Request" where the first=20 > step is usually AD review, and IESG review follows. > Al >=20 _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IZep0-0006FV-Ly; Sun, 23 Sep 2007 23:41:22 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IZeoz-0006B5-1Q for pmol-confirm+ok@megatron.ietf.org; Sun, 23 Sep 2007 23:41:21 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IZeoy-0006Au-Iz for pmol@ietf.org; Sun, 23 Sep 2007 23:41:20 -0400 Received: from mail146.messagelabs.com ([216.82.245.131]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IZeoy-0003Mc-3g for pmol@ietf.org; Sun, 23 Sep 2007 23:41:20 -0400 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-11.tower-146.messagelabs.com!1190605278!18328851!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.128.149] Received: (qmail 15185 invoked from network); 24 Sep 2007 03:41:19 -0000 Received: from sbcsmtp9.sbc.com (HELO flph024.enaf.ffdc.sbc.com) (144.160.128.149) by server-11.tower-146.messagelabs.com with AES256-SHA encrypted SMTP; 24 Sep 2007 03:41:19 -0000 Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8O3fIOF012327 for ; Sun, 23 Sep 2007 20:41:18 -0700 Received: from flph023.ffdc.sbc.com (flph023.ffdc.sbc.com [150.234.117.36]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8O3fDYN012319 for ; Sun, 23 Sep 2007 20:41:13 -0700 Received: from ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph023.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8O3fDIa029935 for ; Sun, 23 Sep 2007 20:41:13 -0700 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by flph023.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8O3f7Tt029923 for ; Sun, 23 Sep 2007 20:41:08 -0700 Message-Id: <200709240341.l8O3f7Tt029923@flph023.ffdc.sbc.com> Received: from acmt.att.com (unknown[135.210.75.109](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20070924034106gw10010gjme>; Mon, 24 Sep 2007 03:41:06 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Sun, 23 Sep 2007 23:40:39 -0400 To: "Romascanu, Dan (Dan)" , From: Al Morton Subject: RE: [PMOL] Draft Charter for discussion In-Reply-To: References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: 1.5 (+) X-Scan-Signature: 30ac594df0e66ffa5a93eb4c48bcb014 Cc: X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org At 10:20 AM 9/23/2007, Romascanu, Dan (Dan) wrote: >- Guidelines should mention submission to the IESG rather than 'AD >review' OK. I was thinking of "Publication Request" where the first step is usually AD review, and IESG review follows. Al _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IZSL5-0000y8-69; Sun, 23 Sep 2007 10:21:39 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IZSL4-0000uF-47 for pmol-confirm+ok@megatron.ietf.org; Sun, 23 Sep 2007 10:21:38 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IZSKz-0000cS-BF for pmol@ietf.org; Sun, 23 Sep 2007 10:21:33 -0400 Received: from de307622-de-outbound.net.avaya.com ([198.152.71.100]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IZSKv-0007ua-CO for pmol@ietf.org; Sun, 23 Sep 2007 10:21:30 -0400 X-IronPort-AV: E=Sophos;i="4.20,288,1186372800"; d="scan'208";a="60299279" Received: from unknown (HELO 307622ANEX5.global.avaya.com) ([135.64.140.16]) by de307622-de-outbound.net.avaya.com with ESMTP; 23 Sep 2007 10:21:25 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PMOL] Draft Charter for discussion Date: Sun, 23 Sep 2007 16:20:40 +0200 Message-ID: In-Reply-To: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PMOL] Draft Charter for discussion Thread-Index: Acf8UxN6uHF5EQ5JR6e7Xp4PZwCHMABl1pFw References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> From: "Romascanu, Dan (Dan)" To: "Al Morton" , X-Spam-Score: 0.0 (/) X-Scan-Signature: b132cb3ed2d4be2017585bf6859e1ede Cc: X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Hi Al,=20 I have a couple of comments:=20 - when describing the mandate of IPPM it would be more complete to mention that IPPM explicitly limits its scope in the charter to a set of metrics related to Internet data delivery=20 - Guidelines should mention submission to the IESG rather than 'AD review' and specify the target status for each of the two documents - BCP for the framework and guidelines document, and Proposed Status for the SIP metrics document.=20 Also language-like it may be better to refer to 'applications and sessions' rather than just 'applications' to make clear that with PMOL we do not target only application layers metrics.=20 Dan =20 =20 > -----Original Message----- > From: Al Morton [mailto:acmorton@att.com]=20 > Sent: Friday, September 21, 2007 3:24 PM > To: pmol@ietf.org > Subject: [PMOL] Draft Charter for discussion >=20 > Here's the initial draft of the charter. > Comments welcome, of course. >=20 > Al >=20 > Proposed Charter (0.0) >=20 > Performance Metrics at Other Layers WG (PMOL) >=20 > There are often uncertainties about the performance and=20 > suitability of new technologies and applications for their=20 > intended audience, and the Internet is no exception. Most=20 > uncertainties are effectively addressed through quantified=20 > assessment of key performance indicators. Standardized=20 > performance metrics add the desirable features of consistent=20 > implementation, interpretation, and comparison. >=20 > Although the IETF has two Working Groups dedicated to the=20 > development of performance metrics, they each have strict=20 > limitations in their > charters: >=20 > - The Benchmarking Methodology WG has addressed a range of=20 > networking technologies and protocols in their long history=20 > (such as IEEE 802.3, ATM, Frame Relay, and Routing=20 > Protocols), but the charter strictly limits their performance=20 > characterizations to the laboratory environment. >=20 > - The IP Performance Metrics WG has the mandate to develop=20 > metrics applicable to live IP networks, but it is=20 > specifically prohibited from developing metrics that=20 > characterize traffic (such as a VoIP stream). >=20 > The IETF also has current and completed activities related to=20 > the reporting of application performance metrics (e.g.=20 > RAQMON) and is also actively involved in the development of=20 > reliable transport protocols which would affect the=20 > relationship between IP performance and application performance. >=20 > Thus there is a gap in the currently chartered coverage of IETF WGs: > development of performance metrics for IP-based applications=20 > that operate over UDP, TCP, SCTP, DCCP, Forward Error=20 > Correction (FEC) and other robust transport protocols, and=20 > that can be used to characterize traffic on live networks. >=20 > The working group will focus on the completion of two RFCs: >=20 > 1. A PMOL framework and guidelines memo that includes the motivation > of work to define performance metrics for applications transported > on IETF-specified protocols, and how that work fills a=20 > need and a gap > in IETF-chartered work. The framework will describe the necessary > elements of performance metric drafts and the various=20 > types of metrics > that may be prepared in this work. The framework will=20 > also address the > need to specify the intended audience and the motivation for the > performance metrics. There will also be guidelines for a=20 > performance > metric development process that includes entry criteria for > new proposals (how a proposal might be evaluated for possible > endorsement by a protocol development working group), and how a > successful proposal will be developed by PMOL WG in=20 > cooperation with a > protocol development WG. >=20 > 2. A proof-of-concept RFC defining performance metrics for=20 > SIP, based on > draft-malas-performance-metrics. This memo would serve=20 > as an example of > the framework and the PMOL development process in the IETF. >=20 > Discussion of new work proposals is strongly discouraged in=20 > the PMOL WG, except to advise a protocol development WG when=20 > they are evaluating a new work proposal for related=20 > performance metrics. >=20 > The PMOL WG will also be guided by a document describing how=20 > memos defining performance metrics are intended to advance=20 > along the IETF Standards track (draft-bradner-metricstest). >=20 > Milestones > June 08 SIP Performance Metrics Draft to AD Review Sept 08 =20 > PMOL Framework and Guidelines Draft to AD Review >=20 >=20 >=20 >=20 > _______________________________________________ > PMOL mailing list > PMOL@ietf.org > https://www1.ietf.org/mailman/listinfo/pmol >=20 _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYq9R-0004J7-MG; Fri, 21 Sep 2007 17:35:05 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IYq9R-0004Iz-Fi for pmol-confirm+ok@megatron.ietf.org; Fri, 21 Sep 2007 17:35:05 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYq9Q-0004G0-J7 for pmol@ietf.org; Fri, 21 Sep 2007 17:35:04 -0400 Received: from mail121.messagelabs.com ([216.82.241.195]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IYq9P-0003Dw-Jv for pmol@ietf.org; Fri, 21 Sep 2007 17:35:03 -0400 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-14.tower-121.messagelabs.com!1190410501!16104755!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.128.149] Received: (qmail 21958 invoked from network); 21 Sep 2007 21:35:02 -0000 Received: from sbcsmtp9.sbc.com (HELO flph024.enaf.ffdc.sbc.com) (144.160.128.149) by server-14.tower-121.messagelabs.com with AES256-SHA encrypted SMTP; 21 Sep 2007 21:35:02 -0000 Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LLZ1sT004462 for ; Fri, 21 Sep 2007 14:35:01 -0700 Received: from flph023.ffdc.sbc.com (flph023.ffdc.sbc.com [150.234.117.36]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LLYume004414 for ; Fri, 21 Sep 2007 14:34:56 -0700 Received: from ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph023.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LLYuUt002544 for ; Fri, 21 Sep 2007 14:34:56 -0700 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by flph023.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LLYsFo002519 for ; Fri, 21 Sep 2007 14:34:54 -0700 Message-Id: <200709212134.l8LLYsFo002519@flph023.ffdc.sbc.com> Received: from acmt.att.com (unknown[135.210.76.249](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20070921213452gw10010gi0e>; Fri, 21 Sep 2007 21:34:53 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 21 Sep 2007 17:34:24 -0400 To: "Loki Jorgenson" , From: Al Morton Subject: RE: [PMOL] Draft Charter for discussion In-Reply-To: References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: -2.5 (--) X-Scan-Signature: 7d33c50f3756db14428398e2bdedd581 Cc: X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Loki, At 04:49 PM 9/21/2007, Loki Jorgenson wrote: >...What would seem missing from the draft charter is a reference to some >sort of framework (instead of the implicit "set of L3 and higher network >protocols") that bridges metrics and performance with at least "types of >applications". ... I avoided an explicit framework because the intent is that this WG be able to work on performance metric proposals both above and below IP (if it continues beyond this initial charter). For example, the latest version of draft-xie-ccamp-lsp-dppm- is a proposal on control plane performance below IP. PMOL might be looking at that in a year or so, if it's successful. In one sentence, this WG might cover vetted work that the other charters don't, but perhaps that's too broad and blunt. Al _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYpRc-0003ym-62; Fri, 21 Sep 2007 16:49:48 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IYpRb-0003wl-Fu for pmol-confirm+ok@megatron.ietf.org; Fri, 21 Sep 2007 16:49:47 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYpRb-0003SG-17 for pmol@ietf.org; Fri, 21 Sep 2007 16:49:47 -0400 Received: from [209.139.228.52] (helo=JSRVR18.jaalam.net) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IYpRO-0001hI-4l for pmol@ietf.org; Fri, 21 Sep 2007 16:49:34 -0400 X-AuditID: ac108108-0000063c0000066c-16-46f42e596c7f Received: from jsrvr8.jaalam.net ([172.16.128.105] RDNS failed) by JSRVR18.jaalam.net with Microsoft SMTPSVC(6.0.3790.1830); Fri, 21 Sep 2007 13:49:29 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PMOL] Draft Charter for discussion Date: Fri, 21 Sep 2007 13:49:32 -0700 Message-ID: In-Reply-To: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PMOL] Draft Charter for discussion Thread-Index: Acf8Uuja+L9utHK7RqG7BFNk7bL4YQAO32GA References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> From: "Loki Jorgenson" To: "Al Morton" , X-Brightmail-Tracker: AAAAAA== X-Spam-Score: 0.1 (/) X-Scan-Signature: 082a9cbf4d599f360ac7f815372a6a15 Cc: X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Al - while I can appreciate the careful intention in charter wording, I would point out that the relevance to "other protocols" to any specific or general application is not addressed. You make reference to "VoIP traffic" as an example of an application case. Without naming a vendor, a codec, or a context, I note that there are possible various high-level characterizations about VoIP as an application - e.g. the usual about loss and jitter sensitivity (and conversely relative insensitivity to available capacity). Similar characterizations are possible for other broad categories of application (e.g. data-intensive applications are very bandwidth and latency sensitive - less sensitive to jitter). Roughly following the ITU's lead (re: Y-1541), five major categories of application can be identified: o Real-time (two sub-categories) o VoIP o Video o Transactional (e.g. banking) o Data (e.g. backup/recover) o Best Effort (e.g. email, Web) A protocol like UDP could conceivably be associated with any of the above (although not typical for all categories). To address the "performance of UDP" would seem less meaningful than the "performance of ". Of course, the "performance of RTP" as a more specific example narrows the implied application set. In some cases, the protocol is one-to-one with a particular application (e.g. NTP). In any case, simply addressing protocol overlooks some of the implications of the type of application utilizing the protocol. What would seem missing from the draft charter is a reference to some sort of framework (instead of the implicit "set of L3 and higher network protocols") that bridges metrics and performance with at least "types of applications". I am NOT recommending that PMOL be a replacement for any application-specific performance initiative. Rather that it naturally bridge the gap between the level of IPPM (and related) and application-specific groups such as AVT or related ITU groups. Otherwise, the breadth of the perceived need, to which PMOL has been conceived as a response, may not be fully addressed. Loki Jorgenson Apparent Networks t 604 433 2333 ext 105 m 604 250-4642 -----Original Message----- From: Al Morton [mailto:acmorton@att.com]=20 Sent: Friday, September 21, 2007 6:24 AM To: pmol@ietf.org Subject: [PMOL] Draft Charter for discussion Here's the initial draft of the charter. Comments welcome, of course. Al Proposed Charter (0.0) Performance Metrics at Other Layers WG (PMOL) _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYmg1-000268-AE; Fri, 21 Sep 2007 13:52:29 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IYmfz-0001wT-BB for pmol-confirm+ok@megatron.ietf.org; Fri, 21 Sep 2007 13:52:27 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYmfy-0001cJ-9O for pmol@ietf.org; Fri, 21 Sep 2007 13:52:26 -0400 Received: from mail146.messagelabs.com ([216.82.245.131]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IYmfm-0004hj-QH for pmol@ietf.org; Fri, 21 Sep 2007 13:52:14 -0400 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-5.tower-146.messagelabs.com!1190397133!20113863!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.20.53] Received: (qmail 30748 invoked from network); 21 Sep 2007 17:52:13 -0000 Received: from sbcsmtp6.sbc.com (HELO mlph073.enaf.sfdc.sbc.com) (144.160.20.53) by server-5.tower-146.messagelabs.com with AES256-SHA encrypted SMTP; 21 Sep 2007 17:52:13 -0000 Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlph073.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LHqDRM001359 for ; Fri, 21 Sep 2007 13:52:13 -0400 Received: from mlph070.sfdc.sbc.com (mlph070.sfdc.sbc.com [144.155.224.139]) by mlph073.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LHq9dM000894 for ; Fri, 21 Sep 2007 13:52:09 -0400 Received: from sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlph070.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LHq9kC020893 for ; Fri, 21 Sep 2007 13:52:09 -0400 Received: from maillennium.att.com (mailgw1.maillennium.att.com [135.25.114.99]) by mlph070.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LHq9Xq020874 for ; Fri, 21 Sep 2007 13:52:09 -0400 Message-Id: <200709211752.l8LHq9Xq020874@mlph070.sfdc.sbc.com> Received: from acmt.att.com (acmt.mt.att.com[135.16.251.73](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20070921175209gw10010gf0e>; Fri, 21 Sep 2007 17:52:09 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 21 Sep 2007 13:51:42 -0400 To: Daryl Malas From: Al Morton Subject: Re: [PMOL] Draft Charter for discussion In-Reply-To: <1190394342.1297.232.camel@montag.eng.level3.com> References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> <200709211402.l8LE2Cnl013502@mlph070.sfdc.sbc.com> <1190394342.1297.232.camel@montag.eng.level3.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: 1.5 (+) X-Scan-Signature: 7a6398bf8aaeabc7a7bb696b6b0a2aad Cc: pmol@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org At 01:05 PM 9/21/2007, Daryl Malas wrote: >In addition to the list below, I think it should be noted the PMOL WG >will also take advantage of expertise in the relative IETF working >groups. I appreciate your point, and I think the emphasis on cross-area review in the publication process covers it to some degree (IOW, it's effectively in every charter). Al _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYlqy-0001Yf-OX; Fri, 21 Sep 2007 12:59:44 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IYlqw-0001NW-Bf for pmol-confirm+ok@megatron.ietf.org; Fri, 21 Sep 2007 12:59:42 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYlqv-0000vm-J3 for pmol@ietf.org; Fri, 21 Sep 2007 12:59:41 -0400 Received: from mail20.messagelabs.com ([216.82.245.67]) by chiedprmail1.ietf.org with smtp (Exim 4.43) id 1IYlqr-0002Np-2M for pmol@ietf.org; Fri, 21 Sep 2007 12:59:37 -0400 X-VirusChecked: Checked X-Env-Sender: daryl@level3.net X-Msg-Ref: server-21.tower-20.messagelabs.com!1190393976!35587892!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [209.245.18.106] Received: (qmail 31225 invoked from network); 21 Sep 2007 16:59:36 -0000 Received: from unknown.level3.net (HELO f10bb8-10) (209.245.18.106) by server-21.tower-20.messagelabs.com with SMTP; 21 Sep 2007 16:59:36 -0000 Received: from montag.eng.level3.com (montag.eng.l3.com [10.1.68.57]) by f10bb8-10 (Postfix) with ESMTP id D8CEC4C89; Fri, 21 Sep 2007 16:59:35 +0000 (GMT) Subject: Re: [PMOL] Draft Charter for discussion From: Daryl Malas To: Al Morton In-Reply-To: <200709211402.l8LE2Cnl013502@mlph070.sfdc.sbc.com> References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> <200709211402.l8LE2Cnl013502@mlph070.sfdc.sbc.com> Content-Type: text/plain Date: Fri, 21 Sep 2007 11:05:41 -0600 Message-Id: <1190394342.1297.232.camel@montag.eng.level3.com> Mime-Version: 1.0 X-Mailer: Evolution 2.6.0 (2.6.0-1) Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 2409bba43e9c8d580670fda8b695204a Cc: pmol@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org In addition to the list below, I think it should be noted the PMOL WG will also take advantage of expertise in the relative IETF working groups. On Fri, 2007-09-21 at 10:01 -0400, Al Morton wrote: > At 09:23 AM 9/21/2007, Al Morton wrote: > >Here's the initial draft of the charter. > >Comments welcome, of course. > > a post-press-send comment: > the charter needs a paragraph at the end on coordinating with > other standards development orgs. Something like > > PMOL WG will take advantage of expertise and seek to avoid overlap with > other standards development organizations, such as ETSI STQ, ITU-T SG 12, > ATIS IIF, ATIS PRQC, and ... > > > > > > > _______________________________________________ > PMOL mailing list > PMOL@ietf.org > https://www1.ietf.org/mailman/listinfo/pmol _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYjek-0005Ov-0P; Fri, 21 Sep 2007 10:38:58 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IYjeh-0005ME-V2 for pmol-confirm+ok@megatron.ietf.org; Fri, 21 Sep 2007 10:38:55 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYjeg-0005GD-Ja for pmol@ietf.org; Fri, 21 Sep 2007 10:38:55 -0400 Received: from mail120.messagelabs.com ([216.82.250.83]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IYjea-0002da-B1 for pmol@ietf.org; Fri, 21 Sep 2007 10:38:54 -0400 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-7.tower-120.messagelabs.com!1190385505!26206974!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.128.149] Received: (qmail 3498 invoked from network); 21 Sep 2007 14:38:26 -0000 Received: from sbcsmtp9.sbc.com (HELO flph024.enaf.ffdc.sbc.com) (144.160.128.149) by server-7.tower-120.messagelabs.com with AES256-SHA encrypted SMTP; 21 Sep 2007 14:38:26 -0000 Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LEcPlm021209 for ; Fri, 21 Sep 2007 07:38:25 -0700 Received: from flph023.ffdc.sbc.com (flph023.ffdc.sbc.com [150.234.117.36]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LEcO4F021187 for ; Fri, 21 Sep 2007 07:38:24 -0700 Received: from ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph023.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LEcO49026928 for ; Fri, 21 Sep 2007 07:38:24 -0700 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by flph023.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LEc2ZS026660 for ; Fri, 21 Sep 2007 07:38:18 -0700 Message-Id: <200709211438.l8LEc2ZS026660@flph023.ffdc.sbc.com> Received: from acmt.att.com (acmt.mt.att.com[135.16.251.73](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20070921143802gw10010gcge>; Fri, 21 Sep 2007 14:38:02 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 21 Sep 2007 10:37:34 -0400 To: "STEPHAN Emile RD-CORE-LAN" From: Al Morton Subject: RE: [PMOL] Draft Charter for discussion In-Reply-To: References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: 1.5 (+) X-Scan-Signature: 79899194edc4f33a41f49410777972f8 Cc: pmol@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Emile, (dropping ippm and psamp lists from cc) At 10:25 AM 9/21/2007, STEPHAN Emile RD-CORE-LAN wrote: >Is the WG pmol created in the transport area? The proposal is sponsored by the Ops and Mgmt Area. >To take in account the last IPPM meeting output, I will remove the >passive metrics stuffs from >http://tools.ietf.org/html/draft-ietf-ippm-multimetrics-04. These >performance metrics are performed on real applications traffic >transported by IETF-specified protocols. Are they an emanation of >PSAMP WG or of the (IETF-specified) protocol actually observed? I suppose it could be either, and depends on how the proposed metrics are constructed. Al _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYjSP-0005Au-G4; Fri, 21 Sep 2007 10:26:13 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IYjSO-0005Ad-IR for pmol-confirm+ok@megatron.ietf.org; Fri, 21 Sep 2007 10:26:12 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYjSO-0005AV-8v; Fri, 21 Sep 2007 10:26:12 -0400 Received: from p-mail1.rd.francetelecom.com ([195.101.245.15]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IYjSH-0002HM-Rq; Fri, 21 Sep 2007 10:26:12 -0400 Received: from ftrdmel1.rd.francetelecom.fr ([10.193.117.152]) by ftrdsmtp2.rd.francetelecom.fr with Microsoft SMTPSVC(6.0.3790.1830); Fri, 21 Sep 2007 16:25:50 +0200 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: [PMOL] Draft Charter for discussion Date: Fri, 21 Sep 2007 16:25:57 +0200 Message-ID: In-Reply-To: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PMOL] Draft Charter for discussion Thread-Index: Acf8UufHTiycZ08lRUK2U70TsSIm0wAAW47w References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> From: "STEPHAN Emile RD-CORE-LAN" To: "Al Morton" X-OriginalArrivalTime: 21 Sep 2007 14:25:50.0659 (UTC) FILETIME=[4F55B930:01C7FC5B] X-Spam-Score: 0.0 (/) X-Scan-Signature: c83ccb5cc10e751496398f1233ca9c3a Cc: psamp@ietf.org, pmol@ietf.org, ippm@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Hi Al, Is the WG pmol created in the transport area? To take in account the last IPPM meeting output, I will remove the = passive metrics stuffs from = http://tools.ietf.org/html/draft-ietf-ippm-multimetrics-04. These = performance metrics are performed on real applications traffic = transported by IETF-specified protocols. Are they an emanation of PSAMP = WG or of the (IETF-specified) protocol actually observed? Regards Emile > -----Message d'origine----- > De=A0: Al Morton [mailto:acmorton@att.com] > Envoy=E9=A0: vendredi 21 septembre 2007 15:24 > =C0=A0: pmol@ietf.org > Objet=A0: [PMOL] Draft Charter for discussion >=20 > Here's the initial draft of the charter. > Comments welcome, of course. >=20 > Al >=20 > Proposed Charter (0.0) >=20 > Performance Metrics at Other Layers WG (PMOL) >=20 > There are often uncertainties about the performance and > suitability of new technologies and applications for their intended > audience, and the Internet is no exception. Most uncertainties are > effectively addressed through quantified assessment of key performance > indicators. Standardized performance metrics add the desirable = features > of consistent implementation, interpretation, and comparison. >=20 > Although the IETF has two Working Groups dedicated to the development > of performance metrics, they each have strict limitations in their > charters: >=20 > - The Benchmarking Methodology WG has addressed a range of = networking > technologies and protocols in their long history (such as IEEE 802.3, > ATM, Frame Relay, and Routing Protocols), but the charter strictly > limits their performance characterizations to the laboratory = environment. >=20 > - The IP Performance Metrics WG has the mandate to develop metrics > applicable to live IP networks, but it is specifically prohibited from > developing metrics that characterize traffic (such as a VoIP stream). >=20 > The IETF also has current and completed activities related to the > reporting of application performance metrics (e.g. RAQMON) and is > also actively involved in the development of reliable transport > protocols which would affect the relationship between IP performance > and application performance. >=20 > Thus there is a gap in the currently chartered coverage of IETF WGs: > development of performance metrics for IP-based applications that > operate over UDP, TCP, SCTP, DCCP, Forward Error Correction (FEC) > and other robust transport protocols, and that can be used to > characterize traffic on live networks. >=20 > The working group will focus on the completion of two RFCs: >=20 > 1. A PMOL framework and guidelines memo that includes the motivation > of work to define performance metrics for applications transported > on IETF-specified protocols, and how that work fills a need and a = gap > in IETF-chartered work. The framework will describe the necessary > elements of performance metric drafts and the various types of = metrics > that may be prepared in this work. The framework will also address = the > need to specify the intended audience and the motivation for the > performance metrics. There will also be guidelines for a = performance > metric development process that includes entry criteria for > new proposals (how a proposal might be evaluated for possible > endorsement by a protocol development working group), and how a > successful proposal will be developed by PMOL WG in cooperation = with a > protocol development WG. >=20 > 2. A proof-of-concept RFC defining performance metrics for SIP, based = on > draft-malas-performance-metrics. This memo would serve as an = example > of > the framework and the PMOL development process in the IETF. >=20 > Discussion of new work proposals is strongly discouraged in the PMOL > WG, except to advise a protocol development WG when they are = evaluating > a new work proposal for related performance metrics. >=20 > The PMOL WG will also be guided by a document describing how memos > defining performance metrics are intended to advance along the IETF > Standards track (draft-bradner-metricstest). >=20 > Milestones > June 08 SIP Performance Metrics Draft to AD Review > Sept 08 PMOL Framework and Guidelines Draft to AD Review >=20 >=20 >=20 >=20 > _______________________________________________ > PMOL mailing list > PMOL@ietf.org > https://www1.ietf.org/mailman/listinfo/pmol _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYj5d-0008JS-5C; Fri, 21 Sep 2007 10:02:41 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IYj5b-0008CE-Bl for pmol-confirm+ok@megatron.ietf.org; Fri, 21 Sep 2007 10:02:39 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYj5b-0008Az-1G for pmol@ietf.org; Fri, 21 Sep 2007 10:02:39 -0400 Received: from mail146.messagelabs.com ([216.82.245.131]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IYj5V-0000p4-PP for pmol@ietf.org; Fri, 21 Sep 2007 10:02:39 -0400 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-10.tower-146.messagelabs.com!1190383342!8702239!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.20.53] Received: (qmail 15436 invoked from network); 21 Sep 2007 14:02:22 -0000 Received: from sbcsmtp6.sbc.com (HELO mlph073.enaf.sfdc.sbc.com) (144.160.20.53) by server-10.tower-146.messagelabs.com with AES256-SHA encrypted SMTP; 21 Sep 2007 14:02:22 -0000 Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlph073.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LE2Mtd018846 for ; Fri, 21 Sep 2007 10:02:22 -0400 Received: from mlph070.sfdc.sbc.com (mlph070.sfdc.sbc.com [144.155.224.139]) by mlph073.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LE2HRN018322 for ; Fri, 21 Sep 2007 10:02:17 -0400 Received: from sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlph070.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LE2Gtj013664 for ; Fri, 21 Sep 2007 10:02:16 -0400 Received: from maillennium.att.com (mailgw1.maillennium.att.com [135.25.114.99]) by mlph070.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LE2Cnl013502 for ; Fri, 21 Sep 2007 10:02:12 -0400 Message-Id: <200709211402.l8LE2Cnl013502@mlph070.sfdc.sbc.com> Received: from acmt.att.com (acmt.mt.att.com[135.16.251.73](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20070921140212gw10010gbre>; Fri, 21 Sep 2007 14:02:12 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 21 Sep 2007 10:01:45 -0400 To: pmol@ietf.org From: Al Morton Subject: Re: [PMOL] Draft Charter for discussion In-Reply-To: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> References: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: -2.5 (--) X-Scan-Signature: 08170828343bcf1325e4a0fb4584481c X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org At 09:23 AM 9/21/2007, Al Morton wrote: >Here's the initial draft of the charter. >Comments welcome, of course. a post-press-send comment: the charter needs a paragraph at the end on coordinating with other standards development orgs. Something like PMOL WG will take advantage of expertise and seek to avoid overlap with other standards development organizations, such as ETSI STQ, ITU-T SG 12, ATIS IIF, ATIS PRQC, and ... _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYiUo-0002r3-Qh; Fri, 21 Sep 2007 09:24:38 -0400 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1IYiUn-0002pK-Jw for pmol-confirm+ok@megatron.ietf.org; Fri, 21 Sep 2007 09:24:37 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IYiUj-0002jT-79 for pmol@ietf.org; Fri, 21 Sep 2007 09:24:33 -0400 Received: from mail120.messagelabs.com ([216.82.250.83]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IYiUi-0003Cj-Dh for pmol@ietf.org; Fri, 21 Sep 2007 09:24:32 -0400 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-13.tower-120.messagelabs.com!1190381068!13545730!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.128.149] Received: (qmail 19162 invoked from network); 21 Sep 2007 13:24:29 -0000 Received: from sbcsmtp9.sbc.com (HELO flph024.enaf.ffdc.sbc.com) (144.160.128.149) by server-13.tower-120.messagelabs.com with AES256-SHA encrypted SMTP; 21 Sep 2007 13:24:29 -0000 Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LDOSOB005841 for ; Fri, 21 Sep 2007 06:24:28 -0700 Received: from flph023.ffdc.sbc.com (flph023.ffdc.sbc.com [150.234.117.36]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LDONYf005811 for ; Fri, 21 Sep 2007 06:24:23 -0700 Received: from ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph023.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LDONaN031146 for ; Fri, 21 Sep 2007 06:24:23 -0700 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by flph023.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id l8LDOG7M031034 for ; Fri, 21 Sep 2007 06:24:19 -0700 Message-Id: <200709211324.l8LDOG7M031034@flph023.ffdc.sbc.com> Received: from acmt.att.com (acmt.mt.att.com[135.16.251.73](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20070921132416gw10010gb1e>; Fri, 21 Sep 2007 13:24:16 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 21 Sep 2007 09:23:49 -0400 To: pmol@ietf.org From: Al Morton Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: 1.5 (+) X-Scan-Signature: 25620135586de10c627e3628c432b04a Subject: [PMOL] Draft Charter for discussion X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Here's the initial draft of the charter. Comments welcome, of course. Al Proposed Charter (0.0) Performance Metrics at Other Layers WG (PMOL) There are often uncertainties about the performance and suitability of new technologies and applications for their intended audience, and the Internet is no exception. Most uncertainties are effectively addressed through quantified assessment of key performance indicators. Standardized performance metrics add the desirable features of consistent implementation, interpretation, and comparison. Although the IETF has two Working Groups dedicated to the development of performance metrics, they each have strict limitations in their charters: - The Benchmarking Methodology WG has addressed a range of networking technologies and protocols in their long history (such as IEEE 802.3, ATM, Frame Relay, and Routing Protocols), but the charter strictly limits their performance characterizations to the laboratory environment. - The IP Performance Metrics WG has the mandate to develop metrics applicable to live IP networks, but it is specifically prohibited from developing metrics that characterize traffic (such as a VoIP stream). The IETF also has current and completed activities related to the reporting of application performance metrics (e.g. RAQMON) and is also actively involved in the development of reliable transport protocols which would affect the relationship between IP performance and application performance. Thus there is a gap in the currently chartered coverage of IETF WGs: development of performance metrics for IP-based applications that operate over UDP, TCP, SCTP, DCCP, Forward Error Correction (FEC) and other robust transport protocols, and that can be used to characterize traffic on live networks. The working group will focus on the completion of two RFCs: 1. A PMOL framework and guidelines memo that includes the motivation of work to define performance metrics for applications transported on IETF-specified protocols, and how that work fills a need and a gap in IETF-chartered work. The framework will describe the necessary elements of performance metric drafts and the various types of metrics that may be prepared in this work. The framework will also address the need to specify the intended audience and the motivation for the performance metrics. There will also be guidelines for a performance metric development process that includes entry criteria for new proposals (how a proposal might be evaluated for possible endorsement by a protocol development working group), and how a successful proposal will be developed by PMOL WG in cooperation with a protocol development WG. 2. A proof-of-concept RFC defining performance metrics for SIP, based on draft-malas-performance-metrics. This memo would serve as an example of the framework and the PMOL development process in the IETF. Discussion of new work proposals is strongly discouraged in the PMOL WG, except to advise a protocol development WG when they are evaluating a new work proposal for related performance metrics. The PMOL WG will also be guided by a document describing how memos defining performance metrics are intended to advance along the IETF Standards track (draft-bradner-metricstest). Milestones June 08 SIP Performance Metrics Draft to AD Review Sept 08 PMOL Framework and Guidelines Draft to AD Review _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol