Subject: Re: [PMOL] Suggested updates to proposed charter wording
From: Al Morton <[email-masked]>
To: Henk Uijterwaal <[email-masked]>, "Romascanu, Dan (Dan)" <[email-masked]>
Cc: [email-masked], "Scott O. Bradner" <[email-masked]>
Date: Wed, 26 Sep 2007 08:28:13 -0400
List-Id: Performance Metrics at Other Layers <pmol.ietf.org>

Henk,

Following-up here to close the loop on this,
because this also addresses the issue Lars raised.
I agree that further discussion could be
on the IPPM list...

At 04:09 AM 9/26/2007, Henk Uijterwaal wrote:
>Romascanu, Dan (Dan) wrote:
>>Actually Scott resubmitted his draft in June and then submitted a
>>revised version in August. Henk's observation is correct however to the
>>extent that this is still an Internet-Draft, so referring to it as
>>anything but as 'work-in-progress' is not appropriate.
>
>Somebody correct me if I'm wrong but the June version was a resubmit of
>an old draft, and the August version says near the end in difference
>between the 02 and 03 versions: "correct author list".

There was a small group who discussed the draft with the authors
before the June release of 02.  One of the changes responded to
my suggestion for a test plan (where the accuracy/equivalency issue
would be dealt with up front, on a case-by-case basis),
and the new text in section 5 is partly based on Vern's reply, too:

    "...The implementation report must include a specific
    plan to test the specific metrics in the RFC in lab or real-world
    networks and reports of the tests performed with two or more
    implementations of the software.  The test plan should cover key
    parts of the specification, speak to the accuracy required for each
    aspect measured and thus define "statistically equivalent" for the
    specific metrics being tested.  Ideally, the test plan would co-
    evolve with the development of the metric, since that's when people
    have the most context in their thinking regarding the different
    subtleties that can arise."

IOW, it made more sense to me to defer this question of equivalency
to a specific metric, and allow the metricstest memo to move on
and define the process (which can be accomplished for a general case).

Lars -
This means that the open issue you identified would be addressed
in the test plan memo, and context of each metric.

There may be other differences, but this is one.

Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

From: Lars Eggert <[email-masked]>
Subject: Re: [PMOL] Suggested updates to proposed charter wording
Date: Wed, 26 Sep 2007 12:09:24 +0300
To: ext Alan Clark <[email-masked]>
Cc: [email-masked]

--===============0666142055==
Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-56--110316069;
	protocol="application/pkcs7-signature"

--Apple-Mail-56--110316069
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset=US-ASCII;
	delsp=yes;
	format=flowed

On 2007-9-25, at 21:22, ext Alan Clark wrote:
> 1. A PMOL framework and guidelines memo that outlines the motivation
>     for work to define performance metrics for applications  
> transported
>     over IETF-specified protocols, how that work fills a need and  
> any gap
>     in IETF-chartered work. The framework will describe the necessary
>     elements of performance metric drafts and the various types of  
> metrics
>     that may be prepared in this work. The framework will also  
> address the
>     need to specify the intended audience and the motivation for the
>     performance metrics. There will also be guidelines for a  
> performance
>     metric development process that includes entry criteria for
>     new proposals (how a proposal might be evaluated for possible
>     endorsement by a protocol development working group), and how a
>     successful proposal will be developed by PMOL WG in cooperation  
> with a
>     protocol development WG.

Some of the content of this proposed WG deliverable is what we  
typically want to see already during the BOF proposal phase:

Specifically, the "motivation for work", "how that work fills a need  
and any gap in IETF-chartered work", "motivation for the performance  
metrics" seem pretty relevant already when asking for a BOF.

Likewise, information on "the various types of metrics that may be  
prepared in this work" seem important to define the scope for the  
planned WG. "Entry criteria for new proposals" is also something that  
should be in the proposed charter already.

Lars
--Apple-Mail-56--110316069
Content-Transfer-Encoding: base64
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Disposition: attachment;
	filename=smime.p7s

--Apple-Mail-56--110316069--

--===============0666142055==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

--===============0666142055==--

From: Lars Eggert <[email-masked]>
Subject: Re: [PMOL] Suggested updates to proposed charter wording
Date: Wed, 26 Sep 2007 12:00:30 +0300
To: ext Henk Uijterwaal <[email-masked]>
Cc: [email-masked], "Scott O. Bradner" <[email-masked]>

--===============0073789384==
Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-55--110850528;
	protocol="application/pkcs7-signature"

--Apple-Mail-55--110850528
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset=US-ASCII;
	delsp=yes;
	format=flowed

Hi,

a quick comment about draft-bradner-metrics-test: I had mentioned  
during the IPPM meeting that the draft is pretty complete, with the  
exception of a generally applicable definition what it would mean for  
two implementations of the same metric to generate "statistically  
equivalent" results. Help from metrics experts is surely welcome.

(This list is probably not the right place to discuss this - I  
suggest the IPPM list.)

Lars

--Apple-Mail-55--110850528
Content-Transfer-Encoding: base64
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Disposition: attachment;
	filename=smime.p7s

--Apple-Mail-55--110850528--

--===============0073789384==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

--===============0073789384==--

Date: Wed, 26 Sep 2007 10:09:06 +0200
From: Henk Uijterwaal <[email-masked]>
To: "Romascanu, Dan (Dan)" <[email-masked]>
Subject: Re: [PMOL] Suggested updates to proposed charter wording
Cc: [email-masked], "Scott O. Bradner" <[email-masked]>

Romascanu, Dan (Dan) wrote:
> Actually Scott resubmitted his draft in June and then submitted a
> revised version in August. Henk's observation is correct however to the
> extent that this is still an Internet-Draft, so referring to it as
> anything but as 'work-in-progress' is not appropriate. 

Somebody correct me if I'm wrong but the June version was a resubmit of
an old draft, and the August version says near the end in difference
between the 02 and 03 versions: "correct author list".

Henk

-- 
------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.amsterdamned.org/~henk
P.O.Box 10096          Singel 258         Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
The Netherlands        The Netherlands    Mobile: +31.6.55861746
------------------------------------------------------------------------------

# Lawyer: "Now sir, I'm sure you are an intelligent and honest man--"
# Witness: "Thank you. If I weren't under oath, I'd return the compliment."

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [PMOL] Suggested updates to proposed charter wording
Date: Wed, 26 Sep 2007 09:44:27 +0200
From: "Romascanu, Dan (Dan)" <[email-masked]>
To: "Henk Uijterwaal" <[email-masked]>, <[email-masked]>
Cc: "Scott O. Bradner" <[email-masked]>

Actually Scott resubmitted his draft in June and then submitted a
revised version in August. Henk's observation is correct however to the
extent that this is still an Internet-Draft, so referring to it as
anything but as 'work-in-progress' is not appropriate.=20

Dan

=20
=20

> -----Original Message-----
> From: Henk Uijterwaal [mailto:[email-masked]]=20
> Sent: Wednesday, September 26, 2007 7:26 AM
> To: [email-masked]
> Subject: Re: [PMOL] Suggested updates to proposed charter wording
>=20
> Al, others,
>=20
> > The PMOL WG will also be guided by a document describing how memos=20
> > defining performance metrics are intended to advance along the IETF=20
> > Standards track (draft-bradner-metrics-test).
>=20
> How can the PMOL group be guided by a draft that hasn't seen any work
> for the last 5+ years and expired a long time ago?   I do agree
> the metrics-test draft is useful, but it should be picked up=20
> by some WG and finished before it serves as guidelines for PMOL.
>=20
> Henk
>=20
>=20
> --
> --------------------------------------------------------------
> ----------------
> Henk Uijterwaal                           Email:=20
> henk.uijterwaal(at)ripe.net
> RIPE Network Coordination Centre         =20
> http://www.amsterdamned.org/~henk
> P.O.Box 10096          Singel 258         Phone: +31.20.5354414
> 1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
> The Netherlands        The Netherlands    Mobile: +31.6.55861746
> --------------------------------------------------------------
> ----------------
>=20
> # Lawyer: "Now sir, I'm sure you are an intelligent and honest man--"
> # Witness: "Thank you. If I weren't under oath, I'd return=20
> the compliment."
>=20
>=20
>=20
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol
>=20

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Wed, 26 Sep 2007 07:26:29 +0200
From: Henk Uijterwaal <[email-masked]>
To: [email-masked]
Subject: Re: [PMOL] Suggested updates to proposed charter wording

Al, others,

> The PMOL WG will also be guided by a document describing how memos 
> defining performance metrics are intended to advance along the IETF 
> Standards track (draft-bradner-metrics-test).

How can the PMOL group be guided by a draft that hasn't seen any work
for the last 5+ years and expired a long time ago?   I do agree
the metrics-test draft is useful, but it should be picked up by some
WG and finished before it serves as guidelines for PMOL.

Henk

-- 
------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.amsterdamned.org/~henk
P.O.Box 10096          Singel 258         Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
The Netherlands        The Netherlands    Mobile: +31.6.55861746
------------------------------------------------------------------------------

# Lawyer: "Now sir, I'm sure you are an intelligent and honest man--"
# Witness: "Thank you. If I weren't under oath, I'd return the compliment."

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

From: "Alan Clark" <[email-masked]>
To: "'Al Morton'" <[email-masked]>
Date: Tue, 25 Sep 2007 14:22:48 -0400
Cc: [email-masked]
Subject: [PMOL] Suggested updates to proposed charter wording

This is a multi-part message in MIME format.

--===============1499824634==
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_010F_01C7FF7F.8D07B310"

This is a multi-part message in MIME format.

------=_NextPart_000_010F_01C7FF7F.8D07B310
Content-Type: text/plain;
	charset="US-ASCII"
Content-Transfer-Encoding: 7bit

Al

A proposed rewording of the initial paragraph and some minor text cleanup

Regards

Alan

 

 

Performance Metrics at Other Layers WG (PMOL)

The successful implementation and operation of IP based applications often
depends on some underlying performance measurement infrastructure that helps
service operators or network managers to recognize when performance is
unsatisfactory and identify problems affecting service quality.
Standardized performance metrics add the desirable features of consistent
implementation, interpretation, and comparison.

The IETF has two Working Groups dedicated to the development of performance
metrics however each has strict limitations in their charters:

  - The Benchmarking Methodology WG has addressed a range of networking
technologies and protocols in their long history (such as IEEE 802.3, ATM,
Frame Relay, and Routing Protocols), but the charter strictly limits their
performance characterizations to the laboratory environment.

  - The IP Performance Metrics WG has the mandate to develop metrics
applicable to the performance of Internet data delivery, but it is
specifically prohibited from developing metrics that characterize traffic
(such as a VoIP stream).

The IETF also has current and completed activities related to the reporting
of application performance metrics (e.g. RAQMON and RTCP XR) and is also
actively involved in the development of reliable transport protocols which
would affect the relationship between IP performance and application
performance.

Thus there is a gap in the currently chartered coverage of IETF WGs:

development of performance metrics for IP-based applications that operate
over UDP, TCP, SCTP, DCCP, Forward Error Correction (FEC) and other robust
transport protocols, and that can be used to characterize traffic on live
networks.

The working group will focus on the completion of two RFCs:

1. A PMOL framework and guidelines memo that outlines the motivation

    for work to define performance metrics for applications transported

    over IETF-specified protocols, how that work fills a need and any gap

    in IETF-chartered work. The framework will describe the necessary

    elements of performance metric drafts and the various types of metrics

    that may be prepared in this work. The framework will also address the

    need to specify the intended audience and the motivation for the

    performance metrics. There will also be guidelines for a performance

    metric development process that includes entry criteria for

    new proposals (how a proposal might be evaluated for possible

    endorsement by a protocol development working group), and how a

    successful proposal will be developed by PMOL WG in cooperation with a

    protocol development WG.

2. A proof-of-concept RFC defining performance metrics for SIP, based on

    draft-malas-performance-metrics.  This memo would serve as an example of

    the framework and the PMOL development process in the IETF.

Discussion of new work proposals is strongly discouraged in the PMOL WG,
except to advise a protocol development WG when they are evaluating a new
work proposal for related performance metrics.

The PMOL WG will also be guided by a document describing how memos defining
performance metrics are intended to advance along the IETF Standards track
(draft-bradner-metrics-test).

Milestones

June 08  SIP Performance Metrics Draft to IESG Review 

Sept 08  PMOL Framework and Guidelines Draft to IESG Review

 

------=_NextPart_000_010F_01C7FF7F.8D07B310
Content-Type: text/html;
	charset="US-ASCII"
Content-Transfer-Encoding: quoted-printable

<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns=3D"http://www.w3.org/TR/REC-html40">

<head>
<meta http-equiv=3DContent-Type content=3D"text/html; =
charset=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 11 (filtered medium)">
<style>
<!--
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:Arial;
	color:windowtext;}
@page Section1
	{size:8.5in 11.0in;
	margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
	{page:Section1;}
-->
</style>

</head>

<body lang=3DEN-US link=3Dblue vlink=3Dpurple>

<div class=3DSection1>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'>Al<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>A proposed =
rewording of the
initial paragraph and some minor text =
cleanup<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'>Regards<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'>Alan<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>Performance Metrics =
at Other
Layers WG (PMOL)<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>The successful
implementation and operation of IP based applications often depends on =
some
underlying performance measurement infrastructure that helps service =
operators
or network managers to recognize when performance is unsatisfactory and =
identify
problems affecting service quality.&nbsp; Standardized performance =
metrics add
the desirable features of consistent implementation, interpretation, and =
comparison.<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>The IETF has two =
Working
Groups dedicated to the development of performance metrics however each =
has
strict limitations in their charters:<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp; - The =
Benchmarking
Methodology WG has addressed a range of networking technologies and =
protocols
in their long history (such as IEEE 802.3, ATM, Frame Relay, and Routing
Protocols), but the charter strictly limits their performance =
characterizations
to the laboratory environment.<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp; - The IP =
Performance
Metrics WG has the mandate to develop metrics applicable to the =
performance of Internet
data delivery, but it is specifically prohibited from developing metrics =
that
characterize traffic (such as a VoIP =
stream).<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>The IETF also has =
current
and completed activities related to the reporting of application =
performance
metrics (e.g. RAQMON and RTCP XR) and is also actively involved in the
development of reliable transport protocols which would affect the =
relationship
between IP performance and application =
performance.<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>Thus there is a gap =
in the
currently chartered coverage of IETF WGs:<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>development of =
performance
metrics for IP-based applications that operate over UDP, TCP, SCTP, =
DCCP,
Forward Error Correction (FEC) and other robust transport protocols, and =
that
can be used to characterize traffic on live =
networks.<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>The working group =
will focus
on the completion of two RFCs:<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>1. A PMOL framework =
and
guidelines memo that outlines the =
motivation<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp; =
for work
to define performance metrics for applications =
transported<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp; =
over
IETF-specified protocols, how that work fills a need and any =
gap<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp; =
&nbsp;in
IETF-chartered work. The framework will describe the =
necessary<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp; =
elements
of performance metric drafts and the various types of =
metrics<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp; =
that may
be prepared in this work. The framework will also address =
the<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp; =
need to
specify the intended audience and the motivation for =
the<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp;
performance metrics. There will also be guidelines for a =
performance<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp; =
metric
development process that includes entry criteria =
for<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp; =
new
proposals (how a proposal might be evaluated for =
possible<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp;
endorsement by a protocol development working group), and how =
a<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp;
successful proposal will be developed by PMOL WG in cooperation with =
a<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp; =
protocol
development WG.<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>2. A =
proof-of-concept RFC
defining performance metrics for SIP, based =
on<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp;
draft-malas-performance-metrics.&nbsp; This memo would serve as an =
example of<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>&nbsp;&nbsp;&nbsp; =
the
framework and the PMOL development process in the =
IETF.<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>Discussion of new =
work
proposals is strongly discouraged in the PMOL WG, except to advise a =
protocol
development WG when they are evaluating a new work proposal for related
performance metrics.<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>The PMOL WG will =
also be
guided by a document describing how memos defining performance metrics =
are
intended to advance along the IETF Standards track =
(draft-bradner-metrics-test).<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'>Milestones<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>June 08&nbsp; SIP
Performance Metrics Draft to IESG Review <o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier New"'>Sept 08&nbsp; PMOL =
Framework
and Guidelines Draft to IESG Review<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'text-autospace:none'><font size=3D2 =
face=3D"Courier New"><span
style=3D'font-size:10.0pt;font-family:"Courier =
New"'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'><o:p>&nbsp;</o:p></span></font></p>

</div>

</body>

</html>

------=_NextPart_000_010F_01C7FF7F.8D07B310--

--===============1499824634==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

--===============1499824634==--

Content-class: urn:content-classes:message
Subject: RE: [PMOL] Draft Charter for discussion
Date: Mon, 24 Sep 2007 09:16:07 +0200
From: "Romascanu, Dan (Dan)" <[email-masked]>
To: "Al Morton" <[email-masked]>, <[email-masked]>
Cc: 

It's the same but better name the WG action. AD review is already out of
the WG responsibility :-)

Dan

=20
=20

> -----Original Message-----
> From: Al Morton [mailto:[email-masked]]=20
> Sent: Monday, September 24, 2007 5:41 AM
> To: Romascanu, Dan (Dan); [email-masked]
> Subject: RE: [PMOL] Draft Charter for discussion
>=20
> At 10:20 AM 9/23/2007, Romascanu, Dan (Dan) wrote:
> >- Guidelines should mention submission to the IESG rather than 'AD=20
> >review'
>=20
> OK. I was thinking of "Publication Request" where the first=20
> step is usually AD review, and IESG review follows.
> Al
>=20

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Sun, 23 Sep 2007 23:40:39 -0400
To: "Romascanu, Dan (Dan)" <[email-masked]>, <[email-masked]>
From: Al Morton <[email-masked]>
Subject: RE: [PMOL] Draft Charter for discussion
Cc: 

At 10:20 AM 9/23/2007, Romascanu, Dan (Dan) wrote:
>- Guidelines should mention submission to the IESG rather than 'AD
>review'

OK. I was thinking of "Publication Request" where
the first step is usually AD review, and IESG review follows.
Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [PMOL] Draft Charter for discussion
Date: Sun, 23 Sep 2007 16:20:40 +0200
From: "Romascanu, Dan (Dan)" <[email-masked]>
To: "Al Morton" <[email-masked]>, <[email-masked]>
Cc: 

Hi Al,=20

I have a couple of comments:=20

- when describing the mandate of IPPM it would be more complete to
mention that IPPM explicitly limits its scope in the charter to a set of
metrics related to Internet data delivery=20
- Guidelines should mention submission to the IESG rather than 'AD
review' and specify the target status for each of the two documents -
BCP for the framework and guidelines document, and Proposed Status for
the SIP metrics document.=20

Also language-like it may be better to refer to 'applications and
sessions' rather than just 'applications' to make clear that with PMOL
we do not target only application layers metrics.=20

Dan

=20
=20

> -----Original Message-----
> From: Al Morton [mailto:[email-masked]]=20
> Sent: Friday, September 21, 2007 3:24 PM
> To: [email-masked]
> Subject: [PMOL] Draft Charter for discussion
>=20
> Here's the initial draft of the charter.
> Comments welcome, of course.
>=20
> Al
>=20
> Proposed Charter (0.0)
>=20
> Performance Metrics at Other Layers WG (PMOL)
>=20
> There are often uncertainties about the performance and=20
> suitability of new technologies and applications for their=20
> intended audience, and the Internet is no exception. Most=20
> uncertainties are effectively addressed through quantified=20
> assessment of key performance indicators.  Standardized=20
> performance metrics add the desirable features of consistent=20
> implementation, interpretation, and comparison.
>=20
> Although the IETF has two Working Groups dedicated to the=20
> development of performance metrics, they each have strict=20
> limitations in their
> charters:
>=20
>   - The Benchmarking Methodology WG has addressed a range of=20
> networking technologies and protocols in their long history=20
> (such as IEEE 802.3, ATM, Frame Relay, and Routing=20
> Protocols), but the charter strictly limits their performance=20
> characterizations to the laboratory environment.
>=20
>   - The IP Performance Metrics WG has the mandate to develop=20
> metrics applicable to live IP networks, but it is=20
> specifically prohibited from developing metrics that=20
> characterize traffic (such as a VoIP stream).
>=20
> The IETF also has current and completed activities related to=20
> the reporting of application performance metrics (e.g.=20
> RAQMON) and is also actively involved in the development of=20
> reliable transport protocols which would affect the=20
> relationship between IP performance and application performance.
>=20
> Thus there is a gap in the currently chartered coverage of IETF WGs:
> development of performance metrics for IP-based applications=20
> that operate over UDP, TCP, SCTP, DCCP, Forward Error=20
> Correction (FEC) and other robust transport protocols, and=20
> that can be used to characterize traffic on live networks.
>=20
> The working group will focus on the completion of two RFCs:
>=20
> 1. A PMOL framework and guidelines memo that includes the motivation
>     of work to define performance metrics for applications transported
>     on IETF-specified protocols, and how that work fills a=20
> need and a gap
>     in IETF-chartered work. The framework will describe the necessary
>     elements of performance metric drafts and the various=20
> types of metrics
>     that may be prepared in this work. The framework will=20
> also address the
>     need to specify the intended audience and the motivation for the
>     performance metrics. There will also be guidelines for a=20
> performance
>     metric development process that includes entry criteria for
>     new proposals (how a proposal might be evaluated for possible
>     endorsement by a protocol development working group), and how a
>     successful proposal will be developed by PMOL WG in=20
> cooperation with a
>     protocol development WG.
>=20
> 2. A proof-of-concept RFC defining performance metrics for=20
> SIP, based on
>     draft-malas-performance-metrics.  This memo would serve=20
> as an example of
>     the framework and the PMOL development process in the IETF.
>=20
> Discussion of new work proposals is strongly discouraged in=20
> the PMOL WG, except to advise a protocol development WG when=20
> they are evaluating a new work proposal for related=20
> performance metrics.
>=20
> The PMOL WG will also be guided by a document describing how=20
> memos defining performance metrics are intended to advance=20
> along the IETF Standards track (draft-bradner-metricstest).
>=20
> Milestones
> June 08  SIP Performance Metrics Draft to AD Review Sept 08 =20
> PMOL Framework and Guidelines Draft to AD Review
>=20
>=20
>=20
>=20
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol
>=20

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Fri, 21 Sep 2007 17:34:24 -0400
To: "Loki Jorgenson" <[email-masked]>, <[email-masked]>
From: Al Morton <[email-masked]>
Subject: RE: [PMOL] Draft Charter for discussion
Cc: 

Loki,

At 04:49 PM 9/21/2007, Loki Jorgenson wrote:
>...What would seem missing from the draft charter is a reference to some
>sort of framework (instead of the implicit "set of L3 and higher network
>protocols") that bridges metrics and performance with at least "types of
>applications". ...

I avoided an explicit framework because the intent is that this WG
be able to work on performance metric proposals both above
and below IP (if it continues beyond this initial charter).

For example, the latest version of  draft-xie-ccamp-lsp-dppm-
is a proposal on control plane performance below IP.
PMOL might be looking at that in a year or so, if it's successful.

In one sentence, this WG might cover vetted work that the other
charters don't, but perhaps that's too broad and blunt.

Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [PMOL] Draft Charter for discussion
Date: Fri, 21 Sep 2007 13:49:32 -0700
From: "Loki Jorgenson" <[email-masked]>
To: "Al Morton" <[email-masked]>, <[email-masked]>
Cc: 

Al - while I can appreciate the careful intention in charter wording, I
would point out that the relevance to "other protocols" to any specific
or general application is not addressed.

You make reference to "VoIP traffic" as an example of an application
case.  Without naming a vendor, a codec, or a context, I note that there
are possible various high-level characterizations about VoIP as an
application - e.g. the usual about loss and jitter sensitivity (and
conversely relative insensitivity to available capacity).  Similar
characterizations are possible for other broad categories of application
(e.g. data-intensive applications are very bandwidth and latency
sensitive - less sensitive to jitter).

Roughly following the ITU's lead (re: Y-1541), five major categories of
application can be identified:
      o Real-time (two sub-categories)
         o VoIP
         o Video
      o Transactional (e.g. banking)
      o Data (e.g. backup/recover)
      o Best Effort (e.g. email, Web)
A protocol like UDP could conceivably be associated with any of the
above (although not typical for all categories).  To address the
"performance of UDP" would seem less meaningful than the "performance of
<application category>".  Of course, the "performance of RTP" as a more
specific example narrows the implied application set.  In some cases,
the protocol is one-to-one with a particular application (e.g. NTP).  In
any case, simply addressing protocol overlooks some of the implications
of the type of application utilizing the protocol.

What would seem missing from the draft charter is a reference to some
sort of framework (instead of the implicit "set of L3 and higher network
protocols") that bridges metrics and performance with at least "types of
applications".

I am NOT recommending that PMOL be a replacement for any
application-specific performance initiative.  Rather that it naturally
bridge the gap between the level of IPPM (and related) and
application-specific groups such as AVT or related ITU groups.
Otherwise, the breadth of the perceived need, to which PMOL has been
conceived as a response, may not be fully addressed.

Loki Jorgenson
Apparent Networks
t   604 433 2333 ext 105
m   604 250-4642

-----Original Message-----
From: Al Morton [mailto:[email-masked]]=20
Sent: Friday, September 21, 2007 6:24 AM
To: [email-masked]
Subject: [PMOL] Draft Charter for discussion

Here's the initial draft of the charter.
Comments welcome, of course.

Al

Proposed Charter (0.0)

Performance Metrics at Other Layers WG (PMOL)

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Fri, 21 Sep 2007 13:51:42 -0400
To: Daryl Malas <[email-masked]>
From: Al Morton <[email-masked]>
Subject: Re: [PMOL] Draft Charter for discussion
Cc: [email-masked]

At 01:05 PM 9/21/2007, Daryl Malas wrote:
>In addition to the list below, I think it should be noted the PMOL WG
>will also take advantage of expertise in the relative IETF working
>groups.

I appreciate your point, and I think the emphasis on
cross-area review in the publication process covers
it to some degree (IOW, it's effectively in every charter).

Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Subject: Re: [PMOL] Draft Charter for discussion
From: Daryl Malas <[email-masked]>
To: Al Morton <[email-masked]>
Date: Fri, 21 Sep 2007 11:05:41 -0600
Cc: [email-masked]

In addition to the list below, I think it should be noted the PMOL WG
will also take advantage of expertise in the relative IETF working
groups.

On Fri, 2007-09-21 at 10:01 -0400, Al Morton wrote:
> At 09:23 AM 9/21/2007, Al Morton wrote:
> >Here's the initial draft of the charter.
> >Comments welcome, of course.
> 
> a post-press-send comment:
> the charter needs a paragraph at the end on coordinating with
> other standards development orgs. Something like
> 
> PMOL WG will take advantage of expertise and seek to avoid overlap with
> other standards development organizations, such as ETSI STQ, ITU-T SG 12,
> ATIS IIF, ATIS PRQC, and ...
> 
> 
> 
> 
> 
> 
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Fri, 21 Sep 2007 10:37:34 -0400
To: "STEPHAN Emile RD-CORE-LAN" <[email-masked]>
From: Al Morton <[email-masked]>
Subject: RE: [PMOL] Draft Charter for discussion
Cc: [email-masked]

Emile,  (dropping ippm and psamp lists from cc)

At 10:25 AM 9/21/2007, STEPHAN Emile RD-CORE-LAN wrote:
>Is the WG pmol created in the transport area?

The proposal is sponsored by the Ops and Mgmt Area.

>To take in account the last IPPM meeting output, I will remove the 
>passive metrics stuffs from 
>http://tools.ietf.org/html/draft-ietf-ippm-multimetrics-04.  These 
>performance metrics are performed on real applications traffic 
>transported by IETF-specified protocols. Are they an emanation of 
>PSAMP WG or of the (IETF-specified) protocol actually observed?

I suppose it could be either, and depends on how the proposed
metrics are constructed.

Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [PMOL] Draft Charter for discussion
Date: Fri, 21 Sep 2007 16:25:57 +0200
From: "STEPHAN Emile RD-CORE-LAN" <[email-masked]>
To: "Al Morton" <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]

Hi Al,

Is the WG pmol created in the transport area?

To take in account the last IPPM meeting output, I will remove the =
passive metrics stuffs from =
http://tools.ietf.org/html/draft-ietf-ippm-multimetrics-04.  These =
performance metrics are performed on real applications traffic =
transported by IETF-specified protocols. Are they an emanation of PSAMP =
WG or of the (IETF-specified) protocol actually observed?

Regards
Emile

> -----Message d'origine-----
> De=A0: Al Morton [mailto:[email-masked]]
> Envoy=E9=A0: vendredi 21 septembre 2007 15:24
> =C0=A0: [email-masked]
> Objet=A0: [PMOL] Draft Charter for discussion
>=20
> Here's the initial draft of the charter.
> Comments welcome, of course.
>=20
> Al
>=20
> Proposed Charter (0.0)
>=20
> Performance Metrics at Other Layers WG (PMOL)
>=20
> There are often uncertainties about the performance and
> suitability of new technologies and applications for their intended
> audience, and the Internet is no exception. Most uncertainties are
> effectively addressed through quantified assessment of key performance
> indicators.  Standardized performance metrics add the desirable =
features
> of consistent implementation, interpretation, and comparison.
>=20
> Although the IETF has two Working Groups dedicated to the development
> of performance metrics, they each have strict limitations in their
> charters:
>=20
>   - The Benchmarking Methodology WG has addressed a range of =
networking
> technologies and protocols in their long history (such as IEEE 802.3,
> ATM, Frame Relay, and Routing Protocols), but the charter strictly
> limits their performance characterizations to the laboratory =
environment.
>=20
>   - The IP Performance Metrics WG has the mandate to develop metrics
> applicable to live IP networks, but it is specifically prohibited from
> developing metrics that characterize traffic (such as a VoIP stream).
>=20
> The IETF also has current and completed activities related to the
> reporting of application performance metrics (e.g. RAQMON) and is
> also actively involved in the development of reliable transport
> protocols which would affect the relationship between IP performance
> and application performance.
>=20
> Thus there is a gap in the currently chartered coverage of IETF WGs:
> development of performance metrics for IP-based applications that
> operate over UDP, TCP, SCTP, DCCP, Forward Error Correction (FEC)
> and other robust transport protocols, and that can be used to
> characterize traffic on live networks.
>=20
> The working group will focus on the completion of two RFCs:
>=20
> 1. A PMOL framework and guidelines memo that includes the motivation
>     of work to define performance metrics for applications transported
>     on IETF-specified protocols, and how that work fills a need and a =
gap
>     in IETF-chartered work. The framework will describe the necessary
>     elements of performance metric drafts and the various types of =
metrics
>     that may be prepared in this work. The framework will also address =
the
>     need to specify the intended audience and the motivation for the
>     performance metrics. There will also be guidelines for a =
performance
>     metric development process that includes entry criteria for
>     new proposals (how a proposal might be evaluated for possible
>     endorsement by a protocol development working group), and how a
>     successful proposal will be developed by PMOL WG in cooperation =
with a
>     protocol development WG.
>=20
> 2. A proof-of-concept RFC defining performance metrics for SIP, based =
on
>     draft-malas-performance-metrics.  This memo would serve as an =
example
> of
>     the framework and the PMOL development process in the IETF.
>=20
> Discussion of new work proposals is strongly discouraged in the PMOL
> WG, except to advise a protocol development WG when they are =
evaluating
> a new work proposal for related performance metrics.
>=20
> The PMOL WG will also be guided by a document describing how memos
> defining performance metrics are intended to advance along the IETF
> Standards track (draft-bradner-metricstest).
>=20
> Milestones
> June 08  SIP Performance Metrics Draft to AD Review
> Sept 08  PMOL Framework and Guidelines Draft to AD Review
>=20
>=20
>=20
>=20
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Fri, 21 Sep 2007 10:01:45 -0400
To: [email-masked]
From: Al Morton <[email-masked]>
Subject: Re: [PMOL] Draft Charter for discussion

At 09:23 AM 9/21/2007, Al Morton wrote:
>Here's the initial draft of the charter.
>Comments welcome, of course.

a post-press-send comment:
the charter needs a paragraph at the end on coordinating with
other standards development orgs. Something like

PMOL WG will take advantage of expertise and seek to avoid overlap with
other standards development organizations, such as ETSI STQ, ITU-T SG 12,
ATIS IIF, ATIS PRQC, and ...

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Fri, 21 Sep 2007 09:23:49 -0400
To: [email-masked]
From: Al Morton <[email-masked]>
Subject: [PMOL] Draft Charter for discussion

Here's the initial draft of the charter.
Comments welcome, of course.

Al

Proposed Charter (0.0)

Performance Metrics at Other Layers WG (PMOL)

There are often uncertainties about the performance and
suitability of new technologies and applications for their intended
audience, and the Internet is no exception. Most uncertainties are
effectively addressed through quantified assessment of key performance
indicators.  Standardized performance metrics add the desirable features
of consistent implementation, interpretation, and comparison.

Although the IETF has two Working Groups dedicated to the development
of performance metrics, they each have strict limitations in their
charters:

  - The Benchmarking Methodology WG has addressed a range of networking
technologies and protocols in their long history (such as IEEE 802.3,
ATM, Frame Relay, and Routing Protocols), but the charter strictly
limits their performance characterizations to the laboratory environment.

  - The IP Performance Metrics WG has the mandate to develop metrics
applicable to live IP networks, but it is specifically prohibited from
developing metrics that characterize traffic (such as a VoIP stream).

The IETF also has current and completed activities related to the
reporting of application performance metrics (e.g. RAQMON) and is
also actively involved in the development of reliable transport
protocols which would affect the relationship between IP performance
and application performance.

Thus there is a gap in the currently chartered coverage of IETF WGs:
development of performance metrics for IP-based applications that
operate over UDP, TCP, SCTP, DCCP, Forward Error Correction (FEC)
and other robust transport protocols, and that can be used to
characterize traffic on live networks.

The working group will focus on the completion of two RFCs:

1. A PMOL framework and guidelines memo that includes the motivation
    of work to define performance metrics for applications transported
    on IETF-specified protocols, and how that work fills a need and a gap
    in IETF-chartered work. The framework will describe the necessary
    elements of performance metric drafts and the various types of metrics
    that may be prepared in this work. The framework will also address the
    need to specify the intended audience and the motivation for the
    performance metrics. There will also be guidelines for a performance
    metric development process that includes entry criteria for
    new proposals (how a proposal might be evaluated for possible
    endorsement by a protocol development working group), and how a
    successful proposal will be developed by PMOL WG in cooperation with a
    protocol development WG.

2. A proof-of-concept RFC defining performance metrics for SIP, based on
    draft-malas-performance-metrics.  This memo would serve as an example of
    the framework and the PMOL development process in the IETF.

Discussion of new work proposals is strongly discouraged in the PMOL
WG, except to advise a protocol development WG when they are evaluating
a new work proposal for related performance metrics.

The PMOL WG will also be guided by a document describing how memos
defining performance metrics are intended to advance along the IETF
Standards track (draft-bradner-metricstest).

Milestones
June 08  SIP Performance Metrics Draft to AD Review
Sept 08  PMOL Framework and Guidelines Draft to AD Review

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol