Subject: [PMOL] AD review of draft-ietf-pmol-metrics-framework-06

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 11 Jan 2011 11:29:00 +0100

Please find below the AD review of draft-ietf-pmol-metrics-framework-06.
This document is in good shape. There is however at least one key issue
which I suggest to be clarified in a revised version of the document
before submitting it to IETF Last Call. 

Meta-issue:

I believe that the document should make the recommendation that the
Performance Metrics Entity more clear. For this purpose I would
recommend a change on the following lines in Section 1.1: 

OLD: 

   This document refers to a Performance Metrics Entity, which will
   become a directorate, whose goal is to coordinate the Performance
   Metric development at the IETF.

NEW: 

   This document refers to a Performance Metrics Entity, whose goal is 
   to advice and support the Performance Metric development at the IETF.
   A recommendation about the Performance Metrics Entity is made in 
   Section 6.6. 

And NEW: 

Section 6.6

    This document recommends that the Performance Metrics Entity be 
    implemented as a directorate in one of the IETF Areas, providing
advice
    and support as described in this document to all areas in the IETF. 

Editorial Issues: 

E1. I suggest to add at the end of Section 1.1 a paragraph that lists
the intended audience for this document including but maybe not
restricted to IETF participants who write performance metrics documents
in the IETF, reviewers of such documents and members of the Performance
Metrics Entity. 

E2. A couple of bullets are duplicated in Section 5.1 (or I cannot see
the differences)

E3. In Section 5.2, (ii) - s/QoS G1000 [G1000]/QoS [G1000]/

E4. In Section 5.3.2 - provide a reference for the G.107 R Factor

E5. Expand BOF, NAT at first occurrences

E6. Section 5.7 - ALL is not a keyword, do not capitalize it

E7. Section 6.1 - 'Proposals SHOULD be vetted by the corresponding
protocol development WG ...' - I think that we should be more clear here
about what 'vetted' means. In most cases this means a charter item
and/or a deliverable in the WG plans

E8. There is some overlap between the 4th and 6th paragraphs in this
section when it comes to relating to other SDOs

E9. In section 6.4 I think that we need to mention that a formal review
is RECOMMENDED by the time the document is reviewed by the Area
Directors, or an IETF Last Call is being conducted - same as expert
reviews are being performed by other directorates

Thanks and Regards,

Dan

---

From: Benoit Claise <[email-masked]>
Date: Thu, 13 Jan 2011 13:44:36 +0100

Thank you Dan for the detailed review.
See inline.
If you agree with the changes, I'll post a new draft version right away.
>     A recommendation about the Performance Metrics Entity is made in
Done.
>      and support as described in this document to all areas in the IETF.
Done. I titled this section 6.6  "Recommendations"

> in the IETF, reviewers of such documents and members of the Performance
       "The intended audience for this document includes, but is not
      restricted to, IETF participants who write performance metrics 
documents
      in the IETF, reviewers of such documents, and members of the 
Performance
      Metrics Entity. "
> E2. A couple of bullets are duplicated in Section 5.1 (or I cannot see
Done.
> E3. In Section 5.2, (ii) - s/QoS G1000 [G1000]/QoS [G1000]/
Done.
> E4. In Section 5.3.2 - provide a reference for the G.107 R Factor
Done.
> E5. Expand BOF, NAT at first occurrences
Done.
> E6. Section 5.7 - ALL is not a keyword, do not capitalize it
Done.
> and/or a deliverable in the WG plans

That makes sense since we're proposing a directorate.
I added "Proposals SHOULD be deliverables of the corresponding protocol 
development WG charters. As such,... " in the text below

OLD:

    Proposals SHOULD be prepared as Internet Drafts, describing the
    Performance Metric and conforming to the qualifications above as much
    as possible.

    Proposals SHOULD be vetted by the corresponding protocol development
    WG prior to discussion by the Performance Metrics Entity.  This
    aspect of the process includes an assessment of the need for the
    Performance Metric proposed and assessment of the support for their
    development in IETF.

NEW:

    Proposals SHOULD be prepared as Internet Drafts, describing the
    Performance Metric and conforming to the qualifications above as much
    as possible.

    _Proposals SHOULD be deliverables of the corresponding protocol development
    WG charters. As such,_the Proposals SHOULD be vetted_by that
    WG_  prior to discussion by the Performance Metrics Entity.  This
    aspect of the process includes an assessment of the need for the
    Performance Metric proposed and assessment of the support for their
    development in IETF.

> section when it comes to relating to other SDOs
Proposal: a combination of the 4th and 6th paragraphs

OLD:

    Proposals SHOULD include an assessment of interaction and/or overlap
    with work in other Standards Development Organizations.

    Proposals SHOULD survey the existing standards work in the area and
    identify additional expertise that might be consulted, or possible
    overlap with other standards development orgs.

NEW:
    Proposals SHOULD include an assessment of interaction and/or overlap
    with work in other Standards Development Organizations.  Proposals SHOULD
    identify additional expertise that might be consulted.

> reviews are being performed by other directorates
Do we actually need to say this, as we wrote

    "Proposals SHOULD be deliverables of the corresponding protocol development
    WG charters"

Regards, Benoit.
> Thanks and Regards,
>
> Dan
>
>
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/pmol

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 13 Jan 2011 16:49:47 +0100

Thanks. As soon as your revised version is submitted I will issue the
IETF Last Call. 
 
To the question at E9 - I believe that the answer is Yes - because this
paragraph emphasizes the need of an MET-DIR (provisional name) review
same as OPS-DIR, SEC-DIR - etc. at the time of an IETF Last Call (the
latest). 
 
Dan

________________________________

	From: Benoit Claise [mailto:[email-masked]] 
	Sent: Thursday, January 13, 2011 2:45 PM
	To: Romascanu, Dan (Dan)
	Cc: [email-masked]
	Subject: Re: [PMOL] AD review of
draft-ietf-pmol-metrics-framework-06

	Thank you Dan for the detailed review.
	See inline.
	If you agree with the changes, I'll post a new draft version
right away.

		Please find below the AD review of
draft-ietf-pmol-metrics-framework-06.
		This document is in good shape. There is however at
least one key issue
		which I suggest to be clarified in a revised version of
the document
		before submitting it to IETF Last Call. 
		
		Meta-issue:
		
		I believe that the document should make the
recommendation that the
		Performance Metrics Entity more clear. For this purpose
I would
		recommend a change on the following lines in Section
1.1: 
		
		OLD: 
		
		   This document refers to a Performance Metrics Entity,
which will
		   become a directorate, whose goal is to coordinate the
Performance
		   Metric development at the IETF.
		
		NEW: 
		
		   This document refers to a Performance Metrics Entity,
whose goal is 
		   to advice and support the Performance Metric
development at the IETF.
		   A recommendation about the Performance Metrics Entity
is made in 
		   Section 6.6. 

	Done.

		And NEW: 
		
		Section 6.6
		
		    This document recommends that the Performance
Metrics Entity be 
		    implemented as a directorate in one of the IETF
Areas, providing
		advice
		    and support as described in this document to all
areas in the IETF. 

	

		Editorial Issues: 
		
		E1. I suggest to add at the end of Section 1.1 a
paragraph that lists
		the intended audience for this document including but
maybe not
		restricted to IETF participants who write performance
metrics documents
		in the IETF, reviewers of such documents and members of
the Performance
		Metrics Entity. 

	      "The intended audience for this document includes, but is
not
	     restricted to, IETF participants who write performance
metrics documents
	     in the IETF, reviewers of such documents, and members of
the Performance
	     Metrics Entity. "

		E2. A couple of bullets are duplicated in Section 5.1
(or I cannot see
		the differences)

	Done.

		E3. In Section 5.2, (ii) - s/QoS G1000 [G1000]/QoS
[G1000]/

	Done.

		E4. In Section 5.3.2 - provide a reference for the G.107
R Factor

	Done.

	Done.

		E6. Section 5.7 - ALL is not a keyword, do not
capitalize it

	Done.

		E7. Section 6.1 - 'Proposals SHOULD be vetted by the
corresponding
		protocol development WG ...' - I think that we should be
more clear here
		about what 'vetted' means. In most cases this means a
charter item
		and/or a deliverable in the WG plans

	That makes sense since we're proposing a directorate.
	I added "Proposals SHOULD be deliverables of the corresponding
protocol development WG charters. As such,... " in the text below
	
	OLD:
	
	   Proposals SHOULD be prepared as Internet Drafts, describing
the
	   Performance Metric and conforming to the qualifications above
as much
	   as possible.
	   Proposals SHOULD be vetted by the corresponding protocol
development
	   WG prior to discussion by the Performance Metrics Entity.
This
	   aspect of the process includes an assessment of the need for
the
	   Performance Metric proposed and assessment of the support for
their
	   development in IETF.
	NEW:

	   Proposals SHOULD be prepared as Internet Drafts, describing
the
	   Performance Metric and conforming to the qualifications above
as much
	   as possible.
	
	   Proposals SHOULD be deliverables of the corresponding
protocol development
	   WG charters. As such, the Proposals SHOULD be vetted by that 
	   WG prior to discussion by the Performance Metrics Entity.
This
	   aspect of the process includes an assessment of the need for
the
	   Performance Metric proposed and assessment of the support for
their
	   development in IETF.

		E8. There is some overlap between the 4th and 6th
paragraphs in this
		section when it comes to relating to other SDOs

	OLD:
	
	   Proposals SHOULD include an assessment of interaction and/or
overlap
	   with work in other Standards Development Organizations.
	
	   Proposals SHOULD survey the existing standards work in the
area and
	   identify additional expertise that might be consulted, or
possible
	   overlap with other standards development orgs.
	
	NEW:
	   Proposals SHOULD include an assessment of interaction and/or
overlap
	   with work in other Standards Development Organizations.
Proposals SHOULD 
	   identify additional expertise that might be consulted.

		E9. In section 6.4 I think that we need to mention that
a formal review
		is RECOMMENDED by the time the document is reviewed by
the Area
		Directors, or an IETF Last Call is being conducted -
same as expert
		reviews are being performed by other directorates

	   "Proposals SHOULD be deliverables of the corresponding
protocol development
	   WG charters"

	Regards, Benoit.

		Thanks and Regards,
		
		Dan