Subject: Re: [PMOL] [ippm] WGLC on pmol-metrics-framework-01

From: Benoit Claise <[email-masked]>
Date: Mon, 26 Oct 2009 15:20:44 +0100

Hi Fardid,

Thanks for your feedback.
> Hi,
>
> I have a number of comments on the PMOL framework draft:
>
> 1.1 Since RAQMON is mentioned, adding an Informative reference to its
> framework is appropriate.
>   
Done.
> 3.3.2 Since Index is a derived metric, it is best to specify how its
> components contribute to its measurement beyond a function definition.
> An index composed of N orthogonal metrics all of which are measurable
> has more "usefulness" than an index whose measurement is based on a
> small subset of these N metrics.
>   
Could you please provide some text for this.
> 3.4.2 (vi) Does measurement point define the measurement domain? If not,
> then measurement domain (intra-, inter-) is another factor to consider.
> Security restrictions may prevent inter-domain measurement of certain
> metrics.
>   
Done.
> 3.4.3 Are both active and passive measurement types within scope? If so,
> then measurement type is another factor to consider.
>   
I realize I forgot this one is the version I just posted.
I'll write this down in my personal to do list.
> 3.5 Some metrics and their measurements may be affected by middleboxes.
> I can think of the following:
>
> 3.5.3 Middlebox presence
>
> Presence of a middlebox, e.g., proxy, NAT, redirect server, session
> border controller (SBC), and application layer gateway (ALG) may add
> variability to or restrict the scope of measurements of a metric.  For
> example, an SBC that does not process RTP loopback packets may block or
> locally terminate this traffic rather then pass it through to its
> target.
>   
Inserted.

Regards, Benoit.
> Thanks,
> Reza Fardid
>
>
>
>
>
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Al Morton
> Sent: Tuesday, December 02, 2008 7:11 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: [ippm] WGLC on pmol-metrics-framework-01
>
> PMOL WG,
> cc IPPM and BMWG WGs,
>
> This message begins a WG Last Call on the following draft:
>
>         Title           : Framework for Performance Metric Development
> 	Author(s)       : A. Clark
> 	Filename        : draft-ietf-pmol-metrics-framework-01.txt
> 	Pages           : 14
> 	Date            : 2008-11-02
>
> 	
> This memo describes a framework and guidelines for the development of
> performance metrics that are beyond the scope of existing working
> group charters in the IETF.  In this version, the memo refers to a
> Performance Metrics Entity, or PM Entity, which may in future be a
> working group or directorate or a combination of these two.
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-pmol-metrics-framework-01
> .txt
>
> There has been discussion of this draft for over a year,
> mostly on the PMOL list and at meetings.  The Author believes he
> has addressed all comments to date.
>
> Please weigh-in on whether or not this Draft should be forwarded
> to the Area Directors for publication as a Standards Track RFC.
> Send your comments to the PMOL list or the co-chairs.
>
> The Last Call will be open till January 2, 2009.
>
> thanks for your review and comments,
> Al Morton
> co-chair, PMOL WG
>
> _______________________________________________
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/pmol
>

---

From: Benoit Claise <[email-masked]>
Date: Mon, 26 Oct 2009 15:17:34 +0100

Loki,

I have added the statement that this framework is valid for both active 
and passive measurements.

All the editorial comments have been taken into account.

Regards, Benoit.
> Forwarded at Loki's request...
>
>> X-VirusChecked: Checked
>> X-Env-Sender: [email-masked]
>> X-Msg-Ref: server-13.tower-121.messagelabs.com!1228755277!24935304!1
>> X-StarScan-Version: 5.5.12.14.2; banners=-,-,-
>> X-Originating-IP: [209.139.228.52]
>> X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG
>> X-AuditID: ac108108-00000e4c00000774-d5-493d514d1d00
>> Subject: RE: [PMOL] WGLC on pmol-metrics-framework-01
>> Date: Mon, 8 Dec 2008 08:54:35 -0800
>> X-MS-Has-Attach:
>> X-MS-TNEF-Correlator:
>> thread-topic: [PMOL] WGLC on pmol-metrics-framework-01
>> thread-index: AclUuJeWbGY6AyloRw+sWjN4K8ltvABneGJgAL/HSyA=
>> From: "Loki Jorgenson" <[email-masked]>
>> To: "Al Morton" <[email-masked]>
>> X-Brightmail-Tracker: AAAAAA==
>>
>> Al - any reason why my post to PMOL did not make it onto the list?
>>
>> Loki Jorgenson
>> Apparent Networks
>> t   604 433 2333 ext 105
>> m   604 250-4642
>>
>> From: Loki Jorgenson
>> Sent: Thursday, December 04, 2008 16:49
>> To: [email-masked]
>> Cc: Alan Clark
>> Subject: RE: [PMOL] WGLC on pmol-metrics-framework-01
>>
>> Pardon my late comments as I have been unavailable for PMOL
>> participation as of late.
>>
>> One request for clarification as well as some tidying of prose.
>>
>> 1) Is there any value in defining the relationship of this framework to
>> active or passive measurement methodologies?
>>
>> Originally I had thought not.  It appeared that this document was
>> intended to be high-level and agnostic to active or passive
>> methodologies, and therefore said nothing differentiate between the two.
>>
>>
>> However, there is a reference in Section 6 (Security Considerations)
>> that states that "security considerations that apply to any active
>> measurements of live networks are relevant here" with a pointer to
>> IPPM-developed RFC4656, OWAMP.  IPPM thus far limits itself to active
>> methodologies.  This very mildly suggests that this framework is
>> specific to active techniques.
>>
>> In Purpose and Scope, the reference is "for... metrics... that can be
>> used to characterize traffic on live networks and services" - which
>> sounds distinctly passive, insofar as "traffic" suggests applications
>> and not active probing.
>>
>> And, various cross-references to other IPPM drafts and RFCs have been
>> directly incorporated (e.g. compagg in at least Section 3.3).  However,
>> examples to the Telchemy burst-loss algorithm and RFC 4546 may be
>> applied to either passive or active.
>>
>> Further, if passive is included, then security considerations specific
>> to passive should apply in Section 6 as well (e.g. anonymization).  I
>> don't have a handy sample reference for that.
>>
>> Not to make more of it than necessary, but maybe it is worth addressing
>> explicitly.  Perhaps a sentence or two within the Purpose and Scope
>> section declaring its applicability with regard to common measurement
>> methodologies.
>>
>> 2) Awkward sentence:
>>
>> Section 3.4.2 (v)
>>
>> "Short intervals or frequent sampling provides a richer source of
>> information that can be helpful in assessing application performance
>> however can lead to excessive measurement data."
>>
>> It is comprehensible but somewhat awkward.  Suggest a couple of changes:
>>
>> "A SHORT SAMPLING INTERVAL or frequent sampling provides a richer source
>> of information that MAY be helpful in assessing application performance
>> BUT MAY ALSO lead to excessive measurement data."
>>
>> (Capitals indicate changes, not use of capitals in the document - e.g.
>> MAY here is not the usual IETF directive convention)
>>
>> 3) Ambiguous sentence:
>>
>> Section 3.4.2 (v)
>>
>> "Long measurement or sampling intervals reduce that amount of reported
>> and collected data however may be insufficient to truly understand
>> application performance or service quality if this varies with time."
>>
>> Semantically challenged although more or less readable.  Suggest:
>>
>> "Long measurement or sampling intervals reduce THE amount of reported
>> and collected data SUCH THAT IT may be insufficient to ___ understand
>> application performance or service quality INSOFAR AS THE MEASURED
>> QUANTITY MAY vary SIGNIFICANTLY with time."
>>
>> 4) Awkward sentence:
>>
>> Section 3.4.3 (iv)
>>
>> "For example, if the metric is a short term running average packet delay
>> variation (e.g.  PPDV as defined in RFC3550) however this value is
>> reported at intervals of 6-10 seconds this results in a sampling model
>> which may have limited accuracy if packet delay variation is
>> non-stationary."
>>
>> A very long sentence with missing or awkward conjunction.  Suggest:
>>
>> "For example, if the metric is a short term running average packet delay
>> variation (e.g.  PPDV as defined in RFC3550) AND (YET) this value is
>> reported at intervals of 6-10 seconds, THE RESULTING MEASUREMENT may
>> have limited accuracy WHEN packet delay variation is non-stationary."
>>
>> 5) Typographic error:
>>
>> Section 3.4.5 (Measurement Timing:)
>>
>> "... due to time-of-day load network load changes."
>>
>> Suggest:
>>
>> "... due to time-of-day changes in network load."
>>
>>
>> 6) Inconsistent notation:
>>
>> Section 3.5.1
>>
>> "... a 10 percent variation ..." (two instances)
>>
>> Could using number word and "percent" for all instances or numeric value
>> and '%'.  Suggest:
>>
>> "... a 10% variation ..." (two instances)
>>
>>
>> End of Comments
>>
>> ------------------------------------------------------------------------
>> -------------
>>
>> Message: 1
>> Date: Tue, 02 Dec 2008 10:11:00 -0500
>> From: Al Morton <[email-masked]>
>> Subject: [PMOL] WGLC on pmol-metrics-framework-01
>> To: [email-masked]
>> Cc: [email-masked], [email-masked]
>>
>> PMOL WG,
>> cc IPPM and BMWG WGs,
>>