Subject: [PMOL] [sipping] New revision to SIP End-to-End Performance Metrics
From: "Daryl Malas" <[email-masked]>
To: <[email-masked]>
Cc: [email-masked]
Date: Fri, 29 Feb 2008 14:32:03 -0700
List-Id: Performance Metrics at Other Layers <pmol.ietf.org>

All,

A new version of the SIP End-to-End Performance Metrics has been posted.
It can be reviewed here:

http://tools.ietf.org/html/draft-ietf-pmol-sip-perf-metrics-00

This update includes the following updated content (mostly minor
clarifications and corrections):

- Updated Introduction and Scope
- Corrected (clarified) Registration Request Delay (RRD) Failures
- Updated Session Disconnect Delay (SDD)
- Updated Session Establishment Efficiency Rate (SEER)

This version includes corrections based on recent feedback on the
mailing list, individual emails, and input from
draft-morton-perf-metrics-framework.

Comments and feedback are appreciated.

Thank you,

Daryl

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

From: "Alan Clark" <[email-masked]>
To: <[email-masked]>
Date: Mon, 25 Feb 2008 10:33:29 -0500
Subject: [PMOL] New Version Notification for draft-morton-perf-metrics-framework-02
Sender: [email-masked]

A new version of I-D, draft-morton-perf-metrics-framework-02.txt has been
successfuly submitted by Alan Clark and posted to the IETF repository.

Filename:	 draft-morton-perf-metrics-framework
Revision:	 02
Title:		 Framework for Performance Metric Development
Creation_date:	 2008-02-25
WG ID:		 Independent Submission
Number_of_pages: 15

Abstract:
This memo describes a framework and guidelines for the development of
performance metrics that are beyond the scope of existing working
group charters in the IETF.  In this version, the memo refers to a
Performance Metrics Entity, or PM Entity, which may in future be a
working group or directorate or a combination of these two.Requirements
Language

_______________________________________________
PMOL mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/pmol

Date: Thu, 21 Feb 2008 20:37:24 -0500
To: [email-masked]
From: Al Morton <[email-masked]>
Subject: [PMOL] Suggestions for the framework draft
Sender: [email-masked]

PMOL,

While reviewing the SIP metrics draft, I found a few places
where the framework draft needed further development.

I suggest some text in several sections below. I'm not
exactly sure where most of them fit in the outline, but
section 4.2 is strawman text for an empty section.

Al
(as a participant)

3.x.  Organization of Results

    The IPPM Framework [RFC2330] organizes the results of metrics into
    three related notions:

    1.  singleton, an elementary instance, or "atomic" value

    2.  sample, a set of singletons with some common properties and some
        varying properties.

    3.  statistic, a value derived from a sample through deterministic
        calculation, such as the mean.

    Many metrics can use this organization for the results, with or
    without the term names used by IPPM working group.  Section 11 of
    [RFC2330] should consulted for further details.

3.y.  Parameters, the variables of a metric

    Metrics are completely defined when all options and input variables
    have been identified and considered.  These variables are sometimes
    left unspecified in a metric definition, and their general name
    indicates that the user must set them and report them with the
    results.  Such variables are called "parameters" in the IPPM metric
    template.  The scope of the metric, the time at which it was
    conducted, the settings for timers and the thresholds for counters
    are all examples of parameters.

    All memos defining performance metrics SHOULD identify ALL key
    parameters for each metric.

3.z.  Packet Measurement Details

    Another key aspect of the IPPM Framework [RFC2330] is the discussion
    of "wire time".  Internet hosts require physical resources to observe
    packet traffic, and they can contribute to the performance observed.
    Further, the observations SHOULD be related to some instant of time
    (such as a specific bit of a packet), because a packet contains many
    bits and exists on physical media long enough to introduce some
    ambiguity.  Section 10.2 of [RFC2330] should be consulted.

3.a.  Identifying and Categorizing the Audience

    Many of the aspects of metric definition and reporting, even the
    selection or determination of the essential metrics, depend on who
    will use the results, and for what purpose.  The question, "How will
    the results be used?" usually yields important factors to consider
    when developing performance metrics.

    All memos defining performance metrics SHOULD identify the primary
    audience and its associated requirements.  The audience can influence
    both the definition of metrics and the methods of measurement.

    The key areas of variation between different metric users include:

    o  Suitability of passive measurements of live traffic, or active
       measurements using dedicated traffic

    o  Measurement in laboratory environment, or on a network of deployed
       devices

    o  Accuracy of the results

    o  Access to measurement points and configuration information

    o  Measurement topology (point-to-point, point-to-multipoint)

    o  Scale of the measurement system

    o  Measurements conducted on-demand, or continuously

    o  Required reporting formats

4.2.  Proposal Approval

    The process depends on the form that the PM Entity ultimately takes,
    Directorate or working group.

    In all cases, the proposal will need to achieve consensus, in the
    corresponding protocol development working group (or alternatively,
    an "Area" working group with broad charter), that there is interest
    and a need for the work.

    IF the PM Entity is a Directorate,

    THEN Approval SHOULD include the following steps

    o  consultation with the PM Directorate, using this framework memo

    o  consultation with Area Director(s)

    o  and possibly IESG approval of a new or revised charter for the
       working group

    IF the PM Entity is a Working Group, and the protocol development
    working group decides to take up the work under its charter,

    THEN the approval is the same as the PM Directorate steps above, with
    the possible additional assignment of a PM Advisor for the work item.

    IF the PM Entity is a Working Group, and the protocol development
    working group decides it does not have sufficient expertise to
    take-up the work, or the proposal falls outside the current charter,

    THEN

    Approval SHOULD include the following steps

    o  identification of protocol expertise to support metric development

    o  consensus in the PM working group that there is interest and a
       need for the work, and that a memo conforming to this framework
       can be successfully developed

    o  consultation with Area Director(s)

    o  IESG approval of a revised charter for the PM working group

_______________________________________________
PMOL mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/pmol