Subject: [PMOL] Comment on framework 01
From: Al Morton <[email-masked]>
To: [email-masked]
Date: Fri, 21 Nov 2008 12:59:06 -0500
List-Id: Performance Metrics at Other Layers <pmol.ietf.org>

Alan,

As we discussed yesterday, please consider this comment as part of the
WG Last Call (message to follow):

Section 3.4.3
...
   (iv) Reporting Model

    The Reporting Model definition is intended to make any relationship
    between the metric and the reporting model clear. There are often
    implied relationships between the method of reporting metrics and the
    metric itself, however these are often not made apparent to the
    implementor. For example, if the metric is a short term running
    average packet delay variation (e.g.  PPDV as defined in RFC3550)
    however this value is reported at intervals of 6-10 seconds
    this results in a sampling model which may have limited accuracy if
    packet delay variation is non-stationary.

I suggest we use the metric name from RFC 3550 to help people find
the reference there, as follows:

    ... For example, if the metric is a short term running
    average source-to-receiver packet spacing difference
    (e.g. D(i,j) as defined in RFC3550), then
    the value is typically reported at intervals of 6-10 seconds.
    This reporting frequency results in a sampling model which
    may have limited accuracy if the inter-packet delay variation
    is non-stationary.

Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

Date: Thu, 20 Nov 2008 17:30:22 -0500
To: [email-masked]
From: Al Morton <[email-masked]>
Subject: Re: [PMOL] Remote Participation URLs IETF-73
Sender: [email-masked]

All the slides are now posted.

At 08:16 PM 11/17/2008, Al Morton wrote:
>For those who wish to participate remotely, we have:
>
>Start time:  Thursday, November 20, 2008, 1740-1930 (GMT-600)
>Afternoon Session III
>Room: Duluth
>
>Much of the info below is available on the tools team agenda:
>http://tools.ietf.org/agenda/73/
>
>Jabber Instructions
>http://jabber.ietf.org/
>Jabber Room
>[email-masked]
>
>Slides, not all posted yet (scroll down to OPS area)
>https://datatracker.ietf.org/meeting/73/materials.html
>
>Unicast Audio
>http://videolab.uoregon.edu/events/ietf/
>pmol is on Channel 7
>
>Al
>pmol co-chair
>
>_______________________________________________
>PMOL mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

Date: Mon, 17 Nov 2008 20:16:31 -0500
To: [email-masked]
From: Al Morton <[email-masked]>
Subject: [PMOL] Remote Participation URLs IETF-73
Sender: [email-masked]

For those who wish to participate remotely, we have:

Start time:  Thursday, November 20, 2008, 1740-1930 (GMT-600)
Afternoon Session III
Room: Duluth

Much of the info below is available on the tools team agenda:
http://tools.ietf.org/agenda/73/

Jabber Instructions
http://jabber.ietf.org/
Jabber Room
[email-masked]

Slides, not all posted yet (scroll down to OPS area)
https://datatracker.ietf.org/meeting/73/materials.html

Unicast Audio
http://videolab.uoregon.edu/events/ietf/
pmol is on Channel 7

Al
pmol co-chair

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

Date: Fri, 14 Nov 2008 08:20:11 -0500
To: "Marian Delkinov" <[email-masked]>, "Alan Clark" <[email-masked]>, "Daryl Malas" <[email-masked]>
From: Al Morton <[email-masked]>
Cc: [email-masked]
Subject: Re: [PMOL] Draft Agenda for pmol at IETF-73
Sender: [email-masked]

Mario,
Thanks for expressing your opinion,
and also for your careful review!
Al

At 04:58 AM 11/14/2008, Marian Delkinov wrote:
>Al, Alan, Daryl,
>
>Since I won't be able to participate in IETF-73, just want to say few
>words about
>2. SIP Metrics Draft
>http://tools.ietf.org/html/draft-ietf-pmol-sip-perf-metrics-02
>
>I went through the document once more and practically I have no more
>comments on it. My last remarks about Chapters 4.7 and 4.8 have been
>taken into account and the document updated. I fully support it.
>
>Best regards!
>Mario.
>
>Marian Delkinov
>Service Layer and Multimedia
>ERICSSON AB
>Sweden
>
>
>
>
>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On Behalf Of
>Al Morton
>Sent: Tuesday, November 04, 2008 10:04 PM
>To: [email-masked]
>Subject: [PMOL] Draft Agenda for pmol at IETF-73
>
>Performance Metrics for Other Layers WG (pmol)
>
>Thursday, November 20, 2008, 1740-1930 Afternoon Session III
>Room: Duluth
>
>Co-Chairs:
>Al Morton <[email-masked]>
>Alan Clark <[email-masked]>
>
>Please help contribute to a successful meeting by reading the drafts and
>references before we meet.
>
>Agenda: Draft of 11/4/08
>
>0. Agenda Bashing
>
>1. WG Status
>http://tools.ietf.org/wg/pmol/
>
>2. SIP Metrics Draft (review WGLC comment resolutions)
>http://tools.ietf.org/html/draft-ietf-pmol-sip-perf-metrics-02
>
>3. Framework and Guidelines Draft
>http://tools.ietf.org/html/draft-ietf-pmol-metrics-framework-01
>
>4. Discussion of Future of PMOL
>   - Is there a continued need for PMOL, as a WG or as a Directorate?
>   - Presentation on Application Performance Monitoring
>
>5. AOB
>
>Please contact the co-chairs directly with any proposals.
>
>To offer comments on PMOL work in progress or the agenda itself, please
>send email to:
>
>                [email-masked]
>
>
>_______________________________________________
>PMOL mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Date: Fri, 14 Nov 2008 10:58:36 +0100
From: "Marian Delkinov" <[email-masked]>
To: "Al Morton" <[email-masked]>, "Alan Clark" <[email-masked]>, "Daryl Malas" <[email-masked]>
Cc: [email-masked]
Subject: Re: [PMOL] Draft Agenda for pmol at IETF-73
Sender: [email-masked]

Al, Alan, Daryl,

Since I won't be able to participate in IETF-73, just want to say few
words about 
2. SIP Metrics Draft
http://tools.ietf.org/html/draft-ietf-pmol-sip-perf-metrics-02

I went through the document once more and practically I have no more
comments on it. My last remarks about Chapters 4.7 and 4.8 have been
taken into account and the document updated. I fully support it.

Best regards!
Mario.

Marian Delkinov 
Service Layer and Multimedia
ERICSSON AB
Sweden

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Al Morton
Sent: Tuesday, November 04, 2008 10:04 PM
To: [email-masked]
Subject: [PMOL] Draft Agenda for pmol at IETF-73

Performance Metrics for Other Layers WG (pmol)

Thursday, November 20, 2008, 1740-1930 Afternoon Session III
Room: Duluth

Co-Chairs:
Al Morton <[email-masked]>
Alan Clark <[email-masked]>

Please help contribute to a successful meeting by reading the drafts and
references before we meet.

Agenda: Draft of 11/4/08

0. Agenda Bashing

1. WG Status
http://tools.ietf.org/wg/pmol/

2. SIP Metrics Draft (review WGLC comment resolutions)
http://tools.ietf.org/html/draft-ietf-pmol-sip-perf-metrics-02

3. Framework and Guidelines Draft
http://tools.ietf.org/html/draft-ietf-pmol-metrics-framework-01

4. Discussion of Future of PMOL
  - Is there a continued need for PMOL, as a WG or as a Directorate?
  - Presentation on Application Performance Monitoring

5. AOB

Please contact the co-chairs directly with any proposals.

To offer comments on PMOL work in progress or the agenda itself, please
send email to:

               [email-masked]

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol
_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

Date: Tue, 04 Nov 2008 16:04:10 -0500
To: [email-masked]
From: Al Morton <[email-masked]>
Subject: [PMOL] Draft Agenda for pmol at IETF-73
Sender: [email-masked]

Performance Metrics for Other Layers WG (pmol)

Thursday, November 20, 2008, 1740-1930 Afternoon Session III
Room: Duluth

Co-Chairs:
Al Morton <[email-masked]>
Alan Clark <[email-masked]>

Please help contribute to a successful meeting by reading the drafts
and references before we meet.

Agenda: Draft of 11/4/08

0. Agenda Bashing

1. WG Status
http://tools.ietf.org/wg/pmol/

2. SIP Metrics Draft (review WGLC comment resolutions)
http://tools.ietf.org/html/draft-ietf-pmol-sip-perf-metrics-02

3. Framework and Guidelines Draft
http://tools.ietf.org/html/draft-ietf-pmol-metrics-framework-01

4. Discussion of Future of PMOL
  - Is there a continued need for PMOL, as a WG or as a Directorate?
  - Presentation on Application Performance Monitoring

5. AOB

Please contact the co-chairs directly with any proposals.

To offer comments on PMOL work in progress or the agenda itself,
please send email to:

               [email-masked]

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Date: Mon, 3 Nov 2008 10:10:36 -0700
From: "Daryl Malas" <[email-masked]>
To: <[email-masked]>
Cc: [email-masked]
Subject: [PMOL] [Sipping] RE: I-D ACTION:draft-ietf-pmol-sip-perf-metrics-02.txt
Sender: [email-masked]

All,

As you might remember, the SIP End-to-End Performance Metrics draft
completed one WGLC round, and I received feedback and additional
comments at the IETF meeting in Dublin.  As you can see below, a new
version of the SIP End-to-End Performance Metrics draft has been posted.
The following is a summary of the changes is this version:

- Separation of failed REGISTER requests from the Registration Request
Delay metric.  These are now within their own metric, Ineffective
Registration Attempts.  These changes are viewed as significantly
improving the RRD metric's output value. 

- Clarification that successful and failed sessions used to calculate
Session Request Delay should not be combined in the same result, as the
two will likely have widely varying results causing a combined result to
be difficult to trend and understand.  Capturing SRD in both failed and
successful session attempts is still perceived as valuable, because the
affect is directly perceived by a human or machine awaiting the result
of a session attempt.

- The metrics resulting in a percentage, have had their calculations
updated to reflect a percentage result.

- A calculation was added for the Session Defects metric.

Regards,

Daryl

----------------
Daryl Malas
CableLabs
(o) +1 303 661 3302
(f) +1 303 661 9199
mailto:[email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of [email-masked]
> Sent: Saturday, November 01, 2008 3:00 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: [PMOL] I-D ACTION:draft-ietf-pmol-sip-perf-metrics-02.txt
> 
> A New Internet-Draft is available from the on-line 
> Internet-Drafts directories.
> This draft is a work item of the Performance Metrics for 
> Other Layers Working Group of the IETF.
> 
> 	Title		: SIP End-to-End Performance Metrics
> 	Author(s)	: D. Malas
> 	Filename	: draft-ietf-pmol-sip-perf-metrics-02.txt
> 	Pages		: 29
> 	Date		: 2008-11-1
> 	
> This document defines a set of metrics and their usage to evaluate 
>    the performance of end-to-end Session Initiation Protocol 
> (SIP) based 
>    services in both production and testing environments.  The 
> purpose of 
>    this document is to combine a set of common metrics, allowing 
>    interoperable performance measurements, easing the comparison of 
>    industry implementations.
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-pmol-sip-perf-m
> etrics-02.txt
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> Below is the data which will enable a MIME compliant mail 
> reader implementation to automatically retrieve the ASCII 
> version of the Internet-Draft.
> 
_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

From: [email-masked]
To: [email-masked]
Date: Sun,  2 Nov 2008 12:15:01 -0800 (PST)
Cc: [email-masked]
Subject: [PMOL] I-D Action:draft-ietf-pmol-metrics-framework-01.txt
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Performance Metrics for Other Layers Working Group of the IETF.

	Title           : Framework for Performance Metric Development
	Author(s)       : A. Clark
	Filename        : draft-ietf-pmol-metrics-framework-01.txt
	Pages           : 14
	Date            : 2008-11-02

This memo describes a framework and guidelines for the development of
performance metrics that are beyond the scope of existing working
group charters in the IETF.  In this version, the memo refers to a
Performance Metrics Entity, or PM Entity, which may in future be a
working group or directorate or a combination of these two.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-pmol-metrics-framework-01.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-pmol-metrics-framework-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID: <[email-masked]>

--NextPart
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

--NextPart--

From: [email-masked]
To: [email-masked]
Date: Sat,  1 Nov 2008 14:00:01 -0700 (PDT)
Cc: [email-masked]
Subject: [PMOL] I-D ACTION:draft-ietf-pmol-sip-perf-metrics-02.txt
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.
This draft is a work item of the Performance Metrics for Other Layers Working Group of the IETF.

	Title		: SIP End-to-End Performance Metrics
	Author(s)	: D. Malas
	Filename	: draft-ietf-pmol-sip-perf-metrics-02.txt
	Pages		: 29
	Date		: 2008-11-1
	
This document defines a set of metrics and their usage to evaluate 
   the performance of end-to-end Session Initiation Protocol (SIP) based 
   services in both production and testing environments.  The purpose of 
   this document is to combine a set of common metrics, allowing 
   interoperable performance measurements, easing the comparison of 
   industry implementations.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-pmol-sip-perf-metrics-02.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-pmol-sip-perf-metrics-02.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID: <[email-masked]>

--NextPart
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

--NextPart--