Return-Path: X-Original-To: ietfarch-pmol-archive@core3.amsl.com Delivered-To: ietfarch-pmol-archive@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6E32328C0D7; Thu, 31 Jan 2008 20:35:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.499 X-Spam-Level: X-Spam-Status: No, score=-6.499 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from core3.amsl.com ([127.0.0.1]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S7zJHcB4VtaW; Thu, 31 Jan 2008 20:35:30 -0800 (PST) Received: from core3.amsl.com (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7785428C11D; Thu, 31 Jan 2008 20:35:30 -0800 (PST) X-Original-To: pmol@core3.amsl.com Delivered-To: pmol@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E765828C11D for ; Thu, 31 Jan 2008 20:35:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com Received: from core3.amsl.com ([127.0.0.1]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oBjhEZfWT5-O for ; Thu, 31 Jan 2008 20:35:28 -0800 (PST) Received: from sj-iport-2.cisco.com (sj-iport-2-in.cisco.com [171.71.176.71]) by core3.amsl.com (Postfix) with ESMTP id CCE4728C0D7 for ; Thu, 31 Jan 2008 20:35:26 -0800 (PST) Received: from sj-dkim-3.cisco.com ([171.71.179.195]) by sj-iport-2.cisco.com with ESMTP; 31 Jan 2008 20:37:00 -0800 Received: from sj-core-5.cisco.com (sj-core-5.cisco.com [171.71.177.238]) by sj-dkim-3.cisco.com (8.12.11/8.12.11) with ESMTP id m114b0I9029638 for ; Thu, 31 Jan 2008 20:37:00 -0800 Received: from [192.168.4.177] (sjc-fluffy-vpn1.cisco.com [10.25.236.82]) by sj-core-5.cisco.com (8.12.10/8.12.6) with SMTP id m114b0qX000745 for ; Fri, 1 Feb 2008 04:37:00 GMT Message-Id: From: Cullen Jennings To: pmol@ietf.org In-Reply-To: <200801231528.m0NFSn6k017415@alph001.aldc.att.com> Impp: xmpp:cullenfluffyjennings@jabber.org Mime-Version: 1.0 (Apple Message framework v915) Date: Thu, 31 Jan 2008 20:36:39 -0800 References: <200801231528.m0NFSn6k017415@alph001.aldc.att.com> X-Mailer: Apple Mail (2.915) Authentication-Results: sj-dkim-3; header.From=fluffy@cisco.com; dkim=pass ( sig from cisco.com/sjdkim3002 verified; ); Subject: Re: [PMOL] Proposed Adoption of draft-malas-performance-metrics-08 X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: pmol-bounces@ietf.org Errors-To: pmol-bounces@ietf.org Hi, I'd like to make a few comments - some in my role as one of the RAI ADs and some just as an individual contributor. With my AD hat on, This draft was bounced around several places before we figured out a good place to get this work done. I worked with Dan in his OPs AD role and with the chairs of the related WGs to try and determine where we wanted this work to happen. We came to the conclusion that this was the best WG to do the work in. Now that does not mean the IETF has to do this work at all, but if we are going to do it, I think this is the right place to do it. I know that multiple people in the RAI area expressed their opinions that this work was worth doing. Of course I want the work to meet the quality level expected by this WG - that was part of the reasons Dan and I believed this group was the best place to do the work. I would expect that would mean evolving this draft to be consistent with the framework document as they both get closer to WGLC. This evolution certainly could be done after the draft was adopted as a WG document. I don't think the WG has to finish the framework document before they can adopt this and start doing work - both can happen in parallel if the WG choices to do that. The important thing to me is that when this WG sends the drafts to IESG, the WG has consensus that they are of adequate quality and consistency that they should be published as RFCs. I note that we IETF Last Called the charter text that has 2. A proof-of-concept RFC defining performance metrics for SIP, based on draft-malas-performance-metrics. This memo would serve as an example of the framework and the PMOL development process in the IETF. and I don't believe I received any comments about this not being a good starting point. From a RAI point of view, I don't claim to know what is the best starting document to get this work done but the RAI folks would like to see some of the issues this draft addresses described in some RFC. WIth my individual contributor hat on, Of all the proposed drafts that this WG could adopted to meet it milestones, this one seems to be the best :-) I support adopting it as a WG item. I'm sure I don't know what needs to change to make it "right" but once it is a WG draft instead of an individual draft, whatever the consensus of the WG is can get put into this draft. I think this will be the fastest path to finishing the work and having people agree on it. Thanks, Cullen On Jan 23, 2008, at 7:28 AM, Al Morton wrote: > PMOL Working Group: > > A comment period for the Internet-Draft on > > "SIP End-to-End Performance Metrics" > > > > will be open from 23 January 2008 through 6 February 2003. > > A URL for this Internet-Draft is: > http://tools.ietf.org/html/draft-malas-performance-metrics-08 > > The purpose of this comment period is to examine whether there is > PMOL WG > consensus to adopt this draft as the basis for the SIP Performance > Metrics > "Proof of Concept" work item described in the PMOL WG Charter, > http://www.ietf.org/html.charters/pmol-charter.html > and if so to progress this as a Working Group Draft. > > > STATUS > > When this topic was raised at the IETF-70 PMOL session, there > was substantial readership and some support expressed to take-up > this version as the WG draft. There were also some > reservations expressed at the meeting about how well > the current draft matches the (evolving and not yet complete) > framework draft. Thus, completing this milestone will require > further development of the framework (and process) draft, as well. > > Also, this draft has enjoyed interest and review of the SIPPING WG, > and members of the SIP development community continue to apply > their expertise to fine-tune the performance metric definitions. > > Please weigh-in with any comments to this list or to the co-chairs: > and . > > Al Morton > Alan Clark > pmol co-chairs > > Note: The current version of the framework and process draft is here: > http://tools.ietf.org/html/draft-morton-perf-metrics-framework-01 > > > > > _______________________________________________ > PMOL mailing list > PMOL@ietf.org > https://www1.ietf.org/mailman/listinfo/pmol _______________________________________________ PMOL mailing list PMOL@ietf.org http://www.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JKbGA-000166-8r; Thu, 31 Jan 2008 10:23:26 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JKbG8-00015o-Gh for pmol-confirm+ok@megatron.ietf.org; Thu, 31 Jan 2008 10:23:24 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JKbG0-00013Z-Tl for pmol@ietf.org; Thu, 31 Jan 2008 10:23:16 -0500 Received: from mail120.messagelabs.com ([216.82.250.83]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1JKbG0-0006Ur-8f for pmol@ietf.org; Thu, 31 Jan 2008 10:23:16 -0500 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-11.tower-120.messagelabs.com!1201792994!20758121!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.128.149] Received: (qmail 25612 invoked from network); 31 Jan 2008 15:23:14 -0000 Received: from sbcsmtp9.sbc.com (HELO flph024.enaf.ffdc.sbc.com) (144.160.128.149) by server-11.tower-120.messagelabs.com with AES256-SHA encrypted SMTP; 31 Jan 2008 15:23:14 -0000 Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id m0VFNE1b027797 for ; Thu, 31 Jan 2008 07:23:14 -0800 Received: from klph001.kcdc.att.com (klph001.kcdc.att.com [135.188.3.11]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id m0VFN9Kg027718 for ; Thu, 31 Jan 2008 07:23:10 -0800 Received: from kcdc.att.com (localhost.localdomain [127.0.0.1]) by klph001.kcdc.att.com (8.14.0/8.14.0) with ESMTP id m0VFN97f006997 for ; Thu, 31 Jan 2008 09:23:09 -0600 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by klph001.kcdc.att.com (8.14.0/8.14.0) with ESMTP id m0VFN159006712 for ; Thu, 31 Jan 2008 09:23:01 -0600 Message-Id: <200801311523.m0VFN159006712@klph001.kcdc.att.com> Received: from acmt.att.com (martym.mt.att.com[135.16.251.71](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20080131152300gw10010ggge>; Thu, 31 Jan 2008 15:23:00 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 31 Jan 2008 10:23:00 -0500 To: "Daryl Malas" , pmol@ietf.org From: Al Morton Subject: Re: [PMOL] SIP performance metrics draft In-Reply-To: <200801300054.m0U0sktT017157@klph001.kcdc.att.com> References: <200801300054.m0U0sktT017157@klph001.kcdc.att.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: 1.5 (+) X-Scan-Signature: b4a0a5f5992e2a4954405484e7717d8c Cc: X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Hi Daryl, A few more points: At 07:54 PM 1/29/2008, Al Morton wrote: >Clock Accuracy and Error: Since most of the SIP metrics are time >intervals measured at a single UA, time sync between two remote >points is not an issue. Time interval accuracy for the single >clock should be identified as a source of error, however. It will help to designate one of the two time stamps (TB or TS) as "the" time each individual test was conducted. Clearly, TB is the most suitable for this use. TB is the time that is controled in an Active test scenario. I suggest adding this to the current definition of TB: ... TB represents the time at which each request-response test begins, and SHALL be used to designate the time when a particular test was conducted (e.g., The Session Request Delay at "TB" and was measured to be X ms.) OR we could add a Date-Time stamp format, and let TB and TS exist in a locally-defined format, as long as their difference produces sufficient resolution. There are plenty of date-time formats that we could specify: RFC 2330 specifies UTC format for time in section 6.1, when discussing measurement units. The IETF reference for time formats appears to be http://tools.ietf.org/html/rfc1123#section-5.2.14, but there are others such as ISO 8601. NTP format has more resolution, but it is expressed relative to 0h Jan 1 1900. Session Setup: and Session Establishment: The presence of these two definitions begs for a general definition of "Session Failure", especially when so many metrics have "failure" versions. The definition should probably include a reference to the IANA list of Response codes (and thereby, the Reference for each code) http://www.iana.org/assignments/sip-parameters Singletons, Samples, Statistics (as defined in RFC 2330): http://tools.ietf.org/html/rfc2330#section-11 Some one-line definitions for these terms are: Singleton: a single instance of a metric, a single result. Sample: a number of single instances, or singletons. Statistic: a metric derived from a sample by computing some statistic. I think it will help to introduce these (or similar) definitions into the SIP metrics draft (and the FRMWK). Each of the message flows illustrate a Singleton metric, while equations like the Average Registration Request Delay in section 3.1 are examples of Statistical metrics. The Data Collection Section 5.4 touches on the topic of Samples in the last paragraph. Would it be useful to include discussion of how the Sample should be collected for summarization with a Statistic? Perhaps this could be addressed in a short section on Methodology? (This is where it helps to associate a date-time with each request-response singleton, so that the time coverage of the sample and resulting statistic are known.) Al _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JK1EF-0005FS-8I; Tue, 29 Jan 2008 19:55:03 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JK1ED-0005BD-Te for pmol-confirm+ok@megatron.ietf.org; Tue, 29 Jan 2008 19:55:01 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JK1ED-00056J-DQ for pmol@ietf.org; Tue, 29 Jan 2008 19:55:01 -0500 Received: from mail121.messagelabs.com ([216.82.241.195]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1JK1EB-0004Mk-CJ for pmol@ietf.org; Tue, 29 Jan 2008 19:55:01 -0500 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-12.tower-121.messagelabs.com!1201654497!29556899!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.128.149] Received: (qmail 16279 invoked from network); 30 Jan 2008 00:54:58 -0000 Received: from sbcsmtp9.sbc.com (HELO flph024.enaf.ffdc.sbc.com) (144.160.128.149) by server-12.tower-121.messagelabs.com with AES256-SHA encrypted SMTP; 30 Jan 2008 00:54:58 -0000 Received: from enaf.ffdc.sbc.com (localhost.localdomain [127.0.0.1]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id m0U0suFV019135 for ; Tue, 29 Jan 2008 16:54:57 -0800 Received: from klph001.kcdc.att.com (klph001.kcdc.att.com [135.188.3.11]) by flph024.enaf.ffdc.sbc.com (8.14.0/8.14.0) with ESMTP id m0U0sogV019096 for ; Tue, 29 Jan 2008 16:54:54 -0800 Received: from kcdc.att.com (localhost.localdomain [127.0.0.1]) by klph001.kcdc.att.com (8.14.0/8.14.0) with ESMTP id m0U0snA8017172 for ; Tue, 29 Jan 2008 18:54:49 -0600 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by klph001.kcdc.att.com (8.14.0/8.14.0) with ESMTP id m0U0sktT017157 for ; Tue, 29 Jan 2008 18:54:46 -0600 Message-Id: <200801300054.m0U0sktT017157@klph001.kcdc.att.com> Received: from acmt.att.com (unknown[135.210.96.244](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20080130005445gw10010g33e>; Wed, 30 Jan 2008 00:54:46 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 29 Jan 2008 19:54:43 -0500 To: "Daryl Malas" , pmol@ietf.org From: Al Morton Mime-Version: 1.0 X-Spam-Score: -0.8 (/) X-Scan-Signature: a2c12dacc0736f14d6b540e805505a86 Cc: Subject: [PMOL] SIP performance metrics draft X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0348309450==" Errors-To: pmol-bounces@ietf.org --===============0348309450== Content-Type: text/html; charset="us-ascii" Hi Daryl,

Here are a few comments on the SIP draft.
I'm raising some metric development related points,
and trying to harmonize the draft with the pmol framework.

Some of the points imply additional development of  
the pmol framework, and are comments for *both* drafts.
(I've marked these with "FRMWK")

I will have a few more comments soon, but this is
a good start.

regards,
Al

===========================================================
Intro

It's useful to have a Scope, and the current statements of scope
can be found in the Introduction section.  I suggest re-titling
the section "1.0  Introduction and Scope", or splitting into 2 sections.


Active and/or Passive in scope of draft:
I think this question came up along the way:
Is the draft applicable to Active/Passive or both?
I think the answer is both, and suggest to add this:
The metrics defined in this document are applicable in scenarios where the SIP messages launched (into a network under test) are dedicated messages for testing purposes, or where the messages are user-initiated and a portion of the live traffic present.  These two scenarios are sometimes referred to as active and passive measurement, respectively.
FRMWK: I'm not sure where to mention this consideration in the text.
 
Section 3.1 of the framework draft highlights the need to
identify the audience, or users of the metrics
(FRMWK: need to flesh this section out).
Here's a suggestion, it could be combined with the paragraph
that begins, "the scope":
The intended audience for this document can be found among network operators, who often collect information on the responsiveness of the network to customer requests for services.

Section 3 candidate material to add

SIP Configuration:  Various timers influence performance, so the
draft must mention them (at least).  Ideally, the supporting info
for any measurement report would include a list of relevant
configuration variables and their settings (in the originating UA).
T1 looks like a good candidate:
(from http://tools.ietf.org/html/rfc3261#section-17.1.1.1 )
17.1.1.1 Overview of INVITE Transaction
   The INVITE transaction consists of a three-way handshake.  The client
   transaction sends an INVITE, the server transaction sends responses,
   and the client transaction sends an ACK.  For unreliable transports
   (such as UDP), the client transaction retransmits requests at an
   interval that starts at T1 seconds and doubles after every
   retransmission.  T1 is an estimate of the round-trip time (RTT), and
   it defaults to 500 ms.  Nearly all of the transaction timers
   described here scale with T1, and changing T1 adjusts their values.

Elsewhere, the default value of T2 is 4 seconds. We usually list all
the configuration parameters in the BMWG drafts, so that everyone
reports the same ones.


Clock Accuracy and Error:  Since most of the SIP metrics are time
intervals measured at a single UA, time sync between two remote
points is not an issue. Time interval accuracy for the single
clock should be identified as a source of error, however.
Section 10 of RFC 2330 http://tools.ietf.org/html/rfc2330#section-10
discusses clock issues in detail (reference in FRMWK).

"Wire Time": This is another issue described well in
section 10.2 of RFC 2330 http://tools.ietf.org/html/rfc2330#section-10.2
The basic problem is that there is a difference between the time
when a host sees a packet for time-stamping, and the time that the
packet actually appears on the "wire". --===============0348309450== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol --===============0348309450==-- Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JI6be-0000bK-52; Thu, 24 Jan 2008 13:15:18 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JI6bc-0000aG-UG for pmol-confirm+ok@megatron.ietf.org; Thu, 24 Jan 2008 13:15:16 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JI6bc-0000Zw-5w; Thu, 24 Jan 2008 13:15:16 -0500 Received: from ondar.cablelabs.com ([192.160.73.61]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1JI6bb-0004u0-4u; Thu, 24 Jan 2008 13:15:16 -0500 Received: from kyzyl.cablelabs.com (kyzyl.cablelabs.com [10.253.0.7]) by ondar.cablelabs.com (8.13.8/8.13.8) with ESMTP id m0OIFB69027679; Thu, 24 Jan 2008 11:15:11 -0700 Received: from srvxchg3.cablelabs.com (10.5.0.25) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/511/kyzyl.cablelabs.com); Thu, 24 Jan 2008 11:15:10 -0700 (MST) X-Virus-Status: clean(F-Secure/fsigk_smtp/511/kyzyl.cablelabs.com) X-Mimeole: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Date: Thu, 24 Jan 2008 11:15:11 -0700 Message-ID: <160DE07A1C4F8E4AA2715DEC577DA4917B0C39@srvxchg3.cablelabs.com> In-Reply-To: <8E3A1C85FE049E4BB5B4C96B207020B101199F68@esealmw106.eemea.ericsson.se> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Sipping] [PMOL] SIP End-to-End Performance Metrics Thread-Index: AchZ7ZVdC+psk80ASTW3x0Ddfzi+cwCJukqwAAre+OAAjaUCkAAPmpxQ References: <1197672127.7806.53.camel@montag.eng.level3.com><19f9b0170801180816p4bcc4f30kd962ae29a9b4c845@mail.gmail.com> <8E3A1C85FE049E4BB5B4C96B207020B10111A2F1@esealmw106.eemea.ericsson.se> <160DE07A1C4F8E4AA2715DEC577DA4917B0C0E@srvxchg3.cablelabs.com> <8E3A1C85FE049E4BB5B4C96B207020B101199F68@esealmw106.eemea.ericsson.se> From: "Daryl Malas" To: "Marian Delkinov" , "Hadriel Kaplan" X-Approved: ondar X-Spam-Score: 0.0 (/) X-Scan-Signature: f0b5a4216bfa030ed8a6f68d1833f8ae Cc: sipping@ietf.org, pmol@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Mario, At this point, I agree with your assertion to keep SEER as is, unless there is a strong argument to change it. Thanks... Daryl=20 -----Original Message----- From: Marian Delkinov [mailto:marian.delkinov@ericsson.com]=20 Sent: Thursday, January 24, 2008 4:01 AM To: Daryl Malas; Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Daryl, Regarding adding 401/407 to the numerator of SEER, my opinion is that it depends on the idea behind SEER definition. 1) If SEER should complement SER, but exclude the called party behaviour, then I support keeping SEER as it is. This case matches the NER definition in PSTN.=20 2) If SEER should represent the ability of the infrastructure to establish sessions, then I would support adding 401/407 codes to the numerator. (If you remember during our discussion over the last year, I proposed adding all 4xx codes except 408 in the numerator of SEER, but we then agreed that it would cover much larger scope than just excluding the customer behaviour. Perhaps we have to define another metric to cover those cases, if the WG supports that.) =20 Best regards! Mario. -----Original Message----- From: Daryl Malas [mailto:D.Malas@cablelabs.com] Sent: Monday, 21 January, 2008 16:19 To: Marian Delkinov; Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Mario, I agree. This is the traditional difference between SER and SEER, much like the difference between ASR and NER (ITU-T e.411). Now, one thing that might make sense is to add 401/407 to the numerator of the SEER metric algorithm. --Daryl -----Original Message----- From: Marian Delkinov [mailto:marian.delkinov@ericsson.com] Sent: Monday, January 21, 2008 3:45 AM To: Daryl Malas; Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Daryl and Hadriel, I have some comments regarding SER and SEER: > SER - need to specify that a 401/407 would also be subtracted in that=20 > denominator, or better yet the Invite would not be double-counted=20 > period. Also, need to point out that "# of INVITE" is for=20 > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) If we go back to the analogy with PSTN and the ASR defined by ITU, I would suggest leaving SER as it is. Thus the metric would really be useful for estimating the %revenue compared to the total volume of invites, regardless of the reason for failure (as it is in PSTN). For analysis of the different aspects of the infrastructure efficiency I suggest using complementary to SER metrics, as SEER (already defined), and why not defining some more. SEER: Section 3.7 defines SEER, however the formula still uses SER=3D... I suggest using "SEER" in the formula in order to avoid confusion with SER. Best regards! Mario. =20 -----Original Message----- From: Daryl Malas [mailto:dmalas@gmail.com] Sent: Friday, 18 January, 2008 17:17 To: Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics Hadriel, Comments in-line... On 1/17/08, Hadriel Kaplan wrote: > > Hey Daryl, > Comments: > > Failed RRD - I'm a little confused by the wording of how we decide what constitutes a failed registration. Clearly a 401/407 challenge round does not, but you state a second round would. Why? It's totally possible it's not a failure. > I'm also confused what the value/purpose of measuring this delay is. > The section says measuring failed RRD is useful for detecting problems with reaching the intended Registrar - that would be true if you counted transaction timeouts, not request/response delays for auth failures, no? [DM] I agree this could create some false indications of failures. I can update the metric to only determine a failed condition based on a timeout scenario. Also, I'm going to put together a suggested list of metrics from this draft. The topic has come up several times on how many metrics should be included. Since, no one has suggested a list, I will throw one out. > > Failed SRD - need to specify a 401/407 is not a failed SRD (you say any 4xx). > [DM] I agree and will update the draft. > Successful SDD - you say at the end "In these two examples, TB and TS are the same even if the UAC/UAS receives the indicated messages instead of sending them." I'm pretty sure the delay in receiving a Bye and sending the 200 ok for that is not really useful. :) Or at least a very different issue than that measured from sending a Bye and receiving a 200. > [DM] I agree. At one point, someone requested this to be exhaustive in all conditions. I can remove this to indicate relevance only in the situation of initiating the Bye from either the UAC or UAS. > Failed SDD - what do we count the time of a Bye to a 4xx (but not 401/407) or 5xx as? Since those are the failure conditions for the other cases, this seems a bit odd to no longer count it as one here. [DM] I'm not sure I understand this question. > > Failed SDT - this seems to be at odds with the definition of successful SDT. Successful SDT is stopped on sending the Bye, but this one is stopped on timing out that sent Bye. (in other words, every call will thus count as a successful SDT, and some will also count as a failed one) Also, what good is this failed SDT info? You already have Failed SDD. It seems to me an SDT can only be of type "successful", and the timer should be stopped at send/receipt of Bye. [DM] I agree. I will update this to only be between the 200 and Bye. It should be the Bye regardless of whether or not it times out based on the intiator. > > AHR - how does the UAC or UAS know this info? I.e., how does the UAS know what max-forwards the UAC started with, and how does the UAC know what max-forwards the UAS received? [DM] The thought is the clients (or some other in-line monitoring device) would have to capture this information at both ends and hold it in order to correlate and make the determination. This metric is a tricky one and has come up much scrutiny. It will work and could be very useful, but will require some capture outside of SIP in all likelihood. > > SER - need to specify that a 401/407 would also be subtracted in that=20 > denominator, or better yet the Invite would not be double-counted=20 > period. Also, need to point out that "# of INVITE" is for=20 > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) [DM] Are you saying 401/407 should be included with this sentence "...to the total number of attempted INVITE requests less INVITE requests resulting in a 3XX response..."? > > -hadriel > > > -----Original Message----- > > From: Daryl Malas [mailto:daryl@level3.net] > > Sent: Friday, December 14, 2007 5:42 PM > > To: pmol@ietf.org > > Cc: sipping@ietf.org > > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics > > > > At the conference, I introduced the SIP End-to-End Performance=20 > > Metrics draft (draft-malas-performance-metrics-08) to the PMOL working group. > > Although this draft is referenced in the PMOL WG charter, I wanted=20 > > to ask everyone to review the draft and provide feedback of whether=20 > > or not you feel the current version is ready for the WG to accept as > > a working group item. A couple of questions came up, which I think=20 > > should be answered regarding this consideration: > > > > Is the SIPPING WG content with the current set of metrics? > > > > Are there too many? > > > > Do these metrics capture the relevant concerns of performance=20 > > regarding the SIP protocol? > > > > Are these metrics depicted accurately from a SIP protocol perspective? > > > > In addition to these questions, I have a couple of tasks from the=20 > > PMOL group to include in the next revision. > > > > Here is a link to the draft: > > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics- > > 08.txt > > > > Thanks... > > > > Daryl > > > > > > > > _______________________________________________ > > Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping > > This list is for NEW development of the application of SIP Use=20 > > sip-implementors@cs.columbia.edu for questions on current sip Use=20 > > sip@ietf.org for new developments of core SIP > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email=20 > ______________________________________________________________________ > _______________________________________________ Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW development of the application of SIP Use sip-implementors@cs.columbia.edu for questions on current sip Use sip@ietf.org for new developments of core SIP _______________________________________________ Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW development of the application of SIP Use sip-implementors@cs.columbia.edu for questions on current sip Use sip@ietf.org for new developments of core SIP _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JHzpl-00013i-Cc; Thu, 24 Jan 2008 06:01:25 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JHzpj-00010m-Vk for pmol-confirm+ok@megatron.ietf.org; Thu, 24 Jan 2008 06:01:24 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JHzpc-0000zI-Hp; Thu, 24 Jan 2008 06:01:16 -0500 Received: from mailgw4.ericsson.se ([193.180.251.62]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1JHzpb-0008DA-94; Thu, 24 Jan 2008 06:01:16 -0500 Received: from mailgw4.ericsson.se (unknown [127.0.0.1]) by mailgw4.ericsson.se (Symantec Mail Security) with ESMTP id 49E39216A9; Thu, 24 Jan 2008 12:00:39 +0100 (CET) X-AuditID: c1b4fb3e-ad5edbb0000007e1-cd-47986fd7e98e Received: from esealmw129.eemea.ericsson.se (unknown [153.88.254.124]) by mailgw4.ericsson.se (Symantec Mail Security) with ESMTP id 20A83216A6; Thu, 24 Jan 2008 12:00:39 +0100 (CET) Received: from esealmw106.eemea.ericsson.se ([153.88.200.69]) by esealmw129.eemea.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Thu, 24 Jan 2008 12:00:38 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Date: Thu, 24 Jan 2008 12:00:37 +0100 Message-ID: <8E3A1C85FE049E4BB5B4C96B207020B101199F68@esealmw106.eemea.ericsson.se> In-Reply-To: <160DE07A1C4F8E4AA2715DEC577DA4917B0C0E@srvxchg3.cablelabs.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Sipping] [PMOL] SIP End-to-End Performance Metrics Thread-Index: AchZ7ZVdC+psk80ASTW3x0Ddfzi+cwCJukqwAAre+OAAjaUCkA== References: <1197672127.7806.53.camel@montag.eng.level3.com><19f9b0170801180816p4bcc4f30kd962ae29a9b4c845@mail.gmail.com> <8E3A1C85FE049E4BB5B4C96B207020B10111A2F1@esealmw106.eemea.ericsson.se> <160DE07A1C4F8E4AA2715DEC577DA4917B0C0E@srvxchg3.cablelabs.com> From: "Marian Delkinov" To: "Daryl Malas" , "Hadriel Kaplan" X-OriginalArrivalTime: 24 Jan 2008 11:00:38.0813 (UTC) FILETIME=[5A89DCD0:01C85E78] X-Brightmail-Tracker: AAAAAA== X-Spam-Score: 0.0 (/) X-Scan-Signature: a4a24b484706be629f915bfb1a3e4771 Cc: sipping@ietf.org, pmol@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Daryl, Regarding adding 401/407 to the numerator of SEER, my opinion is that it depends on the idea behind SEER definition. 1) If SEER should complement SER, but exclude the called party behaviour, then I support keeping SEER as it is. This case matches the NER definition in PSTN.=20 2) If SEER should represent the ability of the infrastructure to establish sessions, then I would support adding 401/407 codes to the numerator. (If you remember during our discussion over the last year, I proposed adding all 4xx codes except 408 in the numerator of SEER, but we then agreed that it would cover much larger scope than just excluding the customer behaviour. Perhaps we have to define another metric to cover those cases, if the WG supports that.) =20 Best regards! Mario. -----Original Message----- From: Daryl Malas [mailto:D.Malas@cablelabs.com]=20 Sent: Monday, 21 January, 2008 16:19 To: Marian Delkinov; Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Mario, I agree. This is the traditional difference between SER and SEER, much like the difference between ASR and NER (ITU-T e.411). Now, one thing that might make sense is to add 401/407 to the numerator of the SEER metric algorithm. --Daryl -----Original Message----- From: Marian Delkinov [mailto:marian.delkinov@ericsson.com] Sent: Monday, January 21, 2008 3:45 AM To: Daryl Malas; Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Daryl and Hadriel, I have some comments regarding SER and SEER: > SER - need to specify that a 401/407 would also be subtracted in that=20 > denominator, or better yet the Invite would not be double-counted=20 > period. Also, need to point out that "# of INVITE" is for=20 > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) If we go back to the analogy with PSTN and the ASR defined by ITU, I would suggest leaving SER as it is. Thus the metric would really be useful for estimating the %revenue compared to the total volume of invites, regardless of the reason for failure (as it is in PSTN). For analysis of the different aspects of the infrastructure efficiency I suggest using complementary to SER metrics, as SEER (already defined), and why not defining some more. SEER: Section 3.7 defines SEER, however the formula still uses SER=3D... I suggest using "SEER" in the formula in order to avoid confusion with SER. Best regards! Mario. =20 -----Original Message----- From: Daryl Malas [mailto:dmalas@gmail.com] Sent: Friday, 18 January, 2008 17:17 To: Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics Hadriel, Comments in-line... On 1/17/08, Hadriel Kaplan wrote: > > Hey Daryl, > Comments: > > Failed RRD - I'm a little confused by the wording of how we decide what constitutes a failed registration. Clearly a 401/407 challenge round does not, but you state a second round would. Why? It's totally possible it's not a failure. > I'm also confused what the value/purpose of measuring this delay is. > The section says measuring failed RRD is useful for detecting problems with reaching the intended Registrar - that would be true if you counted transaction timeouts, not request/response delays for auth failures, no? [DM] I agree this could create some false indications of failures. I can update the metric to only determine a failed condition based on a timeout scenario. Also, I'm going to put together a suggested list of metrics from this draft. The topic has come up several times on how many metrics should be included. Since, no one has suggested a list, I will throw one out. > > Failed SRD - need to specify a 401/407 is not a failed SRD (you say any 4xx). > [DM] I agree and will update the draft. > Successful SDD - you say at the end "In these two examples, TB and TS are the same even if the UAC/UAS receives the indicated messages instead of sending them." I'm pretty sure the delay in receiving a Bye and sending the 200 ok for that is not really useful. :) Or at least a very different issue than that measured from sending a Bye and receiving a 200. > [DM] I agree. At one point, someone requested this to be exhaustive in all conditions. I can remove this to indicate relevance only in the situation of initiating the Bye from either the UAC or UAS. > Failed SDD - what do we count the time of a Bye to a 4xx (but not 401/407) or 5xx as? Since those are the failure conditions for the other cases, this seems a bit odd to no longer count it as one here. [DM] I'm not sure I understand this question. > > Failed SDT - this seems to be at odds with the definition of successful SDT. Successful SDT is stopped on sending the Bye, but this one is stopped on timing out that sent Bye. (in other words, every call will thus count as a successful SDT, and some will also count as a failed one) Also, what good is this failed SDT info? You already have Failed SDD. It seems to me an SDT can only be of type "successful", and the timer should be stopped at send/receipt of Bye. [DM] I agree. I will update this to only be between the 200 and Bye. It should be the Bye regardless of whether or not it times out based on the intiator. > > AHR - how does the UAC or UAS know this info? I.e., how does the UAS know what max-forwards the UAC started with, and how does the UAC know what max-forwards the UAS received? [DM] The thought is the clients (or some other in-line monitoring device) would have to capture this information at both ends and hold it in order to correlate and make the determination. This metric is a tricky one and has come up much scrutiny. It will work and could be very useful, but will require some capture outside of SIP in all likelihood. > > SER - need to specify that a 401/407 would also be subtracted in that=20 > denominator, or better yet the Invite would not be double-counted=20 > period. Also, need to point out that "# of INVITE" is for=20 > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) [DM] Are you saying 401/407 should be included with this sentence "...to the total number of attempted INVITE requests less INVITE requests resulting in a 3XX response..."? > > -hadriel > > > -----Original Message----- > > From: Daryl Malas [mailto:daryl@level3.net] > > Sent: Friday, December 14, 2007 5:42 PM > > To: pmol@ietf.org > > Cc: sipping@ietf.org > > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics > > > > At the conference, I introduced the SIP End-to-End Performance=20 > > Metrics draft (draft-malas-performance-metrics-08) to the PMOL working group. > > Although this draft is referenced in the PMOL WG charter, I wanted=20 > > to ask everyone to review the draft and provide feedback of whether=20 > > or not you feel the current version is ready for the WG to accept as > > a working group item. A couple of questions came up, which I think=20 > > should be answered regarding this consideration: > > > > Is the SIPPING WG content with the current set of metrics? > > > > Are there too many? > > > > Do these metrics capture the relevant concerns of performance=20 > > regarding the SIP protocol? > > > > Are these metrics depicted accurately from a SIP protocol perspective? > > > > In addition to these questions, I have a couple of tasks from the=20 > > PMOL group to include in the next revision. > > > > Here is a link to the draft: > > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics- > > 08.txt > > > > Thanks... > > > > Daryl > > > > > > > > _______________________________________________ > > Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping > > This list is for NEW development of the application of SIP Use=20 > > sip-implementors@cs.columbia.edu for questions on current sip Use=20 > > sip@ietf.org for new developments of core SIP > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email=20 > ______________________________________________________________________ > _______________________________________________ Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW development of the application of SIP Use sip-implementors@cs.columbia.edu for questions on current sip Use sip@ietf.org for new developments of core SIP _______________________________________________ Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW development of the application of SIP Use sip-implementors@cs.columbia.edu for questions on current sip Use sip@ietf.org for new developments of core SIP _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JHhXE-00054V-9c; Wed, 23 Jan 2008 10:29:04 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JHhXC-0004wl-MX for pmol-confirm+ok@megatron.ietf.org; Wed, 23 Jan 2008 10:29:02 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JHhXC-0004wV-Aw for pmol@ietf.org; Wed, 23 Jan 2008 10:29:02 -0500 Received: from mail120.messagelabs.com ([216.82.250.83]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1JHhXB-0002HS-QO for pmol@ietf.org; Wed, 23 Jan 2008 10:29:02 -0500 X-VirusChecked: Checked X-Env-Sender: acmorton@att.com X-Msg-Ref: server-15.tower-120.messagelabs.com!1201102139!3306280!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [144.160.20.54] Received: (qmail 24690 invoked from network); 23 Jan 2008 15:29:00 -0000 Received: from sbcsmtp7.sbc.com (HELO mlpi135.enaf.sfdc.sbc.com) (144.160.20.54) by server-15.tower-120.messagelabs.com with AES256-SHA encrypted SMTP; 23 Jan 2008 15:29:00 -0000 Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlpi135.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id m0NFSxpd015235 for ; Wed, 23 Jan 2008 10:28:59 -0500 Received: from alph001.aldc.att.com (alph001.aldc.att.com [135.53.7.26]) by mlpi135.enaf.sfdc.sbc.com (8.14.0/8.14.0) with ESMTP id m0NFSu4J015208 for ; Wed, 23 Jan 2008 10:28:56 -0500 Received: from aldc.att.com (localhost.localdomain [127.0.0.1]) by alph001.aldc.att.com (8.14.0/8.14.0) with ESMTP id m0NFSta6017478 for ; Wed, 23 Jan 2008 10:28:56 -0500 Received: from maillennium.att.com (dns.maillennium.att.com [135.25.114.99]) by alph001.aldc.att.com (8.14.0/8.14.0) with ESMTP id m0NFSn6k017415 for ; Wed, 23 Jan 2008 10:28:49 -0500 Message-Id: <200801231528.m0NFSn6k017415@alph001.aldc.att.com> Received: from acmt.att.com (martym.mt.att.com[135.16.251.71](misconfigured sender)) by maillennium.att.com (mailgw1) with SMTP id <20080123152849gw10010ghhe>; Wed, 23 Jan 2008 15:28:49 +0000 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 23 Jan 2008 10:28:48 -0500 To: pmol@ietf.org From: Al Morton Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: 1.5 (+) X-Scan-Signature: cab78e1e39c4b328567edb48482b6a69 Subject: [PMOL] Proposed Adoption of draft-malas-performance-metrics-08 X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org PMOL Working Group: A comment period for the Internet-Draft on "SIP End-to-End Performance Metrics" will be open from 23 January 2008 through 6 February 2003. A URL for this Internet-Draft is: http://tools.ietf.org/html/draft-malas-performance-metrics-08 The purpose of this comment period is to examine whether there is PMOL WG consensus to adopt this draft as the basis for the SIP Performance Metrics "Proof of Concept" work item described in the PMOL WG Charter, http://www.ietf.org/html.charters/pmol-charter.html and if so to progress this as a Working Group Draft. STATUS When this topic was raised at the IETF-70 PMOL session, there was substantial readership and some support expressed to take-up this version as the WG draft. There were also some reservations expressed at the meeting about how well the current draft matches the (evolving and not yet complete) framework draft. Thus, completing this milestone will require further development of the framework (and process) draft, as well. Also, this draft has enjoyed interest and review of the SIPPING WG, and members of the SIP development community continue to apply their expertise to fine-tune the performance metric definitions. Please weigh-in with any comments to this list or to the co-chairs: and . Al Morton Alan Clark pmol co-chairs Note: The current version of the framework and process draft is here: http://tools.ietf.org/html/draft-morton-perf-metrics-framework-01 _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JGyR0-00037i-EM; Mon, 21 Jan 2008 10:19:39 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JGyQy-00031V-GJ for pmol-confirm+ok@megatron.ietf.org; Mon, 21 Jan 2008 10:19:36 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JGyQx-000317-7t; Mon, 21 Jan 2008 10:19:35 -0500 Received: from ondar.cablelabs.com ([192.160.73.61]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1JGyQv-0002Xt-VI; Mon, 21 Jan 2008 10:19:34 -0500 Received: from kyzyl.cablelabs.com (kyzyl.cablelabs.com [10.253.0.7]) by ondar.cablelabs.com (8.13.8/8.13.8) with ESMTP id m0LFJQTC032496; Mon, 21 Jan 2008 08:19:26 -0700 Received: from srvxchg3.cablelabs.com (10.5.0.25) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/511/kyzyl.cablelabs.com); Mon, 21 Jan 2008 08:19:25 -0700 (MST) X-Virus-Status: clean(F-Secure/fsigk_smtp/511/kyzyl.cablelabs.com) X-Mimeole: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Date: Mon, 21 Jan 2008 08:19:25 -0700 Message-ID: <160DE07A1C4F8E4AA2715DEC577DA4917B0C0E@srvxchg3.cablelabs.com> In-Reply-To: <8E3A1C85FE049E4BB5B4C96B207020B10111A2F1@esealmw106.eemea.ericsson.se> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Sipping] [PMOL] SIP End-to-End Performance Metrics Thread-Index: AchZ7ZVdC+psk80ASTW3x0Ddfzi+cwCJukqwAAre+OA= References: <1197672127.7806.53.camel@montag.eng.level3.com><19f9b0170801180816p4bcc4f30kd962ae29a9b4c845@mail.gmail.com> <8E3A1C85FE049E4BB5B4C96B207020B10111A2F1@esealmw106.eemea.ericsson.se> From: "Daryl Malas" To: "Marian Delkinov" , "Hadriel Kaplan" X-Approved: ondar X-Spam-Score: 0.0 (/) X-Scan-Signature: 88b11fc64c1bfdb4425294ef5374ca07 Cc: sipping@ietf.org, pmol@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Mario, I agree. This is the traditional difference between SER and SEER, much like the difference between ASR and NER (ITU-T e.411). Now, one thing that might make sense is to add 401/407 to the numerator of the SEER metric algorithm. --Daryl -----Original Message----- From: Marian Delkinov [mailto:marian.delkinov@ericsson.com]=20 Sent: Monday, January 21, 2008 3:45 AM To: Daryl Malas; Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Daryl and Hadriel, I have some comments regarding SER and SEER: > SER - need to specify that a 401/407 would also be subtracted in that=20 > denominator, or better yet the Invite would not be double-counted=20 > period. Also, need to point out that "# of INVITE" is for=20 > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) If we go back to the analogy with PSTN and the ASR defined by ITU, I would suggest leaving SER as it is. Thus the metric would really be useful for estimating the %revenue compared to the total volume of invites, regardless of the reason for failure (as it is in PSTN). For analysis of the different aspects of the infrastructure efficiency I suggest using complementary to SER metrics, as SEER (already defined), and why not defining some more. SEER: Section 3.7 defines SEER, however the formula still uses SER=3D... I suggest using "SEER" in the formula in order to avoid confusion with SER. Best regards! Mario. =20 -----Original Message----- From: Daryl Malas [mailto:dmalas@gmail.com] Sent: Friday, 18 January, 2008 17:17 To: Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics Hadriel, Comments in-line... On 1/17/08, Hadriel Kaplan wrote: > > Hey Daryl, > Comments: > > Failed RRD - I'm a little confused by the wording of how we decide what constitutes a failed registration. Clearly a 401/407 challenge round does not, but you state a second round would. Why? It's totally possible it's not a failure. > I'm also confused what the value/purpose of measuring this delay is. > The section says measuring failed RRD is useful for detecting problems with reaching the intended Registrar - that would be true if you counted transaction timeouts, not request/response delays for auth failures, no? [DM] I agree this could create some false indications of failures. I can update the metric to only determine a failed condition based on a timeout scenario. Also, I'm going to put together a suggested list of metrics from this draft. The topic has come up several times on how many metrics should be included. Since, no one has suggested a list, I will throw one out. > > Failed SRD - need to specify a 401/407 is not a failed SRD (you say any 4xx). > [DM] I agree and will update the draft. > Successful SDD - you say at the end "In these two examples, TB and TS are the same even if the UAC/UAS receives the indicated messages instead of sending them." I'm pretty sure the delay in receiving a Bye and sending the 200 ok for that is not really useful. :) Or at least a very different issue than that measured from sending a Bye and receiving a 200. > [DM] I agree. At one point, someone requested this to be exhaustive in all conditions. I can remove this to indicate relevance only in the situation of initiating the Bye from either the UAC or UAS. > Failed SDD - what do we count the time of a Bye to a 4xx (but not 401/407) or 5xx as? Since those are the failure conditions for the other cases, this seems a bit odd to no longer count it as one here. [DM] I'm not sure I understand this question. > > Failed SDT - this seems to be at odds with the definition of successful SDT. Successful SDT is stopped on sending the Bye, but this one is stopped on timing out that sent Bye. (in other words, every call will thus count as a successful SDT, and some will also count as a failed one) Also, what good is this failed SDT info? You already have Failed SDD. It seems to me an SDT can only be of type "successful", and the timer should be stopped at send/receipt of Bye. [DM] I agree. I will update this to only be between the 200 and Bye. It should be the Bye regardless of whether or not it times out based on the intiator. > > AHR - how does the UAC or UAS know this info? I.e., how does the UAS know what max-forwards the UAC started with, and how does the UAC know what max-forwards the UAS received? [DM] The thought is the clients (or some other in-line monitoring device) would have to capture this information at both ends and hold it in order to correlate and make the determination. This metric is a tricky one and has come up much scrutiny. It will work and could be very useful, but will require some capture outside of SIP in all likelihood. > > SER - need to specify that a 401/407 would also be subtracted in that=20 > denominator, or better yet the Invite would not be double-counted=20 > period. Also, need to point out that "# of INVITE" is for=20 > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) [DM] Are you saying 401/407 should be included with this sentence "...to the total number of attempted INVITE requests less INVITE requests resulting in a 3XX response..."? > > -hadriel > > > -----Original Message----- > > From: Daryl Malas [mailto:daryl@level3.net] > > Sent: Friday, December 14, 2007 5:42 PM > > To: pmol@ietf.org > > Cc: sipping@ietf.org > > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics > > > > At the conference, I introduced the SIP End-to-End Performance=20 > > Metrics draft (draft-malas-performance-metrics-08) to the PMOL working group. > > Although this draft is referenced in the PMOL WG charter, I wanted=20 > > to ask everyone to review the draft and provide feedback of whether=20 > > or not you feel the current version is ready for the WG to accept as > > a working group item. A couple of questions came up, which I think=20 > > should be answered regarding this consideration: > > > > Is the SIPPING WG content with the current set of metrics? > > > > Are there too many? > > > > Do these metrics capture the relevant concerns of performance=20 > > regarding the SIP protocol? > > > > Are these metrics depicted accurately from a SIP protocol perspective? > > > > In addition to these questions, I have a couple of tasks from the=20 > > PMOL group to include in the next revision. > > > > Here is a link to the draft: > > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics- > > 08.txt > > > > Thanks... > > > > Daryl > > > > > > > > _______________________________________________ > > Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping > > This list is for NEW development of the application of SIP Use=20 > > sip-implementors@cs.columbia.edu for questions on current sip Use=20 > > sip@ietf.org for new developments of core SIP > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email=20 > ______________________________________________________________________ > _______________________________________________ Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW development of the application of SIP Use sip-implementors@cs.columbia.edu for questions on current sip Use sip@ietf.org for new developments of core SIP _______________________________________________ Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW development of the application of SIP Use sip-implementors@cs.columbia.edu for questions on current sip Use sip@ietf.org for new developments of core SIP _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JGu9o-0001v4-P6; Mon, 21 Jan 2008 05:45:36 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JGu9l-0001hI-Lz for pmol-confirm+ok@megatron.ietf.org; Mon, 21 Jan 2008 05:45:33 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JGu9l-0001gz-0F; Mon, 21 Jan 2008 05:45:33 -0500 Received: from mailgw3.ericsson.se ([193.180.251.60]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1JGu9j-0001N2-OV; Mon, 21 Jan 2008 05:45:32 -0500 Received: from mailgw3.ericsson.se (unknown [127.0.0.1]) by mailgw3.ericsson.se (Symantec Mail Security) with ESMTP id 8EBAF20A3E; Mon, 21 Jan 2008 11:45:30 +0100 (CET) X-AuditID: c1b4fb3c-a9aebbb0000007e0-67-479477cab9c7 Received: from esealmw129.eemea.ericsson.se (unknown [153.88.254.124]) by mailgw3.ericsson.se (Symantec Mail Security) with ESMTP id 67EAB20802; Mon, 21 Jan 2008 11:45:30 +0100 (CET) Received: from esealmw106.eemea.ericsson.se ([153.88.200.69]) by esealmw129.eemea.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Mon, 21 Jan 2008 11:45:29 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Date: Mon, 21 Jan 2008 11:45:28 +0100 Message-ID: <8E3A1C85FE049E4BB5B4C96B207020B10111A2F1@esealmw106.eemea.ericsson.se> In-Reply-To: <19f9b0170801180816p4bcc4f30kd962ae29a9b4c845@mail.gmail.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Sipping] [PMOL] SIP End-to-End Performance Metrics Thread-Index: AchZ7ZVdC+psk80ASTW3x0Ddfzi+cwCJukqw References: <1197672127.7806.53.camel@montag.eng.level3.com> <19f9b0170801180816p4bcc4f30kd962ae29a9b4c845@mail.gmail.com> From: "Marian Delkinov" To: "Daryl Malas" , "Hadriel Kaplan" X-OriginalArrivalTime: 21 Jan 2008 10:45:29.0878 (UTC) FILETIME=[BD882360:01C85C1A] X-Brightmail-Tracker: AAAAAA== X-Spam-Score: 0.0 (/) X-Scan-Signature: 9af087f15dbdd4c64ae6bbcdbc5b1d44 Cc: sipping@ietf.org, pmol@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Daryl and Hadriel, I have some comments regarding SER and SEER: > SER - need to specify that a 401/407 would also be subtracted in that=20 > denominator, or better yet the Invite would not be double-counted=20 > period. Also, need to point out that "# of INVITE" is for=20 > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) If we go back to the analogy with PSTN and the ASR defined by ITU, I would suggest leaving SER as it is. Thus the metric would really be useful for estimating the %revenue compared to the total volume of invites, regardless of the reason for failure (as it is in PSTN). For analysis of the different aspects of the infrastructure efficiency I suggest using complementary to SER metrics, as SEER (already defined), and why not defining some more. SEER: Section 3.7 defines SEER, however the formula still uses SER=3D... I suggest using "SEER" in the formula in order to avoid confusion with SER. Best regards! Mario. =20 -----Original Message----- From: Daryl Malas [mailto:dmalas@gmail.com]=20 Sent: Friday, 18 January, 2008 17:17 To: Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics Hadriel, Comments in-line... On 1/17/08, Hadriel Kaplan wrote: > > Hey Daryl, > Comments: > > Failed RRD - I'm a little confused by the wording of how we decide what constitutes a failed registration. Clearly a 401/407 challenge round does not, but you state a second round would. Why? It's totally possible it's not a failure. > I'm also confused what the value/purpose of measuring this delay is. > The section says measuring failed RRD is useful for detecting problems with reaching the intended Registrar - that would be true if you counted transaction timeouts, not request/response delays for auth failures, no? [DM] I agree this could create some false indications of failures. I can update the metric to only determine a failed condition based on a timeout scenario. Also, I'm going to put together a suggested list of metrics from this draft. The topic has come up several times on how many metrics should be included. Since, no one has suggested a list, I will throw one out. > > Failed SRD - need to specify a 401/407 is not a failed SRD (you say any 4xx). > [DM] I agree and will update the draft. > Successful SDD - you say at the end "In these two examples, TB and TS are the same even if the UAC/UAS receives the indicated messages instead of sending them." I'm pretty sure the delay in receiving a Bye and sending the 200 ok for that is not really useful. :) Or at least a very different issue than that measured from sending a Bye and receiving a 200. > [DM] I agree. At one point, someone requested this to be exhaustive in all conditions. I can remove this to indicate relevance only in the situation of initiating the Bye from either the UAC or UAS. > Failed SDD - what do we count the time of a Bye to a 4xx (but not 401/407) or 5xx as? Since those are the failure conditions for the other cases, this seems a bit odd to no longer count it as one here. [DM] I'm not sure I understand this question. > > Failed SDT - this seems to be at odds with the definition of successful SDT. Successful SDT is stopped on sending the Bye, but this one is stopped on timing out that sent Bye. (in other words, every call will thus count as a successful SDT, and some will also count as a failed one) Also, what good is this failed SDT info? You already have Failed SDD. It seems to me an SDT can only be of type "successful", and the timer should be stopped at send/receipt of Bye. [DM] I agree. I will update this to only be between the 200 and Bye. It should be the Bye regardless of whether or not it times out based on the intiator. > > AHR - how does the UAC or UAS know this info? I.e., how does the UAS know what max-forwards the UAC started with, and how does the UAC know what max-forwards the UAS received? [DM] The thought is the clients (or some other in-line monitoring device) would have to capture this information at both ends and hold it in order to correlate and make the determination. This metric is a tricky one and has come up much scrutiny. It will work and could be very useful, but will require some capture outside of SIP in all likelihood. > > SER - need to specify that a 401/407 would also be subtracted in that=20 > denominator, or better yet the Invite would not be double-counted=20 > period. Also, need to point out that "# of INVITE" is for=20 > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) [DM] Are you saying 401/407 should be included with this sentence "...to the total number of attempted INVITE requests less INVITE requests resulting in a 3XX response..."? > > -hadriel > > > -----Original Message----- > > From: Daryl Malas [mailto:daryl@level3.net] > > Sent: Friday, December 14, 2007 5:42 PM > > To: pmol@ietf.org > > Cc: sipping@ietf.org > > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics > > > > At the conference, I introduced the SIP End-to-End Performance=20 > > Metrics draft (draft-malas-performance-metrics-08) to the PMOL working group. > > Although this draft is referenced in the PMOL WG charter, I wanted=20 > > to ask everyone to review the draft and provide feedback of whether=20 > > or not you feel the current version is ready for the WG to accept as > > a working group item. A couple of questions came up, which I think=20 > > should be answered regarding this consideration: > > > > Is the SIPPING WG content with the current set of metrics? > > > > Are there too many? > > > > Do these metrics capture the relevant concerns of performance=20 > > regarding the SIP protocol? > > > > Are these metrics depicted accurately from a SIP protocol perspective? > > > > In addition to these questions, I have a couple of tasks from the=20 > > PMOL group to include in the next revision. > > > > Here is a link to the draft: > > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics- > > 08.txt > > > > Thanks... > > > > Daryl > > > > > > > > _______________________________________________ > > Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping > > This list is for NEW development of the application of SIP Use=20 > > sip-implementors@cs.columbia.edu for questions on current sip Use=20 > > sip@ietf.org for new developments of core SIP > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email=20 > ______________________________________________________________________ > _______________________________________________ Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW development of the application of SIP Use sip-implementors@cs.columbia.edu for questions on current sip Use sip@ietf.org for new developments of core SIP _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JFttu-000415-OP; Fri, 18 Jan 2008 11:17:02 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JFttt-0003zN-0C for pmol-confirm+ok@megatron.ietf.org; Fri, 18 Jan 2008 11:17:01 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JFttk-0003cK-8d for pmol@ietf.org; Fri, 18 Jan 2008 11:16:52 -0500 Received: from nz-out-0506.google.com ([64.233.162.234]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1JFttj-00081m-Hy for pmol@ietf.org; Fri, 18 Jan 2008 11:16:52 -0500 Received: by nz-out-0506.google.com with SMTP id n1so816985nzf.4 for ; Fri, 18 Jan 2008 08:16:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=URtdOIyOM450ZM+BLfwg0wejU7xwRoE8uPCh1OJXW0M=; b=hJv5VXkWytA+s3n/1IbNwsW8CxDGcJN2OALTfI3lzod7evRMB9TGRwEu/TEWJ7y5m/Fuwo8KMpY17wdOcGYMkGyqZSopB4bLbQADYPlnW5NAjreH3hfrNuvayPNRpd0HAgc/XIrxKSD64Oi3mXj/PcI+U2MhvcaGRCd1FGS9YVo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Ux6QyK3rznezRkK3HUywqAZOA1R3XSavoYLzlT7v9081SCLTvwGf5sbxOJHX/gq0gTRO5oxYTH4L005/NmhlBcTomfGxkN/xvNaZNGZ1631/pmgz600st+tbXTJi6TXCZDgvaYn2EADDH72CHrgOH9yA4T4N7AlZ6I3UqpQ3Q5o= Received: by 10.114.175.16 with SMTP id x16mr1206761wae.12.1200673010037; Fri, 18 Jan 2008 08:16:50 -0800 (PST) Received: by 10.115.15.16 with HTTP; Fri, 18 Jan 2008 08:16:49 -0800 (PST) Message-ID: <19f9b0170801180816p4bcc4f30kd962ae29a9b4c845@mail.gmail.com> Date: Fri, 18 Jan 2008 09:16:49 -0700 From: "Daryl Malas" To: "Hadriel Kaplan" Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <1197672127.7806.53.camel@montag.eng.level3.com> X-Spam-Score: 0.0 (/) X-Scan-Signature: 287c806b254c6353fcb09ee0e53bbc5e Cc: sipping@ietf.org, pmol@ietf.org X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org Hadriel, Comments in-line... On 1/17/08, Hadriel Kaplan wrote: > > Hey Daryl, > Comments: > > Failed RRD - I'm a little confused by the wording of how we decide what constitutes a failed registration. Clearly a 401/407 challenge round does not, but you state a second round would. Why? It's totally possible it's not a failure. > I'm also confused what the value/purpose of measuring this delay is. > The section says measuring failed RRD is useful for detecting problems with reaching the intended Registrar - that would be true if you counted transaction timeouts, not request/response delays for auth failures, no? [DM] I agree this could create some false indications of failures. I can update the metric to only determine a failed condition based on a timeout scenario. Also, I'm going to put together a suggested list of metrics from this draft. The topic has come up several times on how many metrics should be included. Since, no one has suggested a list, I will throw one out. > > Failed SRD - need to specify a 401/407 is not a failed SRD (you say any 4xx). > [DM] I agree and will update the draft. > Successful SDD - you say at the end "In these two examples, TB and TS are the same even if the UAC/UAS receives the indicated messages instead of sending them." I'm pretty sure the delay in receiving a Bye and sending the 200 ok for that is not really useful. :) Or at least a very different issue than that measured from sending a Bye and receiving a 200. > [DM] I agree. At one point, someone requested this to be exhaustive in all conditions. I can remove this to indicate relevance only in the situation of initiating the Bye from either the UAC or UAS. > Failed SDD - what do we count the time of a Bye to a 4xx (but not 401/407) or 5xx as? Since those are the failure conditions for the other cases, this seems a bit odd to no longer count it as one here. [DM] I'm not sure I understand this question. > > Failed SDT - this seems to be at odds with the definition of successful SDT. Successful SDT is stopped on sending the Bye, but this one is stopped on timing out that sent Bye. (in other words, every call will thus count as a successful SDT, and some will also count as a failed one) Also, what good is this failed SDT info? You already have Failed SDD. It seems to me an SDT can only be of type "successful", and the timer should be stopped at send/receipt of Bye. [DM] I agree. I will update this to only be between the 200 and Bye. It should be the Bye regardless of whether or not it times out based on the intiator. > > AHR - how does the UAC or UAS know this info? I.e., how does the UAS know what max-forwards the UAC started with, and how does the UAC know what max-forwards the UAS received? [DM] The thought is the clients (or some other in-line monitoring device) would have to capture this information at both ends and hold it in order to correlate and make the determination. This metric is a tricky one and has come up much scrutiny. It will work and could be very useful, but will require some capture outside of SIP in all likelihood. > > SER - need to specify that a 401/407 would also be subtracted in that denominator, or better yet the Invite would not be double-counted period. Also, need to point out that "# of INVITE" is for new-sessions only (ie, ones without to-tags) - not re-Invites. (I know it's obvious, but ya never know) [DM] Are you saying 401/407 should be included with this sentence "...to the total number of attempted INVITE requests less INVITE requests resulting in a 3XX response..."? > > -hadriel > > > -----Original Message----- > > From: Daryl Malas [mailto:daryl@level3.net] > > Sent: Friday, December 14, 2007 5:42 PM > > To: pmol@ietf.org > > Cc: sipping@ietf.org > > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics > > > > At the conference, I introduced the SIP End-to-End Performance Metrics > > draft (draft-malas-performance-metrics-08) to the PMOL working group. > > Although this draft is referenced in the PMOL WG charter, I wanted to > > ask everyone to review the draft and provide feedback of whether or not > > you feel the current version is ready for the WG to accept as a working > > group item. A couple of questions came up, which I think should be > > answered regarding this consideration: > > > > Is the SIPPING WG content with the current set of metrics? > > > > Are there too many? > > > > Do these metrics capture the relevant concerns of performance regarding > > the SIP protocol? > > > > Are these metrics depicted accurately from a SIP protocol perspective? > > > > In addition to these questions, I have a couple of tasks from the PMOL > > group to include in the next revision. > > > > Here is a link to the draft: > > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics-08.txt > > > > Thanks... > > > > Daryl > > > > > > > > _______________________________________________ > > Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping > > This list is for NEW development of the application of SIP > > Use sip-implementors@cs.columbia.edu for questions on current sip > > Use sip@ietf.org for new developments of core SIP > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JAZoc-0003Po-AN; Thu, 03 Jan 2008 18:49:34 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JAZob-0003Mq-Co for pmol-confirm+ok@megatron.ietf.org; Thu, 03 Jan 2008 18:49:33 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JAZob-0003Mh-36 for pmol@ietf.org; Thu, 03 Jan 2008 18:49:33 -0500 Received: from mail46.messagelabs.com ([216.82.241.227]) by ietf-mx.ietf.org with smtp (Exim 4.43) id 1JAZoa-0002f1-2z for pmol@ietf.org; Thu, 03 Jan 2008 18:49:33 -0500 X-VirusChecked: Checked X-Env-Sender: daryl@level3.net X-Msg-Ref: server-19.tower-46.messagelabs.com!1199404171!23521804!1 X-StarScan-Version: 5.5.12.14.2; banners=-,-,- X-Originating-IP: [209.245.18.106] Received: (qmail 17806 invoked from network); 3 Jan 2008 23:49:31 -0000 Received: from unknown.level3.net (HELO f10bb8-10) (209.245.18.106) by server-19.tower-46.messagelabs.com with SMTP; 3 Jan 2008 23:49:31 -0000 Received: from montag.eng.level3.com (montag.eng.l3.com [10.1.68.57]) by f10bb8-10 (Postfix) with ESMTP id C67DA4440 for ; Thu, 3 Jan 2008 23:49:30 +0000 (GMT) Subject: [PMOL] [Sipping] Draft Change of Email Address From: Daryl Malas To: pmol@ietf.org Content-Type: text/plain Date: Thu, 03 Jan 2008 17:02:19 -0700 Message-Id: <1199404939.7806.90.camel@montag.eng.level3.com> Mime-Version: 1.0 X-Mailer: Evolution 2.6.0 (2.6.0-1) Content-Transfer-Encoding: 7bit X-Spam-Score: -4.0 (----) X-Scan-Signature: de4f315c9369b71d7dd5909b42224370 X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pmol-bounces@ietf.org As of January 4, 2008, this email address (daryl@level3.net) and daryl.malas@level3.com will no longer be a valid address to reach me regarding comments or questions for the following two drafts: (SIP End-to-End Performance Metrics) draft-malas-performance-metrics-08 (Session Initiation Protocol (SIP) Overload Control) draft-hilt-sipping-overload-03 Please use the interim email address of dmalas@gmail.com until further notice. Thank you, Daryl Malas _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JAX5n-0000Af-Dy; Thu, 03 Jan 2008 15:55:07 -0500 Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JAX5m-0000Aa-Th for pmol-confirm+ok@megatron.ietf.org; Thu, 03 Jan 2008 15:55:06 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JAX5m-0000AR-IA for pmol@ietf.org; Thu, 03 Jan 2008 15:55:06 -0500 Received: from mx.cbeyond.com ([66.180.96.58]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1JAX5m-00045i-15 for pmol@ietf.org; Thu, 03 Jan 2008 15:55:06 -0500 Received: from [72.54.75.1] (port=3025 helo=TELWS143) by mx.cbeyond.com with asmtp (Exim 4.34) id 1JAX5l-0002rF-9P for pmol@ietf.org; Thu, 03 Jan 2008 15:55:05 -0500 From: "Alan Clark" To: Date: Thu, 3 Jan 2008 15:55:05 -0500 MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AchOSukuT2+VliSSQIy9MQPDv0rcdA== X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Spam-Score: 0.0 (/) X-Scan-Signature: b22590c27682ace61775ee7b453b40d3 Message-Id: Subject: [PMOL] Draft minutes posted X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1638233590==" Errors-To: pmol-bounces@ietf.org This is a multi-part message in MIME format. --===============1638233590== Content-Type: multipart/alternative; boundary="----=_NextPart_000_00D8_01C84E21.02BFA320" This is a multi-part message in MIME format. ------=_NextPart_000_00D8_01C84E21.02BFA320 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit The minutes for the December PMOL meeting have been posted to: http://www3.ietf.org/proceedings/07dec/minutes/pmol.txt Please let me know if you have any comments or corrections (prior to 7th Jan) Happy New Year! Alan ------=_NextPart_000_00D8_01C84E21.02BFA320 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

The minutes for the December PMOL meeting have been = posted to:

 

http://www3.ietf.org/proceedings/07dec/minutes/pmol.tx= t

 

Please let me know if you have any comments or = corrections (prior to 7th Jan)

 

Happy New Year!

 

Alan

 

------=_NextPart_000_00D8_01C84E21.02BFA320-- --===============1638233590== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol --===============1638233590==--