Return-Path: X-Original-To: pmol@core3.amsl.com Delivered-To: pmol@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AEEE63A6A08 for ; Mon, 9 Aug 2010 22:01:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.486 X-Spam-Level: X-Spam-Status: No, score=-2.486 tagged_above=-999 required=5 tests=[AWL=0.112, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 22O7peahBTZS for ; Mon, 9 Aug 2010 22:01:01 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by core3.amsl.com (Postfix) with ESMTP id 7947F3A6897 for ; Mon, 9 Aug 2010 22:01:01 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id o7A4voTJ008320 for ; Tue, 10 Aug 2010 06:57:50 +0200 (CEST) Received: from [10.55.43.51] (ams-bclaise-8712.cisco.com [10.55.43.51]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id o7A4vltu008313 for ; Tue, 10 Aug 2010 06:57:47 +0200 (CEST) Message-ID: <4C60DC4B.1050606@cisco.com> Date: Tue, 10 Aug 2010 06:57:47 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.11) Gecko/20100711 Thunderbird/3.0.6 MIME-Version: 1.0 To: "pmol@ietf.org" Content-Type: multipart/alternative; boundary="------------040608040006000303040306" Subject: [PMOL] PMOL framework: changing the "Performance Metrics Entity" definition to directorate X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 05:01:02 -0000 This is a multi-part message in MIME format. --------------040608040006000303040306 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear all, The current PMOL "Framework for Performance Metric Development" draft, draft-ietf-pmol-metrics-framework-04, refers to the PM Entity as such: 3.1. Performance Metrics Entity (PM Entity) The Performance Metrics Entity, or PM Entity, which may in future be a WG or directorate or a combination of these two, must coordinate the performance metric development at the IETF. Similar to The "Guidelines for Considering Operations and Management of New Protocols and Protocol Extensions" RFC 5706 [RFC5706], which is the reference document for the IETF Operations Directorate, this document should be consulted as part of the new performance metric review. The PM Entity should be composed of experts in the performance community, potentially selected from the IPPM, BMWG, and PMOL WGs. Historically, it was decided that the PM Entity would a generic term, directorate or WG, without a clear specification. The point was that we would decide later on the content of this PM entity, after the framework document was published. However, now, a couple of years after the BOF at the IETF-69, a directorate might seem a more suitable solution than a new WG. Would you have some issues to clearly specify the PM Entity as a directorate? This draft could then become a RFC similar to RFC5706 "Guidelines for Considering New Performance Metric Development" Feedback? Regards, Benoit. --------------040608040006000303040306 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear all,

The current PMOL "Framework for Performance Metric Development" draft, draft-ietf-pmol-metrics-framework-04, refers to the PM Entity as such:
3.1.  Performance Metrics Entity (PM Entity)

   The Performance Metrics Entity, or PM Entity, which may in future be
   a WG or directorate or a combination of these two, must coordinate
   the performance metric development at the IETF.  Similar to The
   "Guidelines for Considering Operations and Management of New
   Protocols and Protocol Extensions" RFC 5706 [RFC5706], which is the
   reference document for the IETF Operations Directorate, this document
   should be consulted as part of the new performance metric review.

   The PM Entity should be composed of experts in the performance
   community, potentially selected from the IPPM, BMWG, and PMOL WGs.
Historically, it was decided that the PM Entity would a generic term, directorate or WG, without a clear specification. The point was that we would decide later on the content of this PM entity, after the framework document was published.
However, now, a couple of years after the BOF at the IETF-69, a directorate might seem a more suitable solution than a new WG.

Would you have some issues to clearly specify the PM Entity as a directorate?
This draft could then become a RFC similar to RFC5706 "Guidelines for Considering New Performance Metric Development"

Feedback?

Regards, Benoit.


--------------040608040006000303040306-- Return-Path: X-Original-To: pmol@core3.amsl.com Delivered-To: pmol@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id DEFF63A6B11; Mon, 9 Aug 2010 09:59:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -98.851 X-Spam-Level: X-Spam-Status: No, score=-98.851 tagged_above=-999 required=5 tests=[AWL=-0.988, BAYES_50=0.001, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dbws60dPBcbc; Mon, 9 Aug 2010 09:58:59 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by core3.amsl.com (Postfix) with ESMTP id 32FAB3A6B03; Mon, 9 Aug 2010 09:58:59 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.4/8.14.4) with ESMTP id o79GxVUG021125; Mon, 9 Aug 2010 10:59:32 -0600 Received: from srvxchg.cablelabs.com (10.5.0.15) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/303/kyzyl.cablelabs.com); Mon, 9 Aug 2010 10:59:32 -0700 (MST) X-Virus-Status: clean(F-Secure/fsigk_smtp/303/kyzyl.cablelabs.com) Received: from srvxchg.cablelabs.com ([10.5.0.15]) by srvxchg ([10.5.0.15]) with mapi; Mon, 9 Aug 2010 10:59:32 -0600 From: Daryl Malas To: "pmol@ietf.org" Date: Mon, 9 Aug 2010 10:59:30 -0600 Thread-Topic: [PMOL] SIP Perf Telephony Metrics: SER and SEER Updates Thread-Index: Acsu/2FvroY5LByvTPqBebFJWs5sOgI5LpiQ Message-ID: <114DAD31379DFA438C0A2E39B3B8AF5D01422BF782@srvxchg> References: <114DAD31379DFA438C0A2E39B3B8AF5D01422BEE03@srvxchg> In-Reply-To: <114DAD31379DFA438C0A2E39B3B8AF5D01422BEE03@srvxchg> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Approved: ondar Cc: "rai@ietf.org" Subject: Re: [PMOL] SIP Perf Telephony Metrics: SER and SEER Updates X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Performance Metrics at Other Layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Aug 2010 16:59:01 -0000 All, This email is a duplicate, old message, so please ignore. Regards, Daryl -----Original Message----- From: rai-bounces@ietf.org [mailto:rai-bounces@ietf.org] On Behalf Of Daryl= Malas Sent: Thursday, July 29, 2010 3:21 AM To: pmol@ietf.org Cc: rai@ietf.org Subject: [RAI] [PMOL] SIP Perf Telephony Metrics: SER and SEER Updates All, During the IESG review process, there were some comments about the response= codes used in the SER and SEER metrics. I have updated the text in these = metrics and included the updates below. Please review these and provide an= y feedback. Specifically, are the response codes and their purpose clear, = and are the right set of response codes for the purposes of these metrics. = If it is unclear the purpose of these metrics, please let me know. Regards, Daryl ---------------------- 4.6. Session Establishment Ratio (SER) This metric is used to detect the ability of a terminating UA or downstream proxy to successfully establish sessions per new session INVITE requests. SER is defined as the number of new session INVITE requests resulting in a 200 OK response, to the total number of attempted INVITE requests less the total number of attempted INVITE requests less INVITE requests resulting in a 3XX, 401, 402, and 407 response. The 3XX, 401, 402 and 407 response codes were chosen, because they indicate an acceptable recoverable UA effect without the interaction of an individual user of the UA. This metric is similar to Answer Seizure Ratio (ASR) defined in [E.411]. It is measured at the originating UA only. The output value of this metric is numerical and SHOULD be adjusted to indicate a percentage of successfully established sessions. The SER is calculated using the following formula metric. In order to simplify the formula, the following variable is used to summarize multiple SIP responses: a =3D 3XX, 401, 402, and 407 # of INVITE Requests w/ associated 200 OK SER =3D --------------------------------------------------------- x 100 (Total # of INVITE Requests)-(# of INVITE Requests w/ 'a' Response) The following message exchange provides an example of identifiable events necessary for inputs in determining session establishment as described above: UA1 UA2 | | |INVITE | +----------->|------------------>| | | 180| | |<------------------| Session Established | | | | | | | 200| +----------->|<------------------| | | The following is an example call flow including a SIP 302 Redirect response. UA1 UA2 UA3 | | | +----------->|INVITE | | | |------------------>| | INVITE w/ 'a' Response | | | | | 302| | +----------->|<------------------| | | | | |INVITE | +----------->|-------------------------------------->| | | | | | 180| Session Established |<--------------------------------------| | | | | | 200| +----------->|<--------------------------------------| | | 4.7. Session Establishment Effectiveness Ratio (SEER) This metric is complimentary to SER, but is intended to exclude the potential effects of the target UA from the metric. SER is defined as the number of INVITE requests resulting in a 200 OK response and INVITE requests resulting in a 480, 486, or 600; less the total number of attempted INVITE requests less INVITE requests resulting in a 3XX, 401, 402, and 407 response. The response codes 480, 486 and 600 were chosen, because they clearly indicate the effect of an individual user of the UA. It is possible an individual user could cause a negative effect on the UA. For example, they may have misconfigured the UA causing a response code not directly related to a SSP, but this cannot be easily determined from an intermediary B2BUA somewhere inbetween the originating and terminating UA. With this in consideration, response codes such as 415 and 420 were not included in the numerator of the metric. If future response codes provide similar effects, they should be considered in addition with this metric. This metric is similar to Network Effectiveness Ratio (NER) defined in [E.411]. It is measured at the originating UA only. The output value of this metric is numerical and SHOULD be adjusted to indicate a percentage of successfully established sessions less common UAS failures. In order to simplify the formula, the following variable is used to summarize multiple SIP responses: a =3D 3XX, 401, 402, and 407 The SEER is calculated using the following formula: # of INVITE Requests w/ associated 200 OK, 480, 486, or 600 SEER =3D -------------------------------------------------------- x 100 (Total # of INVITE Requests)-(# of INVITE Requests w/ 'a' Response) Reference the example flow is Section 4.6. _______________________________________________ RAI mailing list RAI@ietf.org https://www.ietf.org/mailman/listinfo/rai