From bclaise@cisco.com Tue Oct 2 07:36:02 2012 Return-Path: X-Original-To: pmol@ietfa.amsl.com Delivered-To: pmol@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 080C121F84D6 for ; Tue, 2 Oct 2012 07:36:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.63 X-Spam-Level: X-Spam-Status: No, score=-8.63 tagged_above=-999 required=5 tests=[AWL=1.968, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MaZ2a-uoBCeG for ; Tue, 2 Oct 2012 07:35:58 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id BF31121F84D4 for ; Tue, 2 Oct 2012 07:35:57 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q92EZjeK026305; Tue, 2 Oct 2012 16:35:46 +0200 (CEST) Received: from [10.149.4.137] (dhcp-10-149-4-137.cisco.com [10.149.4.137]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q92EZisi019102; Tue, 2 Oct 2012 16:35:44 +0200 (CEST) Message-ID: <506AFBC0.2060806@cisco.com> Date: Tue, 02 Oct 2012 16:35:44 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: Alan Clark References: In-Reply-To: Content-Type: multipart/alternative; boundary="------------060409070709040501090905" Cc: draft-ietf-xrblock-rtcp-xr-pdv@tools.ietf.org, "pmol@ietf.org" , Gonzalo Camarillo , xrblock-chairs@tools.ietf.org, Qin Wu Subject: Re: [PMOL] New Version Notification - draft-ietf-xrblock-rtcp-xr-pdv-07.txt X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 14:36:02 -0000 This is a multi-part message in MIME format. --------------060409070709040501090905 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear all, I had a discussion with Dan, and we settle on: - RFC 6390 template is required for new perf metric definition - RFC 6390 template is a nice-to-have when we refer to an existing perf metric Nice-to-have because the performance metric reference doesn't always include all the required information about: measurement points, measurement timing, use and applications, reporting model, etc... but focus only on the "Method of Measurement or Calculation" So basically, at this point, I will clear my DISCUSS-DISCUSS Somehow, I've been trying to solve a growing problem, and potentially bigger problem: Where does the list of performance metric definitions come from at the IETF? We have multiple sources: - IPPM for IP performance metrics. - RTCP for RTP performance metrics: - PMOL: Performance Metrics at Other Layers, with RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics - IPFIX will one day or the other exports performance metrics. I see for example http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03. It's again a redefinition, and it should not be! My concerns are that we start to define performance metrics in different parts of the IETF, without consistency. And even finding the performance metrics specified in the IETF is not an easy task. I've convinced that the community will have solve this problem, and I will propose a meeting during the next IETF to try to come up with a solution. This meeting should include the PMOL directorate, the XRBLOCK chairs, and the IPPM chairs. If someone not in the mentioned list wants to participate, let me know privately. Regards, Benoit. > Re: New Version Notification - draft-ietf-xrblock-rtcp-xr-pdv-07.txt > Hi Dan > > The original intent was that the template would be applied to the > original definition of a metric and not to documents in which the > metric is referenced. If a separate template is created each time a > metric is referenced then there is a risk that we would have multiple > definitions of the same metric - which are likely to be inconsistent. > > Best Regards > > Alan > > > On 10/1/12 2:15 PM, "Dan (Dan)" wrote: > > Hi Benoit and Gonzalo, > > I want to make sure that we understand the advice for this I-D and > for other that are in process. Is the requirement to include > 6390-formatted templates in all documents that define new metrics > or refer to already defined metrics -- even if they are defined by > other SDOs? > > Dan > > > > > *From:* Benoit Claise [mailto:bclaise@cisco.com] > *Sent:* Monday, October 01, 2012 5:51 PM > *To:* Romascanu, Dan (Dan) > *Cc:* Gonzalo Camarillo; Qin Wu; xrblock-chairs@tools.ietf.org; > draft-ietf-xrblock-rtcp-xr-pdv@tools.ietf.org > *Subject:* Re: New Version Notification - > draft-ietf-xrblock-rtcp-xr-pdv-07.txt > > > Hi Dan, Qin, > > [sorry for the delay in getting back to you] > I'm not after duplicating information where it's not necessary. > So basically, you want a new registry with new metrics, and the > first two are > * 0: MAPDV2, Clause 6.2.3.2 of [G.1020 > > ], > > * 1: 2-point PDV, Clause 6.2.4 of [Y.1540 > > ], > So I looked at Clause 6.2.3.2 of [G.1020 > > ], and tried to fill in the RFC 6390 template fro MAPDV2 > Normative > > o Metric Name: > Mean Absolute Packet Delay Variation 2 (MAPDV2) > > o Metric Description: > The Mean Absolute Packet Delay Variation 2 > for RTP applications. > > o Method of Measurement or Calculation: > See Clause 6.2.3.2 of [G.1020 > > ] > > o Units of Measurement: > ms or percentile depending on ... > > o Measurement Point(s) with Potential Measurement Domain: > The measurement of these metrics are made at the > receiving end of the RTP stream. > > o Measurement Timing: > This metric block relies on the measurement > interval given by the value of the "Measurement Duration > (Interval)" > field in the Measurement Information Block to indicate the > span of > the report and MUST be sent in the same compound RTCP > packet as the > Measurement Information Block > > Informative > > o Implementation: > ... > > o Verification: > ... > > o Use and Applications: > For example, applications could use the > measurements of these metrics to help adjust the size of > adaptive > jitter buffers to improve performance. Network managers > can use > these metrics to compare actual delay variation to targets > (i.e., a > numerical objective or Service Level Agreement) to help > ensure the > quality of real-time application performance. > > o Reporting Model: > Report Block Structure, as specified in > > Obviously, I don't complete all the points, but I believe that you > get the message. > > I'm certainly not after redefinitions, but a simple summary, with > references when appropriate, to make the lives of people who will > be searching through perf metrics... easier. > Remember that one day, we should have all these perf metrics > listed somewhere. > > You know, simply by trying to fill in this template, I'm wondering? > Should the metric name be called RTP MAPDV2? > What if IPPM also proposes MAPDV2 for IP packets. > > Regards, Benoit. > > Hi Gonzalo, > > See my previous comment. I respectfully disagree with the need > to reformat the metrics definitions using the template defined > in RFC 6390 when we re-use metrics already defined in other > documents. This is at best a duplication that seems to me > unnecessary and at worst risks to introduce inconsistencies > between the various definitions for the same metrics. > > Regards, > > Dan > > > > > -----Original Message----- > From: Gonzalo Camarillo > [mailto:Gonzalo.Camarillo@ericsson.com] > Sent: Friday, September 21, 2012 11:07 AM > To: Qin Wu > Cc: bclaise@cisco.com; xrblock-chairs@tools.ietf.org; > draft-ietf- > xrblock-rtcp-xr-pdv@tools.ietf.org > Subject: Re: New Version Notification - > draft-ietf-xrblock-rtcp-xr-pdv- > 07.txt > > Hi Qin, > > per my conversation with Benoit in the telechat the other > day, I > understand he wants to make sure all drafts defining this > type of metric > include the relevant information. A good way to clearly > find that > information is to use the template in RFC 6390: > > http://tools.ietf.org/html/rfc6390#section-5.4.4 > > So, could you please include such a template in the draft? > > We also discussed with Benoit how to populate the template > and he was OK > with the template having pointers to sections of documents > where the > information can be found. So, you do not really need to > repeat all the > information you already added to the draft in the > template. You can just > point to the relevant section of the draft. > > Thanks, > > Gonzalo > > On 21/09/2012 7:06 AM, Qin Wu wrote: > > Hi,Benoit: > The PDV draft has already specified what you asked > for. It is a > misundersanding to think PDV doesn't follow > RFC6390 performance metric template. > Personally I think it is a bad idea to seek Pharisaism or > doctrinairism. What is more important is to respect > RFC6390 > > performance metric template by facts. > > If you really want to go doctrinairism, please tell > what need to be > > changed. > > > BTW: The new version has already incorporated extra > proposed change > > by WG after the draft gets back to WG list. > > > Regards! > -Qin > ----- Original Message ----- > From: > > To: > ; > > ; > > ; > > Sent: Friday, September 21, 2012 11:48 AM > Subject: New Version Notification - > draft-ietf-xrblock-rtcp-xr-pdv-07.txt > > > > A new version (-07) has been submitted for > draft-ietf-xrblock-rtcp-xr- > > pdv: > > http://www.ietf.org/internet-drafts/draft-ietf-xrblock-rtcp-xr-pdv-07. > txt > > > The IETF datatracker page for this Internet-Draft is: > https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/ > > Diff from previous version: > http://www.ietf.org/rfcdiff?url2=draft-ietf-xrblock-rtcp-xr-pdv-07 > > IETF Secretariat. > > > > > > --------------060409070709040501090905 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Dear all,

I had a discussion with Dan, and we settle on:
- RFC 6390 template is required for new perf metric definition
- RFC 6390 template is a nice-to-have when we refer to an existing perf metric
Nice-to-have because the performance metric reference doesn't always include all the required information about: measurement points, measurement timing, use and applications, reporting model, etc... but focus only on the "Method of Measurement or Calculation"

So basically, at this point, I will clear my DISCUSS-DISCUSS

Somehow, I've been trying to solve a growing problem, and potentially bigger problem:
Where does the list of performance metric definitions come from at
the IETF? We have multiple sources: 
- IPPM for IP performance metrics. 
- RTCP for RTP performance metrics:
- PMOL: Performance Metrics at Other Layers, with
  RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics
- IPFIX will one day or the other exports performance metrics.
  I see for example
  http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03.
  It's again a redefinition, and it should not be!

My concerns are that we start to define performance metrics in different parts
of the IETF, without consistency. 
And even finding the performance metrics specified in the IETF is not an easy task.
I've convinced that the community will have solve this problem, and I will propose a meeting during the next IETF to try to come up with a solution. This meeting should include the PMOL directorate, the XRBLOCK chairs, and the IPPM chairs.
If someone not in the mentioned list wants to participate, let me know privately.

Regards, Benoit.

Re: New Version Notification - draft-ietf-xrblock-rtcp-xr-pdv-07.txt Hi Dan

The original intent was that the template would be applied to the original definition of a metric and not to documents in which the metric is referenced. If a separate template is created each time a metric is referenced then there is a risk that we would have multiple definitions of the same metric - which are likely to be inconsistent.

Best Regards

Alan


On 10/1/12 2:15 PM, "Dan (Dan)" <dromasca@avaya.com> wrote:
Hi Benoit and Gonzalo,
 
I want to make sure that we understand the advice for this I-D and for other that are in process. Is the requirement to include 6390-formatted templates in all documents that define new metrics or refer to already defined metrics – even if they are defined by other SDOs?
 
Dan
 
 
 

From: Benoit Claise [mailto:bclaise@cisco.com]
Sent: Monday, October 01, 2012 5:51 PM
To: Romascanu, Dan (Dan)
Cc: Gonzalo Camarillo; Qin Wu; xrblock-chairs@tools.ietf.org; draft-ietf-xrblock-rtcp-xr-pdv@tools.ietf.org
Subject: Re: New Version Notification - draft-ietf-xrblock-rtcp-xr-pdv-07.txt

Hi Dan, Qin,

[sorry for the delay in getting back to you]
I'm not after duplicating information where it's not necessary.
So basically, you want a new registry with new metrics, and the first two are
      *  0: MAPDV2, Clause 6.2.3.2 of [G.1020 <http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-08#ref-G.1020> ],
 
      *  1: 2-point PDV, Clause 6.2.4 of [Y.1540 <http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-08#ref-Y.1540> ],
So I looked at Clause 6.2.3.2 of [G.1020 <http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-08#ref-G.1020> ], and tried to fill in the RFC 6390 template fro MAPDV2
  Normative
 
      o  Metric Name:
         Mean Absolute Packet Delay Variation 2 (MAPDV2)
 
      o  Metric Description:
         The Mean Absolute Packet Delay Variation 2
         for RTP applications.
 
      o  Method of Measurement or Calculation:
         See Clause 6.2.3.2 of [G.1020 <http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-08#ref-G.1020> ]
 
      o  Units of Measurement:
         ms or percentile depending on ...
 
      o  Measurement Point(s) with Potential Measurement Domain:
         The measurement of these metrics are made at the receiving end of the RTP stream.
 
      o  Measurement Timing:
        This metric block relies on the measurement
        interval given by the value of the "Measurement Duration (Interval)"
        field in the Measurement Information Block to indicate the span of
        the report and MUST be sent in the same compound RTCP packet as the
        Measurement Information Block
 
 Informative
 
      o  Implementation:
         ...
 
      o  Verification:
         ...
 
      o  Use and Applications:
        For example, applications could use the
        measurements of these metrics to help adjust the size of adaptive
        jitter buffers to improve performance.  Network managers can use
        these metrics to compare actual delay variation to targets (i.e., a
        numerical objective or Service Level Agreement) to help ensure the
        quality of real-time application performance.
 
      o  Reporting Model:         
         Report Block Structure, as specified in <this RFC>
 
Obviously, I don't complete all the points, but I believe that you get the message.

I'm certainly not after redefinitions, but a simple summary, with references when appropriate, to make the lives of people who will be searching through perf metrics... easier.
Remember that one day, we should have all these perf metrics listed somewhere.

You know, simply by trying to fill in this template, I'm wondering?
Should the metric name be called RTP MAPDV2?
What if IPPM also proposes MAPDV2 for IP packets.

Regards, Benoit.
Hi Gonzalo,
 
See my previous comment. I respectfully disagree with the need to reformat the metrics definitions using the template defined in RFC 6390 when we re-use metrics already defined in other documents. This is at best a duplication that seems to me unnecessary and at worst risks to introduce inconsistencies between the various definitions for the same metrics.
 
Regards,
 
Dan
 
 
 
 
-----Original Message-----
From: Gonzalo Camarillo [mailto:Gonzalo.Camarillo@ericsson.com]
Sent: Friday, September 21, 2012 11:07 AM
To: Qin Wu
Cc: bclaise@cisco.com; xrblock-chairs@tools.ietf.org; draft-ietf-
xrblock-rtcp-xr-pdv@tools.ietf.org
Subject: Re: New Version Notification - draft-ietf-xrblock-rtcp-xr-pdv-
07.txt
 
Hi Qin,
 
per my conversation with Benoit in the telechat the other day, I
understand he wants to make sure all drafts defining this type of metric
include the relevant information. A good way to clearly find that
information is to use the template in RFC 6390:
 
http://tools.ietf.org/html/rfc6390#section-5.4.4
 
So, could you please include such a template in the draft?
 
We also discussed with Benoit how to populate the template and he was OK
with the template having pointers to sections of documents where the
information can be found. So, you do not really need to repeat all the
information you already added to the draft in the template. You can just
point to the relevant section of the draft.
 
Thanks,
 
Gonzalo
 
On 21/09/2012 7:06 AM, Qin Wu wrote:
Hi,Benoit:
The PDV draft has already specified what you asked for. It is a
misundersanding to think PDV doesn't follow
RFC6390 performance metric template.
Personally I think it is a bad idea to seek Pharisaism or
doctrinairism. What is more important is to respect RFC6390
performance metric template by facts.
If you really want to go doctrinairism, please tell what need to be
changed.

BTW:   The new version has already incorporated extra proposed change
by WG after the draft gets back to WG list.

Regards!
-Qin
----- Original Message -----
From: <internet-drafts@ietf.org> <mailto:internet-drafts@ietf.org>
To: <xrblock-chairs@tools.ietf.org> <mailto:xrblock-chairs@tools.ietf.org> ;
<draft-ietf-xrblock-rtcp-xr-pdv@tools.ietf.org> <mailto:draft-ietf-xrblock-rtcp-xr-pdv@tools.ietf.org> ;
<gonzalo.camarillo@ericsson.com> <mailto:gonzalo.camarillo@ericsson.com> ; <bclaise@cisco.com> <mailto:bclaise@cisco.com>
Sent: Friday, September 21, 2012 11:48 AM
Subject: New Version Notification -
draft-ietf-xrblock-rtcp-xr-pdv-07.txt
 
 
 
A new version (-07) has been submitted for draft-ietf-xrblock-rtcp-xr-
pdv:
http://www.ietf.org/internet-drafts/draft-ietf-xrblock-rtcp-xr-pdv-07.
txt
 
 
The IETF datatracker page for this Internet-Draft is:
https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/
 
Diff from previous version:
http://www.ietf.org/rfcdiff?url2=draft-ietf-xrblock-rtcp-xr-pdv-07
 
IETF Secretariat.
 

 
 



--------------060409070709040501090905-- From bclaise@cisco.com Mon Oct 22 15:29:21 2012 Return-Path: X-Original-To: pmol@ietfa.amsl.com Delivered-To: pmol@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6002711E8099 for ; Mon, 22 Oct 2012 15:29:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -7.543 X-Spam-Level: X-Spam-Status: No, score=-7.543 tagged_above=-999 required=5 tests=[AWL=3.055, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rrrPVpjk0ucG for ; Mon, 22 Oct 2012 15:29:20 -0700 (PDT) Received: from av-tac-sj.cisco.com (av-tac-sj.cisco.com [171.68.227.119]) by ietfa.amsl.com (Postfix) with ESMTP id 8EA6221F8203 for ; Mon, 22 Oct 2012 15:29:20 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from fire.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-sj.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9MMTFD8003602; Mon, 22 Oct 2012 15:29:15 -0700 (PDT) Received: from [10.21.167.60] ([10.21.167.60]) by fire.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9MMTDrm017766; Mon, 22 Oct 2012 15:29:13 -0700 (PDT) Message-ID: <5085C8B9.2050508@cisco.com> Date: Mon, 22 Oct 2012 18:29:13 -0400 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: "pmol@ietf.org" Content-Type: multipart/alternative; boundary="------------010009020903050202060102" Cc: xrblock-chairs@tools.ietf.org, Ron Bonica , "ippm-chairs@tools.ietf.org" , Wesley Eddy Subject: [PMOL] PMOL directorate: meeting at the IETF? X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2012 22:29:21 -0000 This is a multi-part message in MIME format. --------------010009020903050202060102 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear PMOL directorate members, During my review of the latest AVTCORE and XRBLOCK drafts ( draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came to the conclusion that we have an issue in terms of performance metrics at the IETF, and actually in the industry. As background information, here is my DISCUSS on the two drafts ---------------------------------------------------------------------- DISCUSS: ---------------------------------------------------------------------- My entire point is more a DISCUSS-DISCUSS, for both draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt. Sorry to pick on these two drafts, but we need to have an IESG performance metrics discussion. Where does the list of performance metric definitions come from at the IETF? We have multiple sources: - IPPM for IP performance metrics - RTCP for RTP performance metrics: Definitions in the document themselves or potentially referencing some other SDOs Example:http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05 bits 014-011 0: MAPDV2, Clause 6.2.3.2 of [G.1020], 1: 2-point PDV, Clause 6.2.4 of [Y.1540]. - PMOL: Performance Metrics at Other Layers, with RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics - IPFIX will one day or the other exports performance metrics. I see for example http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03 It's again a redefinition, and it should not be! My concerns are that we start to define performance metrics in different parts of the IETF, without consistency. We have defined RFC 6390 on "Guidelines for Considering New Performance Metric Development", which ask for specific definition Seehttp://tools.ietf.org/html/rfc6390#section-5.4.4 I believe that the IETF should at least: - define the performance metrics in a consistent way according to RFC6390. - document those performance metrics in a single location So my questions are: - are we defining the performance metrics the right way? - where is this shared repository of performance metrics (at least for the ones created in the IETF)? - is the PMOL directorate (RFC 6390) used effectively? After discussing with Dan Romascanu, we came to this conclusion I had a discussion with Dan Romascanu, and we settled on: - RFC 6390 template is required for new perf metric definition - RFC 6390 template is a nice-to-have when we refer to an existing perf metric Nice-to-have because the performance metric reference doesn't always include all the required information about: measurement points, measurement timing, use and applications, reporting model, etc... but focus only on the "Method of Measurement or Calculation" I would like to have a meeting during the IETF, with the following agenda - are we defining the performance metrics the right way? - where is this shared repository of performance metrics (at least for the ones created in the IETF)? - is the PMOL directorate (RFC 6390) used effectively? - conclusion discussed with Dan Here is a doodle invite. Please let me know if/when you are available. Your feedback on this mailer is also welcome. Regards, Benoit --------------010009020903050202060102 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear PMOL directorate members,

During my review of the latest AVTCORE and  XRBLOCK drafts ( draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came to the conclusion that we have an issue in terms of performance metrics at the IETF, and actually in the industry.

As background information, here is my DISCUSS on the two drafts
----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

My entire point is more a DISCUSS-DISCUSS, for both
draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt.
Sorry to pick on these two drafts, but we need to have an IESG
performance metrics discussion.
Where does the list of performance metric definitions come from at the
IETF?
We have multiple sources:
- IPPM for IP performance metrics
- RTCP for RTP performance metrics: 
  Definitions in the document themselves or potentially referencing some
other SDOs
  Example: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05
   bits 014-011
            0: MAPDV2, Clause 6.2.3.2 of [G.1020],
            1: 2-point PDV, Clause 6.2.4 of [Y.1540].
- PMOL: Performance Metrics at Other Layers, with 
  RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics
- IPFIX will one day or the other exports performance metrics.
  I see for example
  http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03
  It's again a redefinition, and it should not be!

My concerns are that we start to define performance metrics in different
parts of the IETF, without consistency.

We have defined RFC 6390 on "Guidelines for Considering New Performance
Metric Development", which ask for specific definition
See http://tools.ietf.org/html/rfc6390#section-5.4.4

I believe that the IETF should at least:
- define the performance metrics in a consistent way according to
RFC6390.
- document those performance metrics in a single location

So my questions are:
- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for
the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?
After discussing with Dan Romascanu, we came to this conclusion
I had a discussion with Dan Romascanu, and we settled on:
- RFC 6390 template is required for new perf metric definition
- RFC 6390 template is a nice-to-have when we refer to an existing perf metric
Nice-to-have because the performance metric reference doesn't always include
all the required information about: measurement points, measurement timing, use
and applications, reporting model, etc... but focus only on the "Method of
Measurement or Calculation"
I would like to have a meeting during the IETF, with the following agenda
- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for
the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?
- conclusion discussed with Dan

Here is a doodle invite. Please let me know if/when you are available.
Your feedback on this mailer is also welcome.
Regards, Benoit

--------------010009020903050202060102-- From bclaise@cisco.com Mon Oct 22 15:48:27 2012 Return-Path: X-Original-To: pmol@ietfa.amsl.com Delivered-To: pmol@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5FB7911E808D for ; Mon, 22 Oct 2012 15:48:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -7.685 X-Spam-Level: X-Spam-Status: No, score=-7.685 tagged_above=-999 required=5 tests=[AWL=2.913, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eQ+I+lh3M8X0 for ; Mon, 22 Oct 2012 15:48:26 -0700 (PDT) Received: from av-tac-sj.cisco.com (av-tac-sj.cisco.com [171.68.227.119]) by ietfa.amsl.com (Postfix) with ESMTP id 6F38B11E8099 for ; Mon, 22 Oct 2012 15:48:26 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from fire.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-sj.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9MMmPs6005540; Mon, 22 Oct 2012 15:48:25 -0700 (PDT) Received: from [10.21.167.60] ([10.21.167.60]) by fire.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9MMmOSQ019780; Mon, 22 Oct 2012 15:48:24 -0700 (PDT) Message-ID: <5085CD38.7010409@cisco.com> Date: Mon, 22 Oct 2012 18:48:24 -0400 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: Benoit Claise References: <5085C8B9.2050508@cisco.com> In-Reply-To: <5085C8B9.2050508@cisco.com> Content-Type: multipart/alternative; boundary="------------000606010206090905020407" Cc: xrblock-chairs@tools.ietf.org, Ron Bonica , "ippm-chairs@tools.ietf.org" , "pmol@ietf.org" , Wesley Eddy Subject: Re: [PMOL] PMOL directorate: meeting at the IETF? X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2012 22:48:27 -0000 This is a multi-part message in MIME format. --------------000606010206090905020407 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit With doodle this time http://www.doodle.com/fz3gxri76ngzqygv Thanks Carlos B. > Dear PMOL directorate members, > > During my review of the latest AVTCORE and XRBLOCK drafts ( > draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came > to the conclusion that we have an issue in terms of performance > metrics at the IETF, and actually in the industry. > > As background information, here is my DISCUSS on the two drafts > > ---------------------------------------------------------------------- > DISCUSS: > ---------------------------------------------------------------------- > > My entire point is more a DISCUSS-DISCUSS, for both > draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt. > Sorry to pick on these two drafts, but we need to have an IESG > performance metrics discussion. > Where does the list of performance metric definitions come from at the > IETF? > We have multiple sources: > - IPPM for IP performance metrics > - RTCP for RTP performance metrics: > Definitions in the document themselves or potentially referencing some > other SDOs > Example:http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05 > bits 014-011 > 0: MAPDV2, Clause 6.2.3.2 of [G.1020], > 1: 2-point PDV, Clause 6.2.4 of [Y.1540]. > - PMOL: Performance Metrics at Other Layers, with > RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics > - IPFIX will one day or the other exports performance metrics. > I see for example > http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03 > It's again a redefinition, and it should not be! > > My concerns are that we start to define performance metrics in different > parts of the IETF, without consistency. > > We have defined RFC 6390 on "Guidelines for Considering New Performance > Metric Development", which ask for specific definition > Seehttp://tools.ietf.org/html/rfc6390#section-5.4.4 > > I believe that the IETF should at least: > - define the performance metrics in a consistent way according to > RFC6390. > - document those performance metrics in a single location > > So my questions are: > - are we defining the performance metrics the right way? > - where is this shared repository of performance metrics (at least for > the ones created in the IETF)? > - is the PMOL directorate (RFC 6390) used effectively? > > After discussing with Dan Romascanu, we came to this conclusion > > I had a discussion with Dan Romascanu, and we settled on: > - RFC 6390 template is required for new perf metric definition > - RFC 6390 template is a nice-to-have when we refer to an existing perf metric > Nice-to-have because the performance metric reference doesn't always include > all the required information about: measurement points, measurement timing, use > and applications, reporting model, etc... but focus only on the "Method of > Measurement or Calculation" > > I would like to have a meeting during the IETF, with the following agenda > - are we defining the performance metrics the right way? > - where is this shared repository of performance metrics (at least for > the ones created in the IETF)? > - is the PMOL directorate (RFC 6390) used effectively? > - conclusion discussed with Dan > > Here is a doodle invite. Please let me know if/when you are available. > Your feedback on this mailer is also welcome. > Regards, Benoit > --------------000606010206090905020407 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
With doodle this time http://www.doodle.com/fz3gxri76ngzqygv
Thanks Carlos

B.
Dear PMOL directorate members,

During my review of the latest AVTCORE and  XRBLOCK drafts ( draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came to the conclusion that we have an issue in terms of performance metrics at the IETF, and actually in the industry.

As background information, here is my DISCUSS on the two drafts
----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

My entire point is more a DISCUSS-DISCUSS, for both
draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt.
Sorry to pick on these two drafts, but we need to have an IESG
performance metrics discussion.
Where does the list of performance metric definitions come from at the
IETF?
We have multiple sources:
- IPPM for IP performance metrics
- RTCP for RTP performance metrics: 
  Definitions in the document themselves or potentially referencing some
other SDOs
  Example: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05
   bits 014-011
            0: MAPDV2, Clause 6.2.3.2 of [G.1020],
            1: 2-point PDV, Clause 6.2.4 of [Y.1540].
- PMOL: Performance Metrics at Other Layers, with 
  RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics
- IPFIX will one day or the other exports performance metrics.
  I see for example
  http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03
  It's again a redefinition, and it should not be!

My concerns are that we start to define performance metrics in different
parts of the IETF, without consistency.

We have defined RFC 6390 on "Guidelines for Considering New Performance
Metric Development", which ask for specific definition
See http://tools.ietf.org/html/rfc6390#section-5.4.4

I believe that the IETF should at least:
- define the performance metrics in a consistent way according to
RFC6390.
- document those performance metrics in a single location

So my questions are:
- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for
the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?
After discussing with Dan Romascanu, we came to this conclusion
I had a discussion with Dan Romascanu, and we settled on:
- RFC 6390 template is required for new perf metric definition
- RFC 6390 template is a nice-to-have when we refer to an existing perf metric
Nice-to-have because the performance metric reference doesn't always include
all the required information about: measurement points, measurement timing, use
and applications, reporting model, etc... but focus only on the "Method of
Measurement or Calculation"
I would like to have a meeting during the IETF, with the following agenda
- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for
the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?
- conclusion discussed with Dan

Here is a doodle invite. Please let me know if/when you are available.
Your feedback on this mailer is also welcome.
Regards, Benoit


--------------000606010206090905020407-- From dromasca@avaya.com Wed Oct 24 05:02:57 2012 Return-Path: X-Original-To: pmol@ietfa.amsl.com Delivered-To: pmol@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 47F9921F89A8 for ; Wed, 24 Oct 2012 05:02:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.354 X-Spam-Level: X-Spam-Status: No, score=-103.354 tagged_above=-999 required=5 tests=[AWL=0.244, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1mhwYwAopy4e for ; Wed, 24 Oct 2012 05:02:56 -0700 (PDT) Received: from co300216-co-outbound.net.avaya.com (co300216-co-outbound.net.avaya.com [198.152.13.100]) by ietfa.amsl.com (Postfix) with ESMTP id 0CDEB21F8B90 for ; Wed, 24 Oct 2012 05:02:55 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgIFAObXh1CHCzI1/2dsb2JhbABEgkq2QwGIboEIgh4BAQEBAxIKEQNJEAIBCA0BAwQBAQsGDAsBBgFFCQgBAQQTCBqHYgueHZ0Gi2CGC2ADlwqEb4ozgnE X-IronPort-AV: E=Sophos;i="4.80,639,1344225600"; d="scan'208,217";a="373039731" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by co300216-co-outbound.net.avaya.com with ESMTP; 24 Oct 2012 07:56:34 -0400 Received: from unknown (HELO 307622ANEX5.global.avaya.com) ([135.64.140.14]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 24 Oct 2012 07:40:14 -0400 x-mimeole: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CDB1DF.7EFEB4EA" Date: Wed, 24 Oct 2012 14:02:52 +0200 Message-ID: In-Reply-To: <5085CD38.7010409@cisco.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: thread-topic: [PMOL] PMOL directorate: meeting at the IETF? Thread-Index: Ac2wp13V6f6pbcRISpKtgzh6Z1ibMQBN90BA References: <5085C8B9.2050508@cisco.com> <5085CD38.7010409@cisco.com> From: "Romascanu, Dan (Dan)" To: "Benoit Claise" Cc: xrblock-chairs@tools.ietf.org, Ron Bonica , ippm-chairs@tools.ietf.org, pmol@ietf.org, Wesley Eddy Subject: Re: [PMOL] PMOL directorate: meeting at the IETF? X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 12:02:57 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01CDB1DF.7EFEB4EA Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Benoit,=20 =20 Wednesday at 3:40PM does not seem right, as it enters plenary time, unless we believe that we shall not need more than 20 min.=20 =20 Dan =20 =20 =20 From: pmol-bounces@ietf.org [mailto:pmol-bounces@ietf.org] On Behalf Of Benoit Claise Sent: Tuesday, October 23, 2012 12:48 AM To: Benoit Claise Cc: xrblock-chairs@tools.ietf.org; Ron Bonica; ippm-chairs@tools.ietf.org; pmol@ietf.org; Wesley Eddy Subject: Re: [PMOL] PMOL directorate: meeting at the IETF? =20 With doodle this time http://www.doodle.com/fz3gxri76ngzqygv Thanks Carlos B. Dear PMOL directorate members, =09 During my review of the latest AVTCORE and XRBLOCK drafts ( draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came to the conclusion that we have an issue in terms of performance metrics at the IETF, and actually in the industry. =09 As background information, here is my DISCUSS on the two drafts =09 ---------------------------------------------------------------------- DISCUSS: =09 ---------------------------------------------------------------------- =20 My entire point is more a DISCUSS-DISCUSS, for both draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt. Sorry to pick on these two drafts, but we need to have an IESG performance metrics discussion. Where does the list of performance metric definitions come from at the IETF? We have multiple sources: - IPPM for IP performance metrics - RTCP for RTP performance metrics:=20 Definitions in the document themselves or potentially referencing some other SDOs Example: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05 bits 014-011 0: MAPDV2, Clause 6.2.3.2 of [G.1020], 1: 2-point PDV, Clause 6.2.4 of [Y.1540]. - PMOL: Performance Metrics at Other Layers, with=20 RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics - IPFIX will one day or the other exports performance metrics. I see for example =09 http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03 =20 It's again a redefinition, and it should not be! =20 My concerns are that we start to define performance metrics in different parts of the IETF, without consistency. =20 We have defined RFC 6390 on "Guidelines for Considering New Performance Metric Development", which ask for specific definition See http://tools.ietf.org/html/rfc6390#section-5.4.4 =20 I believe that the IETF should at least: - define the performance metrics in a consistent way according to RFC6390. - document those performance metrics in a single location =20 So my questions are: - are we defining the performance metrics the right way? - where is this shared repository of performance metrics (at least for the ones created in the IETF)? - is the PMOL directorate (RFC 6390) used effectively? After discussing with Dan Romascanu, we came to this conclusion I had a discussion with Dan Romascanu, and we settled on: - RFC 6390 template is required for new perf metric definition - RFC 6390 template is a nice-to-have when we refer to an existing perf metric Nice-to-have because the performance metric reference doesn't always include all the required information about: measurement points, measurement timing, use and applications, reporting model, etc... but focus only on the "Method of Measurement or Calculation" I would like to have a meeting during the IETF, with the following agenda =09 =09 - are we defining the performance metrics the right way? - where is this shared repository of performance metrics (at least for the ones created in the IETF)? - is the PMOL directorate (RFC 6390) used effectively? - conclusion discussed with Dan =20 Here is a doodle invite. Please let me know if/when you are available. Your feedback on this mailer is also welcome. Regards, Benoit =20 ------_=_NextPart_001_01CDB1DF.7EFEB4EA Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Benoit,

 

Wednesday at 3:40PM does not seem right, as it enters plenary time, = unless we believe that we shall not need more than 20 min. =

 

Dan

 

 

 

From: pmol-bounces@ietf.org [mailto:pmol-bounces@ietf.org] On Behalf = Of Benoit Claise
Sent: Tuesday, October 23, 2012 12:48 = AM
To: Benoit Claise
Cc: = xrblock-chairs@tools.ietf.org; Ron Bonica; ippm-chairs@tools.ietf.org; = pmol@ietf.org; Wesley Eddy
Subject: Re: [PMOL] PMOL = directorate: meeting at the IETF?

 

With = doodle this time http://www.doodle.com/fz3= gxri76ngzqygv
Thanks = Carlos

B.

Dear = PMOL directorate members,

During my review of the latest AVTCORE = and  XRBLOCK drafts ( draft-ietf-xrblock-rtcp-xr-pdv and = draft-ietf-avtcore-monarch), I came to the conclusion that we have an = issue in terms of performance metrics at the IETF, and actually in the = industry.

As background information, here is my DISCUSS on the = two = drafts

-----------------------------------------------=
-----------------------
DISCUSS:
----------------------------------------------------------------------<= o:p>
 
My entire point is more =
a DISCUSS-DISCUSS, for =
both
draft-ietf-avtcore-monarch-19 and =
draft-ietf-xrblock-rtcp-xr-pdv-05.txt.
Sorry to =
pick on these two drafts, but we need to have an =
IESG
performance metrics =
discussion.
Where does the list of performance =
metric definitions come from at =
the
IETF?
We have multiple =
sources:
- IPPM for IP performance =
metrics
- RTCP for RTP performance metrics: =

  Definitions in the document themselves =
or potentially referencing some
other =
SDOs
  Example: htt=
p://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05<=
/pre>
   bits =
014-011
       &n=
bsp;    0: MAPDV2, Clause 6.2.3.2 of =
[G.1020],
       =
     1: 2-point PDV, Clause 6.2.4 of =
[Y.1540].
- PMOL: Performance Metrics at Other =
Layers, with 
  RFC 6076 on Basic =
Telephony SIP End-to-End Performance Metrics
- =
IPFIX will one day or the other exports performance =
metrics.
  I see for =
example
=
  =
http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03<=
/o:p>
  It's again a redefinition, and it should not =
be!
 
My concerns are =
that we start to define performance metrics in =
different
parts of the IETF, without =
consistency.
 
We have =
defined RFC 6390 on "Guidelines for Considering New =
Performance
Metric Development", which ask for =
specific definition
See http://tools.ie=
tf.org/html/rfc6390#section-5.4.4
 
I believe that the IETF should at =
least:
- define the performance metrics in a =
consistent way according =
to
RFC6390.
- document those =
performance metrics in a single =
location
 
So my =
questions are:
- are we defining the performance =
metrics the right way?
- where is this shared =
repository of performance metrics (at least for
the =
ones created in the IETF)?
- is the PMOL =
directorate (RFC 6390) used effectively?
After discussing with Dan Romascanu, we came to this =
conclusion
I had a discussion with Dan Romascanu, and =
we settled on:
- RFC 6390 template is required for =
new perf metric definition
- RFC 6390 template is a =
nice-to-have when we refer to an existing perf =
metric
Nice-to-have because the performance metric =
reference doesn't always include
all the required =
information about: measurement points, measurement timing, =
use
and applications, reporting model, etc... but =
focus only on the "Method of
Measurement or =
Calculation"
I would like to =
have a meeting during the IETF, with the following =
agenda

- are we defining the performance =
metrics the right way?
- where is this shared =
repository of performance metrics (at least for
the =
ones created in the IETF)?
- is the PMOL =
directorate (RFC 6390) used effectively?
- =
conclusion discussed with =
Dan
 
Here is a doodle =
invite. Please let me know if/when you are =
available.
Your feedback on this mailer is also =
welcome.
Regards, =
Benoit

 

------_=_NextPart_001_01CDB1DF.7EFEB4EA-- From bclaise@cisco.com Sun Oct 28 18:07:54 2012 Return-Path: X-Original-To: pmol@ietfa.amsl.com Delivered-To: pmol@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DF51A21F84AB for ; Sun, 28 Oct 2012 18:07:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.598 X-Spam-Level: X-Spam-Status: No, score=-10.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YDZW4qDM6Qxr for ; Sun, 28 Oct 2012 18:07:53 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 490D621F849E for ; Sun, 28 Oct 2012 18:07:53 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9T17noL025579; Mon, 29 Oct 2012 02:07:49 +0100 (CET) Received: from [10.60.67.92] (ams-bclaise-89111.cisco.com [10.60.67.92]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9T17kfg000346; Mon, 29 Oct 2012 02:07:47 +0100 (CET) Message-ID: <508DD6E2.30405@cisco.com> Date: Sun, 28 Oct 2012 20:07:46 -0500 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: "Romascanu, Dan (Dan)" References: <5085C8B9.2050508@cisco.com> <5085CD38.7010409@cisco.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------020505020106000809050208" Cc: xrblock-chairs@tools.ietf.org, Ron Bonica , ippm-chairs@tools.ietf.org, pmol@ietf.org, Wesley Eddy Subject: Re: [PMOL] PMOL directorate: meeting at the IETF? X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2012 01:07:55 -0000 This is a multi-part message in MIME format. --------------020505020106000809050208 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi Dan, You're right. Daylight savings yesterday in Belgium + daylight saving next Sunday in Atlanta + a little bit of distraction of my side = my mistake. I was unable to remove the entry, so I marked myself unavailable for that slot. All, please don't forgot to insert your availabilities. Regards, Benoit > > Hi Benoit, > > Wednesday at 3:40PM does not seem right, as it enters plenary time, > unless we believe that we shall not need more than 20 min. > > Dan > > *From:*pmol-bounces@ietf.org [mailto:pmol-bounces@ietf.org] *On Behalf > Of *Benoit Claise > *Sent:* Tuesday, October 23, 2012 12:48 AM > *To:* Benoit Claise > *Cc:* xrblock-chairs@tools.ietf.org; Ron Bonica; > ippm-chairs@tools.ietf.org; pmol@ietf.org; Wesley Eddy > *Subject:* Re: [PMOL] PMOL directorate: meeting at the IETF? > > With doodle this time http://www.doodle.com/fz3gxri76ngzqygv > Thanks Carlos > > B. > > Dear PMOL directorate members, > > During my review of the latest AVTCORE and XRBLOCK drafts ( > draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I > came to the conclusion that we have an issue in terms of > performance metrics at the IETF, and actually in the industry. > > As background information, here is my DISCUSS on the two drafts > > ---------------------------------------------------------------------- > > DISCUSS: > > ---------------------------------------------------------------------- > > > > My entire point is more a DISCUSS-DISCUSS, for both > > draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt. > > Sorry to pick on these two drafts, but we need to have an IESG > > performance metrics discussion. > > Where does the list of performance metric definitions come from at the > > IETF? > > We have multiple sources: > > - IPPM for IP performance metrics > > - RTCP for RTP performance metrics: > > Definitions in the document themselves or potentially referencing some > > other SDOs > > Example:http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05 > > bits 014-011 > > 0: MAPDV2, Clause 6.2.3.2 of [G.1020], > > 1: 2-point PDV, Clause 6.2.4 of [Y.1540]. > > - PMOL: Performance Metrics at Other Layers, with > > RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics > > - IPFIX will one day or the other exports performance metrics. > > I see for example > > http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03 > > It's again a redefinition, and it should not be! > > > > My concerns are that we start to define performance metrics in different > > parts of the IETF, without consistency. > > > > We have defined RFC 6390 on "Guidelines for Considering New Performance > > Metric Development", which ask for specific definition > > Seehttp://tools.ietf.org/html/rfc6390#section-5.4.4 > > > > I believe that the IETF should at least: > > - define the performance metrics in a consistent way according to > > RFC6390. > > - document those performance metrics in a single location > > > > So my questions are: > > - are we defining the performance metrics the right way? > > - where is this shared repository of performance metrics (at least for > > the ones created in the IETF)? > > - is the PMOL directorate (RFC 6390) used effectively? > > After discussing with Dan Romascanu, we came to this conclusion > > I had a discussion with Dan Romascanu, and we settled on: > > - RFC 6390 template is required for new perf metric definition > > - RFC 6390 template is a nice-to-have when we refer to an existing perf metric > > Nice-to-have because the performance metric reference doesn't always include > > all the required information about: measurement points, measurement timing, use > > and applications, reporting model, etc... but focus only on the "Method of > > Measurement or Calculation" > > I would like to have a meeting during the IETF, with the following > agenda > > - are we defining the performance metrics the right way? > > - where is this shared repository of performance metrics (at least for > > the ones created in the IETF)? > > - is the PMOL directorate (RFC 6390) used effectively? > > - conclusion discussed with Dan > > > > Here is a doodle invite. Please let me know if/when you are available. > > Your feedback on this mailer is also welcome. > > Regards, Benoit > --------------020505020106000809050208 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Hi Dan,

You're right. Daylight savings yesterday in Belgium + daylight saving next Sunday in Atlanta + a little bit of distraction of my side = my mistake. I was unable to remove the entry, so I marked myself unavailable for that slot.

All, please don't forgot to insert your availabilities.

Regards, Benoit

Hi Benoit,

 

Wednesday at 3:40PM does not seem right, as it enters plenary time, unless we believe that we shall not need more than 20 min.

 

Dan

 

 

 

From: pmol-bounces@ietf.org [mailto:pmol-bounces@ietf.org] On Behalf Of Benoit Claise
Sent: Tuesday, October 23, 2012 12:48 AM
To: Benoit Claise
Cc: xrblock-chairs@tools.ietf.org; Ron Bonica; ippm-chairs@tools.ietf.org; pmol@ietf.org; Wesley Eddy
Subject: Re: [PMOL] PMOL directorate: meeting at the IETF?

 

With doodle this time http://www.doodle.com/fz3gxri76ngzqygv
Thanks Carlos

B.

Dear PMOL directorate members,

During my review of the latest AVTCORE and  XRBLOCK drafts ( draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came to the conclusion that we have an issue in terms of performance metrics at the IETF, and actually in the industry.

As background information, here is my DISCUSS on the two drafts

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------
 
My entire point is more a DISCUSS-DISCUSS, for both
draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt.
Sorry to pick on these two drafts, but we need to have an IESG
performance metrics discussion.
Where does the list of performance metric definitions come from at the
IETF?
We have multiple sources:
- IPPM for IP performance metrics
- RTCP for RTP performance metrics: 
  Definitions in the document themselves or potentially referencing some
other SDOs
  Example: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05
   bits 014-011
            0: MAPDV2, Clause 6.2.3.2 of [G.1020],
            1: 2-point PDV, Clause 6.2.4 of [Y.1540].
- PMOL: Performance Metrics at Other Layers, with 
  RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics
- IPFIX will one day or the other exports performance metrics.
  I see for example
  http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03
  It's again a redefinition, and it should not be!
 
My concerns are that we start to define performance metrics in different
parts of the IETF, without consistency.
 
We have defined RFC 6390 on "Guidelines for Considering New Performance
Metric Development", which ask for specific definition
See http://tools.ietf.org/html/rfc6390#section-5.4.4
 
I believe that the IETF should at least:
- define the performance metrics in a consistent way according to
RFC6390.
- document those performance metrics in a single location
 
So my questions are:
- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for
the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?

After discussing with Dan Romascanu, we came to this conclusion

I had a discussion with Dan Romascanu, and we settled on:
- RFC 6390 template is required for new perf metric definition
- RFC 6390 template is a nice-to-have when we refer to an existing perf metric
Nice-to-have because the performance metric reference doesn't always include
all the required information about: measurement points, measurement timing, use
and applications, reporting model, etc... but focus only on the "Method of
Measurement or Calculation"

I would like to have a meeting during the IETF, with the following agenda

- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for
the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?
- conclusion discussed with Dan
 
Here is a doodle invite. Please let me know if/when you are available.
Your feedback on this mailer is also welcome.

Regards, Benoit

 


--------------020505020106000809050208-- From bclaise@cisco.com Tue Oct 30 17:40:17 2012 Return-Path: X-Original-To: pmol@ietfa.amsl.com Delivered-To: pmol@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 50F0D21F8767 for ; Tue, 30 Oct 2012 17:40:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.527 X-Spam-Level: X-Spam-Status: No, score=-10.527 tagged_above=-999 required=5 tests=[AWL=0.071, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gJmR-QOwMRBV for ; Tue, 30 Oct 2012 17:40:16 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 1117321F861B for ; Tue, 30 Oct 2012 17:40:15 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9V0eCCH012919; Wed, 31 Oct 2012 01:40:13 +0100 (CET) Received: from [10.60.67.92] (ams-bclaise-89111.cisco.com [10.60.67.92]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9V0eBIK016944; Wed, 31 Oct 2012 01:40:11 +0100 (CET) Message-ID: <5090736A.5030108@cisco.com> Date: Tue, 30 Oct 2012 19:40:10 -0500 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: Benoit Claise References: <5085C8B9.2050508@cisco.com> In-Reply-To: <5085C8B9.2050508@cisco.com> Content-Type: multipart/alternative; boundary="------------080909080802080100060302" Cc: xrblock-chairs@tools.ietf.org, Ron Bonica , "ippm-chairs@tools.ietf.org" , "pmol@ietf.org" , Wesley Eddy Subject: Re: [PMOL] PMOL directorate: meeting at the IETF? X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2012 00:40:17 -0000 This is a multi-part message in MIME format. --------------080909080802080100060302 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear PMOL directorate members, Based on doodle, this meeting will take place on Wed Nov 7th at 1440 - 1540 in the IESG breakout room, to be confirmed. I hope to see you all. Regards, Benoit > Dear PMOL directorate members, > > During my review of the latest AVTCORE and XRBLOCK drafts ( > draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came > to the conclusion that we have an issue in terms of performance > metrics at the IETF, and actually in the industry. > > As background information, here is my DISCUSS on the two drafts > > ---------------------------------------------------------------------- > DISCUSS: > ---------------------------------------------------------------------- > > My entire point is more a DISCUSS-DISCUSS, for both > draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt. > Sorry to pick on these two drafts, but we need to have an IESG > performance metrics discussion. > Where does the list of performance metric definitions come from at the > IETF? > We have multiple sources: > - IPPM for IP performance metrics > - RTCP for RTP performance metrics: > Definitions in the document themselves or potentially referencing some > other SDOs > Example:http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05 > bits 014-011 > 0: MAPDV2, Clause 6.2.3.2 of [G.1020], > 1: 2-point PDV, Clause 6.2.4 of [Y.1540]. > - PMOL: Performance Metrics at Other Layers, with > RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics > - IPFIX will one day or the other exports performance metrics. > I see for example > http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03 > It's again a redefinition, and it should not be! > > My concerns are that we start to define performance metrics in different > parts of the IETF, without consistency. > > We have defined RFC 6390 on "Guidelines for Considering New Performance > Metric Development", which ask for specific definition > Seehttp://tools.ietf.org/html/rfc6390#section-5.4.4 > > I believe that the IETF should at least: > - define the performance metrics in a consistent way according to > RFC6390. > - document those performance metrics in a single location > > So my questions are: > - are we defining the performance metrics the right way? > - where is this shared repository of performance metrics (at least for > the ones created in the IETF)? > - is the PMOL directorate (RFC 6390) used effectively? > > After discussing with Dan Romascanu, we came to this conclusion > > I had a discussion with Dan Romascanu, and we settled on: > - RFC 6390 template is required for new perf metric definition > - RFC 6390 template is a nice-to-have when we refer to an existing perf metric > Nice-to-have because the performance metric reference doesn't always include > all the required information about: measurement points, measurement timing, use > and applications, reporting model, etc... but focus only on the "Method of > Measurement or Calculation" > > I would like to have a meeting during the IETF, with the following agenda > - are we defining the performance metrics the right way? > - where is this shared repository of performance metrics (at least for > the ones created in the IETF)? > - is the PMOL directorate (RFC 6390) used effectively? > - conclusion discussed with Dan > > Here is a doodle invite. Please let me know if/when you are available. > Your feedback on this mailer is also welcome. > Regards, Benoit > --------------080909080802080100060302 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Dear PMOL directorate members,

Based on doodle, this meeting will take place on Wed Nov 7th at 1440 - 1540 in the IESG breakout room, to be confirmed.
I hope to see you all.

Regards, Benoit
Dear PMOL directorate members,

During my review of the latest AVTCORE and  XRBLOCK drafts ( draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came to the conclusion that we have an issue in terms of performance metrics at the IETF, and actually in the industry.

As background information, here is my DISCUSS on the two drafts
----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

My entire point is more a DISCUSS-DISCUSS, for both
draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt.
Sorry to pick on these two drafts, but we need to have an IESG
performance metrics discussion.
Where does the list of performance metric definitions come from at the
IETF?
We have multiple sources:
- IPPM for IP performance metrics
- RTCP for RTP performance metrics: 
  Definitions in the document themselves or potentially referencing some
other SDOs
  Example: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05
   bits 014-011
            0: MAPDV2, Clause 6.2.3.2 of [G.1020],
            1: 2-point PDV, Clause 6.2.4 of [Y.1540].
- PMOL: Performance Metrics at Other Layers, with 
  RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics
- IPFIX will one day or the other exports performance metrics.
  I see for example
  http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03
  It's again a redefinition, and it should not be!

My concerns are that we start to define performance metrics in different
parts of the IETF, without consistency.

We have defined RFC 6390 on "Guidelines for Considering New Performance
Metric Development", which ask for specific definition
See http://tools.ietf.org/html/rfc6390#section-5.4.4

I believe that the IETF should at least:
- define the performance metrics in a consistent way according to
RFC6390.
- document those performance metrics in a single location

So my questions are:
- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for
the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?
After discussing with Dan Romascanu, we came to this conclusion
I had a discussion with Dan Romascanu, and we settled on:
- RFC 6390 template is required for new perf metric definition
- RFC 6390 template is a nice-to-have when we refer to an existing perf metric
Nice-to-have because the performance metric reference doesn't always include
all the required information about: measurement points, measurement timing, use
and applications, reporting model, etc... but focus only on the "Method of
Measurement or Calculation"
I would like to have a meeting during the IETF, with the following agenda
- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for
the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?
- conclusion discussed with Dan

Here is a doodle invite. Please let me know if/when you are available.
Your feedback on this mailer is also welcome.
Regards, Benoit


--------------080909080802080100060302-- From aakhter@cisco.com Wed Oct 31 08:59:31 2012 Return-Path: X-Original-To: pmol@ietfa.amsl.com Delivered-To: pmol@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2806121F87D5 for ; Wed, 31 Oct 2012 08:59:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.598 X-Spam-Level: X-Spam-Status: No, score=-110.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X0e9ubr82e5a for ; Wed, 31 Oct 2012 08:59:28 -0700 (PDT) Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) by ietfa.amsl.com (Postfix) with ESMTP id D345721F87C4 for ; Wed, 31 Oct 2012 08:59:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=17222; q=dns/txt; s=iport; t=1351699162; x=1352908762; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=GaZFpgjUELWDMr3UUAKYuSxYLqHtynViAUmtOrLVv1A=; b=BElwr+F7i+rMLb5/0JDS2e56fh8dts0ur86YOz4uNq248JboGxtcbxcj sBbwCn7g7TfC5afM62O3uEfWlxxy5cKNQvsGQue/Dv2w9Nr8x4FWj90mi LviD7agX3zkiWFf8DMGGIu1pX3U64ZA94YYB8Ct0v3v4I+jxDU8txqFnR E=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhoFAGBKkVCtJV2Z/2dsb2JhbABEgkm4JQGJAIEIgh4BAQEEEgEaTBACAQgRBAEBCx0HMhQJCAIEAQ0FCBqHZAucN6AWi3iFWmEDlxCNPYFrgm+CGQ X-IronPort-AV: E=Sophos;i="4.80,687,1344211200"; d="scan'208,217";a="134383223" Received: from rcdn-core-2.cisco.com ([173.37.93.153]) by rcdn-iport-9.cisco.com with ESMTP; 31 Oct 2012 15:59:21 +0000 Received: from xhc-aln-x14.cisco.com (xhc-aln-x14.cisco.com [173.36.12.88]) by rcdn-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id q9VFxL1r013144 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 31 Oct 2012 15:59:21 GMT Received: from xmb-rcd-x15.cisco.com ([169.254.5.107]) by xhc-aln-x14.cisco.com ([173.36.12.88]) with mapi id 14.02.0318.001; Wed, 31 Oct 2012 10:59:20 -0500 From: "Aamer Akhter (aakhter)" To: "Benoit Claise (bclaise)" , "pmol@ietf.org" Thread-Topic: [PMOL] PMOL directorate: meeting at the IETF? Thread-Index: AQHNsKSxCjgAeyQ9vEOO+J2GR0p4+pfTnbrg Date: Wed, 31 Oct 2012 15:59:20 +0000 Message-ID: <75C0E47A1889264493A2DCB2869AC09631985500@xmb-rcd-x15.cisco.com> References: <5085C8B9.2050508@cisco.com> In-Reply-To: <5085C8B9.2050508@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [64.102.41.112] x-tm-as-product-ver: SMEX-10.2.0.1135-7.000.1014-19324.001 x-tm-as-result: No--56.559800-8.000000-31 x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No Content-Type: multipart/alternative; boundary="_000_75C0E47A1889264493A2DCB2869AC09631985500xmbrcdx15ciscoc_" MIME-Version: 1.0 Cc: "xrblock-chairs@tools.ietf.org" , Ron Bonica , "ippm-chairs@tools.ietf.org" , Wesley Eddy Subject: Re: [PMOL] PMOL directorate: meeting at the IETF? X-BeenThere: pmol@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2012 15:59:31 -0000 --_000_75C0E47A1889264493A2DCB2869AC09631985500xmbrcdx15ciscoc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Just as an FYI: " http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03" Explicitly does NOT define any performance metrics-just the IPFIX transport= . There are references to other documents (the XRBLOCK, RFC3550, the perf-m= on methodology draft, etc.) that do define the metrics. Both this draft and= the perf-mon methodology draft (it's expired currently, and I need to upda= te it), are using the templates defined RFC6390. There might be some mistak= es here and there, but that is the intent anyway and I'm hopeful that furth= er reviews will make these drafts sharper. =3D=3D That said, I do not disagree with the premise below that the entire perform= ance monitoring (IMHO) creation and dissemination is (frankly) a mess. I wo= uld like to propose an enforced clean separation between metrics definition= (the methodology and usage etc-which should follow the templates in RFC639= 0) from the transport-expression of these metrics (RTCP, MIB, IPFIX, syslog= whatever), which refer back to the methodology. Unfortunately, I've got other commitments and will not be in ATL, but very = interested in moving this discussion along. From: pmol-bounces@ietf.org [mailto:pmol-bounces@ietf.org] On Behalf Of Ben= oit Claise (bclaise) Sent: Monday, October 22, 2012 6:29 PM To: pmol@ietf.org Cc: xrblock-chairs@tools.ietf.org; Ron Bonica; ippm-chairs@tools.ietf.org; = Wesley Eddy Subject: [PMOL] PMOL directorate: meeting at the IETF? Dear PMOL directorate members, During my review of the latest AVTCORE and XRBLOCK drafts ( draft-ietf-xrb= lock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came to the conclusion = that we have an issue in terms of performance metrics at the IETF, and actu= ally in the industry. As background information, here is my DISCUSS on the two drafts ---------------------------------------------------------------------- DISCUSS: ---------------------------------------------------------------------- My entire point is more a DISCUSS-DISCUSS, for both draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt. Sorry to pick on these two drafts, but we need to have an IESG performance metrics discussion. Where does the list of performance metric definitions come from at the IETF? We have multiple sources: - IPPM for IP performance metrics - RTCP for RTP performance metrics: Definitions in the document themselves or potentially referencing some other SDOs Example: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05 bits 014-011 0: MAPDV2, Clause 6.2.3.2 of [G.1020], 1: 2-point PDV, Clause 6.2.4 of [Y.1540]. - PMOL: Performance Metrics at Other Layers, with RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics - IPFIX will one day or the other exports performance metrics. I see for example http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03 It's again a redefinition, and it should not be! My concerns are that we start to define performance metrics in different parts of the IETF, without consistency. We have defined RFC 6390 on "Guidelines for Considering New Performance Metric Development", which ask for specific definition See http://tools.ietf.org/html/rfc6390#section-5.4.4 I believe that the IETF should at least: - define the performance metrics in a consistent way according to RFC6390. - document those performance metrics in a single location So my questions are: - are we defining the performance metrics the right way? - where is this shared repository of performance metrics (at least for the ones created in the IETF)? - is the PMOL directorate (RFC 6390) used effectively? After discussing with Dan Romascanu, we came to this conclusion I had a discussion with Dan Romascanu, and we settled on: - RFC 6390 template is required for new perf metric definition - RFC 6390 template is a nice-to-have when we refer to an existing perf met= ric Nice-to-have because the performance metric reference doesn't always includ= e all the required information about: measurement points, measurement timing,= use and applications, reporting model, etc... but focus only on the "Method of Measurement or Calculation" I would like to have a meeting during the IETF, with the following agenda - are we defining the performance metrics the right way? - where is this shared repository of performance metrics (at least for the ones created in the IETF)? - is the PMOL directorate (RFC 6390) used effectively? - conclusion discussed with Dan Here is a doodle invite. Please let me know if/when you are available. Your feedback on this mailer is also welcome. Regards, Benoit --_000_75C0E47A1889264493A2DCB2869AC09631985500xmbrcdx15ciscoc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

 <= /p>

Just as an FYI:

 <= /p> 

“  http://tools.ietf.org=
/html/draft-akhter-opsawg-perfmon-ipfix-03&#=
8221;

 <= /p>

Explicitly does NOT defin= e any performance metrics—just the IPFIX transport. There are referen= ces to other documents (the XRBLOCK, RFC3550, the perf-mon methodology draft, etc.) that do define the metrics. Both this draft and the perf-mon = methodology draft (it’s expired currently, and I need to update it), = are using the templates defined RFC6390. There might be some mistakes here = and there, but that is the intent anyway and I’m hopeful that further reviews will make these drafts sharper.=

 <= /p>

=3D=3D<= /p>

 <= /p>

That said, I do not disag= ree with the premise below that the entire performance monitoring (IMHO) cr= eation and dissemination is (frankly) a mess. I would like to propose an enforced clean separation between metrics definition (the me= thodology and usage etc—which should follow the templates in RFC6390)= from the transport-expression of these metrics (RTCP, MIB, IPFIX, syslog w= hatever), which refer back to the methodology.

 <= /p>

Unfortunately, I’ve= got other commitments and will not be in ATL, but very interested in movin= g this discussion along.

 <= /p>

 <= /p>

From: pmol-bounces@ietf.org [mailto:pmol-bounces@ietf.o= rg] On Behalf Of Benoit Claise (bclaise)
Sent: Monday, October 22, 2012 6:29 PM
To: pmol@ietf.org
Cc: xrblock-chairs@tools.ietf.org; Ron Bonica; ippm-chairs@tools.iet= f.org; Wesley Eddy
Subject: [PMOL] PMOL directorate: meeting at the IETF?

 

Dear PMOL directorate members,

During my review of the latest AVTCORE and  XRBLOCK drafts ( draft-iet= f-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came to the conclu= sion that we have an issue in terms of performance metrics at the IETF, and= actually in the industry.

As background information, here is my DISCUSS on the two drafts<= /p> 

----------------------------------------------------------------------=

DISCUSS:
----------------------------------------------------------------------=

 
My entire point is more a DISCUSS-DISCUSS, for both
draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.tx=
t.
Sorry to pick on these two drafts, but we need to have an IESG
performance metrics discussion.
Where does the list of performance metric definitions come from at the=

IETF?
We have multiple sources:
- IPPM for IP performance metrics
- RTCP for RTP performance metrics: 
  Definitions in the document themselves or potentially refe=
rencing some
other SDOs
  Example: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pd=
v-05
   bits 014-011
            0: =
MAPDV2, Clause 6.2.3.2 of [G.1020],
            1: =
2-point PDV, Clause 6.2.4 of [Y.1540].
- PMOL: Performance Metrics at Other Layers, with 
  RFC 6076 on Basic Telephony SIP End-to-End Performance Met=
rics
- IPFIX will one day or the other exports performance metrics.
  I see for example
  http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-0=
3
  It's again a redefinition, and it should not be!

 


My concerns are that we start to define performance metrics in differe=
nt


parts of the IETF, without consistency.


 


We have defined RFC 6390 on "Guidelines for Considering New Perfo=
rmance


Metric Development", which ask for specific definition=



See http:=
//tools.ietf.org/html/rfc6390#section-5.4.4


 


I believe that the IETF should at least:


- define the performance metrics in a consistent way according to=



RFC6390.


- document those performance metrics in a single location

 


So my questions are:


- are we defining the performance metrics the right way?

- where is this shared repository of performance metrics (at least for=



the ones created in the IETF)?


- is the PMOL directorate (RFC 6390) used effectively?

After discussing with Dan Romascanu, we came to this=
 conclusion


I had a discussion with Dan Romascanu, and we settled on:

- RFC 6390 template is required for new perf metric definition


- RFC 6390 template is a nice-to-have when we refer to an existing per=
f metric


Nice-to-have because the performance metric reference doesn't always i=
nclude


all the required information about: measurement points, measurement ti=
ming, use


and applications, reporting model, etc... but focus only on the "=
Method of


Measurement or Calculation"


I would like to have a meeting during the IETF, with=
 the following agenda






- are we defining the performance metrics the right way?

- where is this shared repository of performance metrics (at least for=



the ones created in the IETF)?


- is the PMOL directorate (RFC 6390) used effectively?

- conclusion discussed with Dan


 


Here is a doodle invite. Please let me know if/when you are available.=



Your feedback on this mailer is also welcome.


Regards, Benoit<=
/o:p>
--_000_75C0E47A1889264493A2DCB2869AC09631985500xmbrcdx15ciscoc_--