Subject:	[IESG-AGENDA-DIST] Summarized Agenda for the 2013-09-12 IESG Teleconference

Hi Vinayak,<= /div>

Thanks a lot for the feedb= ack! We are currently revising this document, and I agree that Section= 6.3 needs substantial rewording. More comments inline [MS]...

<= span class=3D"265112112-13092013">&nb= sp;

Thanks again!

Michael

From: Vinayak Hegde [mailto:vinayak= h@gmail.com]
Sent: Saturday, September 07, 2013 6:23 PM
To: martin.stiemerling@neclab.eu; ietf-alto@skiesel.de; sprevidi@cis= co.com; Scharf, Michael (Michael)
Cc: pm-dir@ietf.org
Subject: Performance directorate review of draft-ietf-alto-deploymen= ts

Hi,

There was a performance review either requested by you or was triggere= d by Benoit's script. I am reviewing this from a performance metrics point = of view.

I looked at the draft (https://datatracker.ietf.org/doc/dra= ft-ietf-alto-deployments/?include_text=3D1) These comments are only on = the Section 6.3 Monitoring ALTO

Comments on Section 6.3

The authors should define some recommended metrics here (possibly IPPM= Type-P metrics) if not recommended metric here. This section can be substa= ntially expanded.

[MS] I am not sure if this document should really f= ormally define metrics (the current wording implies that). The more importa= nt aspect seems to me the overall effects that an operator may want to look at when deploying ALTO. This significant= ly differs between different use cases of ALTO. For instance, if ALTO = is used for traffic optimization inside one AS, monitoring a= metric that describes inter-domain traffic may not be required. Therefore, the relevant metrics will depend on the use case= , as also noted in your review. An alternative document structure would hav= e some discussion of monitoring per use case, but this would imply a s= ignificant change of the table of content of the document. We currently look int= o this, but this requires feedback from the ALTO WG, too.

Comments on section 6.3.1

There are several metrics defined here but they can be more clearly de= fined by following the formats specified in RFC 6390. Also useful will some= use-cases to explain why these metrics make sense and what needs to be use= d in which scenario.

[MS] Agreed. As I said, one potential solution is t= o move this discussion to the descriptions of the use cases (Section 4, 5, = etc.), maybe with one "monitoring" subsection per use-case.

As regards to the network hop-count to be used indirectly for latency,= it might be better to define a different metric for latency as the average= network loads and buffers on each device can be radically different. For e= xample a hybrid network that contains a backbone network, last mile access network and a mobile network (which i= s not unusual in many ISPs today). The delay for transit (as well as packet= loss) internally from one to other might be drastically different.

[MS] I agree that hop-count and latency may not cor= relate. Also, measuring hop-count can be difficult in some cases. If the In= ternet traffic is carried over an MPLS/IP aggregation/core network, the number of routers that are traversed by the = traffic can be pretty different to the number of IP hops. Reducing the IP h= ops does not necessarily imply a more efficient routing. My current thinkin= g is that the document should more focus on "traffic locality" than path monitoring metrics.=

Also it might be useful to use passive monitoring in the case of some = metrics such as application download rate.

[MS] Yes, the document probably should reference IP= FIX, LMAP, etc.

=

Section 6.3.3

The example given here can be expanded to give a more clearer picture = of which are the active monitoring agents and which are the passive compone= nts. The metrics used for monitoring need to standardized. Also you might w= ant to add some recommendation on the data interchange format and method of delivery. This is especially imp= ortant as the interacting entities (ISP and Application Owner) are present = at different layers and have different priorities.This can also be fixed in= section 6.3.1 above.

[MS] Yes, the text and Figure 19 describes the scen= ario at a very high-level and, indeed, it is not clear. Both the quest= ion what somebody can measure, and whether this is done actively or passively, depends on the use case and who that entity= actually is. But I don't believe that this document can provide = comprehensive guidance on data exchange formats and methods of deliver= y, because inter-organization communication raises a lot of non-technical issues, and even standardization of new protocols m= ight be needed. That seems outside the scope of this document specifically,= and ALTO in general. This seems more related to LMAP.

Thanks

Vinayak

--_000_655C07320163294895BBADA28372AF5D0BEB38FR712WXCHMBA15zeu_-- From bclaise@cisco.com Sun Sep 15 14:06:57 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 807B911E81D5 for ; Sun, 15 Sep 2013 14:06:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.547 X-Spam-Level: X-Spam-Status: No, score=-10.547 tagged_above=-999 required=5 tests=[AWL=0.052, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NfyPwmmbi4-c for ; Sun, 15 Sep 2013 14:06:51 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 62CFF11E81D4 for ; Sun, 15 Sep 2013 14:06:51 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r8FL6nr1003420 for ; Sun, 15 Sep 2013 23:06:49 +0200 (CEST) Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r8FL69G5006438 for ; Sun, 15 Sep 2013 23:06:19 +0200 (CEST) Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r8FL66I5015464 for pm-dir@ietf.org; Sun, 15 Sep 2013 23:06:06 +0200 (CEST) Date: Sun, 15 Sep 2013 23:06:06 +0200 From: Benoit Claise To: pm-dir@ietf.org Message-ID: <20130915210606.GA15460@sweet-brew-5.cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Sep 2013 21:06:57 -0000 Dear all, This is an automatically generated email. It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference. It also lists all the IETF internet-drafts that contain "performance metric". Regards, Benoit =========================================================== Normative References -------------------- draft-ietf-ippm-testplan-rfc2680-03 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state Informative References ---------------------- draft-ietf-ippm-testplan-rfc2680-03 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-loss-conceal-08 Active draft-ietf-xrblock-rtcp-xr-qoe-11 Active draft-ietf-xrblock-rtcp-xr-synchronization-06 Active drafts containing performance metric ------------------------------------ draft-ietf-alto-deployments-07 Active draft-ietf-cdni-footprint-capabilities-semantics-00Active draft-ietf-ippm-2330-update-00 Active draft-ietf-ippm-lmap-path-00 Active draft-ietf-ippm-model-based-metrics-00 Active draft-ietf-ippm-rate-problem-03 Active draft-ietf-ippm-testplan-rfc2680-03 Active draft-ietf-manet-smf-mib-08 In IESG processing - ID Tracker state draft-ietf-nvo3-framework-03 Active draft-ietf-opsawg-oam-overview-09 In IESG processing - ID Tracker state draft-ietf-ppsp-base-tracker-protocol-01 Active draft-ietf-ppsp-peer-protocol-07 Active draft-ietf-rtcweb-rtp-usage-09 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-loss-conceal-08 Active draft-ietf-xrblock-rtcp-xr-qoe-11 Active draft-ietf-xrblock-rtcp-xr-synchronization-06 Active From Jan.Seedorf@neclab.eu Wed Sep 18 05:34:03 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E8E0711E8255 for ; Wed, 18 Sep 2013 05:34:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.599 X-Spam-Level: X-Spam-Status: No, score=-103.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m93yJ7hQQUcj for ; Wed, 18 Sep 2013 05:33:59 -0700 (PDT) Received: from mailer1.neclab.eu (mailer1.neclab.eu [195.37.70.40]) by ietfa.amsl.com (Postfix) with ESMTP id 7037E11E8252 for ; Wed, 18 Sep 2013 05:33:59 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mailer1.neclab.eu (Postfix) with ESMTP id 5607C10581C; Wed, 18 Sep 2013 14:31:32 +0200 (CEST) X-Virus-Scanned: Amavisd on Debian GNU/Linux (netlab.nec.de) Received: from mailer1.neclab.eu ([127.0.0.1]) by localhost (atlas-a.office.hd [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w6-D48ou0xwv; Wed, 18 Sep 2013 14:31:32 +0200 (CEST) Received: from ENCELADUS.office.hd (enceladus.office.hd [192.168.24.52]) by mailer1.neclab.eu (Postfix) with ESMTP id 34CD4105810; Wed, 18 Sep 2013 14:30:52 +0200 (CEST) Received: from DAPHNIS.office.hd ([169.254.2.213]) by ENCELADUS.office.hd ([192.168.24.52]) with mapi id 14.01.0323.003; Wed, 18 Sep 2013 14:33:17 +0200 From: Jan Seedorf To: Vinayak Hegde , "jon.peterson@neustar.biz" , "sprevidi@cisco.com" , "ray.vanbrandenburg@tno.nl" , "kevin.ma@azukisystems.com" Thread-Topic: Performance Directorate Review of cdni footprint capabilities semantics Thread-Index: AQHOq/hRA53lgDw/qU6gMxQano6LzJnLfHmQ Date: Wed, 18 Sep 2013 12:33:17 +0000 Message-ID: <2779C9F0771F974CAD742BAE6D9904FE55604195@DAPHNIS.office.hd> References: In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.1.5.89] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 18 Sep 2013 05:53:13 -0700 Cc: "pm-dir@ietf.org" , "Daryl Malas \(D.Malas@cablelabs.com\)" , "Francois Le Faucheur \(flefauch\)" Subject: Re: [pm-dir] Performance Directorate Review of cdni footprint capabilities semantics X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Sep 2013 12:34:04 -0000 Dear Vinayak, Thank you for your review. The point if the CDNI footprint capabilities sem= antics draft is to give guidance for yet-to-be-specified CDNI footprint/cap= abilities advertisement solution documents. Thus, the draft is not normiati= ve for any metrics, and should also not cause interop issues. Actually the = guidance the CDNI footprint capabilities semantics draft gives is not to us= e capabilities with complex metrics such as streaming bandwidth for downstr= eam CDN selection (the text on streaming bandwidth is taken from the CDNI r= equirements RFC).=20 > At a first glance, several broad categories of capabilities seem > useful to convey via an advertisement interface (and indeed many such > candidate capabilities have been suggested in CDNI drafts, see > Section 2). However, advertising capabilities that change highly > dynamically (e.g. real-time delivery performance metrics, CDN > resource load, or other highly dynamically changing QoS information) > should probably not be in scope for the CDNI FCI. First, out of the > multitude of possible metrics and capabilities, it is hard to agree > on a subset and the precise metrics to be used. Second, and perhaps > more importantly, it seems not feasible to specify such highly > dynamically changing capabilities and the corresponding metrics > within the CDNI charter time-frame. Instead of listing all the possible metrics that the draft advised not to u= se, the draft takes the approach of listing mandatory capabilities (which a= re explained in some detail), and specifying a procedure for adding new cap= abilities later on. In summary, I do not think it is needed to precisely define the metrics of = capabilities the draft explicitly excludes from being relevant for the CDNI= specifications. But maybe I am missing something, or some of the co-author= s have different views. What so you think? - Jan > -----Original Message----- > From: Vinayak Hegde [mailto:vinayakh@gmail.com] > Sent: Saturday, September 07, 2013 8:30 PM > To: jon.peterson@neustar.biz; Jan Seedorf; sprevidi@cisco.com; > ray.vanbrandenburg@tno.nl; kevin.ma@azukisystems.com > Cc: pm-dir@ietf.org > Subject: Performance Directorate Review of cdni footprint capabilities > semantics >=20 > Hi, >=20 > I reviewed the CDNI footprint capabilities semantics draft (as a reviewer= on > the Performance directorate). My comments on the draft follow: >=20 > In section 2 you reference CDNI requirements drafts and mention several > metrics such as streaming bandwidth but they are not explicitly listed or > defined anywhere in the draft. Also the measurement methodology for the > smae is not outlined. Can you please do that using RFC 6390 template to b= ring > more clarity. >=20 > Also in section 6 you mention >=20 > At a first glance, several broad categories of capabilities seem > useful to convey via an advertisement interface (and indeed many such > candidate capabilities have been suggested in CDNI drafts, see > Section 2). However, advertising capabilities that change highly > dynamically (e.g. real-time delivery performance metrics, CDN > resource load, or other highly dynamically changing QoS information) > should probably not be in scope for the CDNI FCI. First, out of the > multitude of possible metrics and capabilities, it is hard to agree > on a subset and the precise metrics to be used. Second, and perhaps > more importantly, it seems not feasible to specify such highly > dynamically changing capabilities and the corresponding metrics > within the CDNI charter time-frame. >=20 >=20 >=20 > It might be useful here to atleast list out the possible metrics (as use- > cases/scenarios such as in static content/streaming capabilities) rather = than > leave it out completely to the discretion of the implementer (which will = cause > interop issues later on). >=20 >=20 > -- Vinayak From bclaise@cisco.com Sun Sep 22 14:06:48 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 144DB11E8151 for ; Sun, 22 Sep 2013 14:06:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.54 X-Spam-Level: X-Spam-Status: No, score=-10.54 tagged_above=-999 required=5 tests=[AWL=0.059, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1HMkxB9-A4oH for ; Sun, 22 Sep 2013 14:06:42 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id DFA9C11E8147 for ; Sun, 22 Sep 2013 14:06:41 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r8ML6d2c015640 for ; Sun, 22 Sep 2013 23:06:40 +0200 (CEST) Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r8ML6AGb014539 for ; Sun, 22 Sep 2013 23:06:20 +0200 (CEST) Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r8ML67QO026695 for pm-dir@ietf.org; Sun, 22 Sep 2013 23:06:07 +0200 (CEST) Date: Sun, 22 Sep 2013 23:06:06 +0200 From: Benoit Claise To: pm-dir@ietf.org Message-ID: <20130922210606.GA26693@sweet-brew-5.cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 21:06:48 -0000 Dear all, This is an automatically generated email. It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference. It also lists all the IETF internet-drafts that contain "performance metric". Regards, Benoit =========================================================== Normative References -------------------- draft-ietf-ippm-testplan-rfc2680-03 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state Informative References ---------------------- draft-ietf-ippm-testplan-rfc2680-03 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-loss-conceal-08 Active draft-ietf-xrblock-rtcp-xr-qoe-11 Active draft-ietf-xrblock-rtcp-xr-synchronization-06 Active drafts containing performance metric ------------------------------------ draft-ietf-alto-deployments-07 Active draft-ietf-cdni-footprint-capabilities-semantics-00Active draft-ietf-ippm-2330-update-00 Active draft-ietf-ippm-lmap-path-00 Active draft-ietf-ippm-model-based-metrics-00 Active draft-ietf-ippm-rate-problem-04 Active draft-ietf-ippm-testplan-rfc2680-03 Active draft-ietf-manet-smf-mib-08 In IESG processing - ID Tracker state draft-ietf-nvo3-framework-03 Active draft-ietf-opsawg-oam-overview-09 Active draft-ietf-ppsp-base-tracker-protocol-01 Active draft-ietf-ppsp-peer-protocol-07 Active draft-ietf-rtcweb-rtp-usage-09 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-loss-conceal-08 Active draft-ietf-xrblock-rtcp-xr-qoe-11 Active draft-ietf-xrblock-rtcp-xr-synchronization-06 Active From vinayakh@gmail.com Mon Sep 23 01:17:40 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 130D411E80E9 for ; Mon, 23 Sep 2013 01:17:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CcNpinupNkun for ; Mon, 23 Sep 2013 01:17:33 -0700 (PDT) Received: from mail-pd0-x22b.google.com (mail-pd0-x22b.google.com [IPv6:2607:f8b0:400e:c02::22b]) by ietfa.amsl.com (Postfix) with ESMTP id 9571111E80DF for ; Mon, 23 Sep 2013 01:17:30 -0700 (PDT) Received: by mail-pd0-f171.google.com with SMTP id g10so2926484pdj.30 for ; Mon, 23 Sep 2013 01:17:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kLsh8+awgYJaMSqMe/qkcIPTDivrEMTYPBizCUI2hVs=; b=VrUyYIPO14wYz5ztcg6SB+vYYfzdJNFzdiduk3VhJ/dAau1rM/W0ufMAUhfYQUPNXR R+5JvgzeKb0xx5WqpNSXZv78blMudb2ivsfsMiC4CDSL8NofrKXAo3kJ1Y9mcFSTLDgd 0W75mbgpCyU4JmxgvOfJuIeEIRkqgpQei0U/Kj4R+1ztGFtS0uz3IGNRtKkPxqIJpsKT bRBUFVNHSipa/7I+GBAUmbEG4PoA/BVhaNYfa1LRyLxCUaIl0dptr7Wlbqf2QM/bLtCL NG8L+A7T42QcWGnwKVtlmajef7VgYJhAIF2A71Ur1WNVbrPmyuhBbniJr3mYCCg6bDLS nUUQ== MIME-Version: 1.0 X-Received: by 10.66.230.138 with SMTP id sy10mr23815704pac.103.1379924248674; Mon, 23 Sep 2013 01:17:28 -0700 (PDT) Received: by 10.66.90.39 with HTTP; Mon, 23 Sep 2013 01:17:28 -0700 (PDT) In-Reply-To: <2779C9F0771F974CAD742BAE6D9904FE55604195@DAPHNIS.office.hd> References: <2779C9F0771F974CAD742BAE6D9904FE55604195@DAPHNIS.office.hd> Date: Mon, 23 Sep 2013 13:47:28 +0530 Message-ID: From: Vinayak Hegde To: Jan Seedorf Content-Type: multipart/alternative; boundary=047d7b15ab8da8c88d04e708a62f Cc: "Francois Le Faucheur \(flefauch\)" , "Daryl Malas \(D.Malas@cablelabs.com\)" , "jon.peterson@neustar.biz" , "kevin.ma@azukisystems.com" , "sprevidi@cisco.com" , "pm-dir@ietf.org" , "ray.vanbrandenburg@tno.nl" Subject: Re: [pm-dir] Performance Directorate Review of cdni footprint capabilities semantics X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 08:17:49 -0000 --047d7b15ab8da8c88d04e708a62f Content-Type: text/plain; charset=ISO-8859-1 Hi Jan, Since end-user performance is one of the main goals of the CDNI group, the draft would be incomplete without specifying and agreeing on metrics for atleast the following decisions: 1. Choosing the downstream CDN 2. Penalising the downstream CDN for faking metrics / not adhering to SLAs (example large number of timeouts). 3. Checking and evaluating the performance of each of the downstream CDN providers in case there are multiple choices. The draft may gloss over these now but the WG will have to specify these (atleast non-normative) to make sense to implementers. I am leaving out the mechanics out of this mail. The metrics can be specified in this draft or another one. Either was this draft should include or point to this information. By not specifying it in this draft or a linked one, you are essentially kicking the can down the road. -- Vinayak On Wed, Sep 18, 2013 at 6:03 PM, Jan Seedorf wrote: > Dear Vinayak, > > Thank you for your review. The point if the CDNI footprint capabilities > semantics draft is to give guidance for yet-to-be-specified CDNI > footprint/capabilities advertisement solution documents. Thus, the draft is > not normiative for any metrics, and should also not cause interop issues. > Actually the guidance the CDNI footprint capabilities semantics draft gives > is not to use capabilities with complex metrics such as streaming bandwidth > for downstream CDN selection (the text on streaming bandwidth is taken from > the CDNI requirements RFC). > > > At a first glance, several broad categories of capabilities seem > > useful to convey via an advertisement interface (and indeed many such > > candidate capabilities have been suggested in CDNI drafts, see > > Section 2). However, advertising capabilities that change highly > > dynamically (e.g. real-time delivery performance metrics, CDN > > resource load, or other highly dynamically changing QoS information) > > should probably not be in scope for the CDNI FCI. First, out of the > > multitude of possible metrics and capabilities, it is hard to agree > > on a subset and the precise metrics to be used. Second, and perhaps > > more importantly, it seems not feasible to specify such highly > > dynamically changing capabilities and the corresponding metrics > > within the CDNI charter time-frame. > Instead of listing all the possible metrics that the draft advised not to > use, the draft takes the approach of listing mandatory capabilities (which > are explained in some detail), and specifying a procedure for adding new > capabilities later on. > > In summary, I do not think it is needed to precisely define the metrics of > capabilities the draft explicitly excludes from being relevant for the CDNI > specifications. But maybe I am missing something, or some of the co-authors > have different views. > > What so you think? > > - Jan > > > -----Original Message----- > > From: Vinayak Hegde [mailto:vinayakh@gmail.com] > > Sent: Saturday, September 07, 2013 8:30 PM > > To: jon.peterson@neustar.biz; Jan Seedorf; sprevidi@cisco.com; > > ray.vanbrandenburg@tno.nl; kevin.ma@azukisystems.com > > Cc: pm-dir@ietf.org > > Subject: Performance Directorate Review of cdni footprint capabilities > > semantics > > > > Hi, > > > > I reviewed the CDNI footprint capabilities semantics draft (as a > reviewer on > > the Performance directorate). My comments on the draft follow: > > > > In section 2 you reference CDNI requirements drafts and mention several > > metrics such as streaming bandwidth but they are not explicitly listed or > > defined anywhere in the draft. Also the measurement methodology for the > > smae is not outlined. Can you please do that using RFC 6390 template to > bring > > more clarity. > > > > Also in section 6 you mention > > > > At a first glance, several broad categories of capabilities seem > > useful to convey via an advertisement interface (and indeed many such > > candidate capabilities have been suggested in CDNI drafts, see > > Section 2). However, advertising capabilities that change highly > > dynamically (e.g. real-time delivery performance metrics, CDN > > resource load, or other highly dynamically changing QoS information) > > should probably not be in scope for the CDNI FCI. First, out of the > > multitude of possible metrics and capabilities, it is hard to agree > > on a subset and the precise metrics to be used. Second, and perhaps > > more importantly, it seems not feasible to specify such highly > > dynamically changing capabilities and the corresponding metrics > > within the CDNI charter time-frame. > > > > > > > > It might be useful here to atleast list out the possible metrics (as use- > > cases/scenarios such as in static content/streaming capabilities) rather > than > > leave it out completely to the discretion of the implementer (which will > cause > > interop issues later on). > > > > > > -- Vinayak > > --047d7b15ab8da8c88d04e708a62f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Hi Jan,

Since e= nd-user performance is one of the main goals of the CDNI group, the draft w= ould be incomplete without specifying and agreeing on metrics for atleast t= he following decisions:

1. Choosing the downstream CDN

2. Penalising the downstream = CDN for faking metrics / not adhering to SLAs (example large number of time= outs).

3. Checking and evaluating the performance of each of the d= ownstream CDN providers in case there are multiple choices.

The draft may gloss over these now but the WG will have to specif= y these (atleast non-normative) to make sense to implementers. I am leaving= out the mechanics out of this mail. The metrics can be specified in this d= raft or another one. Either was this draft should include or point to this = information.

By not specifying it in this draft or a linked one, you are essen= tially kicking the can down the road.

-- Vinayak

On Wed, Sep 18, 2013 at= 6:03 PM, Jan Seedorf <Jan.Seedorf@neclab.eu> wrote:

Dear Vinayak,

Thank you for your review. The point if the CDNI footprint capabilities sem= antics draft is to give guidance for yet-to-be-specified CDNI footprint/cap= abilities advertisement solution documents. Thus, the draft is not normiati= ve for any metrics, and should also not cause interop issues. Actually the = guidance the CDNI footprint capabilities semantics draft gives is not to us= e capabilities with complex metrics such as streaming bandwidth for downstr= eam CDN selection (the text on streaming bandwidth is taken from the CDNI r= equirements RFC).

> =A0 =A0At a first glance, several broad categories of capabilities see= m
> =A0 =A0useful to convey via an advertisement interface (and indeed man= y such
> =A0 =A0candidate capabilities have been suggested in CDNI drafts, see<= br> > =A0 =A0Section 2). =A0However, advertising capabilities that change hi= ghly
> =A0 =A0dynamically (e.g. real-time delivery performance metrics, CDN > =A0 =A0resource load, or other highly dynamically changing QoS informa= tion)
> =A0 =A0should probably not be in scope for the CDNI FCI. =A0First, out= of the
> =A0 =A0multitude of possible metrics and capabilities, it is hard to a= gree
> =A0 =A0on a subset and the precise metrics to be used. =A0Second, and = perhaps
> =A0 =A0more importantly, it seems not feasible to specify such highly<= br> > =A0 =A0dynamically changing capabilities and the corresponding metrics=
> =A0 =A0within the CDNI charter time-frame.

Instead of listing all the possible metrics that the draft advised no= t to use, the draft takes the approach of listing mandatory capabilities (w= hich are explained in some detail), and specifying a procedure for adding n= ew capabilities later on.

In summary, I do not think it is needed to precisely define the metrics of = capabilities the draft explicitly excludes from being relevant for the CDNI= specifications. But maybe I am missing something, or some of the co-author= s have different views.

What so you think?

=A0- Jan

> -----Original Message-----
> From: Vinayak Hegde [mailto:vina= yakh@gmail.com]

> Sent: Saturday, September 07, 2013 8:30= PM
> To: jon.peterson@neustar.b= iz; Jan Seedorf; sprevidi@cisco.c= om;
> ray.vanbrandenburg@tno.nl= ; kevin.ma@azukisystems.co= m
> Cc: pm-dir@ietf.org

> Subject: Performance Directorate Review= of cdni footprint capabilities
> semantics
>

> Hi,
>
> I reviewed the CDNI footprint capabilities semantics draft (as a revie= wer on
> the Performance directorate). My comments on the draft follow:
>
> In section 2 you reference CDNI requirements drafts and mention severa= l
> metrics such as streaming bandwidth but they are not explicitly listed= or
> defined anywhere in the draft. Also the measurement methodology for th= e
> smae is not outlined. Can you please do that using RFC 6390 template t= o bring
> more clarity.
>
> Also in section 6 you mention
>
> =A0 =A0At a first glance, several broad categories of capabilities see= m
> =A0 =A0useful to convey via an advertisement interface (and indeed man= y such
> =A0 =A0candidate capabilities have been suggested in CDNI drafts, see<= br> > =A0 =A0Section 2). =A0However, advertising capabilities that change hi= ghly
> =A0 =A0dynamically (e.g. real-time delivery performance metrics, CDN > =A0 =A0resource load, or other highly dynamically changing QoS informa= tion)
> =A0 =A0should probably not be in scope for the CDNI FCI. =A0First, out= of the
> =A0 =A0multitude of possible metrics and capabilities, it is hard to a= gree
> =A0 =A0on a subset and the precise metrics to be used. =A0Second, and = perhaps
> =A0 =A0more importantly, it seems not feasible to specify such highly<= br> > =A0 =A0dynamically changing capabilities and the corresponding metrics=
> =A0 =A0within the CDNI charter time-frame.
>
>
>
> It might be useful here to atleast list out the possible metrics (as u= se-
> cases/scenarios such as in static content/streaming capabilities) rath= er than
> leave it out completely to the discretion of the implementer (which wi= ll cause
> interop issues later on).
>
>
> -- Vinayak

--047d7b15ab8da8c88d04e708a62f-- From vinayakh@gmail.com Mon Sep 23 02:33:19 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6E37E21F9E9C for ; Mon, 23 Sep 2013 02:33:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ylbld1EgRcGX for ; Mon, 23 Sep 2013 02:33:12 -0700 (PDT) Received: from mail-pb0-x233.google.com (mail-pb0-x233.google.com [IPv6:2607:f8b0:400e:c01::233]) by ietfa.amsl.com (Postfix) with ESMTP id 4B5C221F9E9F for ; Mon, 23 Sep 2013 02:33:09 -0700 (PDT) Received: by mail-pb0-f51.google.com with SMTP id jt11so2966406pbb.38 for ; Mon, 23 Sep 2013 02:33:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cyGEhWxtsuK6N0vbM17lVoktC8e3mr8C/TzY0K5+E3E=; b=iauAq6I/+rddcxus7aqWwJKDRgD3/jhKH+1mCNywTZpb9qZW5g5mY0zyRgFeK6jz6I yLJTqTv04m/jpr3DoOC2sowOwIgAj1FC978Cb4LbsLK+vZ9hbhRjsUeohYFDMQkKVyu+ f5odhIa6bjklv/j6F+mU99RqAZtUPSDDExKdj65UHJhxDB9/frjABDj702NhhocGiihH 9M9toKO+XpsuHNMCCabuOAQZ0fwB7zJpOml70COcab2OMcoasq6kU0Zqz2x7jv0KPrLz wCh3qDJ1PNGP4Ha1mccQSg8HPHjthRz8yb91P0xBtRcWbEUf//d6Y/NTi9CI/aKGfjui Ha9A== MIME-Version: 1.0 X-Received: by 10.68.252.135 with SMTP id zs7mr346708pbc.194.1379928789686; Mon, 23 Sep 2013 02:33:09 -0700 (PDT) Received: by 10.66.90.39 with HTTP; Mon, 23 Sep 2013 02:33:09 -0700 (PDT) In-Reply-To: <655C07320163294895BBADA28372AF5D0BEB38@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <655C07320163294895BBADA28372AF5D0BEB38@FR712WXCHMBA15.zeu.alcatel-lucent.com> Date: Mon, 23 Sep 2013 15:03:09 +0530 Message-ID: From: Vinayak Hegde To: "Scharf, Michael (Michael)" Content-Type: multipart/alternative; boundary=047d7b2e0c315325bc04e709b540 Cc: "ietf-alto@skiesel.de" , "pm-dir@ietf.org" , "sprevidi@cisco.com" , "martin.stiemerling@neclab.eu" Subject: Re: [pm-dir] Performance directorate review of draft-ietf-alto-deployments X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 09:33:19 -0000 --047d7b2e0c315325bc04e709b540 Content-Type: text/plain; charset=ISO-8859-1 On Fri, Sep 13, 2013 at 6:50 PM, Scharf, Michael (Michael) < michael.scharf@alcatel-lucent.com> wrote: > ** > [snipped] > Thanks again! > > You are welcome. Comments inline. > ------------------------------ > *From:* Vinayak Hegde [mailto:vinayakh@gmail.com] > *Sent:* Saturday, September 07, 2013 6:23 PM > *To:* martin.stiemerling@neclab.eu; ietf-alto@skiesel.de; > sprevidi@cisco.com; Scharf, Michael (Michael) > *Cc:* pm-dir@ietf.org > *Subject:* Performance directorate review of draft-ietf-alto-deployments > > Hi, > > There was a performance review either requested by you or was triggered > by Benoit's script. I am reviewing this from a performance metrics point of > view. > > I looked at the draft ( > https://datatracker.ietf.org/doc/draft-ietf-alto-deployments/?include_text=1) > These comments are only on the Section 6.3 Monitoring ALTO > > Comments on Section 6.3 > The authors should define some recommended metrics here (possibly IPPM > Type-P metrics) if not recommended metric here. This section can be > substantially expanded. > > [MS] I am not sure if this document should really formally define metrics > (the current wording implies that). The more important aspect seems to > me the overall effects that an operator may want to look at when deploying > ALTO. This significantly differs between different use cases of ALTO. > For instance, if ALTO is used for traffic optimization inside > one AS, monitoring a metric that describes inter-domain traffic may not be > required. Therefore, the relevant metrics will depend on the use case, as > also noted in your review. An alternative document structure would have > some discussion of monitoring per use case, but this would imply a > significant change of the table of content of the document. We currently > look into this, but this requires feedback from the ALTO WG, too. > > Yeah a couple of use-cases would really help though it is bit more work. It would make the document more complete and usable for implementers. > Comments on section 6.3.1 > There are several metrics defined here but they can be more clearly > defined by following the formats specified in RFC 6390. Also useful will > some use-cases to explain why these metrics make sense and what needs to be > used in which scenario. > > [MS] Agreed. As I said, one potential solution is to move this discussion > to the descriptions of the use cases (Section 4, 5, etc.), maybe with one > "monitoring" subsection per use-case. > > Yup. Agree. Makes sense. > > As regards to the network hop-count to be used indirectly for latency, it > might be better to define a different metric for latency as the average > network loads and buffers on each device can be radically different. For > example a hybrid network that contains a backbone network, last mile > access network and a mobile network (which is not unusual in many ISPs > today). The delay for transit (as well as packet loss) internally from one > to other might be drastically different. > > [MS] I agree that hop-count and latency may not correlate. Also, measuring > hop-count can be difficult in some cases. If the Internet traffic is > carried over an MPLS/IP aggregation/core network, the number of routers > that are traversed by the traffic can be pretty different to the number of > IP hops. Reducing the IP hops does not necessarily imply a more efficient > routing. My current thinking is that the document should more focus on > "traffic locality" than path monitoring metrics. > > Agree. Makes sense. Can you reword the mentioned paragraph ? > Section 6.3.3 > The example given here can be expanded to give a more clearer picture of > which are the active monitoring agents and which are the passive > components. The metrics used for monitoring need to standardized. Also you > might want to add some recommendation on the data interchange format and > method of delivery. This is especially important as the interacting > entities (ISP and Application Owner) are present at different layers and > have different priorities.This can also be fixed in section 6.3.1 above. > > [MS] Yes, the text and Figure 19 describes the scenario at a very > high-level and, indeed, it is not clear. Both the question what somebody > can measure, and whether this is done actively or passively, depends on the > use case and who that entity actually is. But I don't believe that > this document can provide comprehensive guidance on data exchange formats > and methods of delivery, because inter-organization communication raises a > lot of non-technical issues, and even standardization of new protocols > might be needed. That seems outside the scope of this document > specifically, and ALTO in general. This seems more related to LMAP. > > The document need not be comprehensive. That was not the intention of the suggestion. But non-normative language about possible measures / metrics could be useful for guidance. Data interchange formats such as JSON could be used for easy parsing instead of binary serialisation. -- Vinayak --047d7b2e0c315325bc04e709b540 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Fri, Sep 13, 2013 at 6:50 PM, Scharf, Michael (Michael) <michael.scharf@alcatel-lucent.com> wrote:

[snipped]=A0
Thanks again!

<= /div>

You are welcome. Comments inline.

From: Vinayak Hegde [mailto:vinayakh@gmail.com]
Sent: Saturday, September 07, 2013 6:23 PM
To: martin.stiemerling@neclab.eu; ietf-alto@skiesel.de; sprevidi@cisco.com; Scharf, Michael (Michael)=
Cc: pm-dir@ietf= .org
Subject: Performance directorate review of draft-ietf-alto-deploymen= ts

Hi,

There was a performance review either requested by you or was triggere= d by Benoit's script. I am reviewing this from a performance metrics po= int of view.

I looked at the draft (https://datatracke= r.ietf.org/doc/draft-ietf-alto-deployments/?include_text=3D1) These com= ments are only on the Section 6.3 Monitoring ALTO

Comments on Section 6.3

The authors should define some recommended metrics here (possibly IPPM= Type-P metrics) if not recommended metric here. This section can be substa= ntially expanded.
=A0

[MS] I am no= t sure if this document should really formally define metrics (the current = wording implies that). The more important aspect seems to me=A0the overall = effects that an operator may want to look at when deploying ALTO. This significant= ly differs between different use cases of ALTO. For=A0instance, if ALTO is = used for traffic=A0optimization inside one=A0AS,=A0monitoring a metric that= describes inter-domain traffic may=A0not=A0be required.=A0Therefore, the relevant metrics will depend on the use case, a= s also noted in your review. An alternative document structure would have s= ome discussion of monitoring per use case, but this would imply=A0a signifi= cant change of the table of content of the document.=A0We currently look into this, but this requires feedback from the= ALTO WG, too.

Yeah a couple of use-cases would really help though it is bit more wor= k. It would make the document more complete and usable for implementers.
=A0

Comments on section 6.3.1

There are several metrics defined here but they can be more clearly de= fined by following the formats specified in RFC 6390. Also useful will some= use-cases to explain why these metrics make sense and what needs to be use= d in which scenario.
=A0

[MS] Agreed.= As I said, one potential solution is to move this discussion to the descri= ptions of the use cases (Section 4, 5, etc.), maybe with one "monitori= ng" subsection per use-case.

Yup. Agree. Makes sense.

=A0

As regards to the network hop-count to be used indirectly for latency,= it might be better to define a different metric for latency as the average= network loads and buffers on each device can be radically different. For e= xample a hybrid network that contains=A0 a backbone network, last mile access network and a mobile network (which i= s not unusual in many ISPs today). The delay for transit (as well as packet= loss) internally from one to other might be drastically different.
=A0

[MS] I agree= that hop-count and latency may not correlate. Also, measuring hop-count ca= n be difficult in some cases. If the Internet traffic is carried over an MP= LS/IP aggregation/core network, the number of routers that are traversed by the = traffic can be pretty different to the number of IP hops. Reducing the IP h= ops does not necessarily imply a more efficient routing. My current thinkin= g is that the document should more focus on "traffic locality" than path monitoring metrics.=

= Agree. Makes sense. Can you reword the mentioned paragraph ?

Section 6.3.3

The example given here can be expanded to give a more clearer picture = of which are the active monitoring agents and which are the passive compone= nts. The metrics used for monitoring need to standardized. Also you might w= ant to add some recommendation on the data interchange format and method of delivery. This is especially imp= ortant as the interacting entities (ISP and Application Owner) are present = at different layers and have different priorities.This can also be fixed in= section 6.3.1 above.
=A0

[MS] Yes, th= e text and Figure 19 describes the scenario at a very high-level and, indee= d, it=A0is not clear. Both the question what somebody can measure, and whet= her this is done actively or passively, depends on the use case and who that entity= actually is. But I don't believe that this=A0document can=A0provide co= mprehensive guidance on data=A0exchange formats and methods of delivery, be= cause inter-organization communication raises a lot of non-technical issues, and even standardization of new protocols m= ight be needed. That seems outside the scope of this document specifically,= and ALTO in general.=A0This seems more related to LMAP.

The document need= not be comprehensive. That was not the intention of the suggestion. But no= n-normative language about possible measures / metrics could be useful for = guidance. Data interchange formats such as JSON could be used for easy pars= ing instead of binary serialisation.

-- Vinayak

--047d7b2e0c315325bc04e709b540-- From bclaise@cisco.com Mon Sep 23 03:30:59 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 46FD521F9A6C; Mon, 23 Sep 2013 03:30:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.542 X-Spam-Level: X-Spam-Status: No, score=-10.542 tagged_above=-999 required=5 tests=[AWL=0.057, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b9Z0sLpTvu-u; Mon, 23 Sep 2013 03:30:51 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 4313121F9A96; Mon, 23 Sep 2013 03:30:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6885; q=dns/txt; s=iport; t=1379932215; x=1381141815; h=message-id:date:from:mime-version:to:subject:references: in-reply-to:content-transfer-encoding; bh=bmrm/iYJhD9BHAUjzVfJ+vSgfAHL2OLRSVX/EFUD/cs=; b=PjD4iAtQkPRFySlKH0avUMLB2qRRCbTnypmXIDuDc8iPGP+4SWrnOYEw HuL/34wjwiqlg7YREqPhX60Iyngx8aL4W7YLZDXGGXOc5TM9ZRyIbUkbh Z346brITWyWLrJYPTaQoJUgzhe6FhjpqmGlQkc7S+2bXKGgvaPFl1ghwy U=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgQFAEUXQFKQ/khR/2dsb2JhbABZgwc4g3u+CIEcFnSCJgEBBCMVMBARHQgCBRYLAgIJAwIBAgEPKAQKBgEJAwYCAQGHbwMPDKkRiBoNiWYEgSmLToElgVCCaYE1A5QgBYFugWmGMoYShTOBZoFAOoE1 X-IronPort-AV: E=Sophos;i="4.90,961,1371081600"; d="scan'208";a="86863112" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-2.cisco.com with ESMTP; 23 Sep 2013 10:30:12 +0000 Received: from [10.60.67.87] (ams-bclaise-8916.cisco.com [10.60.67.87]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id r8NAUAxq002575; Mon, 23 Sep 2013 10:30:10 GMT Message-ID: <52401832.9010904@cisco.com> Date: Mon, 23 Sep 2013 12:30:10 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: "aaa-doctors@ietf.org" , "MIB Doctors (E-mail)" , YANG Doctors , "ops-dir@ietf.org" , "pm-dir@ietf.org" , IETF DNS Directorate References: <20130919215216.8575.29762.idtracker@ietfa.amsl.com> In-Reply-To: <20130919215216.8575.29762.idtracker@ietfa.amsl.com> X-Forwarded-Message-Id: <20130919215216.8575.29762.idtracker@ietfa.amsl.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: [pm-dir] Fwd: PRELIMINARY Agenda and Package for the September 26, 2013 IESG Teleconference X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 10:30:59 -0000 Dear all, Please find below the agenda of the Sept 26th IESG telechat. Please send your questions, comments and concerns before Sept 25th COB. Thanks and Regards, Benoit. AGENDA PACKAGE FOR 2013-09-26 IESG TELECHAT ------------------------------------------------------------------------ 2. AGENDA ------------------------------------------------------------------------ INTERNET ENGINEERING STEERING GROUP (IESG) Summarized Agenda for the 2013-09-26 IESG Teleconference 2. Protocol Actions 2.1 WG Submissions 2.1.1 New Items o draft-ietf-mpls-return-path-specified-lsp-ping-13 - IETF stream Return Path Specified LSP Ping (Proposed Standard) Note: Loa Andersson (loa@pi.nu) is the document shepherd. Token: Adrian Farrel IANA Review: IANA OK - Actions Needed Consensus: Yes o draft-ietf-dime-realm-based-redirect-12 - IETF stream Realm-Based Redirection In Diameter (Proposed Standard) Token: Benoit Claise IANA Review: IANA OK - Actions Needed Consensus: Yes o draft-ietf-ccamp-gmpls-signaling-g709v3-12 - IETF stream Generalized Multi-Protocol Label Switching (GMPLS) Signaling Extensions for the evolving G.709 Optical Transport Networks Control (Proposed Standard) Token: Adrian Farrel IANA Review: IANA - Not OK Consensus: Yes o draft-ietf-netext-update-notifications-08 - IETF stream Update Notifications for Proxy Mobile IPv6 (Proposed Standard) Token: Brian Haberman IANA Review: IANA OK - Actions Needed Consensus: Unknown o draft-ietf-dhc-dhcpv6-solmaxrt-update-05 - IETF stream Modification to Default Values of SOL_MAX_RT and INF_MAX_RT (Proposed Standard) Token: Ted Lemon IANA Review: IANA OK - Actions Needed Consensus: Yes o draft-ietf-ccamp-swcaps-update-03 - IETF stream Revised Definition of The GMPLS Switching Capability and Type Fields (Proposed Standard) Token: Adrian Farrel IANA Review: IANA OK - Actions Needed Consensus: Yes 2.1.2 Returning Items o draft-ietf-spfbis-4408bis-20 - IETF stream Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1 (Proposed Standard) Note: You may wish to view the summary on the IETF list: https://www.ietf.org/ibin/c5i?mid=6&rid=49&gid=0&k1=933&k2=72183&tid=1378561974 Token: Pete Resnick IANA Review: Version Changed - Review Needed Consensus: Unknown 2.2 Individual Submissions 2.2.1 New Items o draft-nandakumar-rtcweb-stun-uri-07 - IETF stream URI Scheme for Session Traversal Utilities for NAT (STUN) Protocol (Proposed Standard) Token: Gonzalo Camarillo IANA Review: IANA OK - Actions Needed Consensus: Yes o draft-petithuguenin-behave-turn-uris-07 - IETF stream Traversal Using Relays around NAT (TURN) Uniform Resource Identifiers (Proposed Standard) Token: Gonzalo Camarillo IANA Review: Version Changed - Review Needed Consensus: Yes o draft-boucadair-rfc6153-update-01 - IETF stream Updates to DHCPv4 and DHCPv6 Options for Access Network Discovery and Selection Function (ANDSF) Discovery (Proposed Standard) Note: This document corrects a clerical error in the assignment of DHCP option codes for rfc6153. I don't think it should be at all controversial, since the problem is obvious and the correction straightforward. Token: Ted Lemon IANA Review: IANA OK - Actions Needed Consensus: Yes 2.2.2 Returning Items NONE 2.3 Status Changes 2.3.1 New Items NONE 2.3.2 Returning Items NONE 3. Document Actions 3.1 WG Submissions 3.1.1 New Items o draft-ietf-dime-overload-reqs-12 - IETF stream Diameter Overload Control Requirements (Informational) Note: Jouni Korhonen (jouni.nospam@gmail.com) is the document shepherd. Token: Benoit Claise IANA Review: Version Changed - Review Needed Consensus: Yes 3.1.2 Returning Items o draft-ietf-homenet-arch-10 - IETF stream Home Networking Architecture for IPv6 (Informational) Token: Ted Lemon IANA Review: IANA OK - No Actions Needed Consensus: Yes Was deferred by Pete Resnick on 2013-09-11 3.2 Individual Submissions Via AD 3.2.1 New Items o draft-ivov-grouptextchat-purpose-03 - IETF stream A Group Text Chat Purpose for Conference and Service URIs in the Session Initiation Protocol (SIP) Event Package for Conference State (Informational) Token: Gonzalo Camarillo IANA Review: IANA OK - Actions Needed Consensus: Unknown 3.2.2 Returning Items NONE 3.3 Status Changes 3.3.1 New Items NONE 3.3.2 Returning Items NONE 3.4 IRTF and Independent Submission Stream Documents 3.4.1 New Items o conflict-review-joseph-pkix-p6rsshextension-00 IETF conflict review for draft-joseph-pkix-p6rsshextension draft-joseph-pkix-p6rsshextension-03 P6R's Secure Shell Public Key Subsystem (ISE: Informational) Token: Sean Turner o conflict-review-morand-http-digest-2g-aka-00 IETF conflict review for draft-morand-http-digest-2g-aka draft-morand-http-digest-2g-aka-03 Hypertext Transfer Protocol (HTTP) Digest Authentication Using GSM 2G Authentication and Key Agreement (AKA) (ISE: Informational) Token: Sean Turner 3.4.2 Returning Items o conflict-review-alimi-decade-00 IETF conflict review for draft-alimi-decade draft-alimi-decade-04 DECoupled Application Data Enroute (DECADE) (ISE: Informational) Token: Martin Stiemerling 3.4.3 For Action o conflict-review-vandesompel-memento-00 IETF conflict review for draft-vandesompel-memento draft-vandesompel-memento-09 HTTP framework for time-based access to resource states -- Memento (ISE: Informational) Token: Jari Arkko o conflict-review-hausenblas-csv-fragment-00 IETF conflict review for draft-hausenblas-csv-fragment draft-hausenblas-csv-fragment-05 URI Fragment Identifiers for the text/csv Media Type (ISE: Informational) Token: Jari Arkko 4. Working Group Actions 4.1 WG Creation 4.1.1 Proposed for IETF Review o Extensions for Scalable DNS Service Discovery (dnssdext) 4.1.2 Proposed for Approval o DTLS In Constrained Environments (dice) o Active Queue Management and Packet Scheduling (aqm) o IPv6 over the TSCH mode of IEEE 802.15.4e (6tisch) 4.2 WG Rechartering 4.2.1 Under Evaluation for IETF Review o INtermediary-safe SIP session ID (insipid) o IPv6 Maintenance (6man) 4.2.2 Proposed for Approval NONE From bclaise@cisco.com Sun Sep 29 14:06:50 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8E0911E8140 for ; Sun, 29 Sep 2013 14:06:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.546 X-Spam-Level: X-Spam-Status: No, score=-10.546 tagged_above=-999 required=5 tests=[AWL=0.053, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DxOOL3k9pPLX for ; Sun, 29 Sep 2013 14:06:44 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 7865121F9477 for ; Sun, 29 Sep 2013 14:06:43 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r8TL6eJ5029002 for ; Sun, 29 Sep 2013 23:06:42 +0200 (CEST) Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r8TL6BLY003711 for ; Sun, 29 Sep 2013 23:06:21 +0200 (CEST) Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r8TL681e010606 for pm-dir@ietf.org; Sun, 29 Sep 2013 23:06:08 +0200 (CEST) Date: Sun, 29 Sep 2013 23:06:08 +0200 From: Benoit Claise To: pm-dir@ietf.org Message-ID: <20130929210608.GA10604@sweet-brew-5.cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Sep 2013 21:06:51 -0000 Dear all, This is an automatically generated email. It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference. It also lists all the IETF internet-drafts that contain "performance metric". Regards, Benoit =========================================================== Normative References -------------------- draft-ietf-ippm-testplan-rfc2680-03 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state Informative References ---------------------- draft-ietf-ippm-testplan-rfc2680-03 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-loss-conceal-08 Active draft-ietf-xrblock-rtcp-xr-qoe-12 Active draft-ietf-xrblock-rtcp-xr-synchronization-06 Active drafts containing performance metric ------------------------------------ draft-ietf-alto-deployments-07 Active draft-ietf-cdni-footprint-capabilities-semantics-00Active draft-ietf-ippm-2330-update-00 Active draft-ietf-ippm-lmap-path-01 Active draft-ietf-ippm-model-based-metrics-00 Active draft-ietf-ippm-rate-problem-04 Active draft-ietf-ippm-testplan-rfc2680-03 Active draft-ietf-manet-smf-mib-08 In IESG processing - ID Tracker state draft-ietf-nvo3-framework-03 Active draft-ietf-opsawg-oam-overview-09 Active draft-ietf-ppsp-base-tracker-protocol-01 Active draft-ietf-ppsp-peer-protocol-07 Active draft-ietf-rtcweb-rtp-usage-09 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-loss-conceal-08 Active draft-ietf-xrblock-rtcp-xr-qoe-12 Active draft-ietf-xrblock-rtcp-xr-synchronization-06 Active