From nagami@wide.ad.jp Mon Jul 1 06:09:53 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C2AB411E8484 for ; Mon, 1 Jul 2013 06:09:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.378 X-Spam-Level: X-Spam-Status: No, score=-1.378 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, J_CHICKENPOX_82=0.6, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qRi9RAQpu4qb for ; Mon, 1 Jul 2013 06:09:53 -0700 (PDT) Received: from sh.wide.ad.jp (sh.wide.ad.jp [IPv6:2001:200:0:1001::6]) by ietfa.amsl.com (Postfix) with ESMTP id BE36C11E8853 for ; Mon, 1 Jul 2013 05:59:42 -0700 (PDT) Received: from mail-pb0-x22f.google.com (mail-pb0-x22f.google.com [IPv6:2607:f8b0:400e:c01::22f]) (authenticated (0 bits)) by mail.wide.ad.jp (8.14.1+3.5Wbeta/8.14.1/smtpfeed 1.21) with ESMTP id r61CxeLg003410 (using TLSv1/SSLv3 with cipher RC4-SHA (128 bits) verified FAIL) for ; Mon, 1 Jul 2013 21:59:41 +0900 (JST) Received: by mail-pb0-f47.google.com with SMTP id rr13so4749735pbb.20 for ; Mon, 01 Jul 2013 05:59:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9Ox0HavnmSrE0Xzv+mX6gZ1qlzhRfkntjngZV5ISa2o=; b=XpubFaZ21ZE4JRo3XfOPQV0lZbQlq+E3KI8z1p6dI6nLQXNNhHTiJv1bVfiQOnKAfX WpUDkbDA0QzXkSnPzx/c1o2bux4Xoe/Qi8S4zV936fAfwAAMH70V67A8lpoFbibv9VN1 ZRchS0at3UJPvEXk0QoUaSYvWFmbwr5B57ZfDpV17piPwP3Na6/DWUlhFSz3GH1CDbJU LYnRvcdhqIq5cKsF8sSFXWUkkhKpRoeGlvMj53GJ1qRa/rFC48lQMcX5MflgKF1yH3xC siG4hKm1xXu1YoOAhqCny/hTd8NKsdlkqVy0KP9yI+PWGO9e9dXolj72kS4ZZr9K4+49 rOFg== MIME-Version: 1.0 X-Received: by 10.66.232.196 with SMTP id tq4mr24162918pac.167.1372683574949; Mon, 01 Jul 2013 05:59:34 -0700 (PDT) Received: by 10.70.102.176 with HTTP; Mon, 1 Jul 2013 05:59:34 -0700 (PDT) In-Reply-To: References: Date: Mon, 1 Jul 2013 21:59:34 +0900 Message-ID: From: Nagami Kenichi To: "MORTON JR., ALFRED C (AL)" Content-Type: text/plain; charset=ISO-8859-1 Cc: lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2013 13:09:53 -0000 Dear Al and Dan, We would like to contribute some use case for LMAP. We have measured the broadband performance in Japan using measurement software. We have distributed 500,000 applications. Regards, Ken Nagami 2013/6/22 MORTON JR., ALFRED C (AL) : > LMAP, > > Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87, > either as a Working Group if approved in time, or as a BoF if not. > If charter-related issues remain to be resolved, they will be > discussed during the session with highest priority. > > However, ... > > If you have new/revised contributions covered under the current > draft of the LMAP charter, please reply to the list identifying the topic, > the filename(if applicable) and the related issues you would like to discuss at > a face-to-face session, and try to do this by June 28, 2013. > > thanks and regards, > Dan and Al > > > >> -----Original Message----- >> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- >> bounces@ietf.org] On Behalf Of The IESG >> Sent: Friday, June 14, 2013 11:42 AM >> To: IETF-Announce >> Cc: lmap WG >> Subject: WG Review: Large-Scale Measurement of Broadband Performance >> (lmap) >> >> A new IETF working group has been proposed in the Operations and >> Management Area. The IESG has not made any determination yet. The >> following draft charter was submitted, and is provided for informational >> purposes only. Please send your comments to the IESG mailing list (iesg >> at ietf.org) by 2013-06-24. >> >> Large-Scale Measurement of Broadband Performance (lmap) >> ------------------------------------------------ >> Current Status: Proposed WG >> >> Assigned Area Director: >> Benoit Claise >>... > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > From Michael.K.Bugenhagen@centurylink.com Mon Jul 1 06:27:40 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 537A621F9F77 for ; Mon, 1 Jul 2013 06:27:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_82=0.6] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E50RjjYIma9I for ; Mon, 1 Jul 2013 06:27:35 -0700 (PDT) Received: from sudnp799.qwest.com (sudnp799.qwest.com [155.70.32.99]) by ietfa.amsl.com (Postfix) with ESMTP id F351521F98AC for ; Mon, 1 Jul 2013 06:27:34 -0700 (PDT) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by sudnp799.qwest.com (8.14.4/8.14.4) with ESMTP id r61DRVU3003737 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 1 Jul 2013 07:27:31 -0600 (MDT) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 0E9501E004F; Mon, 1 Jul 2013 08:27:26 -0500 (CDT) Received: from sudnp797.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id D7E211E004D; Mon, 1 Jul 2013 08:27:25 -0500 (CDT) Received: from sudnp797.qintra.com (localhost [127.0.0.1]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r61DRP8W001512; Mon, 1 Jul 2013 07:27:25 -0600 (MDT) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.qintra.com [151.117.206.28]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r61DRPsZ001504 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 1 Jul 2013 07:27:25 -0600 (MDT) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.02.0318.001; Mon, 1 Jul 2013 08:27:24 -0500 From: "Bugenhagen, Michael K" To: "'Nagami Kenichi'" , "MORTON JR., ALFRED C (AL)" Thread-Topic: [lmap] Call for contributions and Agenda time in Berlin Thread-Index: Ac5ukbwsnWWTUZX3TBq4VwGNgR/klAH8wIUAAAmuyiA= Date: Mon, 1 Jul 2013 13:27:24 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.8] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2013 13:27:40 -0000 Al, Dan, I have a conflict for the July Meeting so I don't think I'll make this = specific meeting.=20 I'm still working on the draft I was going to submit for this meeting, but = there are a few things to work out so it looks like I'll target the Nov. me= eting for this work. Thanks, Mike =20 -----Original Message----- From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Nag= ami Kenichi Sent: Monday, July 01, 2013 8:00 AM To: MORTON JR., ALFRED C (AL) Cc: lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin Dear Al and Dan, We would like to contribute some use case for LMAP. We have measured the broadband performance in Japan using measurement softw= are. We have distributed 500,000 applications. Regards, Ken Nagami 2013/6/22 MORTON JR., ALFRED C (AL) : > LMAP, > > Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87, either as=20 > a Working Group if approved in time, or as a BoF if not. > If charter-related issues remain to be resolved, they will be=20 > discussed during the session with highest priority. > > However, ... > > If you have new/revised contributions covered under the current draft=20 > of the LMAP charter, please reply to the list identifying the topic,=20 > the filename(if applicable) and the related issues you would like to=20 > discuss at a face-to-face session, and try to do this by June 28, 2013. > > thanks and regards, > Dan and Al > > > >> -----Original Message----- >> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce-=20 >> bounces@ietf.org] On Behalf Of The IESG >> Sent: Friday, June 14, 2013 11:42 AM >> To: IETF-Announce >> Cc: lmap WG >> Subject: WG Review: Large-Scale Measurement of Broadband Performance >> (lmap) >> >> A new IETF working group has been proposed in the Operations and=20 >> Management Area. The IESG has not made any determination yet. The=20 >> following draft charter was submitted, and is provided for=20 >> informational purposes only. Please send your comments to the IESG=20 >> mailing list (iesg at ietf.org) by 2013-06-24. >> >> Large-Scale Measurement of Broadband Performance (lmap) >> ------------------------------------------------ >> Current Status: Proposed WG >> >> Assigned Area Director: >> Benoit Claise >>... > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap From jbustos@niclabs.cl Mon Jul 1 06:32:19 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3241C11E8125 for ; Mon, 1 Jul 2013 06:32:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.376 X-Spam-Level: X-Spam-Status: No, score=-2.376 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nXsMijYwtYlA for ; Mon, 1 Jul 2013 06:32:15 -0700 (PDT) Received: from mail-vb0-f44.google.com (mail-vb0-f44.google.com [209.85.212.44]) by ietfa.amsl.com (Postfix) with ESMTP id CC6DF11E8109 for ; Mon, 1 Jul 2013 06:32:14 -0700 (PDT) Received: by mail-vb0-f44.google.com with SMTP id e15so3585830vbg.31 for ; Mon, 01 Jul 2013 06:32:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=2IZrkjCwT6hN4YCfVU/tg/qrEdhEtC4ok4rFore6oQc=; b=jdd8scgye5jhmZuytRCOHQ88WmA1kBdGsdfWd7+Hp3a1qDnkpFDNz61/TTvC7xIsSc qQW7cEbebPGAxTjIdRWQuzEShbMvce0Ee76yu3KIeTat56douNZ4Ja+cMBryGrYGkpIZ l1l/YM3dffkn1vXnvmPP5+K2YbaFLaHTq8yDyxPOOL87bj1vvvQ5q3K71USG0lDzz+az 6wXKuE8Qy5vgpCm9bZY+2WZU4P9+pn1u0uC+3LDV9StUXuChiD04g/D1q8UAQINWLV5W oRAQuMhGZar1HsLNaIoatHLYBDo7kC8oDvvCbNdrziCXnEVRi5rh4I2FyHVYkW1mt/tL nxAQ== MIME-Version: 1.0 X-Received: by 10.52.90.50 with SMTP id bt18mr8108619vdb.35.1372685534155; Mon, 01 Jul 2013 06:32:14 -0700 (PDT) Received: by 10.220.37.79 with HTTP; Mon, 1 Jul 2013 06:32:14 -0700 (PDT) In-Reply-To: References: Date: Mon, 1 Jul 2013 09:32:14 -0400 Message-ID: From: Javier Bustos To: Nagami Kenichi Content-Type: multipart/alternative; boundary=bcaec50162afa6e8b604e0734139 X-Gm-Message-State: ALoCoQml8PBNfZ6PNwHL8RiPqZC1D+z491gn/2NfzDwUCnNhIZpATvMFRF1+FxT4Vd5Cr3qK/rmS Cc: "MORTON JR., ALFRED C \(AL\)" , lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2013 13:32:19 -0000 --bcaec50162afa6e8b604e0734139 Content-Type: text/plain; charset=ISO-8859-1 Dear Al and Dan We would like to contribute for LMAP too. Here in Chile we have performed active measurements for wired internet (around 10,000 probes) and passive measurements for mobile internet (around 750 probes). Regards, Javier Bustos. 2013/7/1 Nagami Kenichi > Dear Al and Dan, > > We would like to contribute some use case for LMAP. > We have measured the broadband performance in Japan using measurement > software. We have distributed 500,000 applications. > > Regards, > Ken Nagami > > 2013/6/22 MORTON JR., ALFRED C (AL) : > > LMAP, > > > > Last week, the draft LMAP charter went forward for external review. > > LMAP has been approved for a placeholder session at IETF-87, > > either as a Working Group if approved in time, or as a BoF if not. > > If charter-related issues remain to be resolved, they will be > > discussed during the session with highest priority. > > > > However, ... > > > > If you have new/revised contributions covered under the current > > draft of the LMAP charter, please reply to the list identifying the > topic, > > the filename(if applicable) and the related issues you would like to > discuss at > > a face-to-face session, and try to do this by June 28, 2013. > > > > thanks and regards, > > Dan and Al > > > > > > > >> -----Original Message----- > >> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- > >> bounces@ietf.org] On Behalf Of The IESG > >> Sent: Friday, June 14, 2013 11:42 AM > >> To: IETF-Announce > >> Cc: lmap WG > >> Subject: WG Review: Large-Scale Measurement of Broadband Performance > >> (lmap) > >> > >> A new IETF working group has been proposed in the Operations and > >> Management Area. The IESG has not made any determination yet. The > >> following draft charter was submitted, and is provided for informational > >> purposes only. Please send your comments to the IESG mailing list (iesg > >> at ietf.org) by 2013-06-24. > >> > >> Large-Scale Measurement of Broadband Performance (lmap) > >> ------------------------------------------------ > >> Current Status: Proposed WG > >> > >> Assigned Area Director: > >> Benoit Claise > >>... > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > --bcaec50162afa6e8b604e0734139 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Dear Al and Dan

We would like to contri= bute for LMAP too.=A0

Here in Chile we have perfor= med active measurements for wired internet (around 10,000 probes) and passi= ve measurements for mobile internet (around 750 probes).

Regards,
Javier Bustos.


2013/7/1 Na= gami Kenichi <nagami@wide.ad.jp>
Dear Al and Dan,

We would like to contribute some use case for LMAP.
We have measured the broadband performance in Japan using measurement
software. We have distributed 500,000 applications.

Regards,
Ken Nagami

2013/6/22 MORTON JR., ALFRED C (AL) <acmorton@att.com>:
> LMAP,
>
> Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87,
> either as a Working Group if approved in time, or as a BoF if not.
> If charter-related issues remain to be resolved, they will be
> discussed during the session with highest priority.
>
> However, ...
>
> If you have new/revised contributions covered under the current
> draft of the LMAP charter, please reply to the list identifying the to= pic,
> the filename(if applicable) and the related issues you would like to d= iscuss at
> a face-to-face session, and try to do this by June 28, 2013.
>
> thanks and regards,
> Dan and Al
>
>
>
>> -----Original Message-----
>> From: ietf-annou= nce-bounces@ietf.org [mailto:ietf-ann= ounce-
>> bounces@ietf.org] On Behal= f Of The IESG
>> Sent: Friday, June 14, 2013 11:42 AM
>> To: IETF-Announce
>> Cc: lmap WG
>> Subject: WG Review: Large-Scale Measurement of Broadband Performan= ce
>> (lmap)
>>
>> A new IETF working group has been proposed in the Operations and >> Management Area. The IESG has not made any determination yet. The<= br> >> following draft charter was submitted, and is provided for informa= tional
>> purposes only. Please send your comments to the IESG mailing list = (iesg
>> at ietf.org) by = 2013-06-24.
>>
>> Large-Scale Measurement of Broadband Performance (lmap)
>> ------------------------------------------------
>> Current Status: Proposed WG
>>
>> Assigned Area Director:
>> =A0 Benoit Claise <bclaise= @cisco.com>
>>...
> _______________________________________________
> lmap mailing list
> lmap@ietf.org
> https://www.ietf.org/mailman/listinfo/lmap
>
_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap

--bcaec50162afa6e8b604e0734139-- From bclaise@cisco.com Wed Jul 3 18:19:11 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 18CDB11E8113 for ; Wed, 3 Jul 2013 18:19:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.221 X-Spam-Level: X-Spam-Status: No, score=-10.221 tagged_above=-999 required=5 tests=[AWL=0.378, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bOwK3gw5px2r for ; Wed, 3 Jul 2013 18:19:07 -0700 (PDT) Received: from av-tac-rtp.cisco.com (av-tac-rtp.cisco.com [64.102.19.209]) by ietfa.amsl.com (Postfix) with ESMTP id CEFF911E80E4 for ; Wed, 3 Jul 2013 18:19:06 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from fire.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-rtp.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r641J2Z7013986 for ; Wed, 3 Jul 2013 21:19:02 -0400 (EDT) Received: from [10.154.208.115] ([10.154.208.115]) by fire.cisco.com (8.14.5+Sun/8.13.8) with ESMTP id r641J1vH010038 for ; Wed, 3 Jul 2013 18:19:01 -0700 (PDT) Message-ID: <51D4CD85.80000@cisco.com> Date: Wed, 03 Jul 2013 18:19:01 -0700 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; WOW64; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: lmap WG References: <20130628164927.29882.71687.idtracker@ietfa.amsl.com> <51CDDE6E.7050309@cisco.com> In-Reply-To: <51CDDE6E.7050309@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [lmap] WG Action: Formed Large-Scale Measurement of Broadband Performance (lmap) X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jul 2013 01:19:11 -0000 Dear all, Jason Weil has been selected as the LMAP co-chair, next to Dan Romascanu. Thanks Jason for stepping forward. Regards, Joel and Benoit > Dear all, > > This WG is now approved. Let the fun (and the work) begin. > > Many thanks to a couple of people. > First of all, to Henning (and others) for the introduction of this > important work to the IETF. > Then, to Al Morton and Dan Romascanu for running an excellent BoF. As > recognized by some IESG and IAB members: kudos on the BoF preparation, > the BoF session management, and helping in the WG creation > Finally, thanks again to Dan, who accepted to become a LMAP chair. > Note: Joel and I are currently working on the second chair selection. > > Regards, Benoit >> A new IETF working group has been formed in the Operations and >> Management >> Area. For additional information please contact the Area Directors or >> the >> WG Chair. >> >> Large-Scale Measurement of Broadband Performance (lmap) >> ------------------------------------------------ >> Current Status: Proposed WG >> >> Chair: >> Dan Romascanu >> >> Assigned Area Director: >> Benoit Claise >> >> Mailing list >> Address: lmap@ietf.org >> To Subscribe: http://www.ietf.org/mailman/listinfo/lmap >> Archive: http://www.ietf.org/mail-archive/web/lmap >> >> Charter: >> >> The Large-Scale Measurement of Broadband Performance (LMAP) working >> group >> standardizes the LMAP measurement system for performance measurements of >> broadband access devices such as home and enterprise edge routers, >> personal computers, mobile devices, set top box, whether wired or >> wireless. >> >> Measuring portions of the Internet on a large scale is essential for >> accurate characterizations of performance over time and geography, for >> network diagnostic investigations by providers and their users, and for >> collecting information to support public policy development. The goal is >> to have the measurements (made using the same metrics and mechanisms) >> for a large number of points on the Internet, and to have the results >> collected and stored in the same form. >> The LMAP working group is chartered to specify an information >> model, the >> associated data models, and select/extend one or more protocols for the >> secure communication: >> 1. A Control Protocol, from a Controller to instruct Measurement >> Agents >> what performance metrics to measure, when to measure them, how/when to >> report the measurement results to a Collector, >> 2. A Report Protocol, for a Measurement Agent to report the >> results to >> the Collector. >> The data models should be extensible for new and additional >> measurements. >> LMAP will consider re-use of existing data models languages. >> >> A key assumption constraining the initial work is that the measurement >> system is under the control of a single organization (for example, an >> Internet Service Provider or a regulator). However, the components of an >> LMAP measurement system can be deployed in administrative domains that >> are not owned by the measuring organization. Thus, the system of >> functions deployed by a single organization constitutes a single LMAP >> domain which may span ownership or other administrative boundaries. >> >> The LMAP architecture will allow for measurements that utilize either >> IPv4 or IPv6, or possibly both. Devices containing Measurement Agents >> may >> have several interfaces using different link technologies. Multiple >> address families and interfaces must be considered in the Control and >> Report protocols. >> >> It is assumed that different organization's LMAP measurement domains can >> overlap, and that active measurement packets appear along with normal >> user traffic when crossing another organization's network. There is no >> requirement to specify a mechanism for coordination between the LMAP >> measurements in overlapping domains (for instance a home network with >> MAs >> on the home gateway, set top box and laptop). In principle, there are no >> restrictions on the type of device in which the MA function resides. >> >> Both active and passive measurements are in scope, although there may be >> differences in their applicability to specific use cases, or in the >> security measures needed according to the threats specific to each >> measurement category. LMAP will not standardize performance metrics. >> >> The LMAP WG will consider privacy as a core requirement and will ensure >> that by default measurement and collection mechanisms and protocols >> standardized operate in a privacy-sensitive manner, for example, >> ensuring that measurements are not personally identifying except where >> permission for such has been granted by identified subjects. Note that >> this does not mean that all uses of LMAP need to turn on all privacy >> features but it does mean that privacy features do need to be at least >> well-defined. >> >> Standardizing control of end users Measurement Agents is out of scope. >> However, end users can obtain an MA to run measurement tasks if desired >> and report their results to whomever they want, most likely the supplier >> of the MA. This provides for user-initiated on-demand measurement, which >> is an important component of the ISP use case. >> >> Inter-organization communication of results is out of scope of the LMAP >> charter. >> >> The management protocol to bootstrap the MAs in measurement devices is >> out of scope of the LMAP charter. >> >> Service parameters, such as product category, can be useful to decide >> which measurements to run and how to interpret the results. These >> parameters are already gathered and stored by existing operations >> systems. >> Discovering the service parameters on the MAs or sharing the service >> parameters between MAs are out of the scope. However, if the service >> parameters are available to the MAs, they could be reported with the >> measurement results in the Report Protocol. >> >> Deciding the set of measurements to run is a business decision and is >> out >> of scope of the LMAP charter. >> >> Protection against the intentional or malicious insertion of >> inaccuracies >> into the overall system or measurement process (sometimes known as >> "gaming the system") is outside the scope of work. >> >> The LMAP working group will coordinate with other standards bodies >> working in this area (e.g., BBF, IEEE 802.16, ETSI) regarding the >> information model, and with other IETF working groups in the areas of >> data models, protocols, multiple interface management, and >> measurement of >> performance metrics. >> >> The LMAP WG will produce the following work items: >> >> 1. The LMAP Framework - provides common terminology, basic architecture >> elements, and justifies the simplifying constraints >> 2. The LMAP Use Cases - provides the motivating use cases as a basis for >> the work >> 3. Information Model, the abstract definition of the information carried >> from the Controller to the MA and the information carried from the MA to >> the Collector. It includes >> * The metric(s) that can be measured and values for its parameters >> such as the Peer MA participating in the measurement and the desired >> environmental conditions (for example, only conduct the measurement when >> there is no user traffic observed) >> * The schedule: when the measurement should be run and how the >> results >> should be reported (when and to which Collector) >> * The report: the metric(s) measured and when, the actual result, and >> supporting metadata such as location. Result reports may be organized in >> batches or may be reported immediately, such as for an on-demand >> measurement. >> 4. The Control protocol and the associated data model: The definition of >> how instructions are delivered from a Controller to a MA; this >> includes a >> Data Model consistent with the Information Model plus a transport >> protocol. This may be a simple instruction - response protocol, and >> LMAP >> will specify how it operates over an existing protocol (to be selected, >> perhaps REST-style HTTP(s) or NETCONF). >> 5. The Report protocol and the associated data model: The definition of >> how the Report is delivered from a MA to a Collector; this includes a >> Data Model consistent with the Information Model plus a transport >> protocol (to be selected, perhaps REST-style HTTP(s) or IPFIX). >> >> >> Milestones: >> Sep 2013 - Initial WG I-D for the LMAP Framework including >> terminology >> Sep 2013 - Initial WG I-D for the LMAP Use cases >> Dec 2013 - Submit the LMAP Framework I-D to the IESG for >> consideration >> as Informational RFC >> Dec 2013 - Submit I-D on the LMAP Use cases to the IESG for >> consideration as Informational RFC >> Jan 2014 - Initial WG I-D for LMAP Information models >> Apr 2014 - Initial WG I-D for the Control protocol >> Apr 2014 - Initial WG I-D for the Report protocol >> Apr 2014 - Initial WG I-D for a Data model for LMAP control >> information >> Apr 2014 - Initial WG I-D for a Data model for LMAP report >> information >> Jul 2014 - Submit the LMAP Information models I-D to the IESG for >> consideration as Standards track RFC >> Dec 2014 - Submit the Control protocol I-D to the IESG for >> consideration as Standards track RFC >> Dec 2014 - Submit the Report protocol I-D to the IESG for >> consideration >> as Standards track RFC >> Dec 2014 - Submit the Data model for LMAP control I-D to the IESG for >> consideration as Standards track RFC >> Dec 2014 - Submit the Data model for LMAP report I-D to the IESG for >> consideration as Standards track RFC >> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> >> > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > > From v.bajpai@jacobs-university.de Thu Jul 4 04:45:05 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0EC3A21F9ED8 for ; Thu, 4 Jul 2013 04:45:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.949 X-Spam-Level: X-Spam-Status: No, score=-2.949 tagged_above=-999 required=5 tests=[AWL=0.300, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DFaeS8Vb+0Tx for ; Thu, 4 Jul 2013 04:45:00 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id AA7F621F9E63 for ; Thu, 4 Jul 2013 04:45:00 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id BDADC20BE9 for ; Thu, 4 Jul 2013 13:44:59 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id ZlJKGyIRQGBQ for ; Thu, 4 Jul 2013 13:44:59 +0200 (CEST) Received: from exchange.jacobs-university.de (unknown [10.70.0.123]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by hermes.jacobs-university.de (Postfix) with ESMTPS id 9E15920BE6 for ; Thu, 4 Jul 2013 13:44:59 +0200 (CEST) Received: from SXCHMB01.jacobs.jacobs-university.de ([fe80::c1f:c30f:99ac:df0c]) by SHUBCAS02.jacobs.jacobs-university.de ([::1]) with mapi id 14.02.0342.003; Thu, 4 Jul 2013 13:44:55 +0200 From: "Bajpai, Vaibhav" To: "" Thread-Topic: A Tutorial on Large-Scale Measurement Platforms Thread-Index: AQHOeKvmFr1OvVG5NEK38ByYhP9GJg== Date: Thu, 4 Jul 2013 11:44:54 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.50.226.26] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] A Tutorial on Large-Scale Measurement Platforms X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jul 2013 11:45:05 -0000 Hello, We recently gave a tutorial on Large-Scale Measurement Platforms at=20 AIMS 2013 [1]. The tutorial is available at [2]. [1] http://www.aims-conference.org/2013/labs.html [2] http://cnds.eecs.jacobs-university.de/tutorials Abstract: --------- This tutorial provides an introduction into large-scale measurement platfor= ms. It begins by surveying active measurement studies that paved way for the deployment of such platforms. It then describes the internals of the RIPE Atlas and SamKnows platforms, their architecture and surveys the work published from insights discovered from the collected data. The first part concludes by introducing the current Large Scale Measurement of Access Netw= ork Performance (LMAP) standardisation efforts in the IETF. The second part of = the tutorial provides practical training on how to virtualise an OpenWrt-based Customer Premises Equipment (CPE) and bootstrap it to register as a Measurement Agent (MA) with a controller. The MA is then used to install a measurement test, schedule the reporting of the measurement test results an= d use a REST API to send and retrieve the measurement results back from the collector. The tutorial also provides training on how to use the RIPE Atlas REST API to create User Defined Measurements (UDMs), and retrieve the UDM measurement results. It concludes with a description of the UDM result data structures and a brief of how programming tools can be used to analyse such= data. Best Regards, Vaibhav Bajpai and Nikolay Melnikov About authors: Vaibhav Bajpai is a PhD student at Jacobs University Bremen, Germany. His research focuses on understanding the impact of network infrastructure chan= ges using Large-Scale Measurement Platforms. http://vaibhavbajpai.com Nikolay Melnikov is a PhD student at Jacobs University Bremen, Germany. His research focuses on identification and differentiation of Internet users based on their browsing patterns and behaviors. http://nmelnikov.com= From trevor.burbridge@bt.com Fri Jul 5 03:15:07 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A56A321F95EF for ; Fri, 5 Jul 2013 03:15:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[AWL=0.300, BAYES_00=-2.599, J_CHICKENPOX_21=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gKhkNAB8V-ae for ; Fri, 5 Jul 2013 03:15:03 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp62.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id 57D7311E827A for ; Fri, 5 Jul 2013 03:15:03 -0700 (PDT) Received: from EVMHT63-UKRD.domain1.systemhost.net (10.36.3.100) by RDW083A006ED62.smtp-e2.hygiene.service (10.187.98.11) with Microsoft SMTP Server (TLS) id 8.3.298.1; Fri, 5 Jul 2013 11:15:01 +0100 Received: from EMV64-UKRD.domain1.systemhost.net ([169.254.2.35]) by EVMHT63-UKRD.domain1.systemhost.net ([10.36.3.100]) with mapi; Fri, 5 Jul 2013 11:15:00 +0100 From: To: Date: Fri, 5 Jul 2013 11:14:59 +0100 Thread-Topic: Information Model draft Thread-Index: Ac55aIE+OwzoyewBT527P8/qrbzdgA== Message-ID: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] Information Model draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jul 2013 10:15:07 -0000 I have submitted a first version of an Information Model for LMAP that we h= ope to be able to present and discuss at the Berlin LMAP group: http://www.ietf.org/id/draft-burbridge-lmap-information-model-00.txt Trevor. Trevor Burbridge Network Infrastructure & Innovation | BT Innovate & Design Tel: 01473 645115 Fax: 01473 640929 This email contains BT information, which may be privileged or confidential= . It's meant only for the individual(s) or entity named above. If you're no= t the intended recipient, note that disclosing, copying, distributing or us= ing this information is prohibited. If you've received this email in error,= please let me know immediately on the email address above. Thank you. We monitor our email system, and may record your emails.=20 British Telecommunications plc Registered office: 81 Newgate Street London EC1A 7AJ Registered in England no: 1800000=20 From jason.weil@twcable.com Fri Jul 5 15:06:36 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4159421F9F93 for ; Fri, 5 Jul 2013 15:06:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.463 X-Spam-Level: X-Spam-Status: No, score=-1.463 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JBf8AtRd+igw for ; Fri, 5 Jul 2013 15:06:32 -0700 (PDT) Received: from cdpipgw01.twcable.com (cdpipgw01.twcable.com [165.237.59.22]) by ietfa.amsl.com (Postfix) with ESMTP id CCCE421F9F7F for ; Fri, 5 Jul 2013 15:06:31 -0700 (PDT) X-SENDER-IP: 10.136.163.10 X-SENDER-REPUTATION: None X-IronPort-AV: E=Sophos;i="4.87,1004,1363147200"; d="scan'208";a="103744957" Received: from unknown (HELO PRVPEXHUB01.corp.twcable.com) ([10.136.163.10]) by cdpipgw01.twcable.com with ESMTP/TLS/RC4-MD5; 05 Jul 2013 18:04:25 -0400 Received: from PRVPEXVS17.corp.twcable.com ([10.136.163.95]) by PRVPEXHUB01.corp.twcable.com ([10.136.163.10]) with mapi; Fri, 5 Jul 2013 18:06:30 -0400 From: "Weil, Jason" To: Benoit Claise , lmap WG Date: Fri, 5 Jul 2013 18:06:31 -0400 Thread-Topic: [lmap] WG Action: Formed Large-Scale Measurement of Broadband Performance (lmap) Thread-Index: Ac55y+bg/WovBgnuSD2nVp/9uhwZiQ== Message-ID: In-Reply-To: <51D4CD85.80000@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.4.130416 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] WG Action: Formed Large-Scale Measurement of Broadband Performance (lmap) X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jul 2013 22:06:36 -0000 Thank You Benoit for the opportunity to assist in this working group. I look forward to working with everyone interested in the creation of a standardized information and data model and a communication protocol for the broadband measurement platforms being deployed for Internet services. Jason On 7/3/13 9:19 PM, "Benoit Claise" wrote: >Dear all, > >Jason Weil has been selected as the LMAP co-chair, next to Dan Romascanu. >Thanks Jason for stepping forward. > >Regards, Joel and Benoit >> Dear all, >> >> This WG is now approved. Let the fun (and the work) begin. >> >> Many thanks to a couple of people. >> First of all, to Henning (and others) for the introduction of this >> important work to the IETF. >> Then, to Al Morton and Dan Romascanu for running an excellent BoF. As >> recognized by some IESG and IAB members: kudos on the BoF preparation, >> the BoF session management, and helping in the WG creation >> Finally, thanks again to Dan, who accepted to become a LMAP chair. >> Note: Joel and I are currently working on the second chair selection. >> >> Regards, Benoit >>> A new IETF working group has been formed in the Operations and >>> Management >>> Area. For additional information please contact the Area Directors or >>> the >>> WG Chair. >>> >>> Large-Scale Measurement of Broadband Performance (lmap) >>> ------------------------------------------------ >>> Current Status: Proposed WG >>> >>> Chair: >>> Dan Romascanu >>> >>> Assigned Area Director: >>> Benoit Claise >>> >>> Mailing list >>> Address: lmap@ietf.org >>> To Subscribe: http://www.ietf.org/mailman/listinfo/lmap >>> Archive: http://www.ietf.org/mail-archive/web/lmap >>> >>> Charter: >>> >>> The Large-Scale Measurement of Broadband Performance (LMAP) working >>> group >>> standardizes the LMAP measurement system for performance measurements >>>of >>> broadband access devices such as home and enterprise edge routers, >>> personal computers, mobile devices, set top box, whether wired or >>> wireless. >>> >>> Measuring portions of the Internet on a large scale is essential for >>> accurate characterizations of performance over time and geography, for >>> network diagnostic investigations by providers and their users, and for >>> collecting information to support public policy development. The goal >>>is >>> to have the measurements (made using the same metrics and mechanisms) >>> for a large number of points on the Internet, and to have the results >>> collected and stored in the same form. >>> The LMAP working group is chartered to specify an information >>> model, the >>> associated data models, and select/extend one or more protocols for the >>> secure communication: >>> 1. A Control Protocol, from a Controller to instruct Measurement >>> Agents >>> what performance metrics to measure, when to measure them, how/when to >>> report the measurement results to a Collector, >>> 2. A Report Protocol, for a Measurement Agent to report the >>> results to >>> the Collector. >>> The data models should be extensible for new and additional >>> measurements. >>> LMAP will consider re-use of existing data models languages. >>> >>> A key assumption constraining the initial work is that the measurement >>> system is under the control of a single organization (for example, an >>> Internet Service Provider or a regulator). However, the components of >>>an >>> LMAP measurement system can be deployed in administrative domains that >>> are not owned by the measuring organization. Thus, the system of >>> functions deployed by a single organization constitutes a single LMAP >>> domain which may span ownership or other administrative boundaries. >>> >>> The LMAP architecture will allow for measurements that utilize either >>> IPv4 or IPv6, or possibly both. Devices containing Measurement Agents >>> may >>> have several interfaces using different link technologies. Multiple >>> address families and interfaces must be considered in the Control and >>> Report protocols. >>> >>> It is assumed that different organization's LMAP measurement domains >>>can >>> overlap, and that active measurement packets appear along with normal >>> user traffic when crossing another organization's network. There is no >>> requirement to specify a mechanism for coordination between the LMAP >>> measurements in overlapping domains (for instance a home network with >>> MAs >>> on the home gateway, set top box and laptop). In principle, there are >>>no >>> restrictions on the type of device in which the MA function resides. >>> >>> Both active and passive measurements are in scope, although there may >>>be >>> differences in their applicability to specific use cases, or in the >>> security measures needed according to the threats specific to each >>> measurement category. LMAP will not standardize performance metrics. >>> >>> The LMAP WG will consider privacy as a core requirement and will ensure >>> that by default measurement and collection mechanisms and protocols >>> standardized operate in a privacy-sensitive manner, for example, >>> ensuring that measurements are not personally identifying except where >>> permission for such has been granted by identified subjects. Note that >>> this does not mean that all uses of LMAP need to turn on all privacy >>> features but it does mean that privacy features do need to be at least >>> well-defined. >>> >>> Standardizing control of end users Measurement Agents is out of scope. >>> However, end users can obtain an MA to run measurement tasks if desired >>> and report their results to whomever they want, most likely the >>>supplier >>> of the MA. This provides for user-initiated on-demand measurement, >>>which >>> is an important component of the ISP use case. >>> >>> Inter-organization communication of results is out of scope of the LMAP >>> charter. >>> >>> The management protocol to bootstrap the MAs in measurement devices is >>> out of scope of the LMAP charter. >>> >>> Service parameters, such as product category, can be useful to decide >>> which measurements to run and how to interpret the results. These >>> parameters are already gathered and stored by existing operations >>> systems. >>> Discovering the service parameters on the MAs or sharing the service >>> parameters between MAs are out of the scope. However, if the service >>> parameters are available to the MAs, they could be reported with the >>> measurement results in the Report Protocol. >>> >>> Deciding the set of measurements to run is a business decision and is >>> out >>> of scope of the LMAP charter. >>> >>> Protection against the intentional or malicious insertion of >>> inaccuracies >>> into the overall system or measurement process (sometimes known as >>> "gaming the system") is outside the scope of work. >>> >>> The LMAP working group will coordinate with other standards bodies >>> working in this area (e.g., BBF, IEEE 802.16, ETSI) regarding the >>> information model, and with other IETF working groups in the areas of >>> data models, protocols, multiple interface management, and >>> measurement of >>> performance metrics. >>> >>> The LMAP WG will produce the following work items: >>> >>> 1. The LMAP Framework - provides common terminology, basic architecture >>> elements, and justifies the simplifying constraints >>> 2. The LMAP Use Cases - provides the motivating use cases as a basis >>>for >>> the work >>> 3. Information Model, the abstract definition of the information >>>carried >>> from the Controller to the MA and the information carried from the MA >>>to >>> the Collector. It includes >>> * The metric(s) that can be measured and values for its parameters >>> such as the Peer MA participating in the measurement and the desired >>> environmental conditions (for example, only conduct the measurement >>>when >>> there is no user traffic observed) >>> * The schedule: when the measurement should be run and how the >>> results >>> should be reported (when and to which Collector) >>> * The report: the metric(s) measured and when, the actual result, >>>and >>> supporting metadata such as location. Result reports may be organized >>>in >>> batches or may be reported immediately, such as for an on-demand >>> measurement. >>> 4. The Control protocol and the associated data model: The definition >>>of >>> how instructions are delivered from a Controller to a MA; this >>> includes a >>> Data Model consistent with the Information Model plus a transport >>> protocol. This may be a simple instruction - response protocol, and >>> LMAP >>> will specify how it operates over an existing protocol (to be selected, >>> perhaps REST-style HTTP(s) or NETCONF). >>> 5. The Report protocol and the associated data model: The definition of >>> how the Report is delivered from a MA to a Collector; this includes a >>> Data Model consistent with the Information Model plus a transport >>> protocol (to be selected, perhaps REST-style HTTP(s) or IPFIX). >>> >>> >>> Milestones: >>> Sep 2013 - Initial WG I-D for the LMAP Framework including >>> terminology >>> Sep 2013 - Initial WG I-D for the LMAP Use cases >>> Dec 2013 - Submit the LMAP Framework I-D to the IESG for >>> consideration >>> as Informational RFC >>> Dec 2013 - Submit I-D on the LMAP Use cases to the IESG for >>> consideration as Informational RFC >>> Jan 2014 - Initial WG I-D for LMAP Information models >>> Apr 2014 - Initial WG I-D for the Control protocol >>> Apr 2014 - Initial WG I-D for the Report protocol >>> Apr 2014 - Initial WG I-D for a Data model for LMAP control >>> information >>> Apr 2014 - Initial WG I-D for a Data model for LMAP report >>> information >>> Jul 2014 - Submit the LMAP Information models I-D to the IESG for >>> consideration as Standards track RFC >>> Dec 2014 - Submit the Control protocol I-D to the IESG for >>> consideration as Standards track RFC >>> Dec 2014 - Submit the Report protocol I-D to the IESG for >>> consideration >>> as Standards track RFC >>> Dec 2014 - Submit the Data model for LMAP control I-D to the IESG >>>for >>> consideration as Standards track RFC >>> Dec 2014 - Submit the Data model for LMAP report I-D to the IESG for >>> consideration as Standards track RFC >>> >>> >>> _______________________________________________ >>> lmap mailing list >>> lmap@ietf.org >>> https://www.ietf.org/mailman/listinfo/lmap >>> >>> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> >> > >_______________________________________________ >lmap mailing list >lmap@ietf.org >https://www.ietf.org/mailman/listinfo/lmap This E-mail and any of its attachments may contain Time Warner Cable propri= etary information, which is privileged, confidential, or subject to copyrig= ht belonging to Time Warner Cable. This E-mail is intended solely for the u= se of the individual or entity to which it is addressed. If you are not the= intended recipient of this E-mail, you are hereby notified that any dissem= ination, distribution, copying, or action taken in relation to the contents= of and attachments to this E-mail is strictly prohibited and may be unlawf= ul. If you have received this E-mail in error, please notify the sender imm= ediately and permanently delete the original and any copy of this E-mail an= d any printout. From dromasca@avaya.com Tue Jul 9 04:43:27 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9E06921F9FED for ; Tue, 9 Jul 2013 04:43:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.298 X-Spam-Level: X-Spam-Status: No, score=-103.298 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mcDbaAPMspHi for ; Tue, 9 Jul 2013 04:43:22 -0700 (PDT) Received: from co300216-co-outbound.net.avaya.com (co300216-co-outbound.net.avaya.com [198.152.13.100]) by ietfa.amsl.com (Postfix) with ESMTP id 7902221F9F96 for ; Tue, 9 Jul 2013 04:43:09 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AmAFAAH/ZlHGmAcV/2dsb2JhbABQgkIjITa5FgGIMYEHFnSCHwEBAQECAQEBAQ8LCgZBCwUHBAIBCA0BAwQBAQsdBycLFAkIAgQBDQUIGodsBgELoDyMZ5AlF45lJgcEBgEGA4JXYQOYI4UGimiDC4Io X-IronPort-AV: E=Sophos;i="4.87,456,1363147200"; d="scan'208,217";a="18750748" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by co300216-co-outbound.net.avaya.com with ESMTP; 09 Jul 2013 07:43:07 -0400 Received: from unknown (HELO AZ-FFEXHC02.global.avaya.com) ([135.64.58.12]) by co300216-co-erhwest-out.avaya.com with ESMTP; 09 Jul 2013 07:41:28 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC02.global.avaya.com ([135.64.58.12]) with mapi id 14.02.0328.009; Tue, 9 Jul 2013 13:43:05 +0200 From: "Romascanu, Dan (Dan)" To: Javier Bustos , Nagami Kenichi Thread-Topic: [lmap] Call for contributions and Agenda time in Berlin Thread-Index: AQHOdlrVnWWTUZX3TBq4VwGNgR/klJlQFSMAgAwvtIA= Date: Tue, 9 Jul 2013 11:43:05 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA12862C42@AZ-FFEXMB04.global.avaya.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: multipart/alternative; boundary="_000_9904FB1B0159DA42B0B887B7FA8119CA12862C42AZFFEXMB04globa_" MIME-Version: 1.0 Cc: lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jul 2013 11:43:27 -0000 --_000_9904FB1B0159DA42B0B887B7FA8119CA12862C42AZFFEXMB04globa_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Javier and Nagami, Thank you for letting the lmap list know about your applications and for of= fering your help. It may be useful if you would talk with Marc Linsner and = Phil Eardley who have authored the use case document and see if your experi= ence can add to this. The current version in the tracker is https://datatra= cker.ietf.org/doc/draft-linsner-lmap-use-cases/ but there may be a revision= in preparation for the Internet Draft submission cut-off next Monday. Will you attend the Berlin meeting? Thanks and Regards, Dan From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Jav= ier Bustos Sent: Monday, July 01, 2013 4:32 PM To: Nagami Kenichi Cc: MORTON JR., ALFRED C (AL); lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin Dear Al and Dan We would like to contribute for LMAP too. Here in Chile we have performed active measurements for wired internet (aro= und 10,000 probes) and passive measurements for mobile internet (around 750= probes). Regards, Javier Bustos. 2013/7/1 Nagami Kenichi > Dear Al and Dan, We would like to contribute some use case for LMAP. We have measured the broadband performance in Japan using measurement software. We have distributed 500,000 applications. Regards, Ken Nagami 2013/6/22 MORTON JR., ALFRED C (AL) >: > LMAP, > > Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87, > either as a Working Group if approved in time, or as a BoF if not. > If charter-related issues remain to be resolved, they will be > discussed during the session with highest priority. > > However, ... > > If you have new/revised contributions covered under the current > draft of the LMAP charter, please reply to the list identifying the topic= , > the filename(if applicable) and the related issues you would like to disc= uss at > a face-to-face session, and try to do this by June 28, 2013. > > thanks and regards, > Dan and Al > > > >> -----Original Message----- >> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- >> bounces@ietf.org] On Behalf Of The IESG >> Sent: Friday, June 14, 2013 11:42 AM >> To: IETF-Announce >> Cc: lmap WG >> Subject: WG Review: Large-Scale Measurement of Broadband Performance >> (lmap) >> >> A new IETF working group has been proposed in the Operations and >> Management Area. The IESG has not made any determination yet. The >> following draft charter was submitted, and is provided for informational >> purposes only. Please send your comments to the IESG mailing list (iesg >> at ietf.org) by 2013-06-24. >> >> Large-Scale Measurement of Broadband Performance (lmap) >> ------------------------------------------------ >> Current Status: Proposed WG >> >> Assigned Area Director: >> Benoit Claise > >>... > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_9904FB1B0159DA42B0B887B7FA8119CA12862C42AZFFEXMB04globa_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Javier and Nagami,

 <= /p>

Thank you for letting the= lmap list know about your applications and for offering your help. It may = be useful if you would talk with Marc Linsner and Phil Eardley who have authored the use case document and see if your experience can add= to this. The current version in the tracker is = https://datatracker.ietf.org/doc/draft-linsner-lmap-use-cases/ but ther= e may be a revision in preparation for the Internet Draft submission cut-off next Monda= y.

 

Will you attend the Berlin m= eeting?

 

Thanks and Regards,

 

Dan

 <= /p>

 <= /p>

 <= /p>

From: lmap-bou= nces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Javier Bustos
Sent: Monday, July 01, 2013 4:32 PM
To: Nagami Kenichi
Cc: MORTON JR., ALFRED C (AL); lmap WG
Subject: Re: [lmap] Call for contributions and Agenda time in Berlin=

 

Dear Al and Dan

 

We would like to contribute for LMAP too. =

 

Here in Chile we have performed active measurements = for wired internet (around 10,000 probes) and passive measurements for mobi= le internet (around 750 probes).

 

Regards,

Javier Bustos.

 

2013/7/1 Nagami Kenichi <nagami@wide.ad.jp>

Dear Al and Dan,

We would like to contribute some use case for LMAP.
We have measured the broadband performance in Japan using measurement
software. We have distributed 500,000 applications.

Regards,
Ken Nagami

2013/6/22 MORTON JR., ALFRED C (AL) <acmorton@att.com>:

> LMAP,
>
> Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87,
> either as a Working Group if approved in time, or as a BoF if not.
> If charter-related issues remain to be resolved, they will be
> discussed during the session with highest priority.
>
> However, ...
>
> If you have new/revised contributions covered under the current
> draft of the LMAP charter, please reply to the list identifying the to= pic,
> the filename(if applicable) and the related issues you would like to d= iscuss at
> a face-to-face session, and try to do this by June 28, 2013.
>
> thanks and regards,
> Dan and Al
>
>
>
>> -----Original Message-----
>> From: ietf-annou= nce-bounces@ietf.org [mailto:ietf-ann= ounce-
>> bounces@ietf.org] On Behal= f Of The IESG
>> Sent: Friday, June 14, 2013 11:42 AM
>> To: IETF-Announce
>> Cc: lmap WG
>> Subject: WG Review: Large-Scale Measurement of Broadband Performan= ce
>> (lmap)
>>
>> A new IETF working group has been proposed in the Operations and >> Management Area. The IESG has not made any determination yet. The<= br> >> following draft charter was submitted, and is provided for informa= tional
>> purposes only. Please send your comments to the IESG mailing list = (iesg
>> at ietf.org) by = 2013-06-24.
>>
>> Large-Scale Measurement of Broadband Performance (lmap)
>> ------------------------------------------------
>> Current Status: Proposed WG
>>
>> Assigned Area Director:
>>   Benoit Claise <bcla= ise@cisco.com>
>>...
> _______________________________________________
> lmap mailing list
> lmap@ietf.org
> https://www.ietf.org/mailman/listinfo/lmap
>
_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap

 

--_000_9904FB1B0159DA42B0B887B7FA8119CA12862C42AZFFEXMB04globa_-- From mlinsner@cisco.com Tue Jul 9 07:54:22 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3B64D21F9B85 for ; Tue, 9 Jul 2013 07:54:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.999 X-Spam-Level: X-Spam-Status: No, score=-9.999 tagged_above=-999 required=5 tests=[AWL=-0.001, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Cgh7DyCblNVK for ; Tue, 9 Jul 2013 07:54:12 -0700 (PDT) Received: from rcdn-iport-4.cisco.com (rcdn-iport-4.cisco.com [173.37.86.75]) by ietfa.amsl.com (Postfix) with ESMTP id 1C60111E80EE for ; Tue, 9 Jul 2013 07:54:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=15283; q=dns/txt; s=iport; t=1373381652; x=1374591252; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=fVEUHe1lilOxOh+W9mQHLiAHJWobVdtUVwMAUOC6hLc=; b=UiZSdkW53FrZTz4s7b8FUI0rtgdvWbbLYbs+bskEma9GLAjP5efyHUwr 6T2Glu09xfXoVQ+cqKZeZz5/poTdoBjQcU+V/rSlyN7w0Z8wNvHUwnRWD 84Qcqb136nQuZiGx+tzk0YthwxKF5WoH/HGKoIki4pRL0M0vszoTJ7lOO 0=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AmIFADMj3FGtJV2c/2dsb2JhbABagkVEMk24Sog9gRgWdIIjAQEBAwEBAQEaCgZBCwUHBgEIDgMDAQEBCx0uCxQJCAEBBAENBQiIAQYMujSPOgYHEw0EBgEGA4MAaQOYfJAfgxGCKA X-IronPort-AV: E=Sophos;i="4.87,1028,1363132800"; d="scan'208,217";a="232683902" Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by rcdn-iport-4.cisco.com with ESMTP; 09 Jul 2013 14:54:11 +0000 Received: from xhc-aln-x14.cisco.com (xhc-aln-x14.cisco.com [173.36.12.88]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id r69EsBHx023098 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 9 Jul 2013 14:54:11 GMT Received: from xmb-rcd-x08.cisco.com ([169.254.8.87]) by xhc-aln-x14.cisco.com ([173.36.12.88]) with mapi id 14.02.0318.004; Tue, 9 Jul 2013 09:54:10 -0500 From: "Marc Linsner (mlinsner)" To: Javier Bustos , Nagami Kenichi Thread-Topic: [lmap] Call for contributions and Agenda time in Berlin Thread-Index: Ac5ukbwsnWWTUZX3TBq4VwGNgR/klAH8wIUAAAEkEAABjoTzgP//8lSA Date: Tue, 9 Jul 2013 14:54:10 +0000 Message-ID: <581E085DEB6A444093CDAEA4C4EE48181359116D@xmb-rcd-x08.cisco.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA12862C42@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.116.195.125] Content-Type: multipart/alternative; boundary="_000_581E085DEB6A444093CDAEA4C4EE48181359116Dxmbrcdx08ciscoc_" MIME-Version: 1.0 Cc: lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jul 2013 14:54:22 -0000 --_000_581E085DEB6A444093CDAEA4C4EE48181359116Dxmbrcdx08ciscoc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable All, We are working on an update to the Use Case draft. Feel free to send comme= nts to the list and/or authors. -Marc- From: , "Dan (Dan)" > Date: Tuesday, July 9, 2013 7:43 AM To: Javier Bustos >, Nagami K= enichi > Cc: lmap WG > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin Hi Javier and Nagami, Thank you for letting the lmap list know about your applications and for of= fering your help. It may be useful if you would talk with Marc Linsner and = Phil Eardley who have authored the use case document and see if your experi= ence can add to this. The current version in the tracker is https://datatra= cker.ietf.org/doc/draft-linsner-lmap-use-cases/ but there may be a revision= in preparation for the Internet Draft submission cut-off next Monday. Will you attend the Berlin meeting? Thanks and Regards, Dan From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of Javier Bustos Sent: Monday, July 01, 2013 4:32 PM To: Nagami Kenichi Cc: MORTON JR., ALFRED C (AL); lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin Dear Al and Dan We would like to contribute for LMAP too. Here in Chile we have performed active measurements for wired internet (aro= und 10,000 probes) and passive measurements for mobile internet (around 750= probes). Regards, Javier Bustos. 2013/7/1 Nagami Kenichi > Dear Al and Dan, We would like to contribute some use case for LMAP. We have measured the broadband performance in Japan using measurement software. We have distributed 500,000 applications. Regards, Ken Nagami 2013/6/22 MORTON JR., ALFRED C (AL) >: > LMAP, > > Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87, > either as a Working Group if approved in time, or as a BoF if not. > If charter-related issues remain to be resolved, they will be > discussed during the session with highest priority. > > However, ... > > If you have new/revised contributions covered under the current > draft of the LMAP charter, please reply to the list identifying the topic= , > the filename(if applicable) and the related issues you would like to disc= uss at > a face-to-face session, and try to do this by June 28, 2013. > > thanks and regards, > Dan and Al > > > >> -----Original Message----- >> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- >> bounces@ietf.org] On Behalf Of The IESG >> Sent: Friday, June 14, 2013 11:42 AM >> To: IETF-Announce >> Cc: lmap WG >> Subject: WG Review: Large-Scale Measurement of Broadband Performance >> (lmap) >> >> A new IETF working group has been proposed in the Operations and >> Management Area. The IESG has not made any determination yet. The >> following draft charter was submitted, and is provided for informational >> purposes only. Please send your comments to the IESG mailing list (iesg >> at ietf.org) by 2013-06-24. >> >> Large-Scale Measurement of Broadband Performance (lmap) >> ------------------------------------------------ >> Current Status: Proposed WG >> >> Assigned Area Director: >> Benoit Claise > >>... > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_581E085DEB6A444093CDAEA4C4EE48181359116Dxmbrcdx08ciscoc_ Content-Type: text/html; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable
All,

We are working on an update to the Use Case draft.  Feel free to = send comments to the list and/or authors.

-Marc-

From: <Romascanu>, "Dan = (Dan)" <dromasca@avaya.com>
Date: Tuesday, July 9, 2013 7:43 AM=
To: Javier Bustos <jbustos@niclabs.cl>, Nagami Kenichi <nagami@wide.ad.jp>
Cc: lmap WG <lmap@ietf.org>
Subject: Re: [lmap] Call for contri= butions and Agenda time in Berlin

Hi Javier and Nagami,

 

Thank you for letting the lmap lis= t know about your applications and for offering your help. It may be useful= if you would talk with Marc Linsner and Phil Eardley who have authored the use case document and see if your e= xperience can add to this. The current version in the tracker is = https://datatracker.ietf.org/doc/draft-linsner-lmap-use-cases/ but ther= e may be a revision in preparation for the Internet Draft submission cut-off next Monday.

 

Will you attend the Berlin meeting?

 

Thanks and Regards,

 

Dan

 

 

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Javier Bustos
Sent: Monday, July 01, 2013 4:32 PM
To: Nagami Kenichi
Cc: MORTON JR., ALFRED C (AL); lmap WG
Subject: Re: [lmap] Call for contributions and Agenda time in Berlin=

 

Dear Al and Dan

 

We would like to contribute for LMAP too. =

 

Here in Chile we have performed active measurements = for wired internet (around 10,000 probes) and passive measurements for mobi= le internet (around 750 probes).

 

Regards,

Javier Bustos.

 

2013/7/1 Nagami Kenichi <nagami@wide.ad.jp>

Dear Al and Dan,

We would like to contribute some use case for LMAP.
We have measured the broadband performance in Japan using measurement
software. We have distributed 500,000 applications.

Regards,
Ken Nagami

2013/6/22 MORTON JR., ALFRED C (AL) <acmorton@att.com>:

> LMAP,
>
> Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87,
> either as a Working Group if approved in time, or as a BoF if not.
> If charter-related issues remain to be resolved, they will be
> discussed during the session with highest priority.
>
> However, ...
>
> If you have new/revised contributions covered under the current
> draft of the LMAP charter, please reply to the list identifying the to= pic,
> the filename(if applicable) and the related issues you would like to d= iscuss at
> a face-to-face session, and try to do this by June 28, 2013.
>
> thanks and regards,
> Dan and Al
>
>
>
>> -----Original Message-----
>> From: ietf-annou= nce-bounces@ietf.org [mailto:ietf-ann= ounce-
>> bounces@ietf.org] On Behal= f Of The IESG
>> Sent: Friday, June 14, 2013 11:42 AM
>> To: IETF-Announce
>> Cc: lmap WG
>> Subject: WG Review: Large-Scale Measurement of Broadband Performan= ce
>> (lmap)
>>
>> A new IETF working group has been proposed in the Operations and >> Management Area. The IESG has not made any determination yet. The<= br> >> following draft charter was submitted, and is provided for informa= tional
>> purposes only. Please send your comments to the IESG mailing list = (iesg
>> at ietf.org) by = 2013-06-24.
>>
>> Large-Scale Measurement of Broadband Performance (lmap)
>> ------------------------------------------------
>> Current Status: Proposed WG
>>
>> Assigned Area Director:
>>   Benoit Claise <bcla= ise@cisco.com>
>>...
> _______________________________________________
> lmap mailing list
> lmap@ietf.org
> https://www.ietf.org/mailman/listinfo/lmap
>
_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap

 

 --_000_581E085DEB6A444093CDAEA4C4EE48181359116Dxmbrcdx08ciscoc_-- From jbustos@niclabs.cl Tue Jul 9 16:26:40 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1ACEC21F8624 for ; Tue, 9 Jul 2013 16:26:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.498 X-Spam-Level: X-Spam-Status: No, score=-2.498 tagged_above=-999 required=5 tests=[AWL=-0.800, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_82=0.6, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K6ThkmJYpAam for ; Tue, 9 Jul 2013 16:26:35 -0700 (PDT) Received: from mail-pd0-f180.google.com (mail-pd0-f180.google.com [209.85.192.180]) by ietfa.amsl.com (Postfix) with ESMTP id 3305721F9CF8 for ; Tue, 9 Jul 2013 16:26:34 -0700 (PDT) Received: by mail-pd0-f180.google.com with SMTP id 10so5702675pdi.25 for ; Tue, 09 Jul 2013 16:26:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to:x-mailer:x-gm-message-state; bh=CE1uGeMJqLrn9UUKs5SpOlxOMArU9bzwbY4K69emb6s=; b=I5wfo15+fHGeTqUWwAsJz2z5ZBgeAFuWgS/LriaVC9NbkzuhpxPb7wZf3LVKBaPIWB LQMCyPfa0pmI04akiAAOx/BMaCxE/QL2vdD7xAw6+jCPOrSrxWKOs+LSAvCbQ1nT5dpE Md79ubGc9jaZRJrZOGyJGTODEduUYolAtlCSbANaL38iVD7oA8WlHZT5TzyHGkGer+DE 6yY4VO5EMv62yH7idzVkC6OMlTG99mm1f38htUHVFneU731UpnghbKBCuEQ+zkTkrAqW uGPLXe53FlQjPSVk2Q9Zm/vlpN6XMfjqETPgY9CezdmPHFh6g+lcsnH6s/0FSa5r3SLI ZyKw== X-Received: by 10.66.161.195 with SMTP id xu3mr30543486pab.56.1373412393943; Tue, 09 Jul 2013 16:26:33 -0700 (PDT) Received: from [192.168.0.10] (pc-141-187-46-190.cm.vtr.net. [190.46.187.141]) by mx.google.com with ESMTPSA id zn4sm32472694pac.21.2013.07.09.16.26.31 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 09 Jul 2013 16:26:32 -0700 (PDT) Content-Type: multipart/alternative; boundary="Apple-Mail=_3207E95E-BEB6-458D-804A-F072258F2FAD" Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) From: =?iso-8859-1?Q?Javier_Bustos_Jim=E9nez?= In-Reply-To: <581E085DEB6A444093CDAEA4C4EE48181359116D@xmb-rcd-x08.cisco.com> Date: Tue, 9 Jul 2013 19:26:28 -0400 Message-Id: References: <581E085DEB6A444093CDAEA4C4EE48181359116D@xmb-rcd-x08.cisco.com> To: Marc Linsner (mlinsner) X-Mailer: Apple Mail (2.1508) X-Gm-Message-State: ALoCoQlOOJzKnTKS+miN+PTICA7AcdqMIEC2YtSvNPWMSEM77spZYraN7vu6IMAdvAEc3SbojZpz Cc: Nagami Kenichi , lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jul 2013 23:26:40 -0000 --Apple-Mail=_3207E95E-BEB6-458D-804A-F072258F2FAD Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Well, my 2 cents: 1.- I don't know if we should use the term Quality of Experience, = because that is related more to the user's expectatives and context and, = as far as I know, the proposed infrastructure doesn't measure that. = Quality of Service (QoS), that is more related to technical metrics, = could be better. 2.- I missed in the documents the peer-to-peer measurement. For = instance, it could be very interesting to know how is the = path/connection/performance between two peers to test if the ISP = fulfills some neutrality law (as we have here in Chile), or to optimize = routes between peers, etc. If the collector knows all nodes that are = participating on the measurements, it could be easy to select randomly a = pair and then perform the test between them. Best regards Javier El 09-07-2013, a las 10:54, Marc Linsner (mlinsner) = escribi=F3: > All, >=20 > We are working on an update to the Use Case draft. Feel free to send = comments to the list and/or authors. >=20 > -Marc- >=20 > From: , "Dan (Dan)" > Date: Tuesday, July 9, 2013 7:43 AM > To: Javier Bustos , Nagami Kenichi = > Cc: lmap WG > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin >=20 > Hi Javier and Nagami, > =20 > Thank you for letting the lmap list know about your applications and = for offering your help. It may be useful if you would talk with Marc = Linsner and Phil Eardley who have authored the use case document and see = if your experience can add to this. The current version in the tracker = is https://datatracker.ietf.org/doc/draft-linsner-lmap-use-cases/ but = there may be a revision in preparation for the Internet Draft submission = cut-off next Monday. > =20 > Will you attend the Berlin meeting? > =20 > Thanks and Regards, > =20 > Dan > =20 > =20 > =20 > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf = Of Javier Bustos > Sent: Monday, July 01, 2013 4:32 PM > To: Nagami Kenichi > Cc: MORTON JR., ALFRED C (AL); lmap WG > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin > =20 > Dear Al and Dan > =20 > We would like to contribute for LMAP too.=20 > =20 > Here in Chile we have performed active measurements for wired internet = (around 10,000 probes) and passive measurements for mobile internet = (around 750 probes). > =20 > Regards, > Javier Bustos. > =20 >=20 > 2013/7/1 Nagami Kenichi > Dear Al and Dan, >=20 > We would like to contribute some use case for LMAP. > We have measured the broadband performance in Japan using measurement > software. We have distributed 500,000 applications. >=20 > Regards, > Ken Nagami >=20 > 2013/6/22 MORTON JR., ALFRED C (AL) : > > LMAP, > > > > Last week, the draft LMAP charter went forward for external review. > > LMAP has been approved for a placeholder session at IETF-87, > > either as a Working Group if approved in time, or as a BoF if not. > > If charter-related issues remain to be resolved, they will be > > discussed during the session with highest priority. > > > > However, ... > > > > If you have new/revised contributions covered under the current > > draft of the LMAP charter, please reply to the list identifying the = topic, > > the filename(if applicable) and the related issues you would like to = discuss at > > a face-to-face session, and try to do this by June 28, 2013. > > > > thanks and regards, > > Dan and Al > > > > > > > >> -----Original Message----- > >> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- > >> bounces@ietf.org] On Behalf Of The IESG > >> Sent: Friday, June 14, 2013 11:42 AM > >> To: IETF-Announce > >> Cc: lmap WG > >> Subject: WG Review: Large-Scale Measurement of Broadband = Performance > >> (lmap) > >> > >> A new IETF working group has been proposed in the Operations and > >> Management Area. The IESG has not made any determination yet. The > >> following draft charter was submitted, and is provided for = informational > >> purposes only. Please send your comments to the IESG mailing list = (iesg > >> at ietf.org) by 2013-06-24. > >> > >> Large-Scale Measurement of Broadband Performance (lmap) > >> ------------------------------------------------ > >> Current Status: Proposed WG > >> > >> Assigned Area Director: > >> Benoit Claise > >>... > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > =20 --Apple-Mail=_3207E95E-BEB6-458D-804A-F072258F2FAD Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 Well, = my 2 cents:

1.- I don't know if we should use the = term Quality of Experience, because that is related more to the user's = expectatives and context and, as far as I know, the proposed = infrastructure doesn't measure that. Quality of Service (QoS), that is = more related to technical metrics, could be = better.

2.- I missed in the documents the = peer-to-peer measurement. For instance, it could be very interesting to = know how is the path/connection/performance between two peers to test if = the ISP fulfills some neutrality law (as we have here in Chile), or to = optimize routes between peers, etc.  If the collector knows all = nodes that are participating on the measurements, it could be easy to = select randomly a pair and then perform the test between = them.

Best = regards
Javier

El 09-07-2013, a las = 10:54, Marc Linsner (mlinsner) <mlinsner@cisco.com> = escribi=F3:

All,

We are working on an update to the Use Case draft.  Feel free = to send comments to the list and/or authors.

-Marc-

From: <Romascanu>, "Dan = (Dan)" <dromasca@avaya.com>
Date: Tuesday, July 9, 2013 7:43 = AM
To: Javier Bustos <jbustos@niclabs.cl>, Nagami = Kenichi <nagami@wide.ad.jp>
Cc: lmap WG <lmap@ietf.org>
Subject: Re: [lmap] Call for = contributions and Agenda time in Berlin

Hi Javier and Nagami,

Thank you for letting the lmap list know about your = applications and for offering your help. It may be useful if you would = talk with Marc Linsner and Phil Eardley who have authored the use case document and see if = your experience can add to this. The current version in the tracker is ht= tps://datatracker.ietf.org/doc/draft-linsner-lmap-use-cases/ but = there may be a revision in preparation for the Internet Draft submission cut-off next Monday.

Will you = attend the Berlin meeting?

Thanks = and Regards,

Dan

 

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Javier Bustos
Sent: Monday, July 01, 2013 4:32 PM
To: Nagami Kenichi
Cc: MORTON JR., ALFRED C (AL); lmap WG
Subject: Re: [lmap] Call for contributions and Agenda time in = Berlin

 

Dear Al and Dan

 

We would like to contribute for LMAP = too. 

 

Here in Chile we have performed active = measurements for wired internet (around 10,000 probes) and passive = measurements for mobile internet (around 750 probes).

 

Regards,

Javier Bustos.

 

2013/7/1 Nagami Kenichi <nagami@wide.ad.jp>

Dear Al and Dan,

We would like to contribute some use case for LMAP.
We have measured the broadband performance in Japan using = measurement
software. We have distributed 500,000 applications.

Regards,
Ken Nagami

2013/6/22 MORTON JR., ALFRED C (AL) <acmorton@att.com>:

> LMAP,
>
> Last week, the draft LMAP charter went forward for external = review.
> LMAP has been approved for a placeholder session at IETF-87,
> either as a Working Group if approved in time, or as a BoF if = not.
> If charter-related issues remain to be resolved, they will be
> discussed during the session with highest priority.
>
> However, ...
>
> If you have new/revised contributions covered under the current
> draft of the LMAP charter, please reply to the list identifying the = topic,
> the filename(if applicable) and the related issues you would like = to discuss at
> a face-to-face session, and try to do this by June 28, 2013.
>
> thanks and regards,
> Dan and Al
>
>
>
>> -----Original Message-----
>> From: ietf-announce-bounces@ietf.= org [mailto:ietf-announce-
>> bounces@ietf.org] On = Behalf Of The IESG
>> Sent: Friday, June 14, 2013 11:42 AM
>> To: IETF-Announce
>> Cc: lmap WG
>> Subject: WG Review: Large-Scale Measurement of Broadband = Performance
>> (lmap)
>>
>> A new IETF working group has been proposed in the Operations = and
>> Management Area. The IESG has not made any determination yet. = The
>> following draft charter was submitted, and is provided for = informational
>> purposes only. Please send your comments to the IESG mailing = list (iesg
>> at ietf.org) = by 2013-06-24.
>>
>> Large-Scale Measurement of Broadband Performance (lmap)
>> ------------------------------------------------
>> Current Status: Proposed WG
>>
>> Assigned Area Director:
>>   Benoit Claise <bclaise@cisco.com>
>>...
> _______________________________________________
> lmap mailing list
> lmap@ietf.org
> https://www.ietf.org/mailman/listinfo/lmap
>
_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

 

= --Apple-Mail=_3207E95E-BEB6-458D-804A-F072258F2FAD-- From marcelo@it.uc3m.es Tue Jul 9 22:35:16 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EBB0721F9A90 for ; Tue, 9 Jul 2013 22:35:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.141 X-Spam-Level: X-Spam-Status: No, score=-105.141 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_HTML_ONLY=1.457, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qYP16-DLxPOj for ; Tue, 9 Jul 2013 22:35:12 -0700 (PDT) Received: from smtp01.uc3m.es (smtp01.uc3m.es [163.117.176.131]) by ietfa.amsl.com (Postfix) with ESMTP id EA88721F9A87 for ; Tue, 9 Jul 2013 22:35:11 -0700 (PDT) Received: from smtp01.uc3m.es (localhost [127.0.0.1]) by localhost.uc3m.es (Postfix) with ESMTP id 824C0CC5BEA for ; Wed, 10 Jul 2013 07:35:10 +0200 (CEST) X-uc3m-safe: yes Received: from [163.117.139.56] (dummyhost19.it.uc3m.es [163.117.139.56]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: marcelo@smtp01.uc3m.es) by smtp01.uc3m.es (Postfix) with ESMTPSA id 76162CB7C3A for ; Wed, 10 Jul 2013 07:35:10 +0200 (CEST) Message-ID: <51DCF290.20906@it.uc3m.es> Date: Wed, 10 Jul 2013 07:35:12 +0200 From: marcelo bagnulo braun User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: "lmap@ietf.org" References: <20130710053057.19859.26080.idtracker@ietfa.amsl.com> In-Reply-To: <20130710053057.19859.26080.idtracker@ietfa.amsl.com> X-Forwarded-Message-Id: <20130710053057.19859.26080.idtracker@ietfa.amsl.com> Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelistedACL 138 matched, not delayed by milter-greylist-4.2.7 (smtp01.uc3m.es); Wed, 10 Jul 2013 07:35:10 +0200 (CEST) X-TM-AS-Product-Ver: IMSS-7.1.0.1224-7.0.0.1014-20006.005 Subject: [lmap] Fwd: I-D Action: draft-bagnulo-lmap-http-00.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 05:35:17 -0000 Hi,

We have posted this draft exploring the possibility of using http as an LMAP protocol.

We would be interested in hearing you comments.

Regards, marcelo


-------- Mensaje original --------
Asunto: I-D Action: draft-bagnulo-lmap-http-00.txt
Fecha: Tue, 09 Jul 2013 22:30:57 -0700
De: internet-drafts@ietf.org
Responder a: internet-drafts@ietf.org
Para: i-d-announce@ietf.org 


A New Internet-Draft is available from the on-line Internet-Drafts directories.


	Title           : Large MeAsurement Platform Protocol
	Author(s)       : Marcelo Bagnulo
                          Trevor Burbridge
                          Sam Crawford
                          Juergen Schoenwaelder
                          Vaibhav Bajpai
	Filename        : draft-bagnulo-lmap-http-00.txt
	Pages           : 20
	Date            : 2013-07-09

Abstract:
   This documents specifies the LMAP protocol based on HTTP for the
   Control and Report in Large Scale Measurement Platforms.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-bagnulo-lmap-http

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-bagnulo-lmap-http-00


Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
I-D-Announce@ietf.org
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt


From michael.faath@hs-augsburg.de Wed Jul 10 03:08:39 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1A09911E811D for ; Wed, 10 Jul 2013 03:08:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.201 X-Spam-Level: X-Spam-Status: No, score=-0.201 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, HELO_MISMATCH_DE=1.448, J_CHICKENPOX_21=0.6] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0nva6+6xfgIs for ; Wed, 10 Jul 2013 03:08:34 -0700 (PDT) Received: from fly-neu.RZ.HS-Augsburg.DE (fly-neu.RZ.HS-Augsburg.DE [141.82.217.48]) by ietfa.amsl.com (Postfix) with ESMTP id 5B30211E8119 for ; Wed, 10 Jul 2013 03:08:34 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by fly-neu.RZ.HS-Augsburg.DE (Postfix) with ESMTP id A79D21D6482; Wed, 10 Jul 2013 12:08:32 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at hs-augsburg.de Received: from fly-neu.RZ.HS-Augsburg.DE ([127.0.0.1]) by localhost (fly-neu.rz.hs-augsburg.de [127.0.0.1]) (amavisd-new, port 10024) with LMTP id CLaqM6ATXoCP; Wed, 10 Jul 2013 12:08:31 +0200 (CEST) Received: from [141.82.175.9] (unknown [141.82.175.9]) by fly-neu.RZ.HS-Augsburg.DE (Postfix) with ESMTPSA id 2DDD31D6468; Wed, 10 Jul 2013 12:08:31 +0200 (CEST) Message-ID: <51DD329E.1030508@hs-augsburg.de> Date: Wed, 10 Jul 2013 12:08:30 +0200 From: Michael Faath User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: trevor.burbridge@bt.com, lmap@ietf.org References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [lmap] Information Model draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 10:08:39 -0000 Hi Trevor, one question concerning the Measurement Suppression: As I understand the Measurement Suppression suppresses one or multiple Measurement Task Configurations for a specified period of time. How about not suppressing an entire Measurement Task Configuration but specific Measurement Schedules? One may have multiple Measurement Schedules using the same Measurement Task Configuration with different Reporting Channels and/or Measurement Timings and would like to suppress only some of the Measurement Schedules and not the entire Measurement Task Configuration. An (additional and optional) reference to the Schedule within the Suppression could solve this. Regards, Michael On 05.07.2013 12:14, trevor.burbridge@bt.com wrote: > I have submitted a first version of an Information Model for LMAP that we hope to be able to present and discuss at the Berlin LMAP group: > > http://www.ietf.org/id/draft-burbridge-lmap-information-model-00.txt > > Trevor. > > Trevor Burbridge > Network Infrastructure & Innovation | BT Innovate & Design > Tel: 01473 645115 > Fax: 01473 640929 > > This email contains BT information, which may be privileged or confidential. It's meant only for the individual(s) or entity named above. If you're not the intended recipient, note that disclosing, copying, distributing or using this information is prohibited. If you've received this email in error, please let me know immediately on the email address above. Thank you. > We monitor our email system, and may record your emails. > British Telecommunications plc > Registered office: 81 Newgate Street London EC1A 7AJ > Registered in England no: 1800000 > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > From trevor.burbridge@bt.com Wed Jul 10 04:05:28 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 748E121F8C38 for ; Wed, 10 Jul 2013 04:05:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.549 X-Spam-Level: X-Spam-Status: No, score=-2.549 tagged_above=-999 required=5 tests=[AWL=-0.150, BAYES_00=-2.599, J_CHICKENPOX_21=0.6, J_CHICKENPOX_91=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AfaJswLkS4dF for ; Wed, 10 Jul 2013 04:05:24 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 254D921F8C9F for ; Wed, 10 Jul 2013 04:05:23 -0700 (PDT) Received: from EVMHT63-UKRD.domain1.systemhost.net (10.36.3.100) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.298.1; Wed, 10 Jul 2013 12:05:21 +0100 Received: from EMV64-UKRD.domain1.systemhost.net ([169.254.1.127]) by EVMHT63-UKRD.domain1.systemhost.net ([10.36.3.100]) with mapi; Wed, 10 Jul 2013 12:05:21 +0100 From: To: , Date: Wed, 10 Jul 2013 12:05:20 +0100 Thread-Topic: [lmap] Information Model draft Thread-Index: Ac59VXCK5VpO7gCdRSWVXs51ieMWVAABs7nQ Message-ID: References: <51DD329E.1030508@hs-augsburg.de> In-Reply-To: <51DD329E.1030508@hs-augsburg.de> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] Information Model draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 11:05:28 -0000 It's certainly an option. I was considering the Measurement Suppression to mainly be an emergency mea= sure when a particular Measurement Configuration was causing problems (e.g.= too many tests to a particular target or a problem in a code revision for = a particular measurement). In this case I think you want to turn of the who= le thing without having to identify which schedules were using that Measure= ment Configuration. Also you could disable a Measurment Configuration and t= he other measurements (which aren't causing any trouble) within the schedul= es would still operate. You always have the option to just delete a particular shcedule in any case= . One possibility would be to allow the Measurement Suppression information t= o carry an optional list of Measurement Configurations AND an optional list= of Measurement Schedules. Trevor. >-----Original Message----- >From: Michael Faath [mailto:michael.faath@hs-augsburg.de] >Sent: 10 July 2013 11:09 >To: Burbridge,T,Trevor,TUB8 R; lmap@ietf.org >Subject: Re: [lmap] Information Model draft > >Hi Trevor, > >one question concerning the Measurement Suppression: > >As I understand the Measurement Suppression suppresses one or multiple >Measurement Task Configurations for a specified period of time. How >about not suppressing an entire Measurement Task Configuration but >specific Measurement Schedules? One may have multiple Measurement >Schedules using the same Measurement Task Configuration with different >Reporting Channels and/or Measurement Timings and would like to >suppress >only some of the Measurement Schedules and not the entire Measurement >Task Configuration. An (additional and optional) reference to the >Schedule within the Suppression could solve this. > >Regards, >Michael > >On 05.07.2013 12:14, trevor.burbridge@bt.com wrote: >> I have submitted a first version of an Information Model for LMAP that w= e >hope to be able to present and discuss at the Berlin LMAP group: >> >> http://www.ietf.org/id/draft-burbridge-lmap-information-model-00.txt >> >> Trevor. >> >> Trevor Burbridge >> Network Infrastructure & Innovation | BT Innovate & Design >> Tel: 01473 645115 >> Fax: 01473 640929 >> >> This email contains BT information, which may be privileged or >confidential. It's meant only for the individual(s) or entity named above.= If >you're not the intended recipient, note that disclosing, copying, distribu= ting >or using this information is prohibited. If you've received this email in = error, >please let me know immediately on the email address above. Thank you. >> We monitor our email system, and may record your emails. >> British Telecommunications plc >> Registered office: 81 Newgate Street London EC1A 7AJ >> Registered in England no: 1800000 >> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> From paitken@cisco.com Wed Jul 10 05:06:26 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C3F7521F9E7B for ; Wed, 10 Jul 2013 05:06:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HCLZ44xwKV9j for ; Wed, 10 Jul 2013 05:06:15 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 0244021F9B9C for ; Wed, 10 Jul 2013 05:06:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=310; q=dns/txt; s=iport; t=1373457974; x=1374667574; h=message-id:date:from:mime-version:to:cc:subject: content-transfer-encoding; bh=cdnvBNg81wLuz+FHRgQ8M3DEJgRru6/e+W0BGknMFJM=; b=b2D8NjkS1WJO7cip24hWhxho/lf1DTf+Eu+rZLxhppiwKCMeICJjmTol n8+r6iDDvvF6ohv/HbmL6QVW+Nsr1zzzEcY81TxKRlIbwlmzbB+cYgjVU Q6rRGmQ2/YGcLmWmegH0dISr+iRvyMqRt3A34ulziUJp1KMsiXma5XYh7 I=; X-IronPort-AV: E=Sophos;i="4.87,1035,1363132800"; d="scan'208";a="84178095" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-2.cisco.com with ESMTP; 10 Jul 2013 12:06:12 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r6AC6CWE030767 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 10 Jul 2013 12:06:12 GMT Received: from [144.254.153.36] (dhcp-144-254-153-36.cisco.com [144.254.153.36]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6AC6BbV024286; Wed, 10 Jul 2013 13:06:11 +0100 (BST) Message-ID: <51DD4E33.5000101@cisco.com> Date: Wed, 10 Jul 2013 13:06:11 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: lmap-chairs@tools.ietf.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Aamer Akhter , lmap@ietf.org Subject: [lmap] LMAP framework draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 12:06:27 -0000 LMAP chairs, We've been writing an LMAP framework document which discusses the building blocks, looking at what we already have in the IETF and what's missing, where the gaps are. It will be published by Monday's cut-off. We'd welcome the opportunity to present the draft in Berlin. Thanks, P. From dromasca@avaya.com Wed Jul 10 05:09:34 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 25B9B21F9E7B for ; Wed, 10 Jul 2013 05:09:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.439 X-Spam-Level: X-Spam-Status: No, score=-103.439 tagged_above=-999 required=5 tests=[AWL=0.160, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9Mw8-ShPKBgp for ; Wed, 10 Jul 2013 05:09:27 -0700 (PDT) Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id 6B18021F9F0A for ; Wed, 10 Jul 2013 05:09:27 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjAFAMFN3VGHCzI1/2dsb2JhbABagmghf8EQgREWdIIjAQEBAQMSKD8MBAIBCA0BAwQBAQsUCQcyFAkIAgQBDQUIGodtAZ1/m0EXjzIxBwaDA2wDnh2LBIMRgig X-IronPort-AV: E=Sophos;i="4.87,1035,1363147200"; d="scan'208";a="16018752" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by de307622-de-outbound.net.avaya.com with ESMTP; 10 Jul 2013 08:09:25 -0400 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 10 Jul 2013 08:03:35 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Wed, 10 Jul 2013 08:09:22 -0400 From: "Romascanu, Dan (Dan)" To: Paul Aitken , "lmap-chairs@tools.ietf.org" Thread-Topic: LMAP framework draft Thread-Index: AQHOfWXoMwAA0GKX80uTiSsuxfOGrpld0R4Q Date: Wed, 10 Jul 2013 12:09:21 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1286526E@AZ-FFEXMB04.global.avaya.com> References: <51DD4E33.5000101@cisco.com> In-Reply-To: <51DD4E33.5000101@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Aamer Akhter , "lmap@ietf.org" Subject: Re: [lmap] LMAP framework draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 12:09:34 -0000 Hi Paul,=20 Thank you for the contribution.=20 How much meeting time would you need to spend on this? As a rule we would s= trongly prefer to avoid presentations, ask participants to come prepared wi= th the I-Ds already read, and use the precious meeting time for discussions= .=20 Regards, Dan > -----Original Message----- > From: Paul Aitken [mailto:paitken@cisco.com] > Sent: Wednesday, July 10, 2013 3:06 PM > To: lmap-chairs@tools.ietf.org > Cc: lmap@ietf.org; Aamer Akhter > Subject: LMAP framework draft >=20 > LMAP chairs, >=20 > We've been writing an LMAP framework document which discusses the > building blocks, looking at what we already have in the IETF and what's > missing, where the gaps are. > It will be published by Monday's cut-off. >=20 > We'd welcome the opportunity to present the draft in Berlin. >=20 > Thanks, > P. From v.bajpai@jacobs-university.de Wed Jul 10 05:09:53 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 941D921F9F0A for ; Wed, 10 Jul 2013 05:09:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.089 X-Spam-Level: X-Spam-Status: No, score=-2.089 tagged_above=-999 required=5 tests=[AWL=-0.860, BAYES_00=-2.599, HELO_EQ_DE=0.35, LOCALPART_IN_SUBJECT=2.02, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hghT2ywdLIjA for ; Wed, 10 Jul 2013 05:09:49 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 5400721F9E7B for ; Wed, 10 Jul 2013 05:09:48 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 5556120BE5; Wed, 10 Jul 2013 14:09:47 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id RIB0D4ivVCBu; Wed, 10 Jul 2013 14:09:47 +0200 (CEST) Received: from exchange.jacobs-university.de (unknown [10.70.0.122]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by hermes.jacobs-university.de (Postfix) with ESMTPS id 30CB020BDC; Wed, 10 Jul 2013 14:09:47 +0200 (CEST) Received: from SXCHMB01.jacobs.jacobs-university.de ([fe80::c1f:c30f:99ac:df0c]) by SHUBCAS01.jacobs.jacobs-university.de ([::1]) with mapi id 14.02.0342.003; Wed, 10 Jul 2013 14:09:44 +0200 From: "Bajpai, Vaibhav" To: "draft-burbridge-lmap-information-model@tools.ietf.org" Thread-Topic: draft-burbridge-lmap-information-model Thread-Index: AQHOfWZcrPw3mZy0/0SMZq8lkykNNA== Date: Wed, 10 Jul 2013 12:09:43 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.50.226.26] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "Bajpai, Vaibhav" , "" Subject: [lmap] draft-burbridge-lmap-information-model X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 12:09:53 -0000 Hello, I read the draft [1] and made some notes. Thought to share them along: [1] http://tools.ietf.org/html/draft-burbridge-lmap-information-model-00 - version of measurement test: 1. Measurement Task Configuration: Object 1. Task Name 2. Registry Entry: URN 3. Options: Set (optional) 4. Measurement Cycle ID: String (optional) It would be useful to convey the version of measurement test no? A version bump will usually come due to a bugfix in a specific measurement test implementation. The version information will help later segregate measurement results coming out from different measurement test implementations. hmm? - measurement time in result report 5. Measurement Task: Set 1. Measurement Task Configuration: Object 2. Result Headers: List 1. Column Name: String 3. Result Data: List 1. Result Row: Object 1. Measurement Time: datetime 2. Cross-traffic: Integer (optional) 3. Result Columns: List 1. Column Data Would it be useful to have explicit START (datetime) and END (datetime) in replacement for the Measurement Time (datetime)? I mean the START and END times when the measurement was performed. - pre-configured information: Detail of the information model elements: 1. MA MAC: MAC Address 2. Controller: FQDN 3. MA Certificate: Certificate 4. Controller Communication Timing: Timing maybe we should add these too, no? (it is discussed in the previous paragra= ph, just seems to be missing from the list of items) 5. MA Private Key 6. CA certificate - configuration information (retrieved from controller): 1. Measurement Agent ID: String 2. Measurement Group ID (optional): String 3. Report MA ID flag (optional): Boolean If I understand correctly, this is additional information sent, when the MA registers with the controller for the first time. In the LMAP protocol draf= t [draft-bagnulo-lmap-http-00], however, we assume that the MA UUID is baked = in as part of the pre-configured information in the MA. [draft-bagnulo-lmap-http-00] - In particular each MA has a version 4 UUID, which is randomly or pseudo randomly generated. We assume that the UUID is preconfigured in the MA before deployment. This could also be because in the LMAP protocol draft [draft-bagnulo-lmap-http-00], we only have pre-configured information, and the controller is only contacted to receive measurement instructions (measurements, schedule, reporting channels). Probably, we should discuss = and synchronize this no? - measurement protocol and several measurement protocols between the MAs used to actually perfor= m the measurements. I searched the terminology document [draft-eardley-lmap-terminology-01], bu= t could not find a description of measurment protocol. - measurement suppression instruction is defined in the information model, = but is not discussed in the terminology [draft-eardley-lmap-terminology-01] a= nd LMAP protocol drafts [draft-bagnulo-lmap-http-00]. Best, Vaibhav ----------------------------------------------------- Vaibhav Bajpai Research I, Room 86 Computer Networks and Distributed Systems (CNDS) Lab School of Engineering and Sciences Jacobs University Bremen, Germany www.vaibhavbajpai.com= From acmorton@att.com Wed Jul 10 05:34:19 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 87A5621F9AFE for ; Wed, 10 Jul 2013 05:34:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.148 X-Spam-Level: X-Spam-Status: No, score=-106.148 tagged_above=-999 required=5 tests=[AWL=-0.450, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_82=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JuwBNU-Zwdqq for ; Wed, 10 Jul 2013 05:34:14 -0700 (PDT) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 29AF821F9FFE for ; Wed, 10 Jul 2013 05:34:14 -0700 (PDT) Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id D64E312045F; Wed, 10 Jul 2013 08:34:10 -0400 (EDT) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-green.research.att.com (Postfix) with ESMTP id BCA84E0211; Wed, 10 Jul 2013 08:32:35 -0400 (EDT) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Wed, 10 Jul 2013 08:34:11 -0400 From: "MORTON JR., ALFRED C (AL)" To: =?iso-8859-1?Q?Javier_Bustos_Jim=E9nez?= , Marc Linsner Date: Wed, 10 Jul 2013 08:34:09 -0400 Thread-Topic: [lmap] Call for contributions and Agenda time in Berlin Thread-Index: Ac58+86nxZUUknX5T2yq3ePRFmTb4wAaa8CA Message-ID: References: <581E085DEB6A444093CDAEA4C4EE48181359116D@xmb-rcd-x08.cisco.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4806D57njfpsrvexg7re_" MIME-Version: 1.0 Cc: Nagami Kenichi , lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 12:34:19 -0000 --_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4806D57njfpsrvexg7re_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Javier wrote: 1.- I don't know if we should use the term Quality of Experience, because t= hat is related more to the user's expectatives and context and, as far as I= know, the proposed infrastructure doesn't measure that. Quality of Service= (QoS), that is more related to technical metrics, could be better. +1.0 I've made this comment previously in connection with LMAP. We haven't discussed any perceptual estimation measurements. We have been talking about Network Performance and QoS. There's no doubt that making measurements is a growing part of user experience, but we have yet to establish how this influences user's QoE w.r.t. an application and context... Al From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Jav= ier Bustos Jim=E9nez Sent: Tuesday, July 09, 2013 7:26 PM To: Marc Linsner Cc: Nagami Kenichi; lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin Well, my 2 cents: 1.- I don't know if we should use the term Quality of Experience, because t= hat is related more to the user's expectatives and context and, as far as I= know, the proposed infrastructure doesn't measure that. Quality of Service= (QoS), that is more related to technical metrics, could be better. 2.- I missed in the documents the peer-to-peer measurement. For instance, i= t could be very interesting to know how is the path/connection/performance = between two peers to test if the ISP fulfills some neutrality law (as we ha= ve here in Chile), or to optimize routes between peers, etc. If the collec= tor knows all nodes that are participating on the measurements, it could be= easy to select randomly a pair and then perform the test between them. Best regards Javier El 09-07-2013, a las 10:54, Marc Linsner (mlinsner) > escribi=F3: All, We are working on an update to the Use Case draft. Feel free to send comme= nts to the list and/or authors. -Marc- From: , "Dan (Dan)" > Date: Tuesday, July 9, 2013 7:43 AM To: Javier Bustos >, Nagami K= enichi > Cc: lmap WG > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin Hi Javier and Nagami, Thank you for letting the lmap list know about your applications and for of= fering your help. It may be useful if you would talk with Marc Linsner and = Phil Eardley who have authored the use case document and see if your experi= ence can add to this. The current version in the tracker is https://datatra= cker.ietf.org/doc/draft-linsner-lmap-use-cases/ but there may be a revision= in preparation for the Internet Draft submission cut-off next Monday. Will you attend the Berlin meeting? Thanks and Regards, Dan From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of Javier Bustos Sent: Monday, July 01, 2013 4:32 PM To: Nagami Kenichi Cc: MORTON JR., ALFRED C (AL); lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin Dear Al and Dan We would like to contribute for LMAP too. Here in Chile we have performed active measurements for wired internet (aro= und 10,000 probes) and passive measurements for mobile internet (around 750= probes). Regards, Javier Bustos. 2013/7/1 Nagami Kenichi > Dear Al and Dan, We would like to contribute some use case for LMAP. We have measured the broadband performance in Japan using measurement software. We have distributed 500,000 applications. Regards, Ken Nagami 2013/6/22 MORTON JR., ALFRED C (AL) >: > LMAP, > > Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87, > either as a Working Group if approved in time, or as a BoF if not. > If charter-related issues remain to be resolved, they will be > discussed during the session with highest priority. > > However, ... > > If you have new/revised contributions covered under the current > draft of the LMAP charter, please reply to the list identifying the topic= , > the filename(if applicable) and the related issues you would like to disc= uss at > a face-to-face session, and try to do this by June 28, 2013. > > thanks and regards, > Dan and Al > > > >> -----Original Message----- >> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- >> bounces@ietf.org] On Behalf Of The IESG >> Sent: Friday, June 14, 2013 11:42 AM >> To: IETF-Announce >> Cc: lmap WG >> Subject: WG Review: Large-Scale Measurement of Broadband Performance >> (lmap) >> >> A new IETF working group has been proposed in the Operations and >> Management Area. The IESG has not made any determination yet. The >> following draft charter was submitted, and is provided for informational >> purposes only. Please send your comments to the IESG mailing list (iesg >> at ietf.org) by 2013-06-24. >> >> Large-Scale Measurement of Broadband Performance (lmap) >> ------------------------------------------------ >> Current Status: Proposed WG >> >> Assigned Area Director: >> Benoit Claise > >>... > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4806D57njfpsrvexg7re_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Javier wrote:<= /p>

1.- I don't know if we should use the term Quality = of Experience, because that is related more to the user's expectatives and = context and, as far as I know, the proposed infrastructure doesn't measure = that. Quality of Service (QoS), that is more related to technical metrics, = could be better.

 

+1.0= =A0

I've made this comment previously in connect= ion with LMAP.

We haven't discussed any perceptua= l estimation measurements.

We have been talking a= bout Network Performance and QoS.

There's no doub= t that making measurements is a growing part

of = user experience, but we have yet to establish how this

influences user's QoE w.r.t. an application and context...=

 

Al

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of = Javier Bustos Jim=E9nez
Sent: Tuesday, July 09, 2013 7:26 PM
<= b>To: Marc Linsner
Cc: Nagami Kenichi; lmap WG
Subject:= Re: [lmap] Call for contributions and Agenda time in Berlin=

 

Well, my 2 cents:

 

1.- I don't know if we sho= uld use the term Quality of Experience, because that is related more to the= user's expectatives and context and, as far as I know, the proposed infras= tructure doesn't measure that. Quality of Service (QoS), that is more relat= ed to technical metrics, could be better.

 

2.- I mis= sed in the documents the peer-to-peer measurement. For instance, it could b= e very interesting to know how is the path/connection/performance between t= wo peers to test if the ISP fulfills some neutrality law (as we have here i= n Chile), or to optimize routes between peers, etc.  If the collector = knows all nodes that are participating on the measurements, it could be eas= y to select randomly a pair and then perform the test between them.

 

Best regards

Javier

 

=

El 09-07-2013, a las 10:54, Marc Linsner (ml= insner) <mlinsner@cisco.com>= ; escribi=F3:



<= /p>

All,

 

We are working on = an update to the Use Case draft.  Feel free to send comments to the li= st and/or authors.

 = ;

-Marc-

 

From: <Romascanu>, "Dan (Dan)" <dromasca@avaya.com>
Date: Tuesday,= July 9, 2013 7:43 AM
To: Javier Bustos <jbustos@niclabs.cl>, Nagami Kenichi <nagami@wide.ad.jp>
Cc: lmap WG &= lt;lmap@ietf.org>
Subject: Re: [lmap] Call for contributions and Agenda time in Berlin

 

Hi Javier and Nagami,

 

<= p class=3DMsoNormal>Thank you for letting the lmap list know about y= our applications and for offering your help. It may be useful if you would = talk with Marc Linsner and Phil Eardley who have authored the use case docu= ment and see if your experience can add to this. The current version in the= tracker is https://datatracker.ietf.org/doc/draft-linsner-lmap-use-cases/<= /a> but there may be a revision in preparation for the Internet Draft subm= ission cut-off next Monday.

 

Will you attend the Berlin meeting?

 

<= span style=3D'font-size:9.5pt;font-family:"Verdana","sans-serif"'>Thanks an= d Regards,

 

Dan

 

 

 

From: lmap-bounces@ietf.org [m= ailto:lmap-bounces@ietf.org] On Behalf Of Javier Bustos
Se= nt: Monday, July 01, 2013 4:32 PM
To: Nagami Kenichi
Cc= : MORTON JR., ALFRED C (AL); lmap WG
Subject: Re: [lmap] Call= for contributions and Agenda time in Berlin

 

Dear= Al and Dan

 

We would like to contribute for LMAP too.&nbs= p;

 

Here in Chile we have performed active measureme= nts for wired internet (around 10,000 probes) and passive measurements for = mobile internet (around 750 probes).

 

Regards,<= /o:p>

Javier Bustos.

 =

2013/7/1 Nagami Kenichi <nagami@wide.ad.jp><= /o:p>

Dear Al and Dan,

We would like to cont= ribute some use case for LMAP.
We have measured the broadband performanc= e in Japan using measurement
software. We have distributed 500,000 appli= cations.

Regards,
Ken Nagami

2013/6/22 MORTON JR., ALFRED = C (AL) <acmorton@att.com>:

> LMAP,
>
> Last = week, the draft LMAP charter went forward for external review.
> LMAP= has been approved for a placeholder session at IETF-87,
> either as = a Working Group if approved in time, or as a BoF if not.
> If charter= -related issues remain to be resolved, they will be
> discussed durin= g the session with highest priority.
>
> However, ...
>> If you have new/revised contributions covered under the current
&= gt; draft of the LMAP charter, please reply to the list identifying the top= ic,
> the filename(if applicable) and the related issues you would li= ke to discuss at
> a face-to-face session, and try to do this by June= 28, 2013.
>
> thanks and regards,
> Dan and Al
>>
>
>> -----Original Message-----
>> From: ietf-announce-bounces@ietf.o= rg [mailto:ietf-announce-
>= > bounces@ietf.org] On Behalf Of= The IESG
>> Sent: Friday, June 14, 2013 11:42 AM
>> To: = IETF-Announce
>> Cc: lmap WG
>> Subject: WG Review: Large= -Scale Measurement of Broadband Performance
>> (lmap)
>><= br>>> A new IETF working group has been proposed in the Operations an= d
>> Management Area. The IESG has not made any determination yet.= The
>> following draft charter was submitted, and is provided for= informational
>> purposes only. Please send your comments to the = IESG mailing list (iesg
>> at ietf.org) by 2013-06-24.
>>
>> Large-Scal= e Measurement of Broadband Performance (lmap)
>> -----------------= -------------------------------
>> Current Status: Proposed WG
= >>
>> Assigned Area Director:
>>   Benoit Clai= se <bclaise@cisco.com>
&g= t;>...
> _______________________________________________
> l= map mailing list
> lmap@ietf.org=
> https://www.ietf.org/mailman/listinfo/lmap
>
__________= _____________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinf= o/lmap

 =

 

= --_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4806D57njfpsrvexg7re_-- From trevor.burbridge@bt.com Wed Jul 10 05:35:49 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 313A221F8FF3 for ; Wed, 10 Jul 2013 05:35:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.099 X-Spam-Level: X-Spam-Status: No, score=-3.099 tagged_above=-999 required=5 tests=[AWL=0.500, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SWla8R1CKwgq for ; Wed, 10 Jul 2013 05:35:45 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 9779921F8FAA for ; Wed, 10 Jul 2013 05:35:43 -0700 (PDT) Received: from EVMHT65-UKRD.domain1.systemhost.net (10.36.3.102) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.298.1; Wed, 10 Jul 2013 13:35:40 +0100 Received: from EMV64-UKRD.domain1.systemhost.net ([169.254.1.127]) by EVMHT65-UKRD.domain1.systemhost.net ([10.36.3.102]) with mapi; Wed, 10 Jul 2013 13:35:40 +0100 From: To: , Date: Wed, 10 Jul 2013 13:35:40 +0100 Thread-Topic: draft-burbridge-lmap-information-model Thread-Index: AQHOfWZcrPw3mZy0/0SMZq8lkykNNJld1OQQ Message-ID: References: In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: lmap@ietf.org Subject: Re: [lmap] draft-burbridge-lmap-information-model X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 12:35:49 -0000 Hi Vaibhav, thanks for the very useful comments/discussion. See inline for my answers.... Trevor. >-----Original Message----- >From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of >Bajpai, Vaibhav >Sent: 10 July 2013 13:10 >To: draft-burbridge-lmap-information-model@tools.ietf.org >Cc: Bajpai, Vaibhav; >Subject: [lmap] draft-burbridge-lmap-information-model > >Hello, > >I read the draft [1] and made some notes. Thought to share them along: > >[1] http://tools.ietf.org/html/draft-burbridge-lmap-information-model-00 > > >- version of measurement test: > > 1. Measurement Task Configuration: Object > > 1. Task Name > 2. Registry Entry: URN > 3. Options: Set (optional) > 4. Measurement Cycle ID: String (optional) > > >It would be useful to convey the version of measurement test no? > >A version bump will usually come due to a bugfix in a specific measurement >test implementation. The version information will help later segregate >measurement results coming out from different measurement test >implementations. hmm? Yes I agree. I considered and then removed some 'metadata' on the informati= on objects: namely author, creation date, name (where not required by the M= A as a reference, description and version number since I didn't want to clu= tter the first version too much. However I think I was over-zealous since I= agree that the Measurement Configuration Version is something that should = be reported by the MA. >- measurement time in result report > > 5. Measurement Task: Set > > 1. Measurement Task Configuration: Object > 2. Result Headers: List > 1. Column Name: String > 3. Result Data: List > 1. Result Row: Object > 1. Measurement Time: datetime > 2. Cross-traffic: Integer (optional) > 3. Result Columns: List > 1. Column Data > >Would it be useful to have explicit START (datetime) and END (datetime) in >replacement for the Measurement Time (datetime)? I mean the START and >END times when the measurement was performed. I think the tests for which this is relevant can report a duration data col= umn or even an end time if they want/prefer. Not against the idea, I just d= on't know it is required in the IM as it would be optional (since many test= s wouldn't want to record or report this). I can see the point in having it= in the standard IM though if you do want to isolate any measurement result= s that were conducted entirely within a period - just not sure how useful t= his is. >- pre-configured information: > > Detail of the information model elements: > > 1. MA MAC: MAC Address > 2. Controller: FQDN > 3. MA Certificate: Certificate > 4. Controller Communication Timing: Timing > >maybe we should add these too, no? (it is discussed in the previous >paragraph, just seems to be missing from the list of items) > > 5. MA Private Key > 6. CA certificate I originally had the certificate on the Controller and Collector in the IM.= However, since these will be provided as required and fetched/validated ou= tside the LMAP framework I removed them. Not against putting them back in, = but not sure if strictly necessary. >- configuration information (retrieved from controller): > > 1. Measurement Agent ID: String > 2. Measurement Group ID (optional): String > 3. Report MA ID flag (optional): Boolean > >If I understand correctly, this is additional information sent, when the M= A >registers with the controller for the first time. In the LMAP protocol dra= ft >[draft-bagnulo-lmap-http-00], however, we assume that the MA UUID is >baked in as part of the pre-configured information in the MA. > >[draft-bagnulo-lmap-http-00] - > > In particular each MA has a version 4 UUID, which is > randomly or pseudo randomly generated. We assume that the UUID is > preconfigured in the MA before deployment. > >This could also be because in the LMAP protocol draft [draft-bagnulo-lmap- >http-00], we only have pre-configured information, and the controller is o= nly >contacted to receive measurement instructions (measurements, schedule, >reporting channels). Probably, we should discuss and synchronize this no? This is an important discussion I think. Either the MA is given an ID when = it registers, or it is pre-configured with and ID. I tended to prefer the f= irst option since it allows IDs to be allocated depending on the registrati= on context. It also doesn't waste ID space on MAs that may never be switche= d on. It also enables multiple vendor devices pre-configured by those vendo= rs to receive guaranteed unique MAs from the Controller. I'm not yet convinced the ID is required in order to contact the Controller= , so the bottom line is that it can be allocated at registration. >- measurement protocol > > and several measurement protocols between the MAs used to actually >perform > the measurements. > >I searched the terminology document [draft-eardley-lmap-terminology-01], >but could not find a description of measurment protocol. > > > > > >- measurement suppression instruction is defined in the information model, >but > is not discussed in the terminology [draft-eardley-lmap-terminology-01] >and > LMAP protocol drafts [draft-bagnulo-lmap-http-00]. > > > > >Best, Vaibhav > > From dromasca@avaya.com Wed Jul 10 05:52:26 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 21BD321F9F26 for ; Wed, 10 Jul 2013 05:52:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.865 X-Spam-Level: X-Spam-Status: No, score=-102.865 tagged_above=-999 required=5 tests=[AWL=-0.466, BAYES_00=-2.599, J_CHICKENPOX_21=0.6, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FyXuB6BqG+O1 for ; Wed, 10 Jul 2013 05:52:15 -0700 (PDT) Received: from co300216-co-outbound.net.avaya.com (co300216-co-outbound.net.avaya.com [198.152.13.100]) by ietfa.amsl.com (Postfix) with ESMTP id BB1DC21F9E2A for ; Wed, 10 Jul 2013 05:52:14 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgUFAAH/ZlGHCzI1/2dsb2JhbABQgmUhNsFIgQcWdIIfAQEBAQMBAQEPCx00FwICAgEIDQQEAQELFAkHFhELFAkIAQEEARIIGodyAQugPJ0MEwQEjVAOgQMmEgaCWmEDnSmKaIMLgXM1 X-IronPort-AV: E=Sophos;i="4.87,456,1363147200"; d="scan'208";a="18952935" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by co300216-co-outbound.net.avaya.com with ESMTP; 10 Jul 2013 08:52:13 -0400 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 10 Jul 2013 08:46:24 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Wed, 10 Jul 2013 08:52:12 -0400 From: "Romascanu, Dan (Dan)" To: "trevor.burbridge@bt.com" , "lmap@ietf.org" Thread-Topic: Call for contributions and Agenda time in Berlin Thread-Index: Ac5ukbwsnWWTUZX3TBq4VwGNgR/klAFjyywwAlLUZqA= Date: Wed, 10 Jul 2013 12:52:11 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1286556D@AZ-FFEXMB04.global.avaya.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 12:52:26 -0000 Hi Trevor, Thank you for the contribution.=20 How much meeting time would you need to spend on this? As a rule we would s= trongly prefer to avoid presentations, ask participants to come prepared wi= th the I-Ds already read, and use the precious meeting time for discussions= .=20 Regards, Dan > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > trevor.burbridge@bt.com > Sent: Friday, June 28, 2013 8:06 PM > To: acmorton@att.com; lmap@ietf.org > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin >=20 > We will be submitting a draft next week on the Information Model for > LMAP (draft-burbridge-lmap-information-model-00 forthcoming). >=20 > It would be good to have a slot to present this and discuss the > information that is necessary to pass to/from the Measurement Agents. > This will enable us to make good progress on revisions on the list for > this early deliverable and frame discussions on any > protocols/commands/interfaces. >=20 > Thanks, >=20 > Trevor. >=20 > Trevor Burbridge > Network Infrastructure & Innovation | BT Innovate & Design > Tel: 01473 645115 > Fax: 01473 640929 >=20 > This email contains BT information, which may be privileged or > confidential. It's meant only for the individual(s) or entity named > above. If you're not the intended recipient, note that disclosing, > copying, distributing or using this information is prohibited. If you've > received this email in error, please let me know immediately on the > email address above. Thank you. > We monitor our email system, and may record your emails. > British Telecommunications plc > Registered office: 81 Newgate Street London EC1A 7AJ Registered in > England no: 1800000 >=20 >=20 > >-----Original Message----- > >From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > >MORTON JR., ALFRED C (AL) > >Sent: 21 June 2013 16:29 > >To: lmap WG > >Subject: [lmap] Call for contributions and Agenda time in Berlin > > > >LMAP, > > > >Last week, the draft LMAP charter went forward for external review. > >LMAP has been approved for a placeholder session at IETF-87, either as > >a Working Group if approved in time, or as a BoF if not. > >If charter-related issues remain to be resolved, they will be discussed > >during the session with highest priority. > > > >However, ... > > > >If you have new/revised contributions covered under the current draft > >of the LMAP charter, please reply to the list identifying the topic, > >the filename(if applicable) and the related issues you would like to > >discuss at a face-to-face session, and try to do this by June 28, 2013. > > > >thanks and regards, > >Dan and Al > > > > > > > >> -----Original Message----- > >> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- > >> bounces@ietf.org] On Behalf Of The IESG > >> Sent: Friday, June 14, 2013 11:42 AM > >> To: IETF-Announce > >> Cc: lmap WG > >> Subject: WG Review: Large-Scale Measurement of Broadband Performance > >> (lmap) > >> > >> A new IETF working group has been proposed in the Operations and > >> Management Area. The IESG has not made any determination yet. The > >> following draft charter was submitted, and is provided for > >> informational purposes only. Please send your comments to the IESG > >> mailing list (iesg at ietf.org) by 2013-06-24. > >> > >> Large-Scale Measurement of Broadband Performance (lmap) > >> ------------------------------------------------ > >> Current Status: Proposed WG > >> > >> Assigned Area Director: > >> Benoit Claise > >>... > >_______________________________________________ > >lmap mailing list > >lmap@ietf.org > >https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From dromasca@avaya.com Wed Jul 10 05:52:59 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 921F211E8121 for ; Wed, 10 Jul 2013 05:52:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.099 X-Spam-Level: X-Spam-Status: No, score=-103.099 tagged_above=-999 required=5 tests=[AWL=-0.100, BAYES_00=-2.599, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w+pd+25yvM3B for ; Wed, 10 Jul 2013 05:52:52 -0700 (PDT) Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id B9ACC21F9E91 for ; Wed, 10 Jul 2013 05:52:51 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjIFAFZY3VGHCzI1/2dsb2JhbABagmghMk3BEIEPFnSCIwEBAQEDAQEBDwsdNAsMBAIBCA0EBAEBCxQJBycLFAkIAQEEAQ0FCBqHbQELngibQxMEjzIxBwaDA2wDnh2LBIMRgig X-IronPort-AV: E=Sophos;i="4.87,1036,1363147200"; d="scan'208";a="16023863" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by de307622-de-outbound.net.avaya.com with ESMTP; 10 Jul 2013 08:52:50 -0400 Received: from unknown (HELO AZ-FFEXHC04.global.avaya.com) ([135.64.58.14]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 10 Jul 2013 08:47:00 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC04.global.avaya.com ([135.64.58.14]) with mapi id 14.02.0328.009; Wed, 10 Jul 2013 08:52:47 -0400 From: "Romascanu, Dan (Dan)" To: Jan Seedorf , "MORTON JR., ALFRED C (AL)" , lmap WG Thread-Topic: Call for contributions and Agenda time in Berlin Thread-Index: Ac5ukbwsnWWTUZX3TBq4VwGNgR/klAFfNoPAAlduseA= Date: Wed, 10 Jul 2013 12:52:47 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1286557F@AZ-FFEXMB04.global.avaya.com> References: <2779C9F0771F974CAD742BAE6D9904FE55562C18@DAPHNIS.office.hd> In-Reply-To: <2779C9F0771F974CAD742BAE6D9904FE55562C18@DAPHNIS.office.hd> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Enrico Marocco , "Vijay K. Gurbani \(vkg@bell-labs.com\)" Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 12:52:59 -0000 Hi Jan, Thank you for the contribution.=20 How much meeting time would you need to spend on this? As a rule we would s= trongly prefer to avoid presentations, ask participants to come prepared wi= th the I-Ds already read, and use the precious meeting time for discussions= .=20 Regards, Dan > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Jan Seedorf > Sent: Friday, June 28, 2013 6:06 PM > To: MORTON JR., ALFRED C (AL); lmap WG > Cc: Enrico Marocco; Vijay K. Gurbani (vkg@bell-labs.com) > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin >=20 > Dear Al and Dan, >=20 > We are currently updating our draft on "using ALTO for Querying LMAP > Results" (draft-seedorf-lmap-alto). We are adding use cases, and will > post the -01 version soon. As discussed in Orlando and at the last BoF, > this draft is not about the core LMAP report protocol from MA to > collector, but about how to make LMAP results available to interested > parties at the collector. If possible, we would be happy to present our > ideas again in Berlin to get feedback if this is something folks regard > as useful or not. >=20 > Thanks, > Jan >=20 > > -----Original Message----- > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf > > Of MORTON JR., ALFRED C (AL) > > Sent: Friday, June 21, 2013 5:29 PM > > To: lmap WG > > Subject: [lmap] Call for contributions and Agenda time in Berlin > > > > LMAP, > > > > Last week, the draft LMAP charter went forward for external review. > > LMAP has been approved for a placeholder session at IETF-87, either as > > a Working Group if approved in time, or as a BoF if not. > > If charter-related issues remain to be resolved, they will be > > discussed during the session with highest priority. > > > > However, ... > > > > If you have new/revised contributions covered under the current draft > > of the LMAP charter, please reply to the list identifying the topic, > > the filename(if applicable) and the related issues you would like to > > discuss at a face-to-face session, and try to do this by June 28, > 2013. > > > > thanks and regards, > > Dan and Al > > > > > > > > > -----Original Message----- > > > From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- > > > bounces@ietf.org] On Behalf Of The IESG > > > Sent: Friday, June 14, 2013 11:42 AM > > > To: IETF-Announce > > > Cc: lmap WG > > > Subject: WG Review: Large-Scale Measurement of Broadband > > Performance > > > (lmap) > > > > > > A new IETF working group has been proposed in the Operations and > > > Management Area. The IESG has not made any determination yet. The > > > following draft charter was submitted, and is provided for > > > informational purposes only. Please send your comments to the IESG > > > mailing list (iesg at ietf.org) by 2013-06-24. > > > > > > Large-Scale Measurement of Broadband Performance (lmap) > > > ------------------------------------------------ > > > Current Status: Proposed WG > > > > > > Assigned Area Director: > > > Benoit Claise > > >... > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From dromasca@avaya.com Wed Jul 10 05:53:30 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 14D1E21F9E78 for ; Wed, 10 Jul 2013 05:53:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.086 X-Spam-Level: X-Spam-Status: No, score=-103.086 tagged_above=-999 required=5 tests=[AWL=-0.087, BAYES_00=-2.599, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zp8k1jDjb7tp for ; Wed, 10 Jul 2013 05:53:24 -0700 (PDT) Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id 0FE6721E8055 for ; Wed, 10 Jul 2013 05:53:19 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AmkKAFZY3VHGmAcV/2dsb2JhbABagmghMk2tDgaTfIEPFnSCIwEBAQEDAQEBDwsdNBcEAgEIDQQEAQELFAkHJwsUCQgBAQQBEggah20BC54Im0MXjzI4BoMDbAOZAIUdiwSDEYIo X-IronPort-AV: E=Sophos;i="4.87,1036,1363147200"; d="scan'208";a="16023953" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by de307622-de-outbound.net.avaya.com with ESMTP; 10 Jul 2013 08:53:17 -0400 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by co300216-co-erhwest-out.avaya.com with ESMTP; 10 Jul 2013 08:51:33 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Wed, 10 Jul 2013 08:53:15 -0400 From: "Romascanu, Dan (Dan)" To: "Huangyihong (Rachel)" , "MORTON JR., ALFRED C (AL)" , lmap WG Thread-Topic: Call for contributions and Agenda time in Berlin Thread-Index: Ac5ukbwsnWWTUZX3TBq4VwGNgR/klAFH7G2gAm69xiA= Date: Wed, 10 Jul 2013 12:53:15 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA12865594@AZ-FFEXMB04.global.avaya.com> References: <51E6A56BD6A85142B9D172C87FC3ABBB45836126@nkgeml501-mbs.china.huawei.com> In-Reply-To: <51E6A56BD6A85142B9D172C87FC3ABBB45836126@nkgeml501-mbs.china.huawei.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 12:53:30 -0000 Hi Rachel, Thank you for the contribution.=20 How much meeting time would you need to spend on this? As a rule we would s= trongly prefer to avoid presentations, ask participants to come prepared wi= th the I-Ds already read, and use the precious meeting time for discussions= .=20 Regards, Dan > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Huangyihong (Rachel) > Sent: Friday, June 28, 2013 1:13 PM > To: MORTON JR., ALFRED C (AL); lmap WG > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin >=20 > Hi all, >=20 > I'm new on this list. I have a contribution to discuss a new usage of > LMAP > http://tools.ietf.org/html/draft-huang-lmap-data-collection-use-case-00 > If anyone is interested in this, I'm happy to present this in the > meeting. >=20 > Best regards, > Rachel >=20 > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > MORTON JR., ALFRED C (AL) > Sent: Friday, June 21, 2013 11:29 PM > To: lmap WG > Subject: [lmap] Call for contributions and Agenda time in Berlin >=20 > LMAP, >=20 > Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87, either as a > Working Group if approved in time, or as a BoF if not. > If charter-related issues remain to be resolved, they will be discussed > during the session with highest priority. >=20 > However, ... >=20 > If you have new/revised contributions covered under the current draft of > the LMAP charter, please reply to the list identifying the topic, the > filename(if applicable) and the related issues you would like to discuss > at a face-to-face session, and try to do this by June 28, 2013. >=20 > thanks and regards, > Dan and Al >=20 >=20 >=20 > > -----Original Message----- > > From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- > > bounces@ietf.org] On Behalf Of The IESG > > Sent: Friday, June 14, 2013 11:42 AM > > To: IETF-Announce > > Cc: lmap WG > > Subject: WG Review: Large-Scale Measurement of Broadband Performance > > (lmap) > > > > A new IETF working group has been proposed in the Operations and > > Management Area. The IESG has not made any determination yet. The > > following draft charter was submitted, and is provided for > > informational purposes only. Please send your comments to the IESG > > mailing list (iesg at ietf.org) by 2013-06-24. > > > > Large-Scale Measurement of Broadband Performance (lmap) > > ------------------------------------------------ > > Current Status: Proposed WG > > > > Assigned Area Director: > > Benoit Claise > >... > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From Jan.Seedorf@neclab.eu Wed Jul 10 06:58:06 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C2B9E21F9E7B for ; Wed, 10 Jul 2013 06:58:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.074 X-Spam-Level: X-Spam-Status: No, score=-103.074 tagged_above=-999 required=5 tests=[AWL=-0.075, BAYES_00=-2.599, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P9DimMuqve2X for ; Wed, 10 Jul 2013 06:58:02 -0700 (PDT) Received: from mailer1.neclab.eu (mailer1.neclab.eu [195.37.70.40]) by ietfa.amsl.com (Postfix) with ESMTP id 0719821F85B3 for ; Wed, 10 Jul 2013 06:58:02 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mailer1.neclab.eu (Postfix) with ESMTP id 94D5A104A3E; Wed, 10 Jul 2013 15:57:28 +0200 (CEST) X-Virus-Scanned: Amavisd on Debian GNU/Linux (netlab.nec.de) Received: from mailer1.neclab.eu ([127.0.0.1]) by localhost (atlas-a.office.hd [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ac97GXb801bG; Wed, 10 Jul 2013 15:57:28 +0200 (CEST) Received: from ENCELADUS.office.hd (enceladus.office.hd [192.168.24.52]) by mailer1.neclab.eu (Postfix) with ESMTP id 7207E104A2C; Wed, 10 Jul 2013 15:57:03 +0200 (CEST) Received: from DAPHNIS.office.hd ([169.254.2.153]) by ENCELADUS.office.hd ([192.168.24.52]) with mapi id 14.01.0323.003; Wed, 10 Jul 2013 15:55:23 +0200 From: Jan Seedorf To: "Romascanu, Dan (Dan)" , "MORTON JR., ALFRED C (AL)" , lmap WG Thread-Topic: Call for contributions and Agenda time in Berlin Thread-Index: Ac5ukbwsnWWTUZX3TBq4VwGNgR/klAFfNoPAAlduseAAATtZ8A== Date: Wed, 10 Jul 2013 13:55:22 +0000 Message-ID: <2779C9F0771F974CAD742BAE6D9904FE55578EE8@DAPHNIS.office.hd> References: <2779C9F0771F974CAD742BAE6D9904FE55562C18@DAPHNIS.office.hd> <9904FB1B0159DA42B0B887B7FA8119CA1286557F@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1286557F@AZ-FFEXMB04.global.avaya.com> Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.1.5.89] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Enrico Marocco , "Vijay K. Gurbani \(vkg@bell-labs.com\)" Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 13:58:06 -0000 Hi Dan, We are also more interested in discussions and getting feedback/comments fr= om the WG. If we could get a 10 minute slot, I would try to briefly summari= ze our ideas in 2-3 slides so that we have most of those 10 minutes for act= ual discussion on these ideas. Would that work? Thanks, Jan > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Romascanu, Dan (Dan) > Sent: Wednesday, July 10, 2013 2:53 PM > To: Jan Seedorf; MORTON JR., ALFRED C (AL); lmap WG > Cc: Enrico Marocco; Vijay K. Gurbani (vkg@bell-labs.com) > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin >=20 > Hi Jan, >=20 > Thank you for the contribution. >=20 > How much meeting time would you need to spend on this? As a rule we > would strongly prefer to avoid presentations, ask participants to come > prepared with the I-Ds already read, and use the precious meeting time fo= r > discussions. >=20 > Regards, >=20 > Dan >=20 >=20 >=20 >=20 >=20 > > -----Original Message----- > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf > Of > > Jan Seedorf > > Sent: Friday, June 28, 2013 6:06 PM > > To: MORTON JR., ALFRED C (AL); lmap WG > > Cc: Enrico Marocco; Vijay K. Gurbani (vkg@bell-labs.com) > > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin > > > > Dear Al and Dan, > > > > We are currently updating our draft on "using ALTO for Querying LMAP > > Results" (draft-seedorf-lmap-alto). We are adding use cases, and will > > post the -01 version soon. As discussed in Orlando and at the last BoF, > > this draft is not about the core LMAP report protocol from MA to > > collector, but about how to make LMAP results available to interested > > parties at the collector. If possible, we would be happy to present our > > ideas again in Berlin to get feedback if this is something folks regard > > as useful or not. > > > > Thanks, > > Jan > > > > > -----Original Message----- > > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf > > > Of MORTON JR., ALFRED C (AL) > > > Sent: Friday, June 21, 2013 5:29 PM > > > To: lmap WG > > > Subject: [lmap] Call for contributions and Agenda time in Berlin > > > > > > LMAP, > > > > > > Last week, the draft LMAP charter went forward for external review. > > > LMAP has been approved for a placeholder session at IETF-87, either a= s > > > a Working Group if approved in time, or as a BoF if not. > > > If charter-related issues remain to be resolved, they will be > > > discussed during the session with highest priority. > > > > > > However, ... > > > > > > If you have new/revised contributions covered under the current draft > > > of the LMAP charter, please reply to the list identifying the topic, > > > the filename(if applicable) and the related issues you would like to > > > discuss at a face-to-face session, and try to do this by June 28, > > 2013. > > > > > > thanks and regards, > > > Dan and Al > > > > > > > > > > > > > -----Original Message----- > > > > From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- > > > > bounces@ietf.org] On Behalf Of The IESG > > > > Sent: Friday, June 14, 2013 11:42 AM > > > > To: IETF-Announce > > > > Cc: lmap WG > > > > Subject: WG Review: Large-Scale Measurement of Broadband > > > Performance > > > > (lmap) > > > > > > > > A new IETF working group has been proposed in the Operations and > > > > Management Area. The IESG has not made any determination yet. The > > > > following draft charter was submitted, and is provided for > > > > informational purposes only. Please send your comments to the IESG > > > > mailing list (iesg at ietf.org) by 2013-06-24. > > > > > > > > Large-Scale Measurement of Broadband Performance (lmap) > > > > ------------------------------------------------ > > > > Current Status: Proposed WG > > > > > > > > Assigned Area Director: > > > > Benoit Claise > > > >... > > > _______________________________________________ > > > lmap mailing list > > > lmap@ietf.org > > > https://www.ietf.org/mailman/listinfo/lmap > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From dromasca@avaya.com Wed Jul 10 07:00:40 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6E68721F8FAF for ; Wed, 10 Jul 2013 07:00:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.077 X-Spam-Level: X-Spam-Status: No, score=-103.077 tagged_above=-999 required=5 tests=[AWL=-0.078, BAYES_00=-2.599, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lO4nsA5rTpTG for ; Wed, 10 Jul 2013 07:00:35 -0700 (PDT) Received: from co300216-co-outbound.net.avaya.com (co300216-co-outbound.net.avaya.com [198.152.13.100]) by ietfa.amsl.com (Postfix) with ESMTP id A6F3A21F8F4F for ; Wed, 10 Jul 2013 07:00:34 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgUFAAH/ZlGHCzI1/2dsb2JhbABQgmUhNsFIgQcWdIIfAQEBAQIBAQEBDwsdNAsMBAIBCA0EBAEBCxQJBycLFAkIAQEEAQ0FCBqHbAYBC6A8nQwTBI5lJgsHBoJaYQOTNolzimiDC4Io X-IronPort-AV: E=Sophos;i="4.87,456,1363147200"; d="scan'208";a="18965000" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by co300216-co-outbound.net.avaya.com with ESMTP; 10 Jul 2013 10:00:33 -0400 Received: from unknown (HELO AZ-FFEXHC04.global.avaya.com) ([135.64.58.14]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 10 Jul 2013 09:54:44 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC04.global.avaya.com ([135.64.58.14]) with mapi id 14.02.0328.009; Wed, 10 Jul 2013 10:00:31 -0400 From: "Romascanu, Dan (Dan)" To: Jan Seedorf , "MORTON JR., ALFRED C (AL)" , lmap WG Thread-Topic: Call for contributions and Agenda time in Berlin Thread-Index: Ac5ukbwsnWWTUZX3TBq4VwGNgR/klAFfNoPAAlduseAAATtZ8AABF0Yg Date: Wed, 10 Jul 2013 14:00:31 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA12866C18@AZ-FFEXMB04.global.avaya.com> References: <2779C9F0771F974CAD742BAE6D9904FE55562C18@DAPHNIS.office.hd> <9904FB1B0159DA42B0B887B7FA8119CA1286557F@AZ-FFEXMB04.global.avaya.com> <2779C9F0771F974CAD742BAE6D9904FE55578EE8@DAPHNIS.office.hd> In-Reply-To: <2779C9F0771F974CAD742BAE6D9904FE55578EE8@DAPHNIS.office.hd> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Enrico Marocco , "Weil, Jason" Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 14:00:40 -0000 Hi Jan, Yes, I believe that 10 min would work, but we will know this more exactly a= fter I will sit with Jason next week in order to build the exact agenda.=20 Regards, Dan > -----Original Message----- > From: Jan Seedorf [mailto:Jan.Seedorf@neclab.eu] > Sent: Wednesday, July 10, 2013 4:55 PM > To: Romascanu, Dan (Dan); MORTON JR., ALFRED C (AL); lmap WG > Cc: Enrico Marocco; Vijay K. Gurbani (vkg@bell-labs.com) > Subject: RE: Call for contributions and Agenda time in Berlin >=20 > Hi Dan, >=20 > We are also more interested in discussions and getting feedback/comments > from the WG. If we could get a 10 minute slot, I would try to briefly > summarize our ideas in 2-3 slides so that we have most of those 10 > minutes for actual discussion on these ideas. Would that work? >=20 > Thanks, > Jan >=20 > > -----Original Message----- > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf > > Of Romascanu, Dan (Dan) > > Sent: Wednesday, July 10, 2013 2:53 PM > > To: Jan Seedorf; MORTON JR., ALFRED C (AL); lmap WG > > Cc: Enrico Marocco; Vijay K. Gurbani (vkg@bell-labs.com) > > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin > > > > Hi Jan, > > > > Thank you for the contribution. > > > > How much meeting time would you need to spend on this? As a rule we > > would strongly prefer to avoid presentations, ask participants to come > > prepared with the I-Ds already read, and use the precious meeting time > > for discussions. > > > > Regards, > > > > Dan > > > > > > > > > > > > > -----Original Message----- > > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf > > Of > > > Jan Seedorf > > > Sent: Friday, June 28, 2013 6:06 PM > > > To: MORTON JR., ALFRED C (AL); lmap WG > > > Cc: Enrico Marocco; Vijay K. Gurbani (vkg@bell-labs.com) > > > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin > > > > > > Dear Al and Dan, > > > > > > We are currently updating our draft on "using ALTO for Querying LMAP > > > Results" (draft-seedorf-lmap-alto). We are adding use cases, and > > > will post the -01 version soon. As discussed in Orlando and at the > > > last BoF, this draft is not about the core LMAP report protocol from > > > MA to collector, but about how to make LMAP results available to > > > interested parties at the collector. If possible, we would be happy > > > to present our ideas again in Berlin to get feedback if this is > > > something folks regard as useful or not. > > > > > > Thanks, > > > Jan > > > > > > > -----Original Message----- > > > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On > > > > Behalf Of MORTON JR., ALFRED C (AL) > > > > Sent: Friday, June 21, 2013 5:29 PM > > > > To: lmap WG > > > > Subject: [lmap] Call for contributions and Agenda time in Berlin > > > > > > > > LMAP, > > > > > > > > Last week, the draft LMAP charter went forward for external > review. > > > > LMAP has been approved for a placeholder session at IETF-87, > > > > either as a Working Group if approved in time, or as a BoF if not. > > > > If charter-related issues remain to be resolved, they will be > > > > discussed during the session with highest priority. > > > > > > > > However, ... > > > > > > > > If you have new/revised contributions covered under the current > > > > draft of the LMAP charter, please reply to the list identifying > > > > the topic, the filename(if applicable) and the related issues you > > > > would like to discuss at a face-to-face session, and try to do > > > > this by June 28, > > > 2013. > > > > > > > > thanks and regards, > > > > Dan and Al > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- > > > > > bounces@ietf.org] On Behalf Of The IESG > > > > > Sent: Friday, June 14, 2013 11:42 AM > > > > > To: IETF-Announce > > > > > Cc: lmap WG > > > > > Subject: WG Review: Large-Scale Measurement of Broadband > > > > Performance > > > > > (lmap) > > > > > > > > > > A new IETF working group has been proposed in the Operations and > > > > > Management Area. The IESG has not made any determination yet. > > > > > The following draft charter was submitted, and is provided for > > > > > informational purposes only. Please send your comments to the > > > > > IESG mailing list (iesg at ietf.org) by 2013-06-24. > > > > > > > > > > Large-Scale Measurement of Broadband Performance (lmap) > > > > > ------------------------------------------------ > > > > > Current Status: Proposed WG > > > > > > > > > > Assigned Area Director: > > > > > Benoit Claise ... > > > > _______________________________________________ > > > > lmap mailing list > > > > lmap@ietf.org > > > > https://www.ietf.org/mailman/listinfo/lmap > > > _______________________________________________ > > > lmap mailing list > > > lmap@ietf.org > > > https://www.ietf.org/mailman/listinfo/lmap > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap From trevor.burbridge@bt.com Wed Jul 10 07:18:39 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 99EFF21E805A for ; Wed, 10 Jul 2013 07:18:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.324 X-Spam-Level: X-Spam-Status: No, score=-2.324 tagged_above=-999 required=5 tests=[AWL=-0.525, BAYES_00=-2.599, J_CHICKENPOX_21=0.6, J_CHICKENPOX_82=0.6, J_CHICKENPOX_91=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aXTapgDyEWz4 for ; Wed, 10 Jul 2013 07:18:35 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id DB35321E8055 for ; Wed, 10 Jul 2013 07:18:34 -0700 (PDT) Received: from EVMHT65-UKRD.domain1.systemhost.net (10.36.3.102) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.298.1; Wed, 10 Jul 2013 15:18:27 +0100 Received: from EMV64-UKRD.domain1.systemhost.net ([169.254.1.127]) by EVMHT65-UKRD.domain1.systemhost.net ([10.36.3.102]) with mapi; Wed, 10 Jul 2013 15:18:27 +0100 From: To: , Date: Wed, 10 Jul 2013 15:18:26 +0100 Thread-Topic: Call for contributions and Agenda time in Berlin Thread-Index: Ac5ukbwsnWWTUZX3TBq4VwGNgR/klAFjyywwAlLUZqAAAuiegA== Message-ID: References: <9904FB1B0159DA42B0B887B7FA8119CA1286556D@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1286556D@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 14:18:39 -0000 I think at this early stage it is critical that we make some progress and g= et consensus on both the Framework and the Information Model. If everyone i= s agreed around these two things it should stop technical protocol work etc= . going off in different incompatible directions. Therefore I suggest we spend 30mins on the Information Model if possible: 1= 0 mins to present and 20 mins to discuss - we will capture the points comin= g up on this list and our own questions for the discussion section, as well= as any further points on the day. Trevor. >-----Original Message----- >From: Romascanu, Dan (Dan) [mailto:dromasca@avaya.com] >Sent: 10 July 2013 13:52 >To: Burbridge,T,Trevor,TUB8 R; lmap@ietf.org >Subject: RE: Call for contributions and Agenda time in Berlin > >Hi Trevor, > >Thank you for the contribution. > >How much meeting time would you need to spend on this? As a rule we >would strongly prefer to avoid presentations, ask participants to come >prepared with the I-Ds already read, and use the precious meeting time for >discussions. > >Regards, > >Dan > > > > > >> -----Original Message----- >> From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf >> Of trevor.burbridge@bt.com >> Sent: Friday, June 28, 2013 8:06 PM >> To: acmorton@att.com; lmap@ietf.org >> Subject: Re: [lmap] Call for contributions and Agenda time in Berlin >> >> We will be submitting a draft next week on the Information Model for >> LMAP (draft-burbridge-lmap-information-model-00 forthcoming). >> >> It would be good to have a slot to present this and discuss the >> information that is necessary to pass to/from the Measurement Agents. >> This will enable us to make good progress on revisions on the list for >> this early deliverable and frame discussions on any >> protocols/commands/interfaces. >> >> Thanks, >> >> Trevor. >> >> Trevor Burbridge >> Network Infrastructure & Innovation | BT Innovate & Design >> Tel: 01473 645115 >> Fax: 01473 640929 >> >> This email contains BT information, which may be privileged or >> confidential. It's meant only for the individual(s) or entity named >> above. If you're not the intended recipient, note that disclosing, >> copying, distributing or using this information is prohibited. If >> you've received this email in error, please let me know immediately on >> the email address above. Thank you. >> We monitor our email system, and may record your emails. >> British Telecommunications plc >> Registered office: 81 Newgate Street London EC1A 7AJ Registered in >> England no: 1800000 >> >> >> >-----Original Message----- >> >From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf >> >Of MORTON JR., ALFRED C (AL) >> >Sent: 21 June 2013 16:29 >> >To: lmap WG >> >Subject: [lmap] Call for contributions and Agenda time in Berlin >> > >> >LMAP, >> > >> >Last week, the draft LMAP charter went forward for external review. >> >LMAP has been approved for a placeholder session at IETF-87, either >> >as a Working Group if approved in time, or as a BoF if not. >> >If charter-related issues remain to be resolved, they will be >> >discussed during the session with highest priority. >> > >> >However, ... >> > >> >If you have new/revised contributions covered under the current draft >> >of the LMAP charter, please reply to the list identifying the topic, >> >the filename(if applicable) and the related issues you would like to >> >discuss at a face-to-face session, and try to do this by June 28, 2013. >> > >> >thanks and regards, >> >Dan and Al >> > >> > >> > >> >> -----Original Message----- >> >> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- >> >> bounces@ietf.org] On Behalf Of The IESG >> >> Sent: Friday, June 14, 2013 11:42 AM >> >> To: IETF-Announce >> >> Cc: lmap WG >> >> Subject: WG Review: Large-Scale Measurement of Broadband >> >> Performance >> >> (lmap) >> >> >> >> A new IETF working group has been proposed in the Operations and >> >> Management Area. The IESG has not made any determination yet. The >> >> following draft charter was submitted, and is provided for >> >> informational purposes only. Please send your comments to the IESG >> >> mailing list (iesg at ietf.org) by 2013-06-24. >> >> >> >> Large-Scale Measurement of Broadband Performance (lmap) >> >> ------------------------------------------------ >> >> Current Status: Proposed WG >> >> >> >> Assigned Area Director: >> >> Benoit Claise ... >> >_______________________________________________ >> >lmap mailing list >> >lmap@ietf.org >> >https://www.ietf.org/mailman/listinfo/lmap >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap From paitken@cisco.com Wed Jul 10 09:08:37 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BE46F21F9D49 for ; Wed, 10 Jul 2013 09:08:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mopogPCMktuJ for ; Wed, 10 Jul 2013 09:08:32 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id 2B44421F9DC2 for ; Wed, 10 Jul 2013 09:08:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=679; q=dns/txt; s=iport; t=1373472511; x=1374682111; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=jsFHiazvHgLH9Gh24Bo5uSIxmhEqYQLu5M6PBi0QD6c=; b=myhqVXeGwONFgSgWM9sz3rjdgM0VKGgduMdCm1QrytsXK9DXaXTqsX/a zxJV0+whaNASCYkiP1Huq5iBJufKRTlFfBCQxjhDhp42TLGATDaWSnUYt wxc6K60yYoxwFPe91K3H4CN/WXP8nxtcYczNgP1uJmGRsGaUxnm4b8iWr s=; X-IronPort-AV: E=Sophos;i="4.87,1036,1363132800"; d="scan'208";a="15066121" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-3.cisco.com with ESMTP; 10 Jul 2013 16:08:30 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id r6AG8Src024134 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 10 Jul 2013 16:08:28 GMT Received: from [144.254.153.36] (dhcp-144-254-153-36.cisco.com [144.254.153.36]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6AG8RBp011243; Wed, 10 Jul 2013 17:08:27 +0100 (BST) Message-ID: <51DD86FC.1040605@cisco.com> Date: Wed, 10 Jul 2013 17:08:28 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: "Romascanu, Dan (Dan)" References: <51DD4E33.5000101@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1286526E@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1286526E@AZ-FFEXMB04.global.avaya.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "lmap-chairs@tools.ietf.org" , Aamer Akhter , "lmap@ietf.org" Subject: Re: [lmap] LMAP framework draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 16:08:37 -0000 Dan, > How much meeting time would you need to spend on this? As a rule we would strongly prefer to avoid presentations, ask participants to come prepared with the I-Ds already read, and use the precious meeting time for discussions. Generally, I would agree. However in this case, this draft would be a good starting point for the discussion. Even if we had not written this draft, it would be worthwhile considering what the necessary building blocks are for LMAP, what we already have, and what's missing so we can focus WG attention on the gaps. I could present the draft in 10 minutes. However, the framework discussion should be longer. Thanks, P. From j.schoenwaelder@jacobs-university.de Wed Jul 10 09:19:40 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B9E7C21F9FDF for ; Wed, 10 Jul 2013 09:19:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.104 X-Spam-Level: X-Spam-Status: No, score=-103.104 tagged_above=-999 required=5 tests=[AWL=0.145, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XG7shykDfvMr for ; Wed, 10 Jul 2013 09:19:35 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 8A6A621F9EDA for ; Wed, 10 Jul 2013 09:19:35 -0700 (PDT) Received: from localhost (demetrius1.jacobs-university.de [212.201.44.46]) by hermes.jacobs-university.de (Postfix) with ESMTP id 57F2620BDC; Wed, 10 Jul 2013 18:19:34 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius1.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id jwn2MINwRq0H; Wed, 10 Jul 2013 18:19:34 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 90C3620BE4; Wed, 10 Jul 2013 18:19:33 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id D538327315FE; Wed, 10 Jul 2013 18:19:30 +0200 (CEST) Date: Wed, 10 Jul 2013 18:19:30 +0200 From: Juergen Schoenwaelder To: Paul Aitken Message-ID: <20130710161930.GA65045@elstar.local> Mail-Followup-To: Paul Aitken , "Romascanu, Dan (Dan)" , "lmap-chairs@tools.ietf.org" , Aamer Akhter , "lmap@ietf.org" References: <51DD4E33.5000101@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1286526E@AZ-FFEXMB04.global.avaya.com> <51DD86FC.1040605@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51DD86FC.1040605@cisco.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "Romascanu, Dan \(Dan\)" , "lmap-chairs@tools.ietf.org" , Aamer Akhter , "lmap@ietf.org" Subject: Re: [lmap] LMAP framework draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 16:19:40 -0000 On Wed, Jul 10, 2013 at 05:08:28PM +0100, Paul Aitken wrote: > Dan, > > >How much meeting time would you need to spend on this? As a rule we would strongly prefer to avoid presentations, ask participants to come prepared with the I-Ds already read, and use the precious meeting time for discussions. > > Generally, I would agree. However in this case, this draft would be > a good starting point for the discussion. > > Even if we had not written this draft, it would be worthwhile > considering what the necessary building blocks are for LMAP, what we > already have, and what's missing so we can focus WG attention on the > gaps. > > I could present the draft in 10 minutes. However, the framework > discussion should be longer. I guess it is too early to ask but how much different is this I-D going to be from draft-eardley-lmap-framework-01.txt? /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From coverdale@sympatico.ca Wed Jul 10 16:25:26 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2037E21F9D31 for ; Wed, 10 Jul 2013 16:25:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.495 X-Spam-Level: X-Spam-Status: No, score=-1.495 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, MSGID_FROM_MTA_HEADER=0.803] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Oi5lz5Zwpt8H for ; Wed, 10 Jul 2013 16:25:19 -0700 (PDT) Received: from blu0-omc1-s16.blu0.hotmail.com (blu0-omc1-s16.blu0.hotmail.com [65.55.116.27]) by ietfa.amsl.com (Postfix) with ESMTP id B030E21F9D15 for ; Wed, 10 Jul 2013 16:25:19 -0700 (PDT) Received: from BLU0-SMTP6 ([65.55.116.8]) by blu0-omc1-s16.blu0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 10 Jul 2013 16:24:44 -0700 X-EIP: [aSE97CEb5if8hKhylqne3/lNaGWmOWnz] X-Originating-Email: [coverdale@sympatico.ca] Message-ID: Received: from PaulNewPC ([70.26.42.175]) by BLU0-SMTP6.phx.gbl over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Wed, 10 Jul 2013 16:24:41 -0700 From: Paul Coverdale To: "'MORTON JR., ALFRED C \(AL\)'" , =?iso-8859-1?Q?'Javier_Bustos_Jim=E9nez'?= References: <581E085DEB6A444093CDAEA4C4EE48181359116D@xmb-rcd-x08.cisco.com> In-Reply-To: Date: Wed, 10 Jul 2013 19:24:36 -0400 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_00AE_01CE7DA3.1F3150F0" X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Ac58+86nxZUUknX5T2yq3ePRFmTb4wAaa8CAABeQKtA= Content-Language: en-us X-OriginalArrivalTime: 10 Jul 2013 23:24:41.0759 (UTC) FILETIME=[A78052F0:01CE7DC4] Cc: 'lmap WG' Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 23:25:26 -0000 ------=_NextPart_000_00AE_01CE7DA3.1F3150F0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Javier wrote: 1.- I don't know if we should use the term Quality of Experience, because that is related more to the user's expectatives and context and, as far as I know, the proposed infrastructure doesn't measure that. Quality of Service (QoS), that is more related to technical metrics, could be better. Al wrote: +1.0 I've made this comment previously in connection with LMAP. We haven't discussed any perceptual estimation measurements. We have been talking about Network Performance and QoS. There's no doubt that making measurements is a growing part of user experience, but we have yet to establish how this influences user's QoE w.r.t. an application and context... Al I agree with Al. Is it in the scope of LMAP to define the link between network measurements and QoE? ..Paul ------=_NextPart_000_00AE_01CE7DA3.1F3150F0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

 

Javier wrote:

1.- I = don't know if we should use the term Quality of Experience, because that = is related more to the user's expectatives and context and, as far as I = know, the proposed infrastructure doesn't measure that. Quality of = Service (QoS), that is more related to technical metrics, could be = better.

 

Al = wrote:

+1.0  =

I've made this = comment previously in connection with LMAP.

We haven't discussed any perceptual estimation = measurements.

We have been = talking about Network Performance and QoS.

There's no doubt that making measurements is a growing = part

of user experience, = but we have yet to establish how this

influences user's QoE w.r.t. an application and = context...

 

Al

 

I agree with Al. Is it in the scope of LMAP to define the link = between network measurements and QoE?

 

..Paul

------=_NextPart_000_00AE_01CE7DA3.1F3150F0-- From rachel.huang@huawei.com Wed Jul 10 18:14:26 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 193B611E80E7 for ; Wed, 10 Jul 2013 18:14:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.149 X-Spam-Level: X-Spam-Status: No, score=-6.149 tagged_above=-999 required=5 tests=[AWL=-0.150, BAYES_00=-2.599, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VjK88a5V340A for ; Wed, 10 Jul 2013 18:14:21 -0700 (PDT) Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) by ietfa.amsl.com (Postfix) with ESMTP id 53F5511E80D1 for ; Wed, 10 Jul 2013 18:14:20 -0700 (PDT) Received: from 172.18.7.190 (EHLO lhreml203-edg.china.huawei.com) ([172.18.7.190]) by lhrrg02-dlp.huawei.com (MOS 4.3.5-GA FastPath queued) with ESMTP id ATI12443; Thu, 11 Jul 2013 01:13:47 +0000 (GMT) Received: from LHREML403-HUB.china.huawei.com (10.201.5.217) by lhreml203-edg.huawei.com (172.18.7.221) with Microsoft SMTP Server (TLS) id 14.1.323.7; Thu, 11 Jul 2013 02:13:07 +0100 Received: from NKGEML404-HUB.china.huawei.com (10.98.56.35) by lhreml403-hub.china.huawei.com (10.201.5.217) with Microsoft SMTP Server (TLS) id 14.1.323.7; Thu, 11 Jul 2013 02:13:41 +0100 Received: from NKGEML501-MBS.china.huawei.com ([169.254.2.43]) by nkgeml404-hub.china.huawei.com ([10.98.56.35]) with mapi id 14.01.0323.007; Thu, 11 Jul 2013 09:13:36 +0800 From: "Huangyihong (Rachel)" To: "Romascanu, Dan (Dan)" , "MORTON JR., ALFRED C (AL)" , lmap WG Thread-Topic: Call for contributions and Agenda time in Berlin Thread-Index: Ac5ukbwsnWWTUZX3TBq4VwGNgR/klAFH7G2gAm69xiAAGaB4gA== Date: Thu, 11 Jul 2013 01:13:36 +0000 Message-ID: <51E6A56BD6A85142B9D172C87FC3ABBB45840FC4@nkgeml501-mbs.china.huawei.com> References: <51E6A56BD6A85142B9D172C87FC3ABBB45836126@nkgeml501-mbs.china.huawei.com> <9904FB1B0159DA42B0B887B7FA8119CA12865594@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA12865594@AZ-FFEXMB04.global.avaya.com> Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.138.41.104] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jul 2013 01:14:26 -0000 Hi Dan, I think 10 minutes will be enough. Actually, it's a small document. I can b= riefly introduce it in 4 or 5 minutes and then leave it for open discussion= . Best regards, Rachel -----Original Message----- From: Romascanu, Dan (Dan) [mailto:dromasca@avaya.com]=20 Sent: Wednesday, July 10, 2013 8:53 PM To: Huangyihong (Rachel); MORTON JR., ALFRED C (AL); lmap WG Subject: RE: Call for contributions and Agenda time in Berlin Hi Rachel, Thank you for the contribution.=20 How much meeting time would you need to spend on this? As a rule we would s= trongly prefer to avoid presentations, ask participants to come prepared wi= th the I-Ds already read, and use the precious meeting time for discussions= .=20 Regards, Dan > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Huangyihong (Rachel) > Sent: Friday, June 28, 2013 1:13 PM > To: MORTON JR., ALFRED C (AL); lmap WG > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin >=20 > Hi all, >=20 > I'm new on this list. I have a contribution to discuss a new usage of > LMAP > http://tools.ietf.org/html/draft-huang-lmap-data-collection-use-case-00 > If anyone is interested in this, I'm happy to present this in the > meeting. >=20 > Best regards, > Rachel >=20 > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > MORTON JR., ALFRED C (AL) > Sent: Friday, June 21, 2013 11:29 PM > To: lmap WG > Subject: [lmap] Call for contributions and Agenda time in Berlin >=20 > LMAP, >=20 > Last week, the draft LMAP charter went forward for external review. > LMAP has been approved for a placeholder session at IETF-87, either as a > Working Group if approved in time, or as a BoF if not. > If charter-related issues remain to be resolved, they will be discussed > during the session with highest priority. >=20 > However, ... >=20 > If you have new/revised contributions covered under the current draft of > the LMAP charter, please reply to the list identifying the topic, the > filename(if applicable) and the related issues you would like to discuss > at a face-to-face session, and try to do this by June 28, 2013. >=20 > thanks and regards, > Dan and Al >=20 >=20 >=20 > > -----Original Message----- > > From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- > > bounces@ietf.org] On Behalf Of The IESG > > Sent: Friday, June 14, 2013 11:42 AM > > To: IETF-Announce > > Cc: lmap WG > > Subject: WG Review: Large-Scale Measurement of Broadband Performance > > (lmap) > > > > A new IETF working group has been proposed in the Operations and > > Management Area. The IESG has not made any determination yet. The > > following draft charter was submitted, and is provided for > > informational purposes only. Please send your comments to the IESG > > mailing list (iesg at ietf.org) by 2013-06-24. > > > > Large-Scale Measurement of Broadband Performance (lmap) > > ------------------------------------------------ > > Current Status: Proposed WG > > > > Assigned Area Director: > > Benoit Claise > >... > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From Jan.Seedorf@neclab.eu Thu Jul 11 01:37:18 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B8CE21F9E68 for ; Thu, 11 Jul 2013 01:37:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.059 X-Spam-Level: X-Spam-Status: No, score=-103.059 tagged_above=-999 required=5 tests=[AWL=-0.060, BAYES_00=-2.599, J_CHICKENPOX_82=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5K9Dhj0qc7yf for ; Thu, 11 Jul 2013 01:37:14 -0700 (PDT) Received: from mailer1.neclab.eu (mailer1.neclab.eu [195.37.70.40]) by ietfa.amsl.com (Postfix) with ESMTP id 1FE1621F9E7B for ; Thu, 11 Jul 2013 01:37:14 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mailer1.neclab.eu (Postfix) with ESMTP id 2CF10104A4F; Thu, 11 Jul 2013 10:36:40 +0200 (CEST) X-Virus-Scanned: Amavisd on Debian GNU/Linux (netlab.nec.de) Received: from mailer1.neclab.eu ([127.0.0.1]) by localhost (atlas-a.office.hd [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tPH6E8u56xGH; Thu, 11 Jul 2013 10:36:40 +0200 (CEST) Received: from ENCELADUS.office.hd (enceladus.office.hd [192.168.24.52]) by mailer1.neclab.eu (Postfix) with ESMTP id 09B24104A43; Thu, 11 Jul 2013 10:36:10 +0200 (CEST) Received: from DAPHNIS.office.hd ([169.254.2.153]) by ENCELADUS.office.hd ([192.168.24.52]) with mapi id 14.01.0323.003; Thu, 11 Jul 2013 10:34:36 +0200 From: Jan Seedorf To: lmap WG Thread-Topic: New version of draft on "ALTO for Querying LMAP Results" Thread-Index: Ac5+ETBaGuWZjzYXT++JlEeZotguSw== Date: Thu, 11 Jul 2013 08:34:35 +0000 Message-ID: <2779C9F0771F974CAD742BAE6D9904FE55584445@DAPHNIS.office.hd> Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.1.5.89] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "Romascanu, Dan \(Dan\)" , Enrico Marocco , "Vijay K. Gurbani \(vkg@bell-labs.com\)" , "MORTON JR., ALFRED C \(AL\)" , "Weil, Jason" Subject: [lmap] New version of draft on "ALTO for Querying LMAP Results" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jul 2013 08:37:18 -0000 Dear all, We have just posted the -01 version of "draft-seedorf-lmap-alto" (see http:= //tools.ietf.org/html/draft-seedorf-lmap-alto-01). The draft contains a new= use case and has been polished. The general idea is to use ALTO for queryi= ng LMAP result (kind of automatically by applications). Comments are welcom= e. Hopefully, we can get some feedback and have a short discussion on our i= deas at the Berlin meeting (see below, we have requested 10 min agenda time= for the document). - Jan > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Romascanu, Dan (Dan) > Sent: Wednesday, July 10, 2013 4:01 PM > To: Jan Seedorf; MORTON JR., ALFRED C (AL); lmap WG > Cc: Enrico Marocco; Weil, Jason > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin >=20 > Hi Jan, >=20 > Yes, I believe that 10 min would work, but we will know this more exactly > after I will sit with Jason next week in order to build the exact agenda. >=20 > Regards, >=20 > Dan >=20 >=20 >=20 >=20 > > -----Original Message----- > > From: Jan Seedorf [mailto:Jan.Seedorf@neclab.eu] > > Sent: Wednesday, July 10, 2013 4:55 PM > > To: Romascanu, Dan (Dan); MORTON JR., ALFRED C (AL); lmap WG > > Cc: Enrico Marocco; Vijay K. Gurbani (vkg@bell-labs.com) > > Subject: RE: Call for contributions and Agenda time in Berlin > > > > Hi Dan, > > > > We are also more interested in discussions and getting > feedback/comments > > from the WG. If we could get a 10 minute slot, I would try to briefly > > summarize our ideas in 2-3 slides so that we have most of those 10 > > minutes for actual discussion on these ideas. Would that work? > > > > Thanks, > > Jan > > > > > -----Original Message----- > > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf > > > Of Romascanu, Dan (Dan) > > > Sent: Wednesday, July 10, 2013 2:53 PM > > > To: Jan Seedorf; MORTON JR., ALFRED C (AL); lmap WG > > > Cc: Enrico Marocco; Vijay K. Gurbani (vkg@bell-labs.com) > > > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin > > > > > > Hi Jan, > > > > > > Thank you for the contribution. > > > > > > How much meeting time would you need to spend on this? As a rule we > > > would strongly prefer to avoid presentations, ask participants to com= e > > > prepared with the I-Ds already read, and use the precious meeting tim= e > > > for discussions. > > > > > > Regards, > > > > > > Dan > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On > Behalf > > > Of > > > > Jan Seedorf > > > > Sent: Friday, June 28, 2013 6:06 PM > > > > To: MORTON JR., ALFRED C (AL); lmap WG > > > > Cc: Enrico Marocco; Vijay K. Gurbani (vkg@bell-labs.com) > > > > Subject: Re: [lmap] Call for contributions and Agenda time in Berli= n > > > > > > > > Dear Al and Dan, > > > > > > > > We are currently updating our draft on "using ALTO for Querying LMA= P > > > > Results" (draft-seedorf-lmap-alto). We are adding use cases, and > > > > will post the -01 version soon. As discussed in Orlando and at the > > > > last BoF, this draft is not about the core LMAP report protocol fro= m > > > > MA to collector, but about how to make LMAP results available to > > > > interested parties at the collector. If possible, we would be happy > > > > to present our ideas again in Berlin to get feedback if this is > > > > something folks regard as useful or not. > > > > > > > > Thanks, > > > > Jan > > > > > > > > > -----Original Message----- > > > > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On > > > > > Behalf Of MORTON JR., ALFRED C (AL) > > > > > Sent: Friday, June 21, 2013 5:29 PM > > > > > To: lmap WG > > > > > Subject: [lmap] Call for contributions and Agenda time in Berlin > > > > > > > > > > LMAP, > > > > > > > > > > Last week, the draft LMAP charter went forward for external > > review. > > > > > LMAP has been approved for a placeholder session at IETF-87, > > > > > either as a Working Group if approved in time, or as a BoF if not= . > > > > > If charter-related issues remain to be resolved, they will be > > > > > discussed during the session with highest priority. > > > > > > > > > > However, ... > > > > > > > > > > If you have new/revised contributions covered under the current > > > > > draft of the LMAP charter, please reply to the list identifying > > > > > the topic, the filename(if applicable) and the related issues you > > > > > would like to discuss at a face-to-face session, and try to do > > > > > this by June 28, > > > > 2013. > > > > > > > > > > thanks and regards, > > > > > Dan and Al > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- > > > > > > bounces@ietf.org] On Behalf Of The IESG > > > > > > Sent: Friday, June 14, 2013 11:42 AM > > > > > > To: IETF-Announce > > > > > > Cc: lmap WG > > > > > > Subject: WG Review: Large-Scale Measurement of Broadband > > > > > Performance > > > > > > (lmap) > > > > > > > > > > > > A new IETF working group has been proposed in the Operations an= d > > > > > > Management Area. The IESG has not made any determination yet. > > > > > > The following draft charter was submitted, and is provided for > > > > > > informational purposes only. Please send your comments to the > > > > > > IESG mailing list (iesg at ietf.org) by 2013-06-24. > > > > > > > > > > > > Large-Scale Measurement of Broadband Performance (lmap) > > > > > > ------------------------------------------------ > > > > > > Current Status: Proposed WG > > > > > > > > > > > > Assigned Area Director: > > > > > > Benoit Claise ... > > > > > _______________________________________________ > > > > > lmap mailing list > > > > > lmap@ietf.org > > > > > https://www.ietf.org/mailman/listinfo/lmap > > > > _______________________________________________ > > > > lmap mailing list > > > > lmap@ietf.org > > > > https://www.ietf.org/mailman/listinfo/lmap > > > _______________________________________________ > > > lmap mailing list > > > lmap@ietf.org > > > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From dromasca@avaya.com Thu Jul 11 02:20:04 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7642C21F9F8A for ; Thu, 11 Jul 2013 02:20:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.89 X-Spam-Level: X-Spam-Status: No, score=-102.89 tagged_above=-999 required=5 tests=[AWL=-0.291, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HTi2vB+9X8JK for ; Thu, 11 Jul 2013 02:19:58 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id AE2E421F9F58 for ; Thu, 11 Jul 2013 02:19:54 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArEIAKR33lGHCzI1/2dsb2JhbABagmghMk2tTZQDgQkWdIIjAQEBAQMSKD8MBAIBCA0BAgEEAQEBChQJBzIUCQgCBAENBQgah20Bmxqbe48wMQcGgwNsA5kDhR2LBIMRgig X-IronPort-AV: E=Sophos;i="4.87,1042,1363147200"; d="scan'208";a="19647082" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 11 Jul 2013 05:19:52 -0400 Received: from unknown (HELO AZ-FFEXHC02.global.avaya.com) ([135.64.58.12]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 11 Jul 2013 05:14:01 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC02.global.avaya.com ([135.64.58.12]) with mapi id 14.02.0328.009; Thu, 11 Jul 2013 11:19:50 +0200 From: "Romascanu, Dan (Dan)" To: Juergen Schoenwaelder , Paul Aitken Thread-Topic: [lmap] LMAP framework draft Thread-Index: AQHOfWXoMwAA0GKX80uTiSsuxfOGrpld0R4QgAAh/ACAAGeqAIAA2ECA Date: Thu, 11 Jul 2013 09:19:50 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1286905E@AZ-FFEXMB04.global.avaya.com> References: <51DD4E33.5000101@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1286526E@AZ-FFEXMB04.global.avaya.com> <51DD86FC.1040605@cisco.com> <20130710161930.GA65045@elstar.local> In-Reply-To: <20130710161930.GA65045@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "lmap-chairs@tools.ietf.org" , Aamer Akhter , "lmap@ietf.org" Subject: Re: [lmap] LMAP framework draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jul 2013 09:20:05 -0000 > -----Original Message----- > From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs- > university.de] > Sent: Wednesday, July 10, 2013 7:19 PM > To: Paul Aitken > Cc: Romascanu, Dan (Dan); lmap-chairs@tools.ietf.org; Aamer Akhter; > lmap@ietf.org > Subject: Re: [lmap] LMAP framework draft >=20 > On Wed, Jul 10, 2013 at 05:08:28PM +0100, Paul Aitken wrote: > > Dan, > > > > >How much meeting time would you need to spend on this? As a rule we > would strongly prefer to avoid presentations, ask participants to come > prepared with the I-Ds already read, and use the precious meeting time > for discussions. > > > > Generally, I would agree. However in this case, this draft would be a > > good starting point for the discussion. > > > > Even if we had not written this draft, it would be worthwhile > > considering what the necessary building blocks are for LMAP, what we > > already have, and what's missing so we can focus WG attention on the > > gaps. > > > > I could present the draft in 10 minutes. However, the framework > > discussion should be longer. >=20 > I guess it is too early to ask but how much different is this I-D going > to be from draft-eardley-lmap-framework-01.txt? >=20 > /js >=20 Individual contributions are welcome at this point. If there are more than = one contribution for a deliverable the WG will discuss what is to be used f= rom each of those.=20 Regards, Dan From dromasca@avaya.com Thu Jul 11 05:19:46 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 02F1611E8100 for ; Thu, 11 Jul 2013 05:19:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.866 X-Spam-Level: X-Spam-Status: No, score=-102.866 tagged_above=-999 required=5 tests=[AWL=-0.267, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TPRlT4x76K7Q for ; Thu, 11 Jul 2013 05:19:39 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id 3D6F011E80F3 for ; Thu, 11 Jul 2013 05:19:39 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ag4FAJid3lGHCzI1/2dsb2JhbABagmghf4MGvkoXbxZ0giMBAQEBAxIREVEGAQgNBAQBAQMCBh0DAgQwFAEGAQEFBQQTCBqHbQGXBIRCijuRJxeBJo4KFiiCUDNsA54giwSDEYIo X-IronPort-AV: E=Sophos;i="4.87,1043,1363147200"; d="scan'208";a="19666668" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 11 Jul 2013 08:19:38 -0400 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 11 Jul 2013 08:13:48 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Thu, 11 Jul 2013 08:19:37 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: lmap - Requested session has been scheduled for IETF 87 Thread-Index: AQHOc3EypnPne6fWV0irqpYZTAgxKZlfeQTw Date: Thu, 11 Jul 2013 12:19:36 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA12869E54@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: [lmap] FW: lmap - Requested session has been scheduled for IETF 87 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jul 2013 12:19:46 -0000 VGhlIExNQVAgV0cgd2FzIGFsbG9jYXRlZCBhIGdlbmVyb3VzIDIuNSBob3VycyBtZWV0aW5nIHNs b3QuIEFzIHdlIGFyZSBidWlsZGluZyBvdXIgYWdlbmRhIEkgc3VnZ2VzdCB0aGUgZm9sbG93aW5n IGdyb3VuZCBydWxlczogDQoNCi0gYW55IGFnZW5kYSBpdGVtIG11c3QgYmUgYmFzZWQgb24gc3Vi bWl0dGVkIEludGVybmV0LURyYWZ0cyAoc3VibWlzc2lvbiBjdXQtb2ZmIGRhdGUgaXMgMjQ6MDAg VVRDIG9uIE1vbmRheSA3LzE1KQ0KLSB3ZSBzaGFsbCBzcGVuZCBhcyBsaXR0bGUgdGltZSBhcyBw b3NzaWJsZSBpbiBwcmVzZW50aW5nIHRoZSBjb250ZW50IG9mIHRoZSBJLURzLCBhc3N1bWluZyB0 aGF0IHRoZSBwYXJ0aWNpcGFudHMgaW4gdGhlIG1lZXRpbmcgaGF2ZSBhbHJlYWR5IHJlYWQgdGhl IEludGVybmV0LURyYWZ0cyANCg0KSWYgeW91IGRpZCBub3QgeWV0IHJlcXVlc3QgYW4gYWdlbmRh IHNsb3QgcGxlYXNlIGxldCBKYXNvbiBhbmQgbWUga25vdyBhcyBzb29uIGFzIHBvc3NpYmxlLiAN Cg0KVGhhbmtzIGFuZCBSZWdhcmRzLA0KDQpEYW4NCg0KIA0KDQoNCg0KLS0tLS1PcmlnaW5hbCBN ZXNzYWdlLS0tLS0NCkZyb206ICJJRVRGIFNlY3JldGFyaWF0IiBbbWFpbHRvOmFnZW5kYUBpZXRm Lm9yZ10gDQpTZW50OiBUaHVyc2RheSwgSnVuZSAyNywgMjAxMyAxMTowMiBQTQ0KVG86IFJvbWFz Y2FudSwgRGFuIChEYW4pDQpDYzogbG1hcC1hZHNAdG9vbHMuaWV0Zi5vcmc7IFJvbWFzY2FudSwg RGFuIChEYW4pOyBjbW9yZ2FuQGFtc2wuY29tDQpTdWJqZWN0OiBsbWFwIC0gUmVxdWVzdGVkIHNl c3Npb24gaGFzIGJlZW4gc2NoZWR1bGVkIGZvciBJRVRGIDg3DQoNCkRlYXIgRGFuIFJvbWFzY2Fu dSwNCg0KVGhlIHNlc3Npb24ocykgdGhhdCB5b3UgaGF2ZSByZXF1ZXN0ZWQgaGF2ZSBiZWVuIHNj aGVkdWxlZC4NCkJlbG93IGlzIHRoZSBzY2hlZHVsZWQgc2Vzc2lvbiBpbmZvcm1hdGlvbiBmb2xs b3dlZCBieSB0aGUgb3JpZ2luYWwgcmVxdWVzdC4gDQoNCmxtYXAgU2Vzc2lvbiAxICgxOjMwOjAw KQ0KICAgIFR1ZXNkYXksIE1vcm5pbmcgU2Vzc2lvbiBJIDA5MDAtMTEzMA0KICAgIFJvb20gTmFt ZTogUG90c2RhbSAyDQogICAgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tDQogICAgDQoNCg0KUmVxdWVzdCBJbmZvcm1hdGlvbjoNCg0KDQotLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCldvcmtpbmcgR3Jv dXAgTmFtZTogDQpBcmVhIE5hbWU6IA0KU2Vzc2lvbiBSZXF1ZXN0ZXI6IA0KDQpOdW1iZXIgb2Yg U2Vzc2lvbnM6IDENCkxlbmd0aCBvZiBTZXNzaW9uKHMpOiAgMS41IEhvdXJzDQpOdW1iZXIgb2Yg QXR0ZW5kZWVzOiA4MA0KQ29uZmxpY3RzIHRvIEF2b2lkOiANCiBGaXJzdCBQcmlvcml0eTogaXBw bSBlY3JpdCBibXdnIGRpc3BhdGNoIGxtYXAgbXB0Y3AgbmV0Y29uZiBuZXRtb2QgZGltZSBvcHNh cmVhIG9wc2F3ZyByYWRleHQgc2FjbSB4cmJsb2NrIHJ0Y3dlYg0KDQoNCg0KDQpTcGVjaWFsIFJl cXVlc3RzOg0KICANCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLQ0KDQo= From philip.eardley@bt.com Thu Jul 11 07:10:47 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 252CC21F9C5D for ; Thu, 11 Jul 2013 07:10:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.939 X-Spam-Level: X-Spam-Status: No, score=-102.939 tagged_above=-999 required=5 tests=[AWL=-0.540, BAYES_00=-2.599, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rJO3Owl0+26k for ; Thu, 11 Jul 2013 07:10:43 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp62.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id CC32A21F9C10 for ; Thu, 11 Jul 2013 07:10:42 -0700 (PDT) Received: from EVMHT63-UKRD.domain1.systemhost.net (10.36.3.100) by RDW083A006ED62.smtp-e2.hygiene.service (10.187.98.11) with Microsoft SMTP Server (TLS) id 8.3.298.1; Thu, 11 Jul 2013 15:10:41 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.228]) by EVMHT63-UKRD.domain1.systemhost.net ([10.36.3.100]) with mapi; Thu, 11 Jul 2013 15:10:40 +0100 From: To: Date: Thu, 11 Jul 2013 15:10:39 +0100 Thread-Topic: New Version Notification for draft-eardley-lmap-terminology-02.txt Thread-Index: Ac5+P8qGiliKn7n1TZ6YpXoXnomvXAAAB0Eg Message-ID: <9510D26531EF184D9017DF24659BB87F34CA8258C3@EMV65-UKRD.domain1.systemhost.net> References: <20130711140541.20962.31537.idtracker@ietfa.amsl.com> In-Reply-To: <20130711140541.20962.31537.idtracker@ietfa.amsl.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: [lmap] FW: New Version Notification for draft-eardley-lmap-terminology-02.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jul 2013 14:10:47 -0000 SSB1cGRhdGVkIHRoZSB0ZXJtaW5vbG9neSBkcmFmdC4NCg0KTWFpbiBjaGFuZ2Ugd2FzIHRvIGFk ZCBhIGZldyB0ZXJtcyB0byAiUzMuMS4gIE90aGVyIHBvdGVudGlhbGx5IHVzZWZ1bCB0ZXJtaW5v bG9neSIsIHdoaWNoIG5vdyBoYXM6IA0KICAgQ3ljbGUtSUQ6IEEgdGFnIHRoYXQgaXMgc2VudCBi eSB0aGUgQ29udHJvbGxlciBpbiBhbiBJbnN0cnVjdGlvbiBhbmQNCiAgIGVjaG9lZCBieSB0aGUg TUEgaW4gaXRzIFJlcG9ydDsgTWVhc3VyZW1lbnQgUmVzdWx0cyB3aXRoIHRoZSBzYW1lDQogICBD eWNsZS1JRCBhcmUgZXhwZWN0ZWQgdG8gYmUgY29tcGFyYWJsZS4NCg0KICAgRW52aXJvbm1lbnRh bCBDb25zdHJhaW50OiBBIHBhcmFtZXRlciB0aGF0IGlzIG1lYXN1cmVkIGFzIHBhcnQgb2YgdGhl DQogICBNZWFzdXJlbWVudCBUYXNrLCBpdHMgdmFsdWUgZGV0ZXJtaW5pbmcgd2hldGhlciB0aGUg cmVzdCBvZiB0aGUNCiAgIE1lYXN1cmVtZW50IFRhc2sgcHJvY2VlZHMuDQoNCiAgIEdyb3VwLUlE OiBBbiBpZGVudGlmaWVyIG9mIGEgZ3JvdXAgb2YgTUFzLg0KDQogICBNZWFzdXJlbWVudCBQYXJh bWV0ZXI6IEEgcGFyYW1ldGVyIHdob3NlIHZhbHVlIGlzIGxlZnQgb3BlbiBieSB0aGUNCiAgIE1l YXN1cmVtZW50IE1ldGhvZC4NCg0KICAgTWVhc3VyZW1lbnQgU3VwcHJlc3Npb246IGEgdHlwZSBv ZiBJbnN0cnVjdGlvbiB0aGF0IHN0b3BzDQogICAoc3VwcHJlc3NlcykgTWVhc3VyZW1lbnQgVGFz a3MuDQoNCiAgIFJlcG9ydCBDaGFubmVsOiBhIHNwZWNpZmljIFJlcG9ydCBTY2hlZHVsZSBhbmQg Q29sbGVjdG9yDQoNCkkgZ3Vlc3MgdGhlIGRlZmF1bHQgaXMgdGhhdCBpbiB0aGUgbmV4dCB2ZXJz aW9uLCB0aGVzZSB0ZXJtcyB3aWxsIGJlIHVwZ3JhZGVkIGludG8gdGhlIG1haW4gc2VjdGlvbiAz Lg0KDQpDb21tZW50cyB3ZWxjb21lIQ0KDQpCZXN0IHdpc2hlcw0KcGhpbA0KDQotLS0tLU9yaWdp bmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogaW50ZXJuZXQtZHJhZnRzQGlldGYub3JnIFttYWlsdG86 aW50ZXJuZXQtZHJhZnRzQGlldGYub3JnXSANClNlbnQ6IDExIEp1bHkgMjAxMyAxNTowNg0KVG86 IEFsIE1vcnRvbjsgRWFyZGxleSxQTCxQaGlsaXAsVFVCOCBSOyBCdXJicmlkZ2UsVCxUcmV2b3Is VFVCOCBSOyBBbCBDLk1vcnRvbjsgTWFyY2VsbyBCYWdudWxvDQpTdWJqZWN0OiBOZXcgVmVyc2lv biBOb3RpZmljYXRpb24gZm9yIGRyYWZ0LWVhcmRsZXktbG1hcC10ZXJtaW5vbG9neS0wMi50eHQN Cg0KDQpBIG5ldyB2ZXJzaW9uIG9mIEktRCwgZHJhZnQtZWFyZGxleS1sbWFwLXRlcm1pbm9sb2d5 LTAyLnR4dA0KaGFzIGJlZW4gc3VjY2Vzc2Z1bGx5IHN1Ym1pdHRlZCBieSBQaGlsaXAgRWFyZGxl eSBhbmQgcG9zdGVkIHRvIHRoZSBJRVRGIHJlcG9zaXRvcnkuDQoNCkZpbGVuYW1lOgkgZHJhZnQt ZWFyZGxleS1sbWFwLXRlcm1pbm9sb2d5DQpSZXZpc2lvbjoJIDAyDQpUaXRsZToJCSBUZXJtaW5v bG9neSBmb3IgTGFyZ2UgTWVBc3VyZW1lbnQgUGxhdGZvcm1zIChMTUFQKQ0KQ3JlYXRpb24gZGF0 ZToJIDIwMTMtMDctMTENCkdyb3VwOgkJIEluZGl2aWR1YWwgU3VibWlzc2lvbg0KTnVtYmVyIG9m IHBhZ2VzOiAxMQ0KVVJMOiAgICAgICAgICAgICBodHRwOi8vd3d3LmlldGYub3JnL2ludGVybmV0 LWRyYWZ0cy9kcmFmdC1lYXJkbGV5LWxtYXAtdGVybWlub2xvZ3ktMDIudHh0DQpTdGF0dXM6ICAg ICAgICAgIGh0dHA6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9kb2MvZHJhZnQtZWFyZGxleS1sbWFw LXRlcm1pbm9sb2d5DQpIdG1saXplZDogICAgICAgIGh0dHA6Ly90b29scy5pZXRmLm9yZy9odG1s L2RyYWZ0LWVhcmRsZXktbG1hcC10ZXJtaW5vbG9neS0wMg0KRGlmZjogICAgICAgICAgICBodHRw Oi8vd3d3LmlldGYub3JnL3JmY2RpZmY/dXJsMj1kcmFmdC1lYXJkbGV5LWxtYXAtdGVybWlub2xv Z3ktMDINCg0KQWJzdHJhY3Q6DQogICBUaGlzIGRvY3VtZW50cyBkZWZpbmVzIHRlcm1pbm9sb2d5 IGZvciBMYXJnZSBTY2FsZSBNZWFzdXJlbWVudA0KICAgUGxhdGZvcm1zLg0KDQogICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgDQoNCg0KVGhlIElFVEYgU2VjcmV0YXJpYXQNCg0K From nagami@wide.ad.jp Thu Jul 11 18:25:51 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B232A21F9A27 for ; Thu, 11 Jul 2013 18:25:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.678 X-Spam-Level: X-Spam-Status: No, score=-1.678 tagged_above=-999 required=5 tests=[AWL=0.300, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YOifm72Wjbn3 for ; Thu, 11 Jul 2013 18:25:51 -0700 (PDT) Received: from sh.wide.ad.jp (sh.wide.ad.jp [IPv6:2001:200:0:1001::6]) by ietfa.amsl.com (Postfix) with ESMTP id CA4CD21F9A17 for ; Thu, 11 Jul 2013 18:25:50 -0700 (PDT) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (authenticated (0 bits)) by mail.wide.ad.jp (8.14.1+3.5Wbeta/8.14.1/smtpfeed 1.21) with ESMTP id r6C1Pm3j012049 (using TLSv1/SSLv3 with cipher RC4-SHA (128 bits) verified FAIL) for ; Fri, 12 Jul 2013 10:25:49 +0900 (JST) Received: by mail-pd0-f181.google.com with SMTP id 14so8066824pdj.26 for ; Thu, 11 Jul 2013 18:25:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6oHAGCX/bbDnYg0ebY3LdLqV2SguhqywwyA+ZheJFgU=; b=gRWCzdQhuBLsvea0tk961NjcbGnqW8i+55wWMhovewZSR/t8sWAQqd1/lGR1SIGCJE yonXfVJCBuerlmy1ZoXTWBGws5WIGCMa+6GOe5aJFnLP6Yokupu6IGZIS3nuOcjqe2qF 8Uq6ptJwCScZoZQlCi/dCINd9Nz456ce6rA7jsg493Y1CIxwiz83xbqKtP9B+jchHujJ GT6zGaTJPjS/c6LmBHKCeNFnGTilMecPLZjdNxQPVaxDN1IncV2PfZkE8QpRf/gR9wYp 0m2OV7iCIuGpBR+BvVp8K4F6WrjjZI+Qx/0zUMbsuEPbQrKP+CDzzGNQ6+Z3VFdv5oEs 8QIg== MIME-Version: 1.0 X-Received: by 10.68.201.226 with SMTP id kd2mr39129277pbc.45.1373592343510; Thu, 11 Jul 2013 18:25:43 -0700 (PDT) Received: by 10.70.102.176 with HTTP; Thu, 11 Jul 2013 18:25:43 -0700 (PDT) In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA12862C42@AZ-FFEXMB04.global.avaya.com> References: <9904FB1B0159DA42B0B887B7FA8119CA12862C42@AZ-FFEXMB04.global.avaya.com> Date: Fri, 12 Jul 2013 10:25:43 +0900 Message-ID: From: Nagami Kenichi To: "Romascanu, Dan (Dan)" Content-Type: text/plain; charset=ISO-8859-1 Cc: Javier Bustos , lmap WG Subject: Re: [lmap] Call for contributions and Agenda time in Berlin X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jul 2013 01:25:51 -0000 Hi Dan, I will attend the Berlin meeting and would like to talk with Marc Linsner and Phil Eardley. I think our case will be classified as "Multi-provider Network Measurements" or an extension to the Multi-Provider use case in the use case document. A single organization performs a measurement from users to multiple OTT providers. Regards, Ken Nagami 2013/7/9 Romascanu, Dan (Dan) : > Hi Javier and Nagami, > > > > Thank you for letting the lmap list know about your applications and for > offering your help. It may be useful if you would talk with Marc Linsner and > Phil Eardley who have authored the use case document and see if your > experience can add to this. The current version in the tracker is > https://datatracker.ietf.org/doc/draft-linsner-lmap-use-cases/ but there may > be a revision in preparation for the Internet Draft submission cut-off next > Monday. > > > > Will you attend the Berlin meeting? > > > > Thanks and Regards, > > > > Dan > > > > > > > > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Javier Bustos > Sent: Monday, July 01, 2013 4:32 PM > To: Nagami Kenichi > Cc: MORTON JR., ALFRED C (AL); lmap WG > > > Subject: Re: [lmap] Call for contributions and Agenda time in Berlin > > > > Dear Al and Dan > > > > We would like to contribute for LMAP too. > > > > Here in Chile we have performed active measurements for wired internet > (around 10,000 probes) and passive measurements for mobile internet (around > 750 probes). > > > > Regards, > > Javier Bustos. > > > > 2013/7/1 Nagami Kenichi > > Dear Al and Dan, > > We would like to contribute some use case for LMAP. > We have measured the broadband performance in Japan using measurement > software. We have distributed 500,000 applications. > > Regards, > Ken Nagami > > 2013/6/22 MORTON JR., ALFRED C (AL) : > >> LMAP, >> >> Last week, the draft LMAP charter went forward for external review. >> LMAP has been approved for a placeholder session at IETF-87, >> either as a Working Group if approved in time, or as a BoF if not. >> If charter-related issues remain to be resolved, they will be >> discussed during the session with highest priority. >> >> However, ... >> >> If you have new/revised contributions covered under the current >> draft of the LMAP charter, please reply to the list identifying the topic, >> the filename(if applicable) and the related issues you would like to >> discuss at >> a face-to-face session, and try to do this by June 28, 2013. >> >> thanks and regards, >> Dan and Al >> >> >> >>> -----Original Message----- >>> From: ietf-announce-bounces@ietf.org [mailto:ietf-announce- >>> bounces@ietf.org] On Behalf Of The IESG >>> Sent: Friday, June 14, 2013 11:42 AM >>> To: IETF-Announce >>> Cc: lmap WG >>> Subject: WG Review: Large-Scale Measurement of Broadband Performance >>> (lmap) >>> >>> A new IETF working group has been proposed in the Operations and >>> Management Area. The IESG has not made any determination yet. The >>> following draft charter was submitted, and is provided for informational >>> purposes only. Please send your comments to the IESG mailing list (iesg >>> at ietf.org) by 2013-06-24. >>> >>> Large-Scale Measurement of Broadband Performance (lmap) >>> ------------------------------------------------ >>> Current Status: Proposed WG >>> >>> Assigned Area Director: >>> Benoit Claise >>>... >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > > > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > From Joachim.Fabini@tuwien.ac.at Mon Jul 15 03:24:17 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3394721F9ABD; Mon, 15 Jul 2013 03:24:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.43 X-Spam-Level: X-Spam-Status: No, score=-5.43 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_AT=0.424, HOST_EQ_AT=0.745, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 25vflHuQeopT; Mon, 15 Jul 2013 03:24:11 -0700 (PDT) Received: from mail1.zserv.tuwien.ac.at (mail1.zserv.tuwien.ac.at [128.130.35.37]) by ietfa.amsl.com (Postfix) with ESMTP id 614E221F9FA7; Mon, 15 Jul 2013 03:24:08 -0700 (PDT) Received: from [128.131.88.241] (priamos.ibk.tuwien.ac.at [128.131.88.241]) (authenticated bits=0) by mail1.zserv.tuwien.ac.at (8.13.8/8.13.8) with ESMTP id r6FAO13o030067 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 15 Jul 2013 12:24:03 +0200 Message-ID: <51E3CDB8.5080105@tuwien.ac.at> Date: Mon, 15 Jul 2013 12:23:52 +0200 From: Joachim Fabini User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: ippm@ietf.org Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 8bit Cc: Al Morton , lmap@ietf.org Subject: [lmap] Draft RFC 2330 Update submission X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2013 10:24:18 -0000 Dear all, we have submitted the RFC2330 Update draft last week, it is available online at https://datatracker.ietf.org/doc/draft-ietf-ippm-2330-update/. As a side-note: the time-slotted randomness cancellation (TSRC) issue, which the draft discusses, questions appropriateness of end-to-end measurement methodologies for sequences of time-slotted links - not only for wireless but also for wired access links. We will present some details/measurement results in Berlin and would welcome a discussion on the potential impact that TSRC can have onto the future lmap architecture. More details on TSRC and its impact on measurements can be found in reference [TSRC] of the draft. The paper is currently available as early-access in IEEExplore, you may contact me (JF) by email in case you have no access to it. regards Joachim and Al. -- --------------------------------------- Dr. Joachim Fabini Institute of Telecommunications Vienna University of Technology Favoritenstraße 9/389 A-1040 Vienna, Austria Tel: +43 1 58801-38813 Fax: +43 1 58801-38898 mailto: Joachim.Fabini@tuwien.ac.at --------------------------------------- From paitken@cisco.com Mon Jul 15 16:49:08 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C90B11E826A for ; Mon, 15 Jul 2013 16:49:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VkZxBaJMJTvc for ; Mon, 15 Jul 2013 16:49:03 -0700 (PDT) Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id C5D0011E8273 for ; Mon, 15 Jul 2013 16:49:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=385; q=dns/txt; s=iport; t=1373932142; x=1375141742; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=H/zypF6LnHLEnVH65FXv8EOO4tHkNPlPtT/LXEHYTI0=; b=KSlKYw4aSVtZKKDeQjtarGFLoidA05kBOm4wfLKHWdFurMh/crINjYgN E3Ma1M7byxoykAUxLmlqPqzRLPT7Isd0zf0cwsGeXuMsX8z/BCThCLuWw YGEtNk1UGE16Wr0DQ1OGYzsg+ksb0oCmIjK77byKCOFbjmGMRAJFgotJl E=; X-IronPort-AV: E=Sophos;i="4.89,672,1367971200"; d="scan'208";a="15832403" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-4.cisco.com with ESMTP; 15 Jul 2013 23:48:57 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id r6FNmtoF009817 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 15 Jul 2013 23:48:55 GMT Received: from [10.61.214.99] ([10.61.214.99]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6FNmsMN026471; Tue, 16 Jul 2013 00:48:55 +0100 (BST) Message-ID: <51E48A6C.8030205@cisco.com> Date: Tue, 16 Jul 2013 00:49:00 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: lmap@ietf.org References: <51DD4E33.5000101@cisco.com> In-Reply-To: <51DD4E33.5000101@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: lmap-chairs@tools.ietf.org, Aamer Akhter Subject: Re: [lmap] LMAP framework draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2013 23:49:08 -0000 > We've been writing an LMAP framework document which discusses the > building blocks, looking at what we already have in the IETF and > what's missing, where the gaps are. > It will be published by Monday's cut-off. The draft is now published: http://www.ietf.org/internet-drafts/draft-akhter-lmap-framework-00.txt We welcome your feedback. Thanks, Aamer and Paul From acmorton@att.com Tue Jul 16 04:51:01 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 171C121E81C7 for ; Tue, 16 Jul 2013 04:51:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.247 X-Spam-Level: X-Spam-Status: No, score=-106.247 tagged_above=-999 required=5 tests=[AWL=0.352, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NSaskVAwmUxw for ; Tue, 16 Jul 2013 04:50:56 -0700 (PDT) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 35ECF21E8198 for ; Tue, 16 Jul 2013 04:50:54 -0700 (PDT) Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id 5DA62120559; Tue, 16 Jul 2013 07:50:52 -0400 (EDT) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-green.research.att.com (Postfix) with ESMTP id BCDB5E0388; Tue, 16 Jul 2013 07:50:45 -0400 (EDT) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Tue, 16 Jul 2013 07:50:53 -0400 From: "MORTON JR., ALFRED C (AL)" To: Juergen Schoenwaelder , Paul Aitken Date: Tue, 16 Jul 2013 07:50:53 -0400 Thread-Topic: [lmap] LMAP framework draft Thread-Index: Ac59iVDa47XVsUuXSA6WwD1wqggcbgEkMENQ Message-ID: References: <51DD4E33.5000101@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1286526E@AZ-FFEXMB04.global.avaya.com> <51DD86FC.1040605@cisco.com> <20130710161930.GA65045@elstar.local> In-Reply-To: <20130710161930.GA65045@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "Romascanu, Dan \(Dan\)" , "lmap-chairs@tools.ietf.org" , Aamer Akhter , "lmap@ietf.org" Subject: Re: [lmap] LMAP framework draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2013 11:51:01 -0000 > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Juergen Schoenwaelder > Sent: Wednesday, July 10, 2013 12:20 PM ... > On Wed, Jul 10, 2013 at 05:08:28PM +0100, Paul Aitken wrote: > > ... > > > > Even if we had not written this draft, it would be worthwhile > > considering what the necessary building blocks are for LMAP, what we > > already have, and what's missing so we can focus WG attention on the > > gaps. > > > > I could present the draft in 10 minutes. However, the framework > > discussion should be longer. >=20 > I guess it is too early to ask but how much different is this I-D > going to be from draft-eardley-lmap-framework-01.txt? >=20 > /js >=20 Despite submission difficulties, we have updated the earlier framework draf= t: http://www.ietf.org/internet-drafts/draft-eardley-lmap-framework-02.txt Revisions address the evolution of the LMAP terminology and the approved ch= arter. comments welcome, Phil, Trevor, Al From mlinsner@cisco.com Tue Jul 16 06:12:32 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D453E21F9AA3 for ; Tue, 16 Jul 2013 06:12:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jotrFI0RtuK4 for ; Tue, 16 Jul 2013 06:12:28 -0700 (PDT) Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) by ietfa.amsl.com (Postfix) with ESMTP id 09F6521F99F7 for ; Tue, 16 Jul 2013 06:12:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1488; q=dns/txt; s=iport; t=1373980348; x=1375189948; h=from:to:subject:date:message-id:in-reply-to:content-id: content-transfer-encoding:mime-version; bh=5GTZlRJwOuPmoUyFxrfrZ0kMNvw51/W9pxvl+LicgHY=; b=bnS6/d8/SfC/5kgCfIivZK4UMnNvqK7++gbGj8ZCZR+Ii3WY9pXkBujq IjyJTjdo8VKKda2h0M8Q07NHXXCqdRikvxxKSA6lLIel61FQkwOKaTU97 0vbKzeLVN8aQ/ei3jPvnVdAIWOLKlx9NrEM4ITx1j2nnmuN1Ywi5sViis Y=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhkFAIJF5VGtJV2b/2dsb2JhbABagwY0SQbBXYEPFnSCIwEBAQQ6PRQBCCIUQhsBBgMCBBsBiAcHBZUtoFqPLjiDDG0DmQWQJIMSgig X-IronPort-AV: E=Sophos;i="4.89,676,1367971200"; d="scan'208";a="235402674" Received: from rcdn-core-4.cisco.com ([173.37.93.155]) by rcdn-iport-7.cisco.com with ESMTP; 16 Jul 2013 13:12:27 +0000 Received: from xhc-rcd-x05.cisco.com (xhc-rcd-x05.cisco.com [173.37.183.79]) by rcdn-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id r6GDCQTs023877 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Tue, 16 Jul 2013 13:12:27 GMT Received: from xmb-rcd-x08.cisco.com ([169.254.8.87]) by xhc-rcd-x05.cisco.com ([173.37.183.79]) with mapi id 14.02.0318.004; Tue, 16 Jul 2013 08:12:27 -0500 From: "Marc Linsner (mlinsner)" To: "lmap@ietf.org" Thread-Topic: New Version Notification for draft-linsner-lmap-use-cases-03.txt Thread-Index: AQHOgY+AM7g/qjGcs0myuR1vECeKEplnWdyA Date: Tue, 16 Jul 2013 13:12:26 +0000 Message-ID: <581E085DEB6A444093CDAEA4C4EE48181359BB8A@xmb-rcd-x08.cisco.com> In-Reply-To: <20130715191412.11308.84695.idtracker@ietfa.amsl.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.116.195.115] Content-Type: text/plain; charset="us-ascii" Content-ID: <804E9EF828322949B292B6122CE2B73C@emea.cisco.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] FW: New Version Notification for draft-linsner-lmap-use-cases-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2013 13:12:32 -0000 Updated the Use Case draft to match charter. Comments welcome. > >A new version of I-D, draft-linsner-lmap-use-cases-03.txt >has been successfully submitted by Marc Linsner and posted to the >IETF repository. > >Filename: draft-linsner-lmap-use-cases >Revision: 03 >Title: Large-Scale Broadband Measurement Use Cases >Creation date: 2013-07-15 >Group: Individual Submission >Number of pages: 16 >URL: =20 >http://www.ietf.org/internet-drafts/draft-linsner-lmap-use-cases-03.txt >Status: =20 >http://datatracker.ietf.org/doc/draft-linsner-lmap-use-cases >Htmlized: =20 >http://tools.ietf.org/html/draft-linsner-lmap-use-cases-03 >Diff: =20 >http://www.ietf.org/rfcdiff?url2=3Ddraft-linsner-lmap-use-cases-03 > >Abstract: > Measuring broadband performance on a large scale is important for > network diagnostics by providers and users, as well for as public > policy. To conduct such measurements, user networks gather data, > either on their own initiative or instructed by a measurement > controller, and then upload the measurement results to a designated > measurement server. Understanding the various scenarios and users of > measuring broadband performance is essential to development of the > system requirements. The details of the measurement metrics > themselves are beyond the scope of this document. > > > =20 > =20 > > >The IETF Secretariat > From dromasca@avaya.com Tue Jul 16 13:48:02 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4BFF921F9F13 for ; Tue, 16 Jul 2013 13:48:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.31 X-Spam-Level: X-Spam-Status: No, score=-103.31 tagged_above=-999 required=5 tests=[AWL=0.289, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LampO8dLdCaA for ; Tue, 16 Jul 2013 13:47:57 -0700 (PDT) Received: from co300216-co-outbound.net.avaya.com (co300216-co-outbound.net.avaya.com [198.152.13.100]) by ietfa.amsl.com (Postfix) with ESMTP id E20EA21F9A8F for ; Tue, 16 Jul 2013 13:47:56 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgIFAAH/ZlHGmAcV/2dsb2JhbABQgmUhNsFIgQcWdIIhAQEDEihRARUVFEImAQQbGodyAQucEYQrjGeQOASOZYMYYQOdKYpogwuCKA X-IronPort-AV: E=Sophos;i="4.87,456,1363147200"; d="scan'208";a="19787687" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by co300216-co-outbound.net.avaya.com with ESMTP; 16 Jul 2013 16:47:53 -0400 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by co300216-co-erhwest-out.avaya.com with ESMTP; 16 Jul 2013 16:45:48 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Tue, 16 Jul 2013 16:47:52 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: LMAP @ IETF-87 - draft agenda Thread-Index: Ac6CZb0fh6Yd4RHIQSSBt5zyPKQO9g== Date: Tue, 16 Jul 2013 20:47:52 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA128742C3@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] LMAP @ IETF-87 - draft agenda X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2013 20:48:02 -0000 We have posted the draft agenda for the first LMAP meeting at https://datat= racker.ietf.org/meeting/87/agenda/lmap/.=20 Comments and last minute requests for the agenda are welcome. The agenda al= so includes a reading list which will help everybody come prepared to the m= eeting. Although this is the first WG meeting we want to spend as little ti= me as possible with presentations and use at best the expensive face-to-fac= e meeting time for discussions about how to progress the items on the chart= er. Priority will be given to the use cases and framework work (the two fir= st items on the charter) and to the information model (which comes next).=20 See you all in Berlin.=20 Regards, Jason and Dan From dromasca@avaya.com Tue Jul 16 13:57:41 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5CBEF21F8AD5 for ; Tue, 16 Jul 2013 13:57:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.314 X-Spam-Level: X-Spam-Status: No, score=-103.314 tagged_above=-999 required=5 tests=[AWL=0.285, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Gu+7q-4AoryW for ; Tue, 16 Jul 2013 13:57:29 -0700 (PDT) Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id DF81C11E8103 for ; Tue, 16 Jul 2013 13:57:03 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhgFAOuy5VGHCzI1/2dsb2JhbABagmUhgQPBfIESFnSCJQEBAxIoUQEVFRRCJgEEGxqHbgGVHIRCm32OLYEBg0RtA54iiweDEoFxNw X-IronPort-AV: E=Sophos;i="4.89,679,1367985600"; d="scan'208";a="16749663" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by de307622-de-outbound.net.avaya.com with ESMTP; 16 Jul 2013 16:57:00 -0400 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 16 Jul 2013 16:50:58 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Tue, 16 Jul 2013 16:56:58 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: LMAP - welcome breakfast for first-time participants Thread-Index: Ac6CZwI3FGgA+N1OTWKNVX7x/ZHKoA== Date: Tue, 16 Jul 2013 20:56:57 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA12874302@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] LMAP - welcome breakfast for first-time participants X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2013 20:57:41 -0000 Hi,=20 Some of the attendees in the LMAP WG meeting in Berlin will be first time p= articipants in the IETF. We would like to extend them a warm welcome and su= ggest that we meet for breakfast on Tuesday 7/30, at 8AM. It would be a goo= d opportunity for us (LMAP chairs) to meet and know you, hear from you what= are the your areas of interest and expectations from the IETF and from the= LMAP WG. We will be glad to answer any questions related to the WG and to = the IETF.=20 Old timers who are interested to meet with the first-time participants are = also welcome.=20 The exact location of the meeting will be announced in a mail to the list t= he day before the meeting.=20 See you all in Berlin.=20 Regards, Jason and Dan From dromasca@avaya.com Tue Jul 16 14:14:43 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7BFA221F962D for ; Tue, 16 Jul 2013 14:14:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.34 X-Spam-Level: X-Spam-Status: No, score=-103.34 tagged_above=-999 required=5 tests=[AWL=0.259, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y4RWK+zjOFlP for ; Tue, 16 Jul 2013 14:14:38 -0700 (PDT) Received: from co300216-co-outbound.net.avaya.com (co300216-co-outbound.net.avaya.com [198.152.13.100]) by ietfa.amsl.com (Postfix) with ESMTP id E08E521F944C for ; Tue, 16 Jul 2013 14:14:37 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgUFAAH/ZlHGmAcV/2dsb2JhbABQgmUhNsFIgQcWdIIfAQEBAQMBAQEPKDQXBAIBCA0EBAEBCxQJBycLFAkIAQEEEwgah3IBC6A8jGeQJRMEjWSBASYSBoJaYQOdKYpogwuBczU X-IronPort-AV: E=Sophos;i="4.87,456,1363147200"; d="scan'208";a="19791184" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by co300216-co-outbound.net.avaya.com with ESMTP; 16 Jul 2013 17:14:37 -0400 Received: from unknown (HELO AZ-FFEXHC04.global.avaya.com) ([135.64.58.14]) by co300216-co-erhwest-out.avaya.com with ESMTP; 16 Jul 2013 17:12:32 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC04.global.avaya.com ([135.64.58.14]) with mapi id 14.02.0328.009; Tue, 16 Jul 2013 17:14:35 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: LMAP - welcome breakfast for first-time participants Thread-Index: Ac6CZwI3FGgA+N1OTWKNVX7x/ZHKoAAAl1YQ Date: Tue, 16 Jul 2013 21:14:35 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1287434E@AZ-FFEXMB04.global.avaya.com> References: <9904FB1B0159DA42B0B887B7FA8119CA12874302@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA12874302@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] LMAP - welcome breakfast for first-time participants X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2013 21:14:43 -0000 We forgot to mention - folks interested to attend, please send us a message= .=20 > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Romascanu, Dan (Dan) > Sent: Tuesday, July 16, 2013 11:57 PM > To: lmap@ietf.org > Subject: [lmap] LMAP - welcome breakfast for first-time participants >=20 > Hi, >=20 > Some of the attendees in the LMAP WG meeting in Berlin will be first > time participants in the IETF. We would like to extend them a warm > welcome and suggest that we meet for breakfast on Tuesday 7/30, at 8AM. > It would be a good opportunity for us (LMAP chairs) to meet and know > you, hear from you what are the your areas of interest and expectations > from the IETF and from the LMAP WG. We will be glad to answer any > questions related to the WG and to the IETF. >=20 > Old timers who are interested to meet with the first-time participants > are also welcome. >=20 > The exact location of the meeting will be announced in a mail to the > list the day before the meeting. >=20 > See you all in Berlin. >=20 > Regards, >=20 > Jason and Dan >=20 >=20 >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From timothy.carey@alcatel-lucent.com Sun Jul 21 05:28:06 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9AF0421E809C for ; Sun, 21 Jul 2013 05:28:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.999 X-Spam-Level: X-Spam-Status: No, score=-9.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_91=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m6jSn91LNGPH for ; Sun, 21 Jul 2013 05:28:00 -0700 (PDT) Received: from ihemail4.lucent.com (ihemail4.lucent.com [135.245.0.39]) by ietfa.amsl.com (Postfix) with ESMTP id B1CD821E808D for ; Sun, 21 Jul 2013 05:27:57 -0700 (PDT) Received: from us70tusmtp2.zam.alcatel-lucent.com (h135-5-2-64.lucent.com [135.5.2.64]) by ihemail4.lucent.com (8.13.8/IER-o) with ESMTP id r6LCRhpV014024 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Sun, 21 Jul 2013 07:27:43 -0500 (CDT) Received: from US70TWXCHHUB04.zam.alcatel-lucent.com (us70twxchhub04.zam.alcatel-lucent.com [135.5.2.36]) by us70tusmtp2.zam.alcatel-lucent.com (GMO) with ESMTP id r6LCRfrQ018930 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 21 Jul 2013 08:27:41 -0400 Received: from US70UWXCHMBA02.zam.alcatel-lucent.com ([169.254.8.216]) by US70TWXCHHUB04.zam.alcatel-lucent.com ([135.5.2.36]) with mapi id 14.02.0247.003; Sun, 21 Jul 2013 08:27:41 -0400 From: "Carey, Timothy (Timothy)" To: "trevor.burbridge@bt.com" Thread-Topic: LMAP Information Model - Question Thread-Index: AQHOgmOIGGhff3Njp0mtktGvSj0tNZlqbWlQgASn7HA= Date: Sun, 21 Jul 2013 12:27:40 +0000 Message-ID: <9966516C6EB5FC4381E05BF80AA55F77190752@US70UWXCHMBA02.zam.alcatel-lucent.com> References: <9966516C6EB5FC4381E05BF80AA55F7715D059@US70UWXCHMBA05.zam.alcatel-lucent.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.5.27.17] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.57 on 135.245.2.39 Cc: "Scharf, Michael \(Michael\)" , "lmap@ietf.org" , "Michael.K.Bugenhagen@CenturyLink.com" , "KEN.KO@adtran.com" , "Charles.Cook2@CenturyLink.com" , "Barbara.Stark@bellsouth.com" Subject: Re: [lmap] LMAP Information Model - Question X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jul 2013 12:28:06 -0000 Trevor, Thanks for the replies -I added the LMAP list. BR, Tim -----Original Message----- From: trevor.burbridge@bt.com [mailto:trevor.burbridge@bt.com]=20 Sent: Thursday, July 18, 2013 8:40 AM To: Carey, Timothy (Timothy) Cc: KEN.KO@adtran.com; Michael.K.Bugenhagen@CenturyLink.com; Charles.Cook2@= CenturyLink.com; Barbara.Stark@bellsouth.com; Scharf, Michael (Michael) Subject: RE: LMAP Information Model - Question Good questions - some discussion inline..... Trevor. >From: Carey, Timothy (Timothy) [mailto:timothy.carey@alcatel-lucent.com]=20 >Sent: 16 July 2013 21:32 >To: Burbridge,T,Trevor,TUB8 R >Cc: 'KEN KO'; Bugenhagen, Michael K; Cook, Charles; 'Stark, Barbara'; Scha= rf, Michael (Michael) >Subject: LMAP Information Model - Question > >Trevor, > >I was looking at your draft-burbridge-lmap-information-model-00 document a= nd had a couple of questions/comments. > >Configuration information - is this strictly be the Measurement Controller= and MA or can any element have the access rights to configure the MA? My a= ssumption is that any authorized party can configure the MA configuration e= lements not just the controller? Question is are there information elements= that are restricted to only the controller? > Yes, our thoughts were that there is only a single controller to simplify t= he management of the MA. We did not want to get into the complexity of havi= ng to delegate rights to other parties. If other parties are involved they = can relay there request via the authorised controller. >You introduced a concept of group membership for a MA. Can a MA belong to = multiple groups? Looking at the draft it doesn't seem so but I don't know w= hy we would have restricted it. Also how does a MA know which group this te= st belongs to it knows if it should report the MA ID? > We were thinking that a single Group could be used to capture any unique se= t of characteristics. It is really only intended as an anonymous group subs= titute instead of an individual MA ID. Note that other line/device/user cha= racteristics MAY be optionally transmitted in the report. >I'll have to look at how to integrate the registry aspect to the measureme= nt task - seems the attributes of the task changes based on the test but wi= ll be documented in the registry - Maybe we can generalize this somehow in = TR-069. > The registry will specify some parameters of the measurement task. There ar= e other optional parameters that are configured in the Measurement task Con= figuration - e.g. test target MA. >For the schedule - I noticed that there is a list of tasks to execute - Wi= ll these have conditional logic in anyway - goes to being either a list or = a policy of some sort. > To keep things simple again we were expecting this to be a simple list. Whe= n one measurement task is finished then the next on the list is started. Ea= ch task may include its own back-off behaviour in which case the other task= s would also be delayed. Providing a complex conditional language is maybe = too much - e.g. if it fails wait 1 minute and then be able to displace tas= k X but not task Z etc. >I assume the measurement suppression is for tasks within a schedule? The suppression is currently targeted at one or more measurement tasks rega= rdless of which schedules they are included in. I was imagining that most r= easons for suppression would mean that we want to disable a task- e.g. code= or configuration error for the task. If a schedule is wrong then the sched= ule can be fixed directly. >On the status information you will probably want to have a set of defined = codes with the ability for =A0vendor specific codes. Yes - logging and status information is barely started...... >On the Timing randomness - The integers are what unit - milliseconds, seco= nds - this one probably needs more description of how to apply the upper, l= ower and spread to the distribution - Unless this is documented somewhere e= lse - then just point to the reference. > More work in progress - yes I would imagine milliseconds. >In the security section you mentioned the MA context could be excluded but= I only saw how the MA ID would be excluded - did I miss something? In the report information model there is a place for optional MA characteri= stics such as Michael has been talking about. > >BTW - What the information elements do not show is the Read-only vs Read-w= rite aspects of the elements - that would be a good thing to have the in RF= C unless be set you mean there are no read-only parameters; also any sizes = of the string would help. > I think everything is writeable by some entity (either pre-configuration or= controller or MA). I don't think the information model is the right place = to specify access control for who can write the information. > >All-in-all - Nice start; I can use this to start some of the BBF work. > Thanks - it would be good to have this discussion on the LMAP list - if you= are willing can you respond to this email including the LMAP list? >BR, >Tim From bclaise@cisco.com Mon Jul 22 07:19:25 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D18BF21F99FB for ; Mon, 22 Jul 2013 07:19:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.519 X-Spam-Level: X-Spam-Status: No, score=-10.519 tagged_above=-999 required=5 tests=[AWL=0.079, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fQ122zhqOgtJ for ; Mon, 22 Jul 2013 07:19:21 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 39EA111E8115 for ; Mon, 22 Jul 2013 07:19:20 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6MEJHXG018550 for ; Mon, 22 Jul 2013 16:19:18 +0200 (CEST) Received: from [10.60.67.85] (ams-bclaise-8914.cisco.com [10.60.67.85]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6MEJ1PO027438 for ; Mon, 22 Jul 2013 16:19:11 +0200 (CEST) Message-ID: <51ED3F4D.5090403@cisco.com> Date: Mon, 22 Jul 2013 16:18:53 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; WOW64; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: "lmap@ietf.org" Content-Type: multipart/alternative; boundary="------------030604040503080307010804" Subject: [lmap] Feedback regarding draft-linsner-lmap-use-cases X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jul 2013 14:19:25 -0000 This is a multi-part message in MIME format. --------------030604040503080307010804 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear authors, A couple of high level points. One of the goal behind this email is to generate discussions, either on the list, or during the IETF meeting next week. - Large-scale measurement efforts in [LMAP-REQ ] describe three use cases to be considered in deriving the requirements to be used in developing the solution. This documents attempts to describe those use cases in further detail and include additional use cases. Should not refer to the requirements as a starting point. The use case document is supposed to the first document in the LMAP series The charter refers to the provider and regulator use cases. Btw, regarding the end user, the charter mentions: Standardizing control of end users Measurement Agents is out of scope. However, end users can obtain an MA to run measurement tasks if desired and report their results to whomever they want, most likely the supplier of the MA. This provides for user-initiated on-demand measurement, which is an important component of the ISP use case. Btw, these specific aspects of the end user network diagnostics should be stressed in the draft. Therefore, you should focus first on the ISP and regulators use cases in the draft. - o Benchmarking and competitor insight. The operation of sample panels across competitor products can enable and ISP to assess where they play in the market, identify opportunities where other products operate different technology, and assess the performance of network suppliers that are common to both operators. How does it work regarding this sentence in the charter? A key assumption constraining the initial work is that the measurement system is under the control of a single organization (for example, an Internet Service Provider or a regulator). I mean: the regulator can share information about different ISPs, but I doubt that ISP1 will be able to get the ISP2 results by re-directing/duplicating the ISP2 measurement agents to his collector. - I believe that the ISPs use cases miss an entry regarding "checking the service SLA". Maybe this is covered by the entry "Understanding the quality experienced by customers"? - What I don't see in the draft: what is meant by "broadband"? - What I don't see in the draft: IPv4 versus IPv6. Linked to that: what is/are use case(s) for multiple interfaces. The charter says: "Devices containing Measurement Agents may have several interfaces using different link technologies. Multiple address families and interfaces must be considered in the Control and Report protocols." - What I don't see in the draft: how many MAs do we have in mind for LMAP. - Note: the QoS and QoE definition from RFC 6390, along with http://tools.ietf.org/html/rfc6390#section-4 might help. Regards, Benoit --------------030604040503080307010804 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear authors,

A couple of high level points.
One of the goal behind this email is to generate discussions, either on the list, or during the IETF meeting next week.
- 
   Large-scale measurement efforts in [LMAP-REQ] describe three use
   cases to be considered in deriving the requirements to be used in
   developing the solution.  This documents attempts to describe those
   use cases in further detail and include additional use cases.

Should not refer to the requirements as a starting point. 
The use case document is supposed to the first document in the LMAP series
The charter refers to the provider and regulator use cases.

Btw, regarding the end user, the charter mentions:

Standardizing control of end users Measurement Agents is out of scope.
However, end users can obtain an MA to run measurement tasks if desired 
and report their results to whomever they want, most likely the supplier
 of the MA. This provides for user-initiated on-demand measurement, 
which is an important component of the ISP use case.
Btw, these specific aspects of the end user network diagnostics should be stressed in the draft.
Therefore, you should focus first on the ISP and regulators use cases in the draft.
- 
      o Benchmarking and competitor insight. The operation of sample
      panels across competitor products can enable and ISP to assess
      where they play in the market, identify opportunities where other
      products operate different technology, and assess the performance
      of network suppliers that are common to both operators.
How does it work regarding this sentence in the charter?
A key assumption constraining the initial work is that the measurement system is under the control of a single organization (for example, an Internet Service Provider or a regulator).
I mean: the regulator can share information about different ISPs, but I doubt that ISP1 will be able to get the ISP2 results by re-directing/duplicating the ISP2 measurement agents to his collector.

- I believe that the ISPs use cases miss an entry regarding "checking the service SLA".
Maybe this is covered by the entry "Understanding the quality experienced by customers"?

- What I don't see in the draft: what is meant by "broadband"?

- What I don't see in the draft:  IPv4 versus IPv6. Linked to that: what is/are use case(s) for multiple interfaces. The charter says: "Devices containing Measurement Agents may have several interfaces using different link technologies. Multiple address families and interfaces must be considered in the Control and Report protocols."

- What I don't see in the draft: how many MAs do we have in mind for LMAP.

- Note: the QoS and QoE definition from RFC 6390, along with http://tools.ietf.org/html/rfc6390#section-4 might help.

Regards, Benoit
--------------030604040503080307010804-- From bclaise@cisco.com Mon Jul 22 07:31:08 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F2C3D21F8427 for ; Mon, 22 Jul 2013 07:31:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.52 X-Spam-Level: X-Spam-Status: No, score=-10.52 tagged_above=-999 required=5 tests=[AWL=0.079, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HbhsF+-ETEWV for ; Mon, 22 Jul 2013 07:31:01 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id CC11D21E809A for ; Mon, 22 Jul 2013 07:28:47 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6MES6NN019639 for ; Mon, 22 Jul 2013 16:28:06 +0200 (CEST) Received: from [10.60.67.85] (ams-bclaise-8914.cisco.com [10.60.67.85]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6MERa0Q006079 for ; Mon, 22 Jul 2013 16:27:47 +0200 (CEST) Message-ID: <51ED4150.6070907@cisco.com> Date: Mon, 22 Jul 2013 16:27:28 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; WOW64; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: "lmap@ietf.org" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [lmap] Feedback on draft-nagami-lmap-use-case-measurement-provider X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jul 2013 14:31:08 -0000 Dear authors, Next to the points I made regarding to draft-linsner-lmap-use-cases, which are applicable here as well, I have one extra point: I like the fact that the draft covers the smartphone. This use case is completely different. I might not care if a MA generates some packets from my home, but I care if this is from my smartphone and my bill is pay per volume. The roaming price are still expensive. One of the goal behind this email is to generate discussions, either on the list, or during the IETF meeting next week. Regards, Benoit From bclaise@cisco.com Mon Jul 22 09:12:23 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 074B011E80E7 for ; Mon, 22 Jul 2013 09:12:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.523 X-Spam-Level: X-Spam-Status: No, score=-10.523 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MEieh4wsm6s3 for ; Mon, 22 Jul 2013 09:12:18 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 3E0C521E808D for ; Mon, 22 Jul 2013 09:12:16 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6MGCB5N001210 for ; Mon, 22 Jul 2013 18:12:11 +0200 (CEST) Received: from [10.60.67.85] (ams-bclaise-8914.cisco.com [10.60.67.85]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6MGBe7N008135 for ; Mon, 22 Jul 2013 18:11:51 +0200 (CEST) Message-ID: <51ED59B3.3040701@cisco.com> Date: Mon, 22 Jul 2013 18:11:31 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; WOW64; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: "lmap@ietf.org" Content-Type: multipart/alternative; boundary="------------090301080604040703080906" Subject: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jul 2013 16:12:23 -0000 This is a multi-part message in MIME format. --------------090301080604040703080906 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear authors, A couple of high level points. One of the goal behind this email is to generate discussions, either on the list, or during the IETF meeting next week. - The consensus is that the LMAP working group should assume that a Measurement Agent receives Instruction from only a single Controller at any point in time (however it may Report to more than one Collector). Instead of consensus, I would use "a key assumption" The charter says: A key assumption constraining the initial work is that the measurement system is under the control of a single organization (for example, an Internet Service Provider or a regulator). - Same remark for the term "consensus" in The job of a Bootstrap Protocol is to provide an automated way to associate a Measurement Agent to its Controller, including authentication credentials. Similarly, there should be a way to pull the plug on rogue Measurement Agents. The current consensus on the LMAP mailing list is that the working group should define the bootstrap process but not a protocol. The charter mentions: "The management protocol to bootstrap the MAs in measurement devices is out of scope of the LMAP charter." - I would like to draw your attention to claise-ippm-perf-metric-registry , which will be presented in IPPM. - I'm not too clear if the Measurement Peer are dedicated test server, or real server (google.com, SIP server, youtube) Do the Measurement Peer need the functionality of an IP SLA responder or a TWAMP controller? This might be clarified in the use case draft. - This draft is good shape, but it's really a mix of terminology and framework concepts. I'm glad that there is a single delivery in the charter for both: "1. The LMAP Framework - provides common terminology, basic architecture elements, and justifies the simplifying constraints" Regards, Benoit --------------090301080604040703080906 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear authors,

A couple of high level points.
One of the goal behind this email is to generate discussions, either on the list, or during the IETF meeting next week.

- 
   The consensus is that the LMAP working group should assume that a
   Measurement Agent receives Instruction from only a single Controller
   at any point in time (however it may Report to more than one
   Collector).
Instead of consensus, I would use "a key assumption"
The charter says:
A key assumption constraining the initial work is that the measurement system is under the control of a single organization (for example, an Internet Service Provider or a regulator).

-
Same remark for the term "consensus" in 
   The job of a Bootstrap Protocol is to provide an automated way to
   associate a Measurement Agent to its Controller, including
   authentication credentials.  Similarly, there should be a way to pull
   the plug on rogue Measurement Agents.  The current consensus on the
   LMAP mailing list is that the working group should define the
   bootstrap process but not a protocol.
The charter mentions:
"The management protocol to bootstrap the MAs in measurement devices is out of scope of the LMAP charter."
-
I would like to draw your attention to claise-ippm-perf-metric-registry, which will be presented in IPPM.

-
I'm not too clear if the Measurement Peer are dedicated test server, or real server (google.com, SIP server, youtube)
Do the Measurement Peer need the functionality of an IP SLA responder or a TWAMP controller?
This might be clarified in the use case draft.

-
This draft is good shape, but it's really a mix of terminology and framework concepts.
I'm glad that there is a single delivery in the charter for both:
"1. The LMAP Framework - provides common terminology, basic architecture elements, and justifies the simplifying constraints"


Regards, Benoit
--------------090301080604040703080906-- From dromasca@avaya.com Tue Jul 23 06:02:34 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8EE5311E810A for ; Tue, 23 Jul 2013 06:02:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rvDbsoUjE0rK for ; Tue, 23 Jul 2013 06:02:28 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id 05D5A11E80FD for ; Tue, 23 Jul 2013 06:02:27 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AigFAOh97lGHCzI1/2dsb2JhbABbgmUhgQXAOoERFnSCJgEBAxIoUQEVBRAUQhcPAQQbGoduAZd1hEKcJo9ig0huA54jiweDEoIq X-IronPort-AV: E=Sophos;i="4.89,727,1367985600"; d="scan'208";a="21186164" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 23 Jul 2013 09:02:20 -0400 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 23 Jul 2013 08:56:05 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Tue, 23 Jul 2013 09:02:19 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: comments on draft-eardley-lmap-framework-02 Thread-Index: Ac6HpNxWSlKt/J3PRHm+kNP0wHNi/g== Date: Tue, 23 Jul 2013 13:02:18 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1287E1FE@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] comments on draft-eardley-lmap-framework-02 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jul 2013 13:02:34 -0000 Hi, Here are a few comments at the first reading of draft-eardley-lmap-franewor= k-02. I appreciate that this is one of the pre-wg-document contributions, s= o my comments and questions are rather high level than trying to point to d= etails.=20 1. I am confused by the way the I-D describes the interaction between contr= ollers and MAs. In Section 3.2 I see:=20 To avoid problems with NAT and firewalls, it is likely that the MA 'pulls' the configuration from its Controller, as identified by the Initialiser. But then in the diagram in Figure 1 the arrow points from the Controller to= the MA, and later the examples of possible protocols in Section 5.3 includ= e such 'push' protocols as SNMP and NETCONF.=20 2. I have a problem imagining what the 'Initialiser' is. Is it a task (runn= ing where?) aiming to ensure consistent configuration of controllers in an = LMAP system. Or another 'box' in the architecture, part of an hierarchical = system, a 'box' not represented because it is out of the LMAP scope? If the= later I would prefer to add it to the diagram, maybe together with the oth= er non-LMAP entities (Subscriber Parameter Database, Results Database) so t= hat we can have a full picture of the whole system, with clear representati= on of what is in scope for LMAP.=20 3. Do we also need to include in the diagram (and discussion) some kind a c= ertificate authority to ensure that controllers and MAs, MAs and peers, MA = and collectors, are authenticated to each other, as the security considerat= ions section demands.=20 4. In a couple of instances OAM is wrongly expanded as Operations, Administ= rations and Management, and not as per BCP 161.=20 5. In Section 5.1 there are a few instances where some procedures or blocks= are being defined as 'out (or beyond) the scope of the IETF'. It would be = more correct to say 'out of scope for LMAP' as we can make statements about= what we do (or do not do) in LMAP but not for the IETF as a whole.=20 Thanks and Regards,=20 Dan (as contributor)=20 From philip.eardley@bt.com Wed Jul 24 01:00:43 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1FB2711E8396 for ; Wed, 24 Jul 2013 01:00:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.854 X-Spam-Level: X-Spam-Status: No, score=-102.854 tagged_above=-999 required=5 tests=[AWL=0.744, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3xiCoJHmsCrD for ; Wed, 24 Jul 2013 01:00:39 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 81B1B11E80D1 for ; Wed, 24 Jul 2013 01:00:38 -0700 (PDT) Received: from EVMHT66-UKRD.domain1.systemhost.net (10.36.3.103) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.298.1; Wed, 24 Jul 2013 09:00:35 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.253]) by EVMHT66-UKRD.domain1.systemhost.net ([10.36.3.103]) with mapi; Wed, 24 Jul 2013 09:00:35 +0100 From: To: , Date: Wed, 24 Jul 2013 09:00:34 +0100 Thread-Topic: [lmap] Feedback on draft-eardley-lmap-terminology Thread-Index: Ac6G9kJL9LoTtC34Rxe0kMCmPNNKaQBSjDvA Message-ID: <9510D26531EF184D9017DF24659BB87F35B7CD1AF5@EMV65-UKRD.domain1.systemhost.net> References: <51ED59B3.3040701@cisco.com> In-Reply-To: <51ED59B3.3040701@cisco.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F35B7CD1AF5EMV65UKRDdoma_" MIME-Version: 1.0 Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 08:00:43 -0000 --_000_9510D26531EF184D9017DF24659BB87F35B7CD1AF5EMV65UKRDdoma_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Benoit, Thanks for your comments! I agree with you that "key assumption" is more accurate than "consensus" I agree that Section 4, "Commentary and notes" wanders into the territory o= f the Framework. At some point soon, as you say, terminology will be merged= into a framework i-d, so this will need to be fixed. I'll try to ensure th= at the merged doc keeps some text that explains the terminology a bit (pers= onally I find terminology on its own hard to understand, without some accom= panying explanation) Ps your registry i-d is now on my reading list, thanks << I'm not too clear if the Measurement Peer are dedicated test server, or = real server (google.com, SIP server, youtube)>> The Measurement Peer could be either. It's up to the operator of the measur= ement system to decide what Measurement Methods the Measurement Peer needs = and exactly how they're implemented. Thanks phil From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Ben= oit Claise Sent: 22 July 2013 17:12 To: lmap@ietf.org Subject: [lmap] Feedback on draft-eardley-lmap-terminology Dear authors, A couple of high level points. One of the goal behind this email is to generate discussions, either on the= list, or during the IETF meeting next week. - The consensus is that the LMAP working group should assume that a Measurement Agent receives Instruction from only a single Controller at any point in time (however it may Report to more than one Collector). Instead of consensus, I would use "a key assumption" The charter says: A key assumption constraining the initial work is that the measurement syst= em is under the control of a single organization (for example, an Internet = Service Provider or a regulator). - Same remark for the term "consensus" in The job of a Bootstrap Protocol is to provide an automated way to associate a Measurement Agent to its Controller, including authentication credentials. Similarly, there should be a way to pull the plug on rogue Measurement Agents. The current consensus on the LMAP mailing list is that the working group should define the bootstrap process but not a protocol. The charter mentions: "The management protocol to bootstrap the MAs in measurement devices is out= of scope of the LMAP charter." - I would like to draw your attention to claise-ippm-perf-metric-registry, which will= be presented in IPPM. - I'm not too clear if the Measurement Peer are dedicated test server, or rea= l server (google.com, SIP server, youtube) Do the Measurement Peer need the functionality of an IP SLA responder or a = TWAMP controller? This might be clarified in the use case draft. - This draft is good shape, but it's really a mix of terminology and framewor= k concepts. I'm glad that there is a single delivery in the charter for both: "1. The LMAP Framework - provides common terminology, basic architecture el= ements, and justifies the simplifying constraints" Regards, Benoit --_000_9510D26531EF184D9017DF24659BB87F35B7CD1AF5EMV65UKRDdoma_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

= Benoit,

 

Thanks for your comme= nts!

 

I agree w= ith you that “key assumption” is more accurate than “cons= ensus”

 

I= agree that Section 4, “Commentary and notes” wanders into the territory of the Framewor= k. At some point soon, as you say, terminology will be merged into a framew= ork i-d, so this will need to be fixed. I’ll try to ensure that the m= erged doc keeps some text that explains the terminology a bit (personally I= find terminology on its own hard to understand, without some accompanying = explanation)

 

Ps your registry i-d is now on my r= eading list, thanks

 

<<= ; I'm not too clear if the Measurement Pe= er are dedicated test server, or real server (google.com, SIP server, youtu= be)>>

The Measurement Peer cou= ld be either. It’s up to the operator of the measurement system to de= cide what Measurement Methods the Measurement Peer needs and exactly how th= ey’re implemented.

<= span style=3D'font-size:10.0pt;font-family:"Courier New";color:windowtext'>=  

Thanks

<= p class=3DMsoNormal>phil

=  

From: lmap-bounces@ietf.org [mailto:l= map-bounces@ietf.org] On Behalf Of Benoit Claise
Sent: 22 = July 2013 17:12
To: lmap@ietf.org
Subject: [lmap] Feedb= ack on draft-eardley-lmap-terminology

 

Dear authors,
A couple of high level points.
One of the goal behind this email is= to generate discussions, either on the list, or during the IETF meeting ne= xt week.

- 

   The consensus is t=
hat the LMAP working group should assume that a
 =
  Measurement Agent receives Instruction from only a single Controller=

   at any point in time (however it may Rep=
ort to more than one
   Collector).

Instead of consensus, I would use "a key = assumption"
The charter says:

A key assumption constraining the initial work= is that the measurement system is under the control of a single organizati= on (for example, an Internet Service Provider or a regulator).

-
Same remark for the term "consensus"= in 

   The job of a Bootstrap Protocol i=
s to provide an automated way to
   associat=
e a Measurement Agent to its Controller, including
&nb=
sp;  authentication credentials.  Similarly, there should be a wa=
y to pull
   the plug on rogue Measurement A=
gents.  The current consensus on the
  =
 LMAP mailing list is that the working group should define the
   bootstrap process but not a protocol.  
The charter mentions:
"The management protocol to bootstrap the MAs in measurem=
ent devices is out of scope of the LMAP charter."
-
I would like to draw your attention to claise-ippm-p=
erf-metric-registry, which will be presented in IPPM.

- 
I'm =
not too clear if the Measurement Peer are dedicated test server, or real se=
rver (google.com, SIP server, youtube)
Do the Measurement Peer need the =
functionality of an IP SLA responder or a TWAMP controller?
This might b=
e clarified in the use case draft.

- 
This draft is good shape, b=
ut it's really a mix of terminology and framework concepts.
I'm glad tha=
t there is a single delivery in the charter for both:
"1. The LMAP =
Framework - provides common terminology, basic architecture elements, and j=
ustifies the simplifying constraints"


Regards, Benoit<=
/o:p>
= --_000_9510D26531EF184D9017DF24659BB87F35B7CD1AF5EMV65UKRDdoma_-- From bclaise@cisco.com Wed Jul 24 01:32:21 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 113C911E83BB for ; Wed, 24 Jul 2013 01:32:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.598 X-Spam-Level: X-Spam-Status: No, score=-10.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S5k5-gxY1PeP for ; Wed, 24 Jul 2013 01:32:16 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 4060911E83A3 for ; Wed, 24 Jul 2013 01:32:14 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6O8W9fe017397; Wed, 24 Jul 2013 10:32:09 +0200 (CEST) Received: from [10.60.67.85] (ams-bclaise-8914.cisco.com [10.60.67.85]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6O8VlZR014482; Wed, 24 Jul 2013 10:31:57 +0200 (CEST) Message-ID: <51EF90E4.6000907@cisco.com> Date: Wed, 24 Jul 2013 10:31:32 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; WOW64; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: philip.eardley@bt.com References: <51ED59B3.3040701@cisco.com> <9510D26531EF184D9017DF24659BB87F35B7CD1AF5@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F35B7CD1AF5@EMV65-UKRD.domain1.systemhost.net> Content-Type: multipart/alternative; boundary="------------050902070104090009010201" Cc: lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 08:32:21 -0000 This is a multi-part message in MIME format. --------------050902070104090009010201 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Phil, > > ** > > *<<*I'm not too clear if the Measurement Peer are dedicated test > server, or real server (google.com, SIP server, youtube)>> > > *The Measurement Peer could be either. It's up to the operator of the > measurement system to decide what Measurement Methods the Measurement > Peer needs and exactly how they're implemented.* > Sure, but what if I point all my 100.000 MAs to constantly ping http://home.bt.com/news-01363796671918. Not sure if BT will be happy. I could take a different example with DNS, NTP, CDN, ... That might be a point for an applicability section somewhere. Regards, Benoit > > ** > > ** > > Thanks > > phil** > > *From:*lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] *On Behalf > Of *Benoit Claise > *Sent:* 22 July 2013 17:12 > *To:* lmap@ietf.org > *Subject:* [lmap] Feedback on draft-eardley-lmap-terminology > > Dear authors, > > A couple of high level points. > One of the goal behind this email is to generate discussions, either > on the list, or during the IETF meeting next week. > > - > > The consensus is that the LMAP working group should assume that a > Measurement Agent receives Instruction from only a single Controller > at any point in time (however it may Report to more than one > Collector). > > Instead of consensus, I would use "a key assumption" > The charter says: > > A key assumption constraining the initial work is that the measurement > system is under the control of a single organization (for example, an > Internet Service Provider or a regulator). > > - > Same remark for the term "consensus" in > > The job of a Bootstrap Protocol is to provide an automated way to > associate a Measurement Agent to its Controller, including > authentication credentials. Similarly, there should be a way to pull > the plug on rogue Measurement Agents. The current consensus on the > LMAP mailing list is that the working group should define the > bootstrap process but not a protocol. > > The charter mentions: > > "The management protocol to bootstrap the MAs in measurement devices > is out of scope of the LMAP charter." > > - > I would like to draw your attention to > claise-ippm-perf-metric-registry > , > which will be presented in IPPM. > > - > I'm not too clear if the Measurement Peer are dedicated test server, > or real server (google.com, SIP server, youtube) > Do the Measurement Peer need the functionality of an IP SLA responder > or a TWAMP controller? > This might be clarified in the use case draft. > > - > This draft is good shape, but it's really a mix of terminology and > framework concepts. > I'm glad that there is a single delivery in the charter for both: > "1. The LMAP Framework - provides common terminology, basic > architecture elements, and justifies the simplifying constraints" > > > Regards, Benoit > --------------050902070104090009010201 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Phil,

 

<< I'm not too clear if the Measurement Peer are dedicated test server, or real server (google.com, SIP server, youtube)>>

The Measurement Peer could be either. It’s up to the operator of the measurement system to decide what Measurement Methods the Measurement Peer needs and exactly how they’re implemented.

Sure, but what if I point all my 100.000 MAs to constantly ping http://home.bt.com/news-01363796671918. Not sure if BT will be happy.
I could take a different example with DNS, NTP, CDN, ...
That might be a point for an applicability section somewhere.

Regards, Benoit

 

Thanks

phil

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Benoit Claise
Sent: 22 July 2013 17:12
To: lmap@ietf.org
Subject: [lmap] Feedback on draft-eardley-lmap-terminology

 

Dear authors,

A couple of high level points.
One of the goal behind this email is to generate discussions, either on the list, or during the IETF meeting next week.

- 

   The consensus is that the LMAP working group should assume that a
   Measurement Agent receives Instruction from only a single Controller
   at any point in time (however it may Report to more than one
   Collector).

Instead of consensus, I would use "a key assumption"
The charter says:

A key assumption constraining the initial work is that the measurement system is under the control of a single organization (for example, an Internet Service Provider or a regulator).

-
Same remark for the term "consensus" in 

   The job of a Bootstrap Protocol is to provide an automated way to
   associate a Measurement Agent to its Controller, including
   authentication credentials.  Similarly, there should be a way to pull
   the plug on rogue Measurement Agents.  The current consensus on the
   LMAP mailing list is that the working group should define the
   bootstrap process but not a protocol. 

The charter mentions:

"The management protocol to bootstrap the MAs in measurement devices is out of scope of the LMAP charter."

-
I would like to draw your attention to claise-ippm-perf-metric-registry, which will be presented in IPPM.

-
I'm not too clear if the Measurement Peer are dedicated test server, or real server (google.com, SIP server, youtube)
Do the Measurement Peer need the functionality of an IP SLA responder or a TWAMP controller?
This might be clarified in the use case draft.

-
This draft is good shape, but it's really a mix of terminology and framework concepts.
I'm glad that there is a single delivery in the charter for both:
"1. The LMAP Framework - provides common terminology, basic architecture elements, and justifies the simplifying constraints"


Regards, Benoit


--------------050902070104090009010201-- From bclaise@cisco.com Wed Jul 24 03:21:06 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B2A6011E83C5 for ; Wed, 24 Jul 2013 03:21:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.598 X-Spam-Level: X-Spam-Status: No, score=-10.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z5-TwQPDnutA for ; Wed, 24 Jul 2013 03:21:02 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id D165011E83A4 for ; Wed, 24 Jul 2013 03:21:01 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6OAKuql001521; Wed, 24 Jul 2013 12:20:56 +0200 (CEST) Received: from [10.60.67.85] (ams-bclaise-8914.cisco.com [10.60.67.85]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6OAKGSL015417; Wed, 24 Jul 2013 12:20:26 +0200 (CEST) Message-ID: <51EFAA51.6020204@cisco.com> Date: Wed, 24 Jul 2013 12:20:01 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; WOW64; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: "lmap@ietf.org" References: <51EFA59C.7000807@bwijnen.net> In-Reply-To: <51EFA59C.7000807@bwijnen.net> X-Forwarded-Message-Id: <51EFA59C.7000807@bwijnen.net> Content-Type: multipart/alternative; boundary="------------030804020305010309050809" Cc: "Bert \(IETF\) Wijnen" Subject: [lmap] Fwd: Re: [87attendees] Wifi and RIPE ATLAS probes X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 10:21:07 -0000 This is a multi-part message in MIME format. --------------030804020305010309050809 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit This email might be interesting the LMAP people attending the IETF next week. Regards, Benoit -------- Original Message -------- Subject: Re: [87attendees] Wifi and RIPE ATLAS probes Date: Wed, 24 Jul 2013 11:59:56 +0200 From: Bert Wijnen (IETF) To: Nathalie Trenaman CC: 87attendees@ietf.org, Brian Dickson , Paul Aitken Hi, the RIPE Atlas probes (the newer versions) do have some written label on it that may give you the impression they can do wifi. They cannot. They do RIPE ATLAS measurements. By the way, I will be bringing (as Nathalie wrote) some more RIPE ATLAS probes to the IETF87 in Berlin. If people are interested to host one, pls see me )or one of my colleagues. If you want to be sure to get one, pls let me know in advance, no later than this Friday noon UTC and I can make sure I keep one for you. Otherwise it is: FCFS. for details about RIPE ATLAS go to: https://atlas.ripe.net Apologies if anyone sees this as spam/plug/ad Bert On 7/24/13 10:49 AM, Nathalie Trenaman wrote: > Hi Brian, > > On Jul 23, 2013, at 5:01 PM, Brian Dickson wrote: > >> Maybe some of the 150 people who picked up RIPE probes at the last NANOG conference, could bring them along and use them to provide "free" WiFi? > > The probes are not access points, just measurement devices. > >> >> Or, perhaps the folks giving away the RIPE probes could bring some with them to give to overflow guests? > > That is not a problem :) Me and my NCC colleagues will bring some probes. Not 150, though! Maybe around 40. > > >> >> (This will improve RIPE probe coverage when a more diverse set of folks take them home and plug them in, after the IETF). > > Yes, we were hoping for that :) As you can see on the map: https://atlas.ripe.net/active_probes_map the coverage for Europe is very decent but there are still some > places in the world that could use a probe. If you can manage to set up a probe there, please let me know in Berlin :) > > Nathalie Trenaman > RIPE NCC Trainer > > > _______________________________________________ > 87attendees mailing list > 87attendees@ietf.org > https://www.ietf.org/mailman/listinfo/87attendees > _______________________________________________ 87attendees mailing list 87attendees@ietf.org https://www.ietf.org/mailman/listinfo/87attendees --------------030804020305010309050809 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit This email might be interesting the LMAP people attending the IETF next week.

Regards, Benoit


-------- Original Message --------
Subject: Re: [87attendees] Wifi and RIPE ATLAS probes
Date: Wed, 24 Jul 2013 11:59:56 +0200
From: Bert Wijnen (IETF) <bertietf@bwijnen.net>
To: Nathalie Trenaman <nathalie@ripe.net>
CC: 87attendees@ietf.org, Brian Dickson <brian.peter.dickson@gmail.com>, Paul Aitken <paitken@cisco.com> 


Hi, the RIPE Atlas probes (the newer versions) do have some written
label on it that may give you the impression they can do wifi.
They cannot. They do RIPE ATLAS measurements.

By the way, I will be bringing (as Nathalie wrote) some more RIPE
ATLAS probes to the IETF87 in Berlin. If people are interested to
host one, pls see me )or one of my colleagues.

If you want to be sure to get one, pls let me know in advance,
no later than this Friday noon UTC and I can make sure I keep one
for you. Otherwise it is: FCFS.

for details about RIPE ATLAS go to: https://atlas.ripe.net
Apologies if anyone sees this as spam/plug/ad

Bert

On 7/24/13 10:49 AM, Nathalie Trenaman wrote:
> Hi Brian,
>
> On Jul 23, 2013, at 5:01 PM, Brian Dickson wrote:
>
>> Maybe some of the 150 people who picked up RIPE probes at the last NANOG conference, could bring them along and use them to provide "free" WiFi?
>
> The probes are not access points, just measurement devices.
>
>>
>> Or, perhaps the folks giving away the RIPE probes could bring some with them to give to overflow guests?
>
> That is not a problem :) Me and my NCC colleagues will bring some probes. Not 150, though! Maybe around 40.
>
>
>>
>> (This will improve RIPE probe coverage when a more diverse set of folks take them home and plug them in, after the IETF).
>
> Yes, we were hoping for that :) As you can see on the map: https://atlas.ripe.net/active_probes_map the coverage for Europe is very decent but there are still some
> places in the world that could use a probe. If you can manage to set up a probe there, please let me know in Berlin :)
>
> Nathalie Trenaman
> RIPE NCC Trainer
>
>
> _______________________________________________
> 87attendees mailing list
> 87attendees@ietf.org
> https://www.ietf.org/mailman/listinfo/87attendees
>
_______________________________________________
87attendees mailing list
87attendees@ietf.org
https://www.ietf.org/mailman/listinfo/87attendees


--------------030804020305010309050809-- From dromasca@avaya.com Wed Jul 24 04:28:30 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4DD6411E8200 for ; Wed, 24 Jul 2013 04:28:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.099 X-Spam-Level: X-Spam-Status: No, score=-103.099 tagged_above=-999 required=5 tests=[AWL=0.500, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lLKkviAbndIv for ; Wed, 24 Jul 2013 04:28:23 -0700 (PDT) Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id EF3AB11E81D0 for ; Wed, 24 Jul 2013 04:28:19 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhYKAA6671GHCzI1/2dsb2JhbABbgmUhgQXAfAMBgRYWdIImAQEDEihRARUFEAsJQhcPAQQbEweHbgGYWIRCnA+PDjtCgwhuA5QIih2LB4MUgio X-IronPort-AV: E=Sophos;i="4.89,734,1367985600"; d="scan'208";a="17616391" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by de307622-de-outbound.net.avaya.com with ESMTP; 24 Jul 2013 07:28:18 -0400 Received: from unknown (HELO AZ-FFEXHC02.global.avaya.com) ([135.64.58.12]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 24 Jul 2013 07:22:00 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC02.global.avaya.com ([135.64.58.12]) with mapi id 14.02.0328.009; Wed, 24 Jul 2013 13:28:16 +0200 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: my comments on draft-akhter-lmap-framework-00 Thread-Index: Ac6IYOM5ZXCAYUqURvC81oEySuxFpA== Date: Wed, 24 Jul 2013 11:28:15 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 11:28:30 -0000 Please find below a set of comments after my first reading of draft-akhter-= lmap-framework-00. These are made from the perspective of contributor, and = take into account the fact that this is a 00 individual submission, so idni= ts was not run, spelling and references were not checked, etc.=20 1. The name of the I-D speaks about 'Inventory' which is a little bit confu= sing or mis-reading. Actually the authors rather seem to have described som= e applications and protocols that fit into the LMAP architecture, so it's r= ather an incomplete 'taxonomy'. See more when talking about the (lack of) r= equirements.=20 2. The previous contributions on LMAP framework included in draft-eardley-l= map-framework were completely ignored, some material is duplicated (in cont= ent, not in text), but the eardley I-D is not even included in the list of = references. As the two I-Ds (and other relevant contributions) will need to= be consolidated into one single WG I-D pretty soon I will refer in the fol= lowing to some of the similarities and some of the differences.=20 3. The order of Sections 1.2 and 1.3 should be reversed.=20 4. Figure 1 is almost identical to Figure 1 in draft-eardley with two diffe= rences. The first is that the entity named Measurement Peer in draft-eardle= y is called here Remote Measurement Test Target and seems to have a much ri= cher functionality and variants as illustrated in Sections 3.2 and Figure 2= . The option of one measurement agent communicating with multiple Remote Me= asurement Agents needs to bear a warning concerning scalability.=20 5. The second significant difference in the two diagrams is the communicati= on between the Controller and the Collector. Later in the draft this is des= cribed as optional and motivated by the need to optimize for Collector Test= Configuration synchronization (so that each MA needs not report its config= uration to controllers). This addition to the basic architecture needs to b= e carefully discussed by the WG.=20 6. One crisp limitation that is being introduced by draft-eardley is about = each MA being configured only by one Controller (motivated among other by t= he need of simplifying authentication of Controllers with the MA). This lim= itation is not mentioned in this I-D, at least not explicitly. Is this an e= ditorial miss, or the authors take a different stand on this issue?=20 7. Sections 3,4,5 are to some extent written with one set of protocols alre= ady in mind to answer LMAP architectural requirements (OWAMP and TWAMP for = active monitoring, PSAMP for passive monitoring, IPFIX for reporting protoc= ol, IPFIX mediators to solve the scalability issues). Although this may be = a reasonable set of solutions, it seems a little bit too early and too deta= iled at this phase. Missing here are the basic requirements for the Control= protocol, Report protocol, and associated data models. I would prefer to s= ee this being written first.=20 8. The Security Consideration section needs to include not only information= about authentication, but also about the risks of modification of informat= ion (data integrity). Thanks and Regards, Dan (as contributor) From philip.eardley@bt.com Wed Jul 24 04:46:13 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2936411E816B for ; Wed, 24 Jul 2013 04:46:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.926 X-Spam-Level: X-Spam-Status: No, score=-102.926 tagged_above=-999 required=5 tests=[AWL=0.072, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y5JrDC+2ARnn for ; Wed, 24 Jul 2013 04:46:09 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp61.intersmtp.com [62.239.224.234]) by ietfa.amsl.com (Postfix) with ESMTP id B059611E80E0 for ; Wed, 24 Jul 2013 04:46:08 -0700 (PDT) Received: from EVMHT65-UKRD.domain1.systemhost.net (10.36.3.102) by RDW083A005ED61.smtp-e1.hygiene.service (10.187.98.10) with Microsoft SMTP Server (TLS) id 8.3.298.1; Wed, 24 Jul 2013 12:46:03 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.253]) by EVMHT65-UKRD.domain1.systemhost.net ([10.36.3.102]) with mapi; Wed, 24 Jul 2013 12:46:03 +0100 From: To: Date: Wed, 24 Jul 2013 12:46:02 +0100 Thread-Topic: [lmap] Feedback on draft-eardley-lmap-terminology Thread-Index: Ac6ISEypJJA759Q0QNmvxWMfGSmlewAGaYjg Message-ID: <9510D26531EF184D9017DF24659BB87F35B7CD1D59@EMV65-UKRD.domain1.systemhost.net> References: <51ED59B3.3040701@cisco.com> <9510D26531EF184D9017DF24659BB87F35B7CD1AF5@EMV65-UKRD.domain1.systemhost.net> <51EF90E4.6000907@cisco.com> In-Reply-To: <51EF90E4.6000907@cisco.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F35B7CD1D59EMV65UKRDdoma_" MIME-Version: 1.0 Cc: lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 11:46:13 -0000 --_000_9510D26531EF184D9017DF24659BB87F35B7CD1D59EMV65UKRDdoma_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Ok, this should be mentioned in the Use case doc as well as in the framewor= k. incidentally, although the measurement functionality would be in lots of MA= s I wouldn't expect them all to run all the tests at the same time - for th= e regular tests, I think you'd use a subset as a panel, and over the course= of a few weeks say the panel would rotate round the different end hosts I agree that the measurement functionality has the potential to be abused a= s a DDOS attack and we need mechanisms to alleviate it. Perhaps too early for specific proposals, but I'd suggest a combination of: - Security on the Control Protocol (to stop rogue controller asking M= As to launch a DDoS) - Some individual tests to include an initial phase where they check = that the Measurement Peer has capacity (especially for tests that create a = significant load on the Measurement Peer) - Ability for Controller to send a "suppress" msg - so lots of MAs ca= n be told to stop sending test pkts, if there's some problem - Perhaps some guidance on best practice (don't kill bt.com - try cis= co.com instead!!) Thanks phil From: Benoit Claise [mailto:bclaise@cisco.com] Sent: 24 July 2013 09:32 To: Eardley,PL,Philip,TUB8 R Cc: lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology Phil, << I'm not too clear if the Measurement Peer are dedicated test server, or = real server (google.com, SIP server, youtube)>> The Measurement Peer could be either. It's up to the operator of the measur= ement system to decide what Measurement Methods the Measurement Peer needs = and exactly how they're implemented. Sure, but what if I point all my 100.000 MAs to constantly ping http://home= .bt.com/news-01363796671918. Not sure if BT will be happy. I could take a different example with DNS, NTP, CDN, ... That might be a point for an applicability section somewhere. Regards, Benoit Thanks phil From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of Benoit Claise Sent: 22 July 2013 17:12 To: lmap@ietf.org Subject: [lmap] Feedback on draft-eardley-lmap-terminology Dear authors, A couple of high level points. One of the goal behind this email is to generate discussions, either on the= list, or during the IETF meeting next week. - The consensus is that the LMAP working group should assume that a Measurement Agent receives Instruction from only a single Controller at any point in time (however it may Report to more than one Collector). Instead of consensus, I would use "a key assumption" The charter says: A key assumption constraining the initial work is that the measurement syst= em is under the control of a single organization (for example, an Internet = Service Provider or a regulator). - Same remark for the term "consensus" in The job of a Bootstrap Protocol is to provide an automated way to associate a Measurement Agent to its Controller, including authentication credentials. Similarly, there should be a way to pull the plug on rogue Measurement Agents. The current consensus on the LMAP mailing list is that the working group should define the bootstrap process but not a protocol. The charter mentions: "The management protocol to bootstrap the MAs in measurement devices is out= of scope of the LMAP charter." - I would like to draw your attention to claise-ippm-perf-metric-registry, which will= be presented in IPPM. - I'm not too clear if the Measurement Peer are dedicated test server, or rea= l server (google.com, SIP server, youtube) Do the Measurement Peer need the functionality of an IP SLA responder or a = TWAMP controller? This might be clarified in the use case draft. - This draft is good shape, but it's really a mix of terminology and framewor= k concepts. I'm glad that there is a single delivery in the charter for both: "1. The LMAP Framework - provides common terminology, basic architecture el= ements, and justifies the simplifying constraints" Regards, Benoit --_000_9510D26531EF184D9017DF24659BB87F35B7CD1D59EMV65UKRDdoma_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

= Ok, this should= be mentioned in the Use case doc as well as in the framework.

 

incidentally, although the= measurement functionality would be in lots of MAs I wouldn’t expect = them all to run all the tests at the same time – for the regular test= s, I think you’d use a subset as a panel, and over the course of a fe= w weeks say the panel would rotate round the different end hosts=

 

I agree that the measure= ment functionality has the potential to be abused as a DDOS attack and we n= eed mechanisms to alleviate it.

<= span style=3D'font-family:"Arial","sans-serif";color:blue'>Perhaps too earl= y for specific proposals, but I’d suggest a combination of:

-       Security on the Control Protocol (to stop rogue controller asking MAs= to launch a DDoS)

-   = ;    Some individual tests to include an ini= tial phase where they check that the Measurement Peer has capacity (especia= lly for tests that create a significant load on the Measurement Peer)<= /o:p>

-       Ability for Controller to send a “suppress” msg = 211; so lots of MAs can be told to stop sending test pkts, if there’s= some problem

-    = ;   Perhaps some guidance on best practice (don&#= 8217;t kill bt.com – try cisco.com instead!!)

 

Thanks

ph= il

 

From: Benoit Claise [mailto:bclaise@cisco.com]
Sent: 24 = July 2013 09:32
To: Eardley,PL,Philip,TUB8 R
Cc: lmap@i= etf.org
Subject: Re: [lmap] Feedback on draft-eardley-lmap-termin= ology

 

Phil,

 

&= lt;< I'm not too clear if the Measurem= ent Peer are dedicated test server, or real server (google.com, SIP server,= youtube)>>

The Measurement Peer co= uld be either. It’s up to the operator of the measurement system to d= ecide what Measurement Methods the Measurement Peer needs and exactly how t= hey’re implemented.

Sure, but what if I point all my 100.000 MAs to constantly ping http://home.bt.com/news-0136379= 6671918. Not sure if BT will be happy.
I could take a different exam= ple with DNS, NTP, CDN, ...
That might be a point for an applicability s= ection somewhere.

Regards, Benoit

=  

Thanks

phil

 

<= b>From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Benoit Claise
Sent: 22 July 2013 17:12
To: lmap@ietf.org
Subject: [lmap] Feedback = on draft-eardley-lmap-terminology

&= nbsp;

Dear authors,

A couple of high level poin= ts.
One of the goal behind this email is to generate discussions, either= on the list, or during the IETF meeting next week.

- 

   The consensus is that the LMAP working group shoul=
d assume that a
   Measurement Agent receive=
s Instruction from only a single Controller
 &nbs=
p; at any point in time (however it may Report to more than one<=
/pre>
   Collector).
Instead of consen=
sus, I would use "a key assumption"
The charter says:
A key assumption constraining the initial work is that the measureme=
nt system is under the control of a single organization (for example, an In=
ternet Service Provider or a regulator).
- 
Same re=
mark for the term "consensus" in 
  =
 The job of a Bootstrap Protocol is to provide an automated way to
   associate a Measurement Agent to its Control=
ler, including
   authentication credentials=
.  Similarly, there should be a way to pull
 =
;  the plug on rogue Measurement Agents.  The current consensus o=
n the
   LMAP mailing list is that the worki=
ng group should define the
   bootstrap proc=
ess but not a protocol.  
The charter mention=
s:
"The management protocol to bootstrap the MAs =
in measurement devices is out of scope of the LMAP charter."
-
I would like to draw your attention to claise-ippm-perf=
-metric-registry, which will be presented in IPPM.

- 
I'm not=
 too clear if the Measurement Peer are dedicated test server, or real serve=
r (google.com, SIP server, youtube)
Do the Measurement Peer need the fun=
ctionality of an IP SLA responder or a TWAMP controller?
This might be c=
larified in the use case draft.

- 
This draft is good shape, but =
it's really a mix of terminology and framework concepts.
I'm glad that t=
here is a single delivery in the charter for both:
"1. The LMAP Fra=
mework - provides common terminology, basic architecture elements, and just=
ifies the simplifying constraints"


Regards, Benoit

 

= --_000_9510D26531EF184D9017DF24659BB87F35B7CD1D59EMV65UKRDdoma_-- From acmorton@att.com Wed Jul 24 06:04:49 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EB93511E80E6 for ; Wed, 24 Jul 2013 06:04:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.299 X-Spam-Level: X-Spam-Status: No, score=-106.299 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wHOjNw3ycvqP for ; Wed, 24 Jul 2013 06:04:44 -0700 (PDT) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id D7D0111E8210 for ; Wed, 24 Jul 2013 06:04:43 -0700 (PDT) Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id 7CD891204FC; Wed, 24 Jul 2013 09:04:40 -0400 (EDT) Received: from njfpsrvexg1.research.att.com (njfpsrvexg1.research.att.com [135.207.177.20]) by mail-green.research.att.com (Postfix) with ESMTP id 72C05E037F; Wed, 24 Jul 2013 09:04:20 -0400 (EDT) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg1.research.att.com ([fe80::58ce:ca01:5d18:db01%13]) with mapi; Wed, 24 Jul 2013 09:04:42 -0400 From: "MORTON JR., ALFRED C (AL)" To: "philip.eardley@bt.com" , "bclaise@cisco.com" Date: Wed, 24 Jul 2013 09:04:41 -0400 Thread-Topic: [lmap] Feedback on draft-eardley-lmap-terminology Thread-Index: Ac6ISEypJJA759Q0QNmvxWMfGSmlewAGaYjgAAJvJNA= Message-ID: References: <51ED59B3.3040701@cisco.com> <9510D26531EF184D9017DF24659BB87F35B7CD1AF5@EMV65-UKRD.domain1.systemhost.net> <51EF90E4.6000907@cisco.com> <9510D26531EF184D9017DF24659BB87F35B7CD1D59@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F35B7CD1D59@EMV65-UKRD.domain1.systemhost.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4F50061njfpsrvexg7re_" MIME-Version: 1.0 Cc: "odonoghue@isoc.org" , "lmap@ietf.org" Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 13:04:49 -0000 --_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4F50061njfpsrvexg7re_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Phil and Benoit, Even dedicated test server capacity can be overrun at large scale. Case in point: A manufacturer of consumer WLAN access points sold in the US coded the IP of a single NTP server in every device, and rather suddenly the US Naval Observatory NTP server they chose had >10k clients they didn't plan on serving (I've cc'd Karen O'Donoghue who may clarify or add to this story). IIRC, this was a case where IETF guidance (ask permission before adding someone's NTP server to your list, and use DNS names instead of IP addresses) was ignored. The two points above apply to LMAP, especially for tests involving DNS servers, and we can add them to Phil's list below. Al From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of phi= lip.eardley@bt.com Sent: Wednesday, July 24, 2013 7:46 AM To: bclaise@cisco.com Cc: lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology Ok, this should be mentioned in the Use case doc as well as in the framewor= k. incidentally, although the measurement functionality would be in lots of MA= s I wouldn't expect them all to run all the tests at the same time - for th= e regular tests, I think you'd use a subset as a panel, and over the course= of a few weeks say the panel would rotate round the different end hosts I agree that the measurement functionality has the potential to be abused a= s a DDOS attack and we need mechanisms to alleviate it. Perhaps too early for specific proposals, but I'd suggest a combination of: - Security on the Control Protocol (to stop rogue controller asking M= As to launch a DDoS) - Some individual tests to include an initial phase where they check = that the Measurement Peer has capacity (especially for tests that create a = significant load on the Measurement Peer) - Ability for Controller to send a "suppress" msg - so lots of MAs ca= n be told to stop sending test pkts, if there's some problem - Perhaps some guidance on best practice (don't kill bt.com - try cis= co.com instead!!) Thanks phil From: Benoit Claise [mailto:bclaise@cisco.com] Sent: 24 July 2013 09:32 To: Eardley,PL,Philip,TUB8 R Cc: lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology Phil, << I'm not too clear if the Measurement Peer are dedicated test server, or = real server (google.com, SIP server, youtube)>> The Measurement Peer could be either. It's up to the operator of the measur= ement system to decide what Measurement Methods the Measurement Peer needs = and exactly how they're implemented. Sure, but what if I point all my 100.000 MAs to constantly ping http://home= .bt.com/news-01363796671918. Not sure if BT will be happy. I could take a different example with DNS, NTP, CDN, ... That might be a point for an applicability section somewhere. Regards, Benoit Thanks phil From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of Benoit Claise Sent: 22 July 2013 17:12 To: lmap@ietf.org Subject: [lmap] Feedback on draft-eardley-lmap-terminology Dear authors, A couple of high level points. One of the goal behind this email is to generate discussions, either on the= list, or during the IETF meeting next week. - The consensus is that the LMAP working group should assume that a Measurement Agent receives Instruction from only a single Controller at any point in time (however it may Report to more than one Collector). Instead of consensus, I would use "a key assumption" The charter says: A key assumption constraining the initial work is that the measurement syst= em is under the control of a single organization (for example, an Internet = Service Provider or a regulator). - Same remark for the term "consensus" in The job of a Bootstrap Protocol is to provide an automated way to associate a Measurement Agent to its Controller, including authentication credentials. Similarly, there should be a way to pull the plug on rogue Measurement Agents. The current consensus on the LMAP mailing list is that the working group should define the bootstrap process but not a protocol. The charter mentions: "The management protocol to bootstrap the MAs in measurement devices is out= of scope of the LMAP charter." - I would like to draw your attention to claise-ippm-perf-metric-registry, which will= be presented in IPPM. - I'm not too clear if the Measurement Peer are dedicated test server, or rea= l server (google.com, SIP server, youtube) Do the Measurement Peer need the functionality of an IP SLA responder or a = TWAMP controller? This might be clarified in the use case draft. - This draft is good shape, but it's really a mix of terminology and framewor= k concepts. I'm glad that there is a single delivery in the charter for both: "1. The LMAP Framework - provides common terminology, basic architecture el= ements, and justifies the simplifying constraints" Regards, Benoit --_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4F50061njfpsrvexg7re_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

= Hi Phil and Benoit,

 = ;

Even dedicated test server capacity= can be overrun at large scale.

<= span style=3D'font-size:10.0pt;font-family:"Courier New";color:windowtext'>=  

Case in point:=

A manufacturer of consumer WLAN access poi= nts sold in the US

coded the = IP of a single NTP server in every device, and rather

=

suddenly the US Naval Observatory NTP server they cho= se

had >10k clients they did= n't plan on serving

(I've cc'd= Karen O'Donoghue who may clarify or add to this story).<= /p>

IIRC, this was a case where IETF guidance

(ask permission before adding someone's= NTP server to your list,

and u= se DNS names instead of IP addresses) was ignored.

 

The tw= o points above apply to LMAP, especially for tests

involving DNS servers, and we can add them to Phil's lis= t below.

 

Al

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@i= etf.org] On Behalf Of philip.eardley@bt.com
Sent: Wednesda= y, July 24, 2013 7:46 AM
To: bclaise@cisco.com
Cc: lmap= @ietf.org
Subject: Re: [lmap] Feedback on draft-eardley-lmap-term= inology

 

Ok, this should be mentioned in the Use case doc= as well as in the framework.

=  

incidentally, although the measur= ement functionality would be in lots of MAs I wouldn’t expect them al= l to run all the tests at the same time – for the regular tests, I th= ink you’d use a subset as a panel, and over the course of a few weeks= say the panel would rotate round the different end hosts=

 

I agr= ee that the measurement functionality has the potential to be abused as a D= DOS attack and we need mechanisms to alleviate it.

Perhaps too early for specific proposals, but I’d sug= gest a combination of:

-&nb= sp;      Security on = the Control Protocol (to stop rogue controller asking MAs to launch a DDoS)=

-    = ;   Some individual tests to include= an initial phase where they check that the Measurement Peer has capacity (= especially for tests that create a significant load on the Measurement Peer= )

-    = ;   Ability for Controller to send a= “suppress” msg – so lots of MAs can be told to stop send= ing test pkts, if there’s some problem

-       Perhaps some guidance on best practice (don’t kill bt.com = – try cisco.com instead!!)

=  

Thanks

phil

 <= /o:p>

From:= Benoit Claise [mailto:bclaise@cisco.com]
Sent:<= /b> 24 July 2013 09:32
To: Eardley,PL,Philip,TUB8 R
Cc:= lmap@ietf.org
Subject: Re: [lmap] Feedback on draft-eardley-lmap= -terminology

 

Phil,

 

<< I'm not = too clear if the Measurement Peer are dedicated test server, or real server= (google.com, SIP server, youtube)>>

<= span lang=3DEN-GB style=3D'font-size:10.0pt;font-family:"Courier New ;color= :windowtext","serif"'>The Measurement Peer could be either. It’s up t= o the operator of the measurement system to decide what Measurement Methods= the Measurement Peer needs and exactly how they’re implemented.

Sure, but what if I point = all my 100.000 MAs to constantly ping http://home.bt.com/news-01363796671918. Not sure if BT wi= ll be happy.
I could take a different example with DNS, NTP, CDN, ...That might be a point for an applicability section somewhere.

Regar= ds, Benoit

 = ;

Thanks

phil

 

=

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Ben= oit Claise
Sent: 22 July 2013 17:12
To: lmap@ietf.org
Subject: [lmap] Feedback on d= raft-eardley-lmap-terminology

 

= Dear authors,

A couple of high level points.
O= ne of the goal behind this email is to generate discussions, either on the = list, or during the IETF meeting next week.

- 

   The consensus is that the LMAP w=
orking group should assume that a
   Measurement Agent receives Instruction from only a singl=
e Controller
   at=
 any point in time (however it may Report to more than one
   Collector).
Instead of consensus, I would use "a key=
 assumption"
The charter says:
A key assumption constraining the initial work is that the measurement=
 system is under the control of a single organization (for example, an Inte=
rnet Service Provider or a regulator).
- 
Same remark for the term "consensus" in 
   The job of a Bootst=
rap Protocol is to provide an automated way to
=
   associate a Measurement Agent to its Contro=
ller, including
  =
 authentication credentials.  Similarly, there should be a way to pull=

   the plug on ro=
gue Measurement Agents.  The current consensus on the
   LMAP mailing list is that the w=
orking group should define the
   bootstrap process but not a protocol.  
The charter mentions:<=
/p>
"The management protocol to bootstrap the=
 MAs in measurement devices is out of scope of the LMAP charter."=

-
I would like to draw your a=
ttention to claise-ippm-perf-metric-registry, which will be presente=
d in IPPM.

- 
I'm not too clear if the Measurement Peer are dedic=
ated test server, or real server (google.com, SIP server, youtube)
Do th=
e Measurement Peer need the functionality of an IP SLA responder or a TWAMP=
 controller?
This might be clarified in the use case draft.

- This draft is good shape, but it's really a mix of terminology and framewo=
rk concepts.
I'm glad that there is a single delivery in the charter for=
 both:
"1. The LMAP Framework - provides common terminology, basic =
architecture elements, and justifies the simplifying constraints"
<=
br>
Regards, Benoit

 

<= /html>= --_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4F50061njfpsrvexg7re_-- From paitken@cisco.com Wed Jul 24 06:06:51 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E0C0011E80E4 for ; Wed, 24 Jul 2013 06:06:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.151 X-Spam-Level: X-Spam-Status: No, score=-10.151 tagged_above=-999 required=5 tests=[AWL=-0.153, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IAu-T1tuER6F for ; Wed, 24 Jul 2013 06:06:42 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id AA35011E80E0 for ; Wed, 24 Jul 2013 06:06:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=28091; q=dns/txt; s=iport; t=1374671202; x=1375880802; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to; bh=Rq64AY/BtTvOkh+Lz+g4kbGuko8IhSb0YobJU+js+RI=; b=jxvIhaE5Em+OxNR+vYl7s825kEFKTQ01bQC2CWO2Q6A8GMLhlL+iFkpb xRLgjdN/1OcnwIqXEA0TnpKytRf1ajj+mARhcVhMOKaD5Wv4L6ql7lnU9 SbZfNSKd0LrS4e0aJ/DbZBiUE2dwdymYCp/gdJEES1iG06+RD5Oe77Ipq w=; X-IronPort-AV: E=Sophos;i="4.89,729,1367971200"; d="scan'208,217";a="15932093" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-3.cisco.com with ESMTP; 24 Jul 2013 13:06:41 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id r6OD6cVQ003872 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Jul 2013 13:06:38 GMT Received: from [144.254.153.45] (dhcp-144-254-153-45.cisco.com [144.254.153.45]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6OD6bBk013764; Wed, 24 Jul 2013 14:06:38 +0100 (BST) Message-ID: <51EFD15E.1050804@cisco.com> Date: Wed, 24 Jul 2013 14:06:38 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: philip.eardley@bt.com References: <51ED59B3.3040701@cisco.com> <9510D26531EF184D9017DF24659BB87F35B7CD1AF5@EMV65-UKRD.domain1.systemhost.net> <51EF90E4.6000907@cisco.com> <9510D26531EF184D9017DF24659BB87F35B7CD1D59@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F35B7CD1D59@EMV65-UKRD.domain1.systemhost.net> Content-Type: multipart/alternative; boundary="------------020808030205020907010202" Cc: bclaise@cisco.com, lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 13:06:51 -0000 This is a multi-part message in MIME format. --------------020808030205020907010202 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Phil, I've been reviewing the drafts, and these are all points I planned to raise. Inline: > Ok, this should be mentioned in the Use case doc as well as in the > framework. > > incidentally, although the measurement functionality would be in lots > of MAs I wouldn't expect them all to run all the tests at the same > time -- for the regular tests, I think you'd use a subset as a panel, > and over the course of a few weeks say the panel would rotate round > the different end hosts > What I read so far seems to focus on the singular test case, whereas in reality there may be many MAs performing the same test, even at the same time. > I agree that the measurement functionality has the potential to be > abused as a DDOS attack and we need mechanisms to alleviate it. > +1. Similar the reporting infra, if 1,000's of MAs are told to send reports to www.bt.com. > Perhaps too early for specific proposals, but I'd suggest a > combination of: > > -Security on the Control Protocol (to stop rogue controller asking MAs > to launch a DDoS) > +1. > -Some individual tests to include an initial phase where they check > that the Measurement Peer has capacity (especially for tests that > create a significant load on the Measurement Peer) > That might not be possible, eg if the test is a DNS lookup or web page retrieval, the peer may have no knowledge that it's involved in an LMAP measurement. > -Ability for Controller to send a "suppress" msg -- so lots of MAs can > be told to stop sending test pkts, if there's some problem > Also to stop sending reports. This needs to be similarly secure so a rogue controller can't suppress sections of the LMAP network. > -Perhaps some guidance on best practice (don't kill bt.com -- try > cisco.com instead!!) > :-) P. > *From:*Benoit Claise [mailto:bclaise@cisco.com] > *Sent:* 24 July 2013 09:32 > *To:* Eardley,PL,Philip,TUB8 R > *Cc:* lmap@ietf.org > *Subject:* Re: [lmap] Feedback on draft-eardley-lmap-terminology > > Phil, > > ** > > *<<*I'm not too clear if the Measurement Peer are dedicated test > server, or real server (google.com, SIP server, youtube)>> > > *The Measurement Peer could be either. It's up to the operator of the > measurement system to decide what Measurement Methods the Measurement > Peer needs and exactly how they're implemented.* > > Sure, but what if I point all my 100.000 MAs to constantly ping > http://home.bt.com/news-01363796671918. Not sure if BT will be happy. > I could take a different example with DNS, NTP, CDN, ... > That might be a point for an applicability section somewhere. > > Regards, Benoit > > ** > > Thanks > > phil > > *From:*lmap-bounces@ietf.org > [mailto:lmap-bounces@ietf.org] *On Behalf Of *Benoit Claise > *Sent:* 22 July 2013 17:12 > *To:* lmap@ietf.org > *Subject:* [lmap] Feedback on draft-eardley-lmap-terminology > > Dear authors, > > A couple of high level points. > One of the goal behind this email is to generate discussions, either > on the list, or during the IETF meeting next week. > > - > > The consensus is that the LMAP working group should assume that a > Measurement Agent receives Instruction from only a single Controller > at any point in time (however it may Report to more than one > Collector). > > Instead of consensus, I would use "a key assumption" > The charter says: > > A key assumption constraining the initial work is that the measurement > system is under the control of a single organization (for example, an > Internet Service Provider or a regulator). > > - > Same remark for the term "consensus" in > > The job of a Bootstrap Protocol is to provide an automated way to > associate a Measurement Agent to its Controller, including > authentication credentials. Similarly, there should be a way to pull > the plug on rogue Measurement Agents. The current consensus on the > LMAP mailing list is that the working group should define the > bootstrap process but not a protocol. > > The charter mentions: > > "The management protocol to bootstrap the MAs in measurement devices > is out of scope of the LMAP charter." > > - > I would like to draw your attention to > claise-ippm-perf-metric-registry > , > which will be presented in IPPM. > > - > I'm not too clear if the Measurement Peer are dedicated test server, > or real server (google.com, SIP server, youtube) > Do the Measurement Peer need the functionality of an IP SLA responder > or a TWAMP controller? > This might be clarified in the use case draft. > > - > This draft is good shape, but it's really a mix of terminology and > framework concepts. > I'm glad that there is a single delivery in the charter for both: > "1. The LMAP Framework - provides common terminology, basic > architecture elements, and justifies the simplifying constraints" > > > Regards, Benoit > > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap --------------020808030205020907010202 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Phil,

I've been reviewing the drafts, and these are all points I planned to raise.

Inline:

Ok, this should be mentioned in the Use case doc as well as in the framework.

 

incidentally, although the measurement functionality would be in lots of MAs I wouldn’t expect them all to run all the tests at the same time – for the regular tests, I think you’d use a subset as a panel, and over the course of a few weeks say the panel would rotate round the different end hosts


What I read so far seems to focus on the singular test case, whereas in reality there may be many MAs performing the same test, even at the same time.


I agree that the measurement functionality has the potential to be abused as a DDOS attack and we need mechanisms to alleviate it.


+1. Similar the reporting infra, if 1,000's of MAs are told to send reports to www.bt.com.


Perhaps too early for specific proposals, but I’d suggest a combination of:

-       Security on the Control Protocol (to stop rogue controller asking MAs to launch a DDoS)


+1.


-       Some individual tests to include an initial phase where they check that the Measurement Peer has capacity (especially for tests that create a significant load on the Measurement Peer)


That might not be possible, eg if the test is a DNS lookup or web page retrieval, the peer may have no knowledge that it's involved in an LMAP measurement.


-       Ability for Controller to send a “suppress” msg – so lots of MAs can be told to stop sending test pkts, if there’s some problem


Also to stop sending reports. This needs to be similarly secure so a rogue controller can't suppress sections of the LMAP network.


-       Perhaps some guidance on best practice (don’t kill bt.com – try cisco.com instead!!)


:-)

P.

 

From: Benoit Claise [mailto:bclaise@cisco.com]
Sent: 24 July 2013 09:32
To: Eardley,PL,Philip,TUB8 R
Cc: lmap@ietf.org
Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology

 

Phil,

 

<< I'm not too clear if the Measurement Peer are dedicated test server, or real server (google.com, SIP server, youtube)>>

The Measurement Peer could be either. It’s up to the operator of the measurement system to decide what Measurement Methods the Measurement Peer needs and exactly how they’re implemented.

Sure, but what if I point all my 100.000 MAs to constantly ping http://home.bt.com/news-01363796671918. Not sure if BT will be happy.
I could take a different example with DNS, NTP, CDN, ...
That might be a point for an applicability section somewhere.

Regards, Benoit

 

Thanks

phil

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Benoit Claise
Sent: 22 July 2013 17:12
To: lmap@ietf.org
Subject: [lmap] Feedback on draft-eardley-lmap-terminology

 

Dear authors,

A couple of high level points.
One of the goal behind this email is to generate discussions, either on the list, or during the IETF meeting next week.

- 

   The consensus is that the LMAP working group should assume that a
   Measurement Agent receives Instruction from only a single Controller
   at any point in time (however it may Report to more than one
   Collector).

Instead of consensus, I would use "a key assumption"
The charter says:

A key assumption constraining the initial work is that the measurement system is under the control of a single organization (for example, an Internet Service Provider or a regulator).

-
Same remark for the term "consensus" in 

   The job of a Bootstrap Protocol is to provide an automated way to
   associate a Measurement Agent to its Controller, including
   authentication credentials.  Similarly, there should be a way to pull
   the plug on rogue Measurement Agents.  The current consensus on the
   LMAP mailing list is that the working group should define the
   bootstrap process but not a protocol. 

The charter mentions:

"The management protocol to bootstrap the MAs in measurement devices is out of scope of the LMAP charter."

-
I would like to draw your attention to claise-ippm-perf-metric-registry, which will be presented in IPPM.

-
I'm not too clear if the Measurement Peer are dedicated test server, or real server (google.com, SIP server, youtube)
Do the Measurement Peer need the functionality of an IP SLA responder or a TWAMP controller?
This might be clarified in the use case draft.

-
This draft is good shape, but it's really a mix of terminology and framework concepts.
I'm glad that there is a single delivery in the charter for both:
"1. The LMAP Framework - provides common terminology, basic architecture elements, and justifies the simplifying constraints"


Regards, Benoit

 



_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

--------------020808030205020907010202-- From dromasca@avaya.com Wed Jul 24 06:45:14 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6496811E80E0 for ; Wed, 24 Jul 2013 06:45:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.349 X-Spam-Level: X-Spam-Status: No, score=-103.349 tagged_above=-999 required=5 tests=[AWL=0.249, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UZQpuWR7UssR for ; Wed, 24 Jul 2013 06:45:08 -0700 (PDT) Received: from co300216-co-outbound.net.avaya.com (co300216-co-outbound.net.avaya.com [198.152.13.100]) by ietfa.amsl.com (Postfix) with ESMTP id 4DFDC11E8131 for ; Wed, 24 Jul 2013 06:45:03 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AnIbAMrZ71HGmAcV/2dsb2JhbABbgkIjITVQrHYGi0SIPgMBgRUWdIIkAQEBAQMSG0EbAgEIDQEDAQMBAQsLEgcyFAMGCAIEARIIGoduAQudL5t7F449AVM7NwEKgwhuA5QIhQCFHYsHgxSBcTk X-IronPort-AV: E=Sophos;i="4.89,735,1367985600"; d="scan'208,217";a="20831678" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by co300216-co-outbound.net.avaya.com with ESMTP; 24 Jul 2013 09:44:16 -0400 Received: from unknown (HELO AZ-FFEXHC02.global.avaya.com) ([135.64.58.12]) by co300216-co-erhwest-out.avaya.com with ESMTP; 24 Jul 2013 09:41:24 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC02.global.avaya.com ([135.64.58.12]) with mapi id 14.02.0328.009; Wed, 24 Jul 2013 15:43:53 +0200 From: "Romascanu, Dan (Dan)" To: Benoit Claise , "lmap@ietf.org" Thread-Topic: [lmap] Feedback on draft-eardley-lmap-terminology Thread-Index: AQHOhvZB9LoTtC34Rxe0kMCmPNNKaZlz1UIw Date: Wed, 24 Jul 2013 13:43:53 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> References: <51ED59B3.3040701@cisco.com> In-Reply-To: <51ED59B3.3040701@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: multipart/alternative; boundary="_000_9904FB1B0159DA42B0B887B7FA8119CA1287FC5DAZFFEXMB04globa_" MIME-Version: 1.0 Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 13:45:14 -0000 --_000_9904FB1B0159DA42B0B887B7FA8119CA1287FC5DAZFFEXMB04globa_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I agree that the draft is in pretty good shape, and that much of the termin= ology content is ready to be incorporated into the framework draft. But we have two framework I-Ds at this point in time! This is not a bad thi= ng and individual contributions are welcome at this point. However, we have= a tight schedule to consolidate the two in one WG framework draft. One of the points where the framework is still confusing for me and the ter= minology I-D does not help is the controller. Says the terminology I-D: Controller: A function that provides a Measurement Agent with Instruction(s). Colloquially, a Controller is a physical device that performs this function. Do we really envision physical devices that perform this (only) function? More unclarity. The single controller (for a given MA) key assumption is no= t mentioned by draft-akhter-lmap-framework. Why? The other framework draft = draft-eardley-lmap-framework doest take into account the key assumtion, but= then talks about the MA pulling configuration from the Controller in order= to avoid firewall traversal problems. Does this eliminate a-priori the usa= ge of NETCONF as a possible transport for the Control Protocol? Regards, Dan From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Ben= oit Claise Sent: Monday, July 22, 2013 7:12 PM To: lmap@ietf.org Subject: [lmap] Feedback on draft-eardley-lmap-terminology Dear authors, A couple of high level points. One of the goal behind this email is to generate discussions, either on the= list, or during the IETF meeting next week. - The consensus is that the LMAP working group should assume that a Measurement Agent receives Instruction from only a single Controller at any point in time (however it may Report to more than one Collector). Instead of consensus, I would use "a key assumption" The charter says: A key assumption constraining the initial work is that the measurement syst= em is under the control of a single organization (for example, an Internet = Service Provider or a regulator). - Same remark for the term "consensus" in The job of a Bootstrap Protocol is to provide an automated way to associate a Measurement Agent to its Controller, including authentication credentials. Similarly, there should be a way to pull the plug on rogue Measurement Agents. The current consensus on the LMAP mailing list is that the working group should define the bootstrap process but not a protocol. The charter mentions: "The management protocol to bootstrap the MAs in measurement devices is out= of scope of the LMAP charter." - I would like to draw your attention to claise-ippm-perf-metric-registry, which will= be presented in IPPM. - I'm not too clear if the Measurement Peer are dedicated test server, or rea= l server (google.com, SIP server, youtube) Do the Measurement Peer need the functionality of an IP SLA responder or a = TWAMP controller? This might be clarified in the use case draft. - This draft is good shape, but it's really a mix of terminology and framewor= k concepts. I'm glad that there is a single delivery in the charter for both: "1. The LMAP Framework - provides common terminology, basic architecture el= ements, and justifies the simplifying constraints" Regards, Benoit --_000_9904FB1B0159DA42B0B887B7FA8119CA1287FC5DAZFFEXMB04globa_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I agree that the draft is= in pretty good shape, and that much of the terminology content is ready to= be incorporated into the framework draft.

 <= /p>

But we have two framework= I-Ds at this point in time! This is not a bad thing and individual contrib= utions are welcome at this point. However, we have a tight schedule to consolidate the two in one WG framework draft.

 <= /p>

One of the points where t= he framework is still confusing for me and the terminology I-D does not hel= p is the controller.

 <= /p>

Says the terminology I-D:

 <= /p>

Controller: A function that provides a Me= asurement Agent with

   Instruction(s).  Colloq= uially, a Controller is a physical device that

   performs this function.=

 

Do we really envision phy= sical devices that perform this (only) function?

 <= /p>

More unclarity. The singl= e controller (for a given MA) key assumption is not mentioned by draft-akht= er-lmap-framework. Why? The other framework draft draft-eardley-lmap-framew= ork doest take into account the key assumtion, but then talks about the MA pul= ling configuration from the Controller in order to avoid firewall traversal= problems. Does this eliminate a-priori the usage of NETCONF as a possible = transport for the Control Protocol?

 <= /p>

Regards,

 <= /p>

Dan

 <= /p>

 <= /p>

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.o= rg] On Behalf Of Benoit Claise
Sent: Monday, July 22, 2013 7:12 PM
To: lmap@ietf.org
Subject: [lmap] Feedback on draft-eardley-lmap-terminology

 

Dear authors,

A couple of high level points.
One of the goal behind this email is to generate discussions, either on the= list, or during the IETF meeting next week.

- 

   The consensus is that the LMAP working group should =
assume that a
   Measurement Agent receives Instruction from only a single=
 Controller
   at any point in time (however it may Report to more than =
one
   Collector).

Instead of consensus, I would use "a key assump= tion"
The charter says:

A key assumption cons= training the initial work is that the measurement system is under the contr= ol of a single organization (for example, an Internet Service Provider or a= regulator).

-
Same remark for the term "consensus" in 

   The job of a Bootstrap Protocol is to provide an aut=
omated way to
   associate a Measurement Agent to its Controller, includin=
g
   authentication credentials.  Similarly, there should=
 be a way to pull
   the plug on rogue Measurement Agents.  The current c=
onsensus on the
   LMAP mailing list is that the working group should define=
 the
   bootstrap process but not a protocol.  

The charter mentions:


"The management protocol to bootstrap the MAs i=
n measurement devices is out of scope of the LMAP charter."=



-

I would like to draw your attention to 
claise-ippm-perf-metric-registry, which will be presented in IPPM.



- 

I'm not too clear if the Measurement Peer are dedicated test server, or rea=
l server (google.com, SIP server, youtube)

Do the Measurement Peer need the functionality of an IP SLA responder or a =
TWAMP controller?

This might be clarified in the use case draft.



- 

This draft is good shape, but it's really a mix of terminology and framewor=
k concepts.

I'm glad that there is a single delivery in the charter for both:

"1. The LMAP Framework - provides common terminology, basic architectu=
re elements, and justifies the simplifying constraints"





Regards, Benoit
--_000_9904FB1B0159DA42B0B887B7FA8119CA1287FC5DAZFFEXMB04globa_-- From acmorton@att.com Wed Jul 24 07:00:01 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 03AEC11E80ED for ; Wed, 24 Jul 2013 07:00:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.448 X-Spam-Level: X-Spam-Status: No, score=-106.448 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x+pzPL2nHSZt for ; Wed, 24 Jul 2013 06:59:55 -0700 (PDT) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id C6C9711E80FE for ; Wed, 24 Jul 2013 06:59:48 -0700 (PDT) Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id 851AD120447; Wed, 24 Jul 2013 09:59:42 -0400 (EDT) Received: from njfpsrvexg5.research.att.com (njfpsrvexg5.research.att.com [135.207.177.27]) by mail-blue.research.att.com (Postfix) with ESMTP id C9C23F0365; Wed, 24 Jul 2013 09:59:44 -0400 (EDT) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg5.research.att.com ([fe80::a501:da3:2345:4587%10]) with mapi; Wed, 24 Jul 2013 09:59:44 -0400 From: "MORTON JR., ALFRED C (AL)" To: "Romascanu, Dan (Dan)" , Benoit Claise , "lmap@ietf.org" Date: Wed, 24 Jul 2013 09:59:43 -0400 Thread-Topic: [lmap] Feedback on draft-eardley-lmap-terminology Thread-Index: AQHOhvZB9LoTtC34Rxe0kMCmPNNKaZlz1UIwgAAGf8A= Message-ID: References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4F500A3njfpsrvexg7re_" MIME-Version: 1.0 Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 14:00:01 -0000 --_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4F500A3njfpsrvexg7re_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Dan, you wrote: More unclarity. The single controller (for a given MA) key assumption is no= t mentioned by draft-akhter-lmap-framework. Why? The other framework draft = draft-eardley-lmap-framework doest take into account the key assumtion, but= then talks about the MA pulling configuration from the Controller in order= to avoid firewall traversal problems. Does this eliminate a-priori the usa= ge of NETCONF as a possible transport for the Control Protocol? Not necessarily. J=FCrgen proposed one possible fix for MA connection initiation with NETCON= F, "Call Home for TLS". See slides 6 and 7 of his presentation: http://www.ietf.org/proceedings/86/slides/slides-86-lmap-3.pdf IIRC, this proposal was to be discussed elsewhere, and perhaps we can learn the outcome on LMAP list. regards, Al From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Rom= ascanu, Dan (Dan) Sent: Wednesday, July 24, 2013 9:44 AM To: Benoit Claise; lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology I agree that the draft is in pretty good shape, and that much of the termin= ology content is ready to be incorporated into the framework draft. But we have two framework I-Ds at this point in time! This is not a bad thi= ng and individual contributions are welcome at this point. However, we have= a tight schedule to consolidate the two in one WG framework draft. One of the points where the framework is still confusing for me and the ter= minology I-D does not help is the controller. Says the terminology I-D: Controller: A function that provides a Measurement Agent with Instruction(s). Colloquially, a Controller is a physical device that performs this function. Do we really envision physical devices that perform this (only) function? More unclarity. The single controller (for a given MA) key assumption is no= t mentioned by draft-akhter-lmap-framework. Why? The other framework draft = draft-eardley-lmap-framework doest take into account the key assumtion, but= then talks about the MA pulling configuration from the Controller in order= to avoid firewall traversal problems. Does this eliminate a-priori the usa= ge of NETCONF as a possible transport for the Control Protocol? Regards, Dan From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Ben= oit Claise Sent: Monday, July 22, 2013 7:12 PM To: lmap@ietf.org Subject: [lmap] Feedback on draft-eardley-lmap-terminology Dear authors, A couple of high level points. One of the goal behind this email is to generate discussions, either on the= list, or during the IETF meeting next week. - The consensus is that the LMAP working group should assume that a Measurement Agent receives Instruction from only a single Controller at any point in time (however it may Report to more than one Collector). Instead of consensus, I would use "a key assumption" The charter says: A key assumption constraining the initial work is that the measurement syst= em is under the control of a single organization (for example, an Internet = Service Provider or a regulator). - Same remark for the term "consensus" in The job of a Bootstrap Protocol is to provide an automated way to associate a Measurement Agent to its Controller, including authentication credentials. Similarly, there should be a way to pull the plug on rogue Measurement Agents. The current consensus on the LMAP mailing list is that the working group should define the bootstrap process but not a protocol. The charter mentions: "The management protocol to bootstrap the MAs in measurement devices is out= of scope of the LMAP charter." - I would like to draw your attention to claise-ippm-perf-metric-registry, which will= be presented in IPPM. - I'm not too clear if the Measurement Peer are dedicated test server, or rea= l server (google.com, SIP server, youtube) Do the Measurement Peer need the functionality of an IP SLA responder or a = TWAMP controller? This might be clarified in the use case draft. - This draft is good shape, but it's really a mix of terminology and framewor= k concepts. I'm glad that there is a single delivery in the charter for both: "1. The LMAP Framework - provides common terminology, basic architecture el= ements, and justifies the simplifying constraints" Regards, Benoit --_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4F500A3njfpsrvexg7re_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

= Hi Dan,

 

you wrote:

More unclarity. The single controller (for a given MA) key assu= mption is not mentioned by draft-akhter-lmap-framework. Why? The other fram= ework draft draft-eardley-lmap-framework doest take into account the key as= sumtion, but then talks about the MA pulling configuration from the Control= ler in order to avoid firewall traversal problems. Does this eliminate a-pr= iori the usage of NETCONF as a possible transport for the Control Protocol?=

 

Not necessarily.

 

J=FCrgen proposed one= possible fix for MA connection initiation with NETCONF,<= /p>

"Call Home for TLS". See slides 6 and 7 = of his presentation:

http://www= .ietf.org/proceedings/86/slides/slides-86-lmap-3.pdf<= /p>

 

<= span style=3D'font-size:10.0pt;font-family:"Courier New";color:windowtext'>= IIRC, this proposal was to be discussed elsewhere, and perhaps we can =

learn the outcome on LMAP list.=

 

regards,

Al

 

From: lmap-= bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Romasca= nu, Dan (Dan)
Sent: Wednesday, July 24, 2013 9:44 AM
To: Benoit Claise; lmap@ietf.org
Subject: Re: [lmap] Feedback on dr= aft-eardley-lmap-terminology

 

I agree that the d= raft is in pretty good shape, and that much of the terminology content is r= eady to be incorporated into the framework draft.

 

<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'>But we have two framework I-Ds at this point in time! This is not a b= ad thing and individual contributions are welcome at this point. However, w= e have a tight schedule to consolidate the two in one WG framework draft. <= o:p>

 

One of the points where the framework is stil= l confusing for me and the terminology I-D does not help is the controller.=

 =

Says the terminology I-D:

 

Controller: A function that provides a Measurement Agent with<= /o:p>

   Instruction(s).  Co= lloquially, a Controller is a physical device that

   performs this function.

 

Do we really envision physical devices that perform this (only) functi= on?

 

More unclarity. The single controller (f= or a given MA) key assumption is not mentioned by draft-akhter-lmap-framewo= rk. Why? The other framework draft draft-eardley-lmap-framework doest take = into account the key assumtion, but then talks about the MA pulling configu= ration from the Controller in order to avoid firewall traversal problems. D= oes this eliminate a-priori the usage of NETCONF as a possible transport fo= r the Control Protocol?

 

Regards,

 

Dan

 

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Beha= lf Of Benoit Claise
Sent: Monday, July 22, 2013 7:12 PM
To: lmap@ietf.org
Subject: [lmap] Feedback on draft-eardley-= lmap-terminology

 

Dear authors,

A couple of high= level points.
One of the goal behind this email is to generate discussi= ons, either on the list, or during the IETF meeting next week.

- 

   The consensus is that the LMAP working =
group should assume that a
   Measurement Ag=
ent receives Instruction from only a single Controller
   at any point in time (however it may Report to more than one<= o:p>
   Collector).

Instead of consensus, I would use "a key assumption"
T= he charter says:

A key assumption constraining the initial work is that the measurem= ent system is under the control of a single organization (for example, an I= nternet Service Provider or a regulator).

-
Same remark for the term "consensus" in

   The job of a Bootstrap Protocol is to provide an autom= ated way to
   associate a Measurement Agent=
 to its Controller, including
   authenticat=
ion credentials.  Similarly, there should be a way to pull<=
/pre>
   the plug on rogue Measurement Agents.  The curr=
ent consensus on the
   LMAP mailing list is=
 that the working group should define the
  =
 bootstrap process but not a protocol.  
The charter mentions:
"The =
management protocol to bootstrap the MAs in measurement devices is out of s=
cope of the LMAP charter."
-
I w=
ould like to draw your attention to claise-ippm-perf-metric-registry=
, which will be presented in IPPM.

- 
I'm not too clear if the Me=
asurement Peer are dedicated test server, or real server (google.com, SIP s=
erver, youtube)
Do the Measurement Peer need the functionality of an IP =
SLA responder or a TWAMP controller?
This might be clarified in the use =
case draft.

- 
This draft is good shape, but it's really a mix of=
 terminology and framework concepts.
I'm glad that there is a single del=
ivery in the charter for both:
"1. The LMAP Framework - provides co=
mmon terminology, basic architecture elements, and justifies the simplifyin=
g constraints"


Regards, Benoit
= --_000_F1312FAF1A1E624DA0972D1C9A91379A1CA4F500A3njfpsrvexg7re_-- From bclaise@cisco.com Wed Jul 24 07:01:29 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 992A511E8219 for ; Wed, 24 Jul 2013 07:01:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6eEXPv7FqqYG for ; Wed, 24 Jul 2013 07:01:25 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id C1E6711E819E for ; Wed, 24 Jul 2013 07:01:16 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6OE0c0w029442 for ; Wed, 24 Jul 2013 16:00:38 +0200 (CEST) Received: from [10.60.67.85] (ams-bclaise-8914.cisco.com [10.60.67.85]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6OE0N6l007983 for ; Wed, 24 Jul 2013 16:00:33 +0200 (CEST) Message-ID: <51EFDDE7.5030009@cisco.com> Date: Wed, 24 Jul 2013 16:00:07 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; WOW64; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: "lmap@ietf.org" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [lmap] Feedback on draft-eardley-lmap-framework X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 14:01:29 -0000 Dear authors, A couple of high level points (next to the ones Dan made) One of the goal behind this email is to generate discussions, either on the list, or during the IETF meeting next week. - It is possible that communications between two Collectors, two Controllers and a Controller and Collector may be useful in some use cases, perhaps to help a measurement system scale. Work on such a protocol is out of scope of LMAP (?) There are many aspects to this: * Report Protocol load balancing, such as Reliable Server Pooling (http://datatracker.ietf.org/wg/rserpool/) * Results deduplication, if the Report Protocol duplicates results to multiple collector * etc... We need to have simplifying assumptions to start with, otherwise it might prove difficult to solve all problems at once. - I like the facts that the framework clearly labels the constraints, and the items out of scope. - Standardised methods are needed for each metric that is measured. A registry for commonly-used metrics [registry] is also required, so that a test can be defined simply by its identifier in the registry. The methods and registry would hopefully also be referenced by other standards organisations. o Such activities are in scope of the IPPM working group (possibly re-chartered) and not LMAP. IPPM was jut rechartered Regards, Benoit From paitken@cisco.com Wed Jul 24 08:02:20 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2B2D311E821B for ; Wed, 24 Jul 2013 08:02:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.413 X-Spam-Level: X-Spam-Status: No, score=-10.413 tagged_above=-999 required=5 tests=[AWL=0.185, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2ow+VKHpMjGW for ; Wed, 24 Jul 2013 08:02:13 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id B6DC911E8142 for ; Wed, 24 Jul 2013 08:01:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3576; q=dns/txt; s=iport; t=1374678061; x=1375887661; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to; bh=riLioJPtXLsbO91bQtoKcpEG0NRRI54VlVlgqx8B7xQ=; b=inS1mx/tceEc0PzZ8+YbpEkL9IPQEwrWj8ubFQR00iSk8v3pZH5io2XS pjKwr4qhsdF1s/EVhKo40KhjoqZ9Gi9r2oi0EHDqdyyDnLsPhhOKj6A5t EZ67ZYAIs8QBr0QB/NBjFANJWepBkowB+xPueqgZIYNzk9XajnSsZooEh s=; X-IronPort-AV: E=Sophos;i="4.89,735,1367971200"; d="scan'208,217";a="85061852" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-2.cisco.com with ESMTP; 24 Jul 2013 15:01:00 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id r6OF0wpa017826 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Jul 2013 15:00:58 GMT Received: from [144.254.153.45] (dhcp-144-254-153-45.cisco.com [144.254.153.45]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6OF0vME021387; Wed, 24 Jul 2013 16:00:58 +0100 (BST) Message-ID: <51EFEC2A.9010701@cisco.com> Date: Wed, 24 Jul 2013 16:00:58 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: "Romascanu, Dan (Dan)" References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> Content-Type: multipart/alternative; boundary="------------060401090709060606080704" Cc: Benoit Claise , "lmap@ietf.org" Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 15:02:20 -0000 This is a multi-part message in MIME format. --------------060401090709060606080704 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dan, > The single controller (for a given MA) key assumption is not mentioned > by draft-akhter-lmap-framework. Why? The only key assumption (in the LMAP WG charter) is "that the measurement system is under the control of a single organization". Given that assumption, we don't see the necessity of restricting an MA to a single controller. This allows for redundant / backup controllers, or a controller cloud (thinking of rserpool, eg). P. --------------060401090709060606080704 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Dan,

The single controller (for a given MA) key assumption is not mentioned by draft-akhter-lmap-framework. Why?

The only key assumption (in the LMAP WG charter) is "that the measurement system is under the control of a single organization".

Given that assumption, we don't see the necessity of restricting an MA to a single controller.
This allows for redundant / backup controllers, or a controller cloud (thinking of rserpool, eg).

P.
--------------060401090709060606080704-- From j.schoenwaelder@jacobs-university.de Wed Jul 24 08:12:55 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 49C2811E8153 for ; Wed, 24 Jul 2013 08:12:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4v3wJAHPQHz4 for ; Wed, 24 Jul 2013 08:12:50 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 4C7BE11E80AE for ; Wed, 24 Jul 2013 08:12:37 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id A065120968; Wed, 24 Jul 2013 17:12:23 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id yQKe9OFeNp29; Wed, 24 Jul 2013 17:12:23 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 567022095C; Wed, 24 Jul 2013 17:12:22 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 19A1B277835D; Wed, 24 Jul 2013 17:12:17 +0200 (CEST) Date: Wed, 24 Jul 2013 17:12:17 +0200 From: Juergen Schoenwaelder To: "Romascanu, Dan (Dan)" Message-ID: <20130724151217.GA39515@elstar.local> Mail-Followup-To: "Romascanu, Dan (Dan)" , Benoit Claise , "lmap@ietf.org" References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Benoit Claise , "lmap@ietf.org" Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 15:12:55 -0000 On Wed, Jul 24, 2013 at 01:43:53PM +0000, Romascanu, Dan (Dan) wrote: > > One of the points where the framework is still confusing for me and the terminology I-D does not help is the controller. > > Says the terminology I-D: > > Controller: A function that provides a Measurement Agent with > Instruction(s). Colloquially, a Controller is a physical device that > performs this function. > > Do we really envision physical devices that perform this (only) function? For me, the second sentence is rather pointless. I see not reason why a controller may not be a virtual machine. The controller is a function, it does not really matter how it is realized. Note that we have similar sentences for the Collector and Measurement Agents (and I think they all three could go). > More unclarity. The single controller (for a given MA) key assumption is not mentioned by draft-akhter-lmap-framework. Why? The other framework draft draft-eardley-lmap-framework doest take into account the key assumtion, but then talks about the MA pulling configuration from the Controller in order to avoid firewall traversal problems. Does this eliminate a-priori the usage of NETCONF as a possible transport for the Control Protocol? > Perhaps things need to be rephrased but then I am not sure which sentence in draft-eardley-lmap-terminology-02 you think is ruling out NETCONF. The underlying requirement is that sessions need to be initiated by the MAs since they may be behind NATs or other middleboxes. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Wed Jul 24 08:15:49 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4A94E11E8221 for ; Wed, 24 Jul 2013 08:15:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kQzf3+M5n+oT for ; Wed, 24 Jul 2013 08:15:42 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 1FE3011E80AE for ; Wed, 24 Jul 2013 08:15:42 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 4FA8020968; Wed, 24 Jul 2013 17:15:12 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 7Lp0oE0OYn7Y; Wed, 24 Jul 2013 17:15:12 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 80D4020BD5; Wed, 24 Jul 2013 17:15:11 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id EAFE927783B6; Wed, 24 Jul 2013 17:15:07 +0200 (CEST) Date: Wed, 24 Jul 2013 17:15:07 +0200 From: Juergen Schoenwaelder To: "MORTON JR., ALFRED C (AL)" Message-ID: <20130724151507.GB39515@elstar.local> Mail-Followup-To: "MORTON JR., ALFRED C (AL)" , "Romascanu, Dan (Dan)" , Benoit Claise , "lmap@ietf.org" References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Benoit Claise , "Romascanu, Dan \(Dan\)" , "lmap@ietf.org" Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 15:15:49 -0000 On Wed, Jul 24, 2013 at 09:59:43AM -0400, MORTON JR., ALFRED C (AL) wrote: > > Jürgen proposed one possible fix for MA connection initiation with NETCONF, > "Call Home for TLS". See slides 6 and 7 of his presentation: > http://www.ietf.org/proceedings/86/slides/slides-86-lmap-3.pdf > > IIRC, this proposal was to be discussed elsewhere, and perhaps we can > learn the outcome on LMAP list. The NETCONF WG is working on call home solutions, both for NETCONF over TLS and NETCONF over SSH. The relevant I-Ds are: http://tools.ietf.org/wg/netconf/draft-ietf-netconf-reverse-ssh/ http://tools.ietf.org/wg/netconf/draft-ietf-netconf-rfc5539bis/ There are fairly new I-Ds and hence it is too early to declare any outcomes. The NETCONF WG meets in Session 2013-07-31 1510-1610 and Session 2013-07-31 1620-1720. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From marcelo@it.uc3m.es Wed Jul 24 12:22:59 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A3A3021F9A51 for ; Wed, 24 Jul 2013 12:22:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.599 X-Spam-Level: X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7rR34-IjKNBP for ; Wed, 24 Jul 2013 12:22:49 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by ietfa.amsl.com (Postfix) with ESMTP id 7A95321F91B7 for ; Wed, 24 Jul 2013 12:22:47 -0700 (PDT) Received: from smtp02.uc3m.es (localhost [127.0.0.1]) by localhost.uc3m.es (Postfix) with ESMTP id B65FA894D5C for ; Wed, 24 Jul 2013 21:22:44 +0200 (CEST) X-uc3m-safe: yes Received: from [163.117.203.132] (unknown [163.117.203.132]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: marcelo@smtp02.uc3m.es) by smtp02.uc3m.es (Postfix) with ESMTPSA id 8C441894D56 for ; Wed, 24 Jul 2013 21:22:44 +0200 (CEST) Message-ID: <51F0297A.7040407@it.uc3m.es> Date: Wed, 24 Jul 2013 21:22:34 +0200 From: marcelo bagnulo braun User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: lmap@ietf.org References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <51EFEC2A.9010701@cisco.com> In-Reply-To: <51EFEC2A.9010701@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelistedACL 131 matched, not delayed by milter-greylist-4.2.7 (smtp02.uc3m.es); Wed, 24 Jul 2013 21:22:44 +0200 (CEST) X-TM-AS-Product-Ver: IMSS-7.1.0.1224-7.0.0.1014-20036.000 X-TM-AS-Result: No--13.695-7.0-31-1 X-imss-scan-details: No--13.695-7.0-31-1 Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 19:22:59 -0000 El 24/07/13 17:00, Paul Aitken escribió: > Dan, > >> The single controller (for a given MA) key assumption is not >> mentioned by draft-akhter-lmap-framework. Why? > > The only key assumption (in the LMAP WG charter) is "that the > measurement system is under the control of a single organization". > > Given that assumption, we don't see the necessity of restricting an MA > to a single controller. > This allows for redundant / backup controllers, or a controller cloud > (thinking of rserpool, eg). Right, i would express that each MA is under one controller at any point in time. It can change controller (i.e. backup) but i think having more than one controller controlling simoultaneously one MA introduces a lot of complexity and i am not sure it is worth it. MAkes sense? Regards, marcelo > > P. > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From paitken@cisco.com Wed Jul 24 13:18:21 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C699811E8239 for ; Wed, 24 Jul 2013 13:18:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fzOhsbKADzGD for ; Wed, 24 Jul 2013 13:18:16 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 9A03511E80D2 for ; Wed, 24 Jul 2013 13:18:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1620; q=dns/txt; s=iport; t=1374697095; x=1375906695; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=JYOi/gTg3+OuZys8OEcckZtfRWSULX/giy2WHsl8F5E=; b=NWrTWMMBsdWd95Dx1SbZZaySeIdEZu9fZLgwzszBMwoJQlN8iBKsxlYw xycA99o4W1TOI9CYrdhZx7Og3WgNusE9McH3QuAmASpGs9CsUVpPh21Q9 FQoDz98oB0PtqccsUcEKXTGRdZE1e2rz7VF3h5ZfWM/iSN6EDPXyN2fzT 0=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjgFABM28FGQ/khL/2dsb2JhbABbgwY2uzmCfYEWFnSCJAEBAQMBMgEFNAwBBQsLIRYPCQMCAQIBRQYNAQcBAYgGBroDjjWBSAeEAAOXX4YjiyqDFQ X-IronPort-AV: E=Sophos;i="4.89,737,1367971200"; d="scan'208";a="85076695" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-2.cisco.com with ESMTP; 24 Jul 2013 20:18:11 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id r6OKI9V9001016 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Jul 2013 20:18:09 GMT Received: from [10.61.164.215] ([10.61.164.215]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6OKI7Vo009753; Wed, 24 Jul 2013 21:18:08 +0100 (BST) Message-ID: <51F0367F.1060905@cisco.com> Date: Wed, 24 Jul 2013 21:18:07 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: marcelo bagnulo braun References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <51EFEC2A.9010701@cisco.com> <51F0297A.7040407@it.uc3m.es> In-Reply-To: <51F0297A.7040407@it.uc3m.es> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 20:18:22 -0000 Marcelo, > El 24/07/13 17:00, Paul Aitken escribió: >> Dan, >> >>> The single controller (for a given MA) key assumption is not >>> mentioned by draft-akhter-lmap-framework. Why? >> >> The only key assumption (in the LMAP WG charter) is "that the >> measurement system is under the control of a single organization". >> >> Given that assumption, we don't see the necessity of restricting an >> MA to a single controller. >> This allows for redundant / backup controllers, or a controller cloud >> (thinking of rserpool, eg). > > > Right, i would express that each MA is under one controller at any > point in time. It can change controller (i.e. backup) but i think > having more than one controller controlling simoultaneously one MA > introduces a lot of complexity and i am not sure it is worth it. > > MAkes sense? Since the system is under control of a single organization, it's up to that organisation how to organise their controllers and MA's. We should make it possible for them to do what they want, rather than making it possible to only do what we think is best. MA's should do what they're requested by controllers as best as they're able. I can't immediately think of a situation where an MA could be given conflicting instructions. However, if conflicts are possible the MA could enqueue the requested tests to be run sequentially to avoid inter-test conflict. If conflict can arise from two controllers, then it can also arise by requesting the same tests from a single controller - so designing this limitation into the system gains nothing. P. From j.schoenwaelder@jacobs-university.de Wed Jul 24 13:49:36 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A225411E8137 for ; Wed, 24 Jul 2013 13:49:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id atmEhDlc3LP6 for ; Wed, 24 Jul 2013 13:49:31 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id B4AE211E8279 for ; Wed, 24 Jul 2013 13:49:30 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id BCDDC20A6D; Wed, 24 Jul 2013 22:49:29 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 47NhcBj3sOdr; Wed, 24 Jul 2013 22:49:29 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 47B3020AFE; Wed, 24 Jul 2013 22:49:29 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 1459427788B1; Wed, 24 Jul 2013 22:49:24 +0200 (CEST) Date: Wed, 24 Jul 2013 22:49:24 +0200 From: Juergen Schoenwaelder To: Paul Aitken Message-ID: <20130724204924.GA40227@elstar.local> Mail-Followup-To: Paul Aitken , marcelo bagnulo braun , lmap@ietf.org References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <51EFEC2A.9010701@cisco.com> <51F0297A.7040407@it.uc3m.es> <51F0367F.1060905@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51F0367F.1060905@cisco.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: marcelo bagnulo braun , lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 20:49:36 -0000 On Wed, Jul 24, 2013 at 09:18:07PM +0100, Paul Aitken wrote: > MA's should do what they're requested by controllers as best as > they're able. I can't immediately think of a situation where an MA > could be given conflicting instructions. However, if conflicts are > possible the MA could enqueue the requested tests to be run > sequentially to avoid inter-test conflict. If conflict can arise > from two controllers, then it can also arise by requesting the same > tests from a single controller - so designing this limitation into > the system gains nothing. C1: do test X C2: don't do test X or C1: do test X every 10 minutes C2: do test X every 8 minutes What should the MA do? /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From paitken@cisco.com Wed Jul 24 14:07:43 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C541021F90FB for ; Wed, 24 Jul 2013 14:07:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KhmmtCBnoGkj for ; Wed, 24 Jul 2013 14:07:37 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id 93EB811E8268 for ; Wed, 24 Jul 2013 14:07:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1579; q=dns/txt; s=iport; t=1374700048; x=1375909648; h=message-id:date:from:mime-version:to:subject:references: in-reply-to:content-transfer-encoding; bh=eYmmOhpgB7DmMuNQUC8PUhoUMHybXL8Sl+Nd6OXWR/4=; b=HgxXZnFVDSJdiyuyzTo734OUcEzJvpD/WFn/NaX1O6bFF6CPDVDqzIDz DZEyyTosaHjLQDfQf2vylI3blkhse8r7qAIgO/1nw5Nub98mbdunKNN4m tmPs80LKyoIh8GKFIwxal2m/FbGQAE/pOXJAo7DiUNY/Rjawlv+3OU4i/ M=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjcFAEZB8FGQ/khM/2dsb2JhbABbgwY2vjaBFhZ0giQBAQEDATg0DAYLCxgJFg8JAwIBAgFFBgEMCAEBiAYGugyQBIQAA5dfhiOLKoMVgWcE X-IronPort-AV: E=Sophos;i="4.89,737,1367971200"; d="scan'208";a="157502982" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-1.cisco.com with ESMTP; 24 Jul 2013 21:07:14 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r6OL7CFZ020464 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Jul 2013 21:07:12 GMT Received: from [10.61.164.215] ([10.61.164.215]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6OL79kq011962; Wed, 24 Jul 2013 22:07:11 +0100 (BST) Message-ID: <51F041FD.4050408@cisco.com> Date: Wed, 24 Jul 2013 22:07:09 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: marcelo bagnulo braun , lmap@ietf.org References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <51EFEC2A.9010701@cisco.com> <51F0297A.7040407@it.uc3m.es> <51F0367F.1060905@cisco.com> <20130724204924.GA40227@elstar.local> In-Reply-To: <20130724204924.GA40227@elstar.local> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 21:07:43 -0000 Juergen, > On Wed, Jul 24, 2013 at 09:18:07PM +0100, Paul Aitken wrote: > >> MA's should do what they're requested by controllers as best as >> they're able. I can't immediately think of a situation where an MA >> could be given conflicting instructions. However, if conflicts are >> possible the MA could enqueue the requested tests to be run >> sequentially to avoid inter-test conflict. If conflict can arise >> from two controllers, then it can also arise by requesting the same >> tests from a single controller - so designing this limitation into >> the system gains nothing. > C1: do test X > C2: don't do test X You're supposing inter-controller communication, such that C2 knows that C1 requested X ? The MA should do test X as requested by C1. It should ignore the command from C2, because C2 didn't request test X. It's clearer if the sequence was: C2: do test X C1: do test X C2: don't do test X Then C2 is cancelling its previous command, whereas C1's command stands. > or > > C1: do test X every 10 minutes > C2: do test X every 8 minutes > > What should the MA do? The MA should do test X multiple times exactly as requested. If both requests are received at exactly the same instant (which is a corner case, even if we're talking about multiple processors + multiple interfaces), then every 40 minutes test X could be run just once as an optimisation. More realistically, test X will be run at t0, t10, t20, t30, ... for C1 and at t1, t9, t17, t25, ... for C2. These times never coincide. P. From jason.weil@twcable.com Wed Jul 24 15:05:50 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BF5C011E8275 for ; Wed, 24 Jul 2013 15:05:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.397 X-Spam-Level: X-Spam-Status: No, score=0.397 tagged_above=-999 required=5 tests=[BAYES_20=-0.74, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7gRbHwqkzhNy for ; Wed, 24 Jul 2013 15:05:43 -0700 (PDT) Received: from cdpipgw01.twcable.com (cdpipgw01.twcable.com [165.237.59.22]) by ietfa.amsl.com (Postfix) with ESMTP id E392511E815E for ; Wed, 24 Jul 2013 15:05:42 -0700 (PDT) X-SENDER-IP: 10.136.163.14 X-SENDER-REPUTATION: None X-IronPort-AV: E=Sophos;i="4.89,737,1367985600"; d="scan'208,217";a="112560240" Received: from unknown (HELO PRVPEXHUB05.corp.twcable.com) ([10.136.163.14]) by cdpipgw01.twcable.com with ESMTP/TLS/RC4-MD5; 24 Jul 2013 18:04:29 -0400 Received: from PRVPEXVS17.corp.twcable.com ([10.136.163.95]) by PRVPEXHUB05.corp.twcable.com ([10.136.163.14]) with mapi; Wed, 24 Jul 2013 18:05:41 -0400 From: "Weil, Jason" To: "lmap@ietf.org" Date: Wed, 24 Jul 2013 18:05:40 -0400 Thread-Topic: Feedback on draft-eardley-lmap-framework-01 Thread-Index: Ac6Iue+JkB9mX20GQROv2rsQmPcjnQ== Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.4.130416 acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_CE15C7F4191A4jasonweiltwcablecom_" MIME-Version: 1.0 Subject: [lmap] Feedback on draft-eardley-lmap-framework-01 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 22:05:50 -0000 --_000_CE15C7F4191A4jasonweiltwcablecom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Here are my comments and questions on the draft: Section 1. [use-cases] Regulators: I would drop the word 'several' from this description. It reads= as if regulators are targeting certain providers purposefully out of a gro= up which is not typically the case. --- The example given in Diversity seems to be another example supporting the i= nteroperability benefits that come with the earlier bullet on Standardizati= on. A MA on a CPE router and another MA on a WiFi device behind the router= or a MA running as a software client on a laptop/desktop might be a bette= r example? --- Section 2. The MA functions are implemented either in specialised hardware or as code on general purpose devices like a PC, tablet or smartphone. The Measurement Peer may be an LMAP device or a normal, non-LMAP device (for example if the MA measures the time for a DNS response or a webpage download from www.example.com). A Measurement Peer is an LMAP functional element. It would seem to me that = as long as an entity performs in the role of a Measurement Peer it would be= considered an LMAP device whether or not it was purpose built for a single= test or simply performing as a Measurement Peer based on its normal operat= ional role such as a recursive resolver. As such I don=92t believe a Measur= ement Peer would ever be considered a non-LMAP device if it was being used = for a test by the MA. -- Section 3. 3.2. Each MA may only have a single Controller at any point in time =96 The= 'may' in this sentence makes me wonder where the constraint exists. Is the= constraint that it must only take direction from a single Controller at an= y given time? -- The topic on NAT an firewalls suggesting a pull model may be better as a se= parate constraint topic by itself. Thanks, Jason ________________________________ This E-mail and any of its attachments may contain Time Warner Cable propri= etary information, which is privileged, confidential, or subject to copyrig= ht belonging to Time Warner Cable. This E-mail is intended solely for the u= se of the individual or entity to which it is addressed. If you are not the= intended recipient of this E-mail, you are hereby notified that any dissem= ination, distribution, copying, or action taken in relation to the contents= of and attachments to this E-mail is strictly prohibited and may be unlawf= ul. If you have received this E-mail in error, please notify the sender imm= ediately and permanently delete the original and any copy of this E-mail an= d any printout. --_000_CE15C7F4191A4jasonweiltwcablecom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Here are= my comments and questions on the draft:

Section = 1. [use-cases]
Regulato= rs: I would drop the word 'several' from this description. It reads as if r= egulators are targeting certain providers purposefully out of a group which= is not typically the case. 
---
The exam= ple given in Diversity seems to be another example supporting the interoper= ability benefits that come with the earlier bullet on Standardization. &nbs= p;A MA on a CPE router and another MA on a WiFi device behind the router or a MA running as a software client on a = laptop/desktop might be  a better example?
---
Section = 2. 

   The MA functions are=
 implemented either in specialised hardware or as
   code on general purpose devices like a PC, tablet or smartphone.  The
   Measurement Peer may be an LMAP device or a normal, non-LMAP device
   (for example if the MA measures the time for a DNS response or a
   webpage download from www.example.com).

A Measurement Peer is a=
n LMAP functional element. It would seem to me that as long as an entity pe=
rforms in the role of a Measurement Peer it would be considered an LMAP dev=
ice whether or not it was purpose built for a single test or simply perform=
ing as a Measurement Peer based on its normal operational role such as a re=
cursive resolver. As such I don=92t believe a Measurement Peer would ever b=
e considered a non-LMAP device if it was being used for a test by the MA.&n=
bsp;
--
Section 3.

3.2. Each MA may only have a single Controller at any point in time =96 The= 'may' in this sentence makes me wonder where the constraint exists. Is the= constraint that it must only take direction from a single Controller at an= y given time?
--
The topic on NAT an firewalls suggest= ing a pull model may be better as a separate constraint topic by itself.


Thanks,

Jason

This E-mail and any of its a= ttachments may contain Time Warner Cable proprietary information, which is = privileged, confidential, or subject to copyright belonging to Time Warner = Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you a= re not the intended recipient of this E-mail, you are hereby notified that = any dissemination, distribution, copying, or action taken in relation to th= e contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have receiv= ed this E-mail in error, please notify the sender immediately and permanent= ly delete the original and any copy of this E-mail and any printout.
 --_000_CE15C7F4191A4jasonweiltwcablecom_-- From j.schoenwaelder@jacobs-university.de Thu Jul 25 02:16:19 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 087CB21F9B03 for ; Thu, 25 Jul 2013 02:16:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AMcFU4S0Q9f6 for ; Thu, 25 Jul 2013 02:16:14 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 595DB21F9B14 for ; Thu, 25 Jul 2013 02:16:11 -0700 (PDT) Received: from localhost (demetrius2.jacobs-university.de [212.201.44.47]) by hermes.jacobs-university.de (Postfix) with ESMTP id 6B2FD20BE2; Thu, 25 Jul 2013 11:16:10 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius2.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id kLIvEO5VrAHn; Thu, 25 Jul 2013 11:16:10 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 0250420BE1; Thu, 25 Jul 2013 11:16:10 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 646A82779292; Thu, 25 Jul 2013 11:16:06 +0200 (CEST) Date: Thu, 25 Jul 2013 11:16:06 +0200 From: Juergen Schoenwaelder To: Paul Aitken Message-ID: <20130725091606.GB41645@elstar.local> Mail-Followup-To: Paul Aitken , marcelo bagnulo braun , lmap@ietf.org References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <51EFEC2A.9010701@cisco.com> <51F0297A.7040407@it.uc3m.es> <51F0367F.1060905@cisco.com> <20130724204924.GA40227@elstar.local> <51F041FD.4050408@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51F041FD.4050408@cisco.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: marcelo bagnulo braun , lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 09:16:19 -0000 On Wed, Jul 24, 2013 at 10:07:09PM +0100, Paul Aitken wrote: [...] You I think you are proposing that instructions received from different Controllers must be kept separate and the MA must act towards a Controller as if there are no other controllers. A potential problem with this is that there are likely instructions and tests that do impact each other through side effects. If C1 schedules a test to be started every 5 min past the hour that requires no cross traffic while C2 schedules a test to measure video streaming capabilities during the interval 3-7 minutes past the hour, then C1 will be surprised that the scheduled test never executes or produces wrong results. Do you require that the MA has the logic to reject one of the instructions in such a case? Or do you expect that the Controllers will simply accept that MAs may not do what they think they should be doing? Or do you expect to have at least read access to all instructions? Will we end up with an access control model to handle things properly in a configurable manner (like we did with SNMP and NETCONF)? If the assumption is complete separation then I believe this is quite a big one - in particular if I consider devices like home routers or hardware probes of some bigger existing measurement platforms as potential targets for this work (usuall small embedded Linux systems running on cheap hardware). /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From dromasca@avaya.com Thu Jul 25 02:17:34 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D6B221F9AE3 for ; Thu, 25 Jul 2013 02:17:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.939 X-Spam-Level: X-Spam-Status: No, score=-102.939 tagged_above=-999 required=5 tests=[AWL=-0.340, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aKOL3B0uVCGU for ; Thu, 25 Jul 2013 02:17:27 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id 56C0921F9B03 for ; Thu, 25 Jul 2013 02:17:25 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AkMKALTr8FHGmAcV/2dsb2JhbABagmUhgQW9WQMBgRgWdIIkAQEBAQIBEig/BQcEAgEIDQECBQ0LCQkHMhQRAgQODRqHaAYBnhWcCY8ROzEHCoMIbgOUCIodiweDFIIq X-IronPort-AV: E=Sophos;i="4.89,742,1367985600"; d="scan'208";a="21502180" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 25 Jul 2013 05:17:24 -0400 Received: from unknown (HELO AZ-FFEXHC02.global.avaya.com) ([135.64.58.12]) by co300216-co-erhwest-out.avaya.com with ESMTP; 25 Jul 2013 05:14:49 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC02.global.avaya.com ([135.64.58.12]) with mapi id 14.02.0328.009; Thu, 25 Jul 2013 11:17:22 +0200 From: "Romascanu, Dan (Dan)" To: Juergen Schoenwaelder Thread-Topic: [lmap] Feedback on draft-eardley-lmap-terminology Thread-Index: AQHOhvZB9LoTtC34Rxe0kMCmPNNKaZlz1UIwgABgPoCAAOq3wA== Date: Thu, 25 Jul 2013 09:17:22 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA12880D39@AZ-FFEXMB04.global.avaya.com> References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <20130724151217.GA39515@elstar.local> In-Reply-To: <20130724151217.GA39515@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Benoit Claise , "lmap@ietf.org" Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 09:17:34 -0000 > -----Original Message----- > From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs- > > More unclarity. The single controller (for a given MA) key assumption > is not mentioned by draft-akhter-lmap-framework. Why? The other > framework draft draft-eardley-lmap-framework doest take into account the > key assumtion, but then talks about the MA pulling configuration from > the Controller in order to avoid firewall traversal problems. Does this > eliminate a-priori the usage of NETCONF as a possible transport for the > Control Protocol? > > >=20 > Perhaps things need to be rephrased but then I am not sure which > sentence in draft-eardley-lmap-terminology-02 you think is ruling out > NETCONF. The underlying requirement is that sessions need to be > initiated by the MAs since they may be behind NATs or other middleboxes. >=20 [[DR]] The phrase I am refering to is not from draft-eardley-lmap-terminolo= gy-02 but from draft-eardley-lmap-framework-02, section 2. In my reading th= is sentence does not talk about who initiates the session, but about the wa= y the configuration itself is to reach the MAs:=20 > To avoid problems with NAT and firewalls, it is likely that the MA 'pulls' the configuration from its Controller, as identified by the Initialiser. Regards, Dan From dromasca@avaya.com Thu Jul 25 02:24:31 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 99E2621F8E3D for ; Thu, 25 Jul 2013 02:24:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.382 X-Spam-Level: X-Spam-Status: No, score=-103.382 tagged_above=-999 required=5 tests=[AWL=0.217, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mUpD7fQ-3x0b for ; Thu, 25 Jul 2013 02:24:25 -0700 (PDT) Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id CD1BD21F8DDD for ; Thu, 25 Jul 2013 02:24:24 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AkUKAEnu8FHGmAcV/2dsb2JhbABagmUhegu9WgMBgRgWdIIkAQEBAQMSKD8MBAIBCA0DAQQBAQsLCQkHMhQJCAIEAQ0FCBqHbgGeCJwNjxE7MQ0EgwhuA5QIih2LB4MUgio X-IronPort-AV: E=Sophos;i="4.89,742,1367985600"; d="scan'208";a="17743897" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by de307622-de-outbound.net.avaya.com with ESMTP; 25 Jul 2013 05:24:07 -0400 Received: from unknown (HELO AZ-FFEXHC04.global.avaya.com) ([135.64.58.14]) by co300216-co-erhwest-out.avaya.com with ESMTP; 25 Jul 2013 05:21:32 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC04.global.avaya.com ([135.64.58.14]) with mapi id 14.02.0328.009; Thu, 25 Jul 2013 05:24:04 -0400 From: "Romascanu, Dan (Dan)" To: "Romascanu, Dan (Dan)" , Juergen Schoenwaelder Thread-Topic: [lmap] Feedback on draft-eardley-lmap-terminology Thread-Index: AQHOhvZB9LoTtC34Rxe0kMCmPNNKaZlz1UIwgABgPoCAAOq3wIAAAyPQ Date: Thu, 25 Jul 2013 09:24:04 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA12880D66@AZ-FFEXMB04.global.avaya.com> References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <20130724151217.GA39515@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Benoit Claise , "lmap@ietf.org" Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 09:24:31 -0000 Sorry, draft-eardley-lmap-framework-02, section 3.2 > -----Original Message----- > From: Romascanu, Dan (Dan) > Sent: Thursday, July 25, 2013 12:17 PM > To: 'Juergen Schoenwaelder' > Cc: Benoit Claise; lmap@ietf.org > Subject: RE: [lmap] Feedback on draft-eardley-lmap-terminology >=20 >=20 >=20 >=20 >=20 > > -----Original Message----- > > From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs- > > > More unclarity. The single controller (for a given MA) key > > > assumption > > is not mentioned by draft-akhter-lmap-framework. Why? The other > > framework draft draft-eardley-lmap-framework doest take into account > > the key assumtion, but then talks about the MA pulling configuration > > from the Controller in order to avoid firewall traversal problems. > > Does this eliminate a-priori the usage of NETCONF as a possible > > transport for the Control Protocol? > > > > > > > Perhaps things need to be rephrased but then I am not sure which > > sentence in draft-eardley-lmap-terminology-02 you think is ruling out > > NETCONF. The underlying requirement is that sessions need to be > > initiated by the MAs since they may be behind NATs or other > middleboxes. > > >=20 > [[DR]] The phrase I am refering to is not from draft-eardley-lmap- > terminology-02 but from draft-eardley-lmap-framework-02, section 2. In > my reading this sentence does not talk about who initiates the session, > but about the way the configuration itself is to reach the MAs: >=20 > > To avoid problems with NAT and firewalls, it is likely that the MA > 'pulls' the configuration from its Controller, as identified by the > Initialiser. >=20 > Regards, >=20 > Dan From j.schoenwaelder@jacobs-university.de Thu Jul 25 02:45:45 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6510821F9A15 for ; Thu, 25 Jul 2013 02:45:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jZ7mC8Ky-YRb for ; Thu, 25 Jul 2013 02:45:40 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 3F55721F9A96 for ; Thu, 25 Jul 2013 02:45:40 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id 7EAEB207B1; Thu, 25 Jul 2013 11:45:39 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 9p4jUziSDxlB; Thu, 25 Jul 2013 11:45:39 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 15F90206D7; Thu, 25 Jul 2013 11:45:32 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 73248277936D; Thu, 25 Jul 2013 11:45:28 +0200 (CEST) Date: Thu, 25 Jul 2013 11:45:28 +0200 From: Juergen Schoenwaelder To: "Romascanu, Dan (Dan)" Message-ID: <20130725094528.GA41769@elstar.local> Mail-Followup-To: "Romascanu, Dan (Dan)" , "lmap@ietf.org" References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <20130724151217.GA39515@elstar.local> <9904FB1B0159DA42B0B887B7FA8119CA12880D39@AZ-FFEXMB04.global.avaya.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA12880D39@AZ-FFEXMB04.global.avaya.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "lmap@ietf.org" Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 09:45:45 -0000 On Thu, Jul 25, 2013 at 09:17:22AM +0000, Romascanu, Dan (Dan) wrote: > > [[DR]] The phrase I am refering to is not from draft-eardley-lmap-terminology-02 but from draft-eardley-lmap-framework-02, section 2. In my reading this sentence does not talk about who initiates the session, but about the way the configuration itself is to reach the MAs: > > > To avoid problems with NAT and firewalls, it is likely that the MA > 'pulls' the configuration from its Controller, as identified by the > Initialiser. > OK. I guess this should then be reworded to read like this (or something similar): To avoid problems with NAT and firewalls, it is likely that the MA is required to establish the transport layer connection over which it receives its configuration from its Controller, as identified by the Initialiser. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From dromasca@avaya.com Thu Jul 25 02:46:57 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D855621F9AAB for ; Thu, 25 Jul 2013 02:46:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.413 X-Spam-Level: X-Spam-Status: No, score=-103.413 tagged_above=-999 required=5 tests=[AWL=0.186, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UhVWtw-hDMdb for ; Thu, 25 Jul 2013 02:46:52 -0700 (PDT) Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id 1F67721F9A15 for ; Thu, 25 Jul 2013 02:46:49 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Al4NACPz8FHGmAcV/2dsb2JhbABagmUhNVCpUpQIAwGBGBZ0giQBAQEBAxIoPwwEAgEIDQECAQQBAQEKFAkHMhQJCAIEDgUIGoduAZ14nA2PETsxBwaDDG4DmQiFHYsHgxSCKg X-IronPort-AV: E=Sophos;i="4.89,742,1367985600"; d="scan'208";a="17745875" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by de307622-de-outbound.net.avaya.com with ESMTP; 25 Jul 2013 05:46:48 -0400 Received: from unknown (HELO AZ-FFEXHC01.global.avaya.com) ([135.64.58.11]) by co300216-co-erhwest-out.avaya.com with ESMTP; 25 Jul 2013 05:44:13 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC01.global.avaya.com ([135.64.58.11]) with mapi id 14.02.0328.009; Thu, 25 Jul 2013 05:46:46 -0400 From: "Romascanu, Dan (Dan)" To: Juergen Schoenwaelder Thread-Topic: [lmap] Feedback on draft-eardley-lmap-terminology Thread-Index: AQHOhvZB9LoTtC34Rxe0kMCmPNNKaZlz1UIwgABgPoCAAOq3wIAATE4A//+9HKA= Date: Thu, 25 Jul 2013 09:46:46 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA12880DA0@AZ-FFEXMB04.global.avaya.com> References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <20130724151217.GA39515@elstar.local> <9904FB1B0159DA42B0B887B7FA8119CA12880D39@AZ-FFEXMB04.global.avaya.com> <20130725094528.GA41769@elstar.local> In-Reply-To: <20130725094528.GA41769@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "lmap@ietf.org" Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 09:46:58 -0000 > -----Original Message----- > From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs- > university.de] > Sent: Thursday, July 25, 2013 12:45 PM > To: Romascanu, Dan (Dan) > Cc: lmap@ietf.org > Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology >=20 > On Thu, Jul 25, 2013 at 09:17:22AM +0000, Romascanu, Dan (Dan) wrote: > > > > [[DR]] The phrase I am refering to is not from draft-eardley-lmap- > terminology-02 but from draft-eardley-lmap-framework-02, section 2. In > my reading this sentence does not talk about who initiates the session, > but about the way the configuration itself is to reach the MAs: > > > > > To avoid problems with NAT and firewalls, it is likely that the MA > > 'pulls' the configuration from its Controller, as identified by the > > Initialiser. > > >=20 > OK. I guess this should then be reworded to read like this (or something > similar): >=20 > To avoid problems with NAT and firewalls, it is likely that the MA > is required to establish the transport layer connection over which > it receives its configuration from its Controller, as identified by > the Initialiser. >=20 > /js >=20 [[DR]] yes - this makes sense.=20 Dan From dromasca@avaya.com Thu Jul 25 04:27:33 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9A0A521F9993 for ; Thu, 25 Jul 2013 04:27:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.954 X-Spam-Level: X-Spam-Status: No, score=-102.954 tagged_above=-999 required=5 tests=[AWL=-0.355, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9UXIjIADgi0W for ; Thu, 25 Jul 2013 04:27:28 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id 22B2A21F9A3C for ; Thu, 25 Jul 2013 04:27:27 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhkFAG8K8VHGmAcV/2dsb2JhbABagmUhNVC9XYEYFnSCJgEBAxIoUQEVFRRCJgEEGxqHbgELmHCEQpt3EwSPTINKbgOeJYsHgxSCKg X-IronPort-AV: E=Sophos;i="4.89,742,1367985600"; d="scan'208";a="21514522" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 25 Jul 2013 07:27:24 -0400 Received: from unknown (HELO AZ-FFEXHC04.global.avaya.com) ([135.64.58.14]) by co300216-co-erhwest-out.avaya.com with ESMTP; 25 Jul 2013 07:24:49 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC04.global.avaya.com ([135.64.58.14]) with mapi id 14.02.0328.009; Thu, 25 Jul 2013 07:27:23 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: LMAP at IETF-87 Thread-Index: Ac6JJzBSce6/ymYGTkqKxM/WLPaAAg== Date: Thu, 25 Jul 2013 11:27:23 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA128810DE@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] LMAP at IETF-87 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 11:27:33 -0000 Hi,=20 IETF-87 is only a few days away. The LMAP WG is scheduled to meet on Tuesda= y, July 30. The agenda is available at https://datatracker.ietf.org/meeting= /87/agenda/lmap/.=20 All presenters who plan to use slides - please send us your slides BEFORE t= he meeting, to allow remote and on-site participants to download and read t= hem in advance.=20 We will need two note-takers and one jabber scribe. Please volunteer in adv= ance so that we do not waste precious time in the meeting.=20 As announced we shall hold an informal welcome and introduction meeting for= the first time participants in the WG at breakfast time on Tuesday, July 3= 0. Unless announced differently we shall meet at 8AM at the entry of the br= eakfast space at the Intercontinental (meeting hotel).=20 We are looking forward to see you in Berlin.=20 Regards, Jason and Dan =20 From dromasca@avaya.com Thu Jul 25 05:40:02 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 612AF21F94FD for ; Thu, 25 Jul 2013 05:40:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.89 X-Spam-Level: X-Spam-Status: No, score=-102.89 tagged_above=-999 required=5 tests=[AWL=-0.291, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5daLtikraLFQ for ; Thu, 25 Jul 2013 05:39:57 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id E709121F93D4 for ; Thu, 25 Jul 2013 05:39:56 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AkMJAPgb8VGHCzI1/2dsb2JhbAA5IYJlITVQvV6BGBZ0giYBAQMSKFEBFRUUQh8HAQQbGoduAQuYR4RCnAgEj0yDSm4DniWLB4MUgio X-IronPort-AV: E=Sophos;i="4.89,743,1367985600"; d="scan'208";a="21523205" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 25 Jul 2013 08:39:56 -0400 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 25 Jul 2013 08:33:36 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.02.0328.009; Thu, 25 Jul 2013 08:39:55 -0400 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: on use cases and requirements Thread-Index: Ac6JNA/7jjUgkfhDSx2RandUUF8Hhw== Date: Thu, 25 Jul 2013 12:39:54 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA12881146@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] on use cases and requirements X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 12:40:02 -0000 If you have a look at the updated agenda at https://datatracker.ietf.org/me= eting/87/agenda/lmap/ you will realized that Jason and me slightly updated = the section dedicated to use cases and added ten minutes of discussion. Let= me try to explain the motivation and what we would try to achieve. Accordi= ng to the charter in a couple of months the WG will need to submit the firs= t WG Internet-Drafts on Use Cases and Framework. These two documents should= provide clear enough requirements for the following phases where we need t= o proceed to the creation of the Information Model, and discuss about the s= election or development of protocols for Control and Reporting and their as= sociated data models. At this moment in time we have one draft-linsner-lmap= -use-cases-03 as principal document on use cases, and two new contributions= , which is good for this phase. What we need to start doing is to sort out = of the use cases the subset that matches the current WG charter - and we ma= y need to drop or prioritize some of the aspects of the use cases for this = purpose.=20 (To be more specific and take this just as an example, I am not sure that a= ll the characteristics described in Sections 3.2, 3.3, 3.5 in draft-linsner= -lmap-use-cases-03 are LMAP problems or problems that belong to the require= ments of the first phase of LMAP.) Regards, Dan From philip.eardley@bt.com Thu Jul 25 06:17:17 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F89421F99AF for ; Thu, 25 Jul 2013 06:17:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.08 X-Spam-Level: X-Spam-Status: No, score=-103.08 tagged_above=-999 required=5 tests=[AWL=0.019, BAYES_00=-2.599, GB_ABOUTYOU=0.5, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JuIm65CUAgIz for ; Thu, 25 Jul 2013 06:17:13 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id A322521F9A18 for ; Thu, 25 Jul 2013 06:17:12 -0700 (PDT) Received: from EVMHT64-UKRD.domain1.systemhost.net (10.36.3.101) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.298.1; Thu, 25 Jul 2013 14:17:09 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.253]) by EVMHT64-UKRD.domain1.systemhost.net ([10.36.3.101]) with mapi; Thu, 25 Jul 2013 14:17:09 +0100 From: To: , Date: Thu, 25 Jul 2013 14:17:07 +0100 Thread-Topic: on use cases and requirements Thread-Index: Ac6JNA/7jjUgkfhDSx2RandUUF8HhwAAwTCw Message-ID: <9510D26531EF184D9017DF24659BB87F35B80337A5@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA12881146@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA12881146@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] on use cases and requirements X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 13:17:17 -0000 I had a look at the new drafts, couple of questions Ken, http://tools.ietf.org/html/draft-nagami-lmap-use-case-measurement-provider-= 00 you talk about a "measurement provider" - if I get it right, this is someon= e who provides information about different ISPs. this seems to me very simi= lar to the regulator use case - but it's a public interest group, or a comp= any providing the information instead of a regulator. It's probably worth p= ointing this out in http://tools.ietf.org/html/draft-linsner-lmap-use-cases= - would this then cover your use case? You also mention a smartphone, I think it's worth adding this explicitly to= draft-linsner You also have a lot of interesting details about your deployment. My feelin= g is that it would take a lot of work to summarise all the state-of-the-art= , so we shouldn't try (perhaps we could add a few references in one of the = docs) Rachel, http://tools.ietf.org/id/draft-huang-lmap-data-collection-use-case-00.txt I take this as a slight expansion of the ISP use case described in http://t= ools.ietf.org/html/draft-linsner-lmap-use-cases=20 Could we work with you on adding a little text about simulations to our dra= ft? - ie the ISP could use measurements to feed its simulation tools (assum= ing it uses simulations as part of the way it works out where a fault is, o= r where to add capacity, etc).=20 Will you be in Berlin? Marc, Ken and I are meeting in the lmap breakfast to= discuss use cases.=20 Dan, Personally I'm happy to drop as much of Section 3 as people want (even all = of it). I agree we should think carefully in terms of what the output of lmap needs= to do - what are the most important aspects of which use cases. Reading between the lines, you think the use cases doc should only cover th= ose aspects that we're going to (try to) make sure we solve during the init= ial lmap charter. And not be a general use case doc that would need future = lmap capability. Is that right?=20 (I'm ok if the answer is 'yes') Thanks phil > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Romascanu, Dan (Dan) > Sent: 25 July 2013 13:40 > To: lmap@ietf.org > Subject: [lmap] on use cases and requirements >=20 > If you have a look at the updated agenda at > https://datatracker.ietf.org/meeting/87/agenda/lmap/ you will realized > that Jason and me slightly updated the section dedicated to use cases > and added ten minutes of discussion. Let me try to explain the > motivation and what we would try to achieve. According to the charter > in a couple of months the WG will need to submit the first WG Internet- > Drafts on Use Cases and Framework. These two documents should provide > clear enough requirements for the following phases where we need to > proceed to the creation of the Information Model, and discuss about the > selection or development of protocols for Control and Reporting and > their associated data models. At this moment in time we have one draft- > linsner-lmap-use-cases-03 as principal document on use cases, and two > new contributions, which is good for this phase. What we need to start > doing is to sort out of the use cases the subset that matches the > current WG charter - and we may need to drop or prioritize some of the > aspects of the use cases for this purpose. >=20 > (To be more specific and take this just as an example, I am not sure > that all the characteristics described in Sections 3.2, 3.3, 3.5 in > draft-linsner-lmap-use-cases-03 are LMAP problems or problems that > belong to the requirements of the first phase of LMAP.) >=20 > Regards, >=20 > Dan >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From philip.eardley@bt.com Thu Jul 25 06:28:16 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 937F421F933B for ; Thu, 25 Jul 2013 06:28:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.333 X-Spam-Level: X-Spam-Status: No, score=-103.333 tagged_above=-999 required=5 tests=[AWL=0.266, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5ghFAXPPwh7L for ; Thu, 25 Jul 2013 06:28:12 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp61.intersmtp.com [62.239.224.234]) by ietfa.amsl.com (Postfix) with ESMTP id 90B7E21F8436 for ; Thu, 25 Jul 2013 06:27:38 -0700 (PDT) Received: from EVMHT67-UKRD.domain1.systemhost.net (10.36.3.104) by RDW083A005ED61.smtp-e1.hygiene.service (10.187.98.10) with Microsoft SMTP Server (TLS) id 8.3.298.1; Thu, 25 Jul 2013 14:27:37 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.253]) by EVMHT67-UKRD.domain1.systemhost.net ([10.36.3.104]) with mapi; Thu, 25 Jul 2013 14:27:37 +0100 From: To: , Date: Thu, 25 Jul 2013 14:27:35 +0100 Thread-Topic: [lmap] Feedback on draft-eardley-lmap-terminology Thread-Index: AQHOhvZB9LoTtC34Rxe0kMCmPNNKaZlz1UIwgABgPoCAAOq3wIAATE4A//+9HKCAADynEA== Message-ID: <9510D26531EF184D9017DF24659BB87F35B80337B0@EMV65-UKRD.domain1.systemhost.net> References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <20130724151217.GA39515@elstar.local> <9904FB1B0159DA42B0B887B7FA8119CA12880D39@AZ-FFEXMB04.global.avaya.com> <20130725094528.GA41769@elstar.local> <9904FB1B0159DA42B0B887B7FA8119CA12880DA0@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA12880DA0@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 13:28:16 -0000 Dan, Juergen - Thanks, good catch and the re-phrasing seems good to me. Also Dan said << But then in the diagram in Figure 1 the arrow points from the Controller= to the MA, and later the examples of possible protocols in Section 5.3 inc= lude such 'push' protocols as SNMP and NETCONF>> I'll clarify that the arrow is about the Instruction. The control protocol = may need some back and forth (to request or 'pull' the Instruction) > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Romascanu, Dan (Dan) > Sent: 25 July 2013 10:47 > To: Juergen Schoenwaelder > Cc: lmap@ietf.org > Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology >=20 >=20 >=20 >=20 >=20 > > -----Original Message----- > > From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs- > > university.de] > > Sent: Thursday, July 25, 2013 12:45 PM > > To: Romascanu, Dan (Dan) > > Cc: lmap@ietf.org > > Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology > > > > On Thu, Jul 25, 2013 at 09:17:22AM +0000, Romascanu, Dan (Dan) wrote: > > > > > > [[DR]] The phrase I am refering to is not from draft-eardley-lmap- > > terminology-02 but from draft-eardley-lmap-framework-02, section 2. > In > > my reading this sentence does not talk about who initiates the > > session, but about the way the configuration itself is to reach the > MAs: > > > > > > > To avoid problems with NAT and firewalls, it is likely that the > > > > MA > > > 'pulls' the configuration from its Controller, as identified by > the > > > Initialiser. > > > > > > > OK. I guess this should then be reworded to read like this (or > > something > > similar): > > > > To avoid problems with NAT and firewalls, it is likely that the MA > > is required to establish the transport layer connection over which > > it receives its configuration from its Controller, as identified by > > the Initialiser. > > > > /js > > >=20 > [[DR]] yes - this makes sense. >=20 > Dan >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From philip.eardley@bt.com Thu Jul 25 06:34:42 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0FB1721F9A96 for ; Thu, 25 Jul 2013 06:34:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.371 X-Spam-Level: X-Spam-Status: No, score=-103.371 tagged_above=-999 required=5 tests=[AWL=0.228, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4tPyrdZosdYF for ; Thu, 25 Jul 2013 06:34:37 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id BA8D721F99F2 for ; Thu, 25 Jul 2013 06:34:36 -0700 (PDT) Received: from EVMHT65-UKRD.domain1.systemhost.net (10.36.3.102) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.298.1; Thu, 25 Jul 2013 14:34:34 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.253]) by EVMHT65-UKRD.domain1.systemhost.net ([10.36.3.102]) with mapi; Thu, 25 Jul 2013 14:34:34 +0100 From: To: , Date: Thu, 25 Jul 2013 14:34:33 +0100 Thread-Topic: comments on draft-eardley-lmap-framework-02 Thread-Index: Ac6HpNxWSlKt/J3PRHm+kNP0wHNi/gBleUuw Message-ID: <9510D26531EF184D9017DF24659BB87F35B80337B7@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA1287E1FE@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1287E1FE@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] comments on draft-eardley-lmap-framework-02 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 13:34:42 -0000 > 2. I have a problem imagining what the 'Initialiser' is. Is it a task > (running where?) aiming to ensure consistent configuration of > controllers in an LMAP system. Or another 'box' in the architecture, > part of an hierarchical system, a 'box' not represented because it is > out of the LMAP scope? If the later I would prefer to add it to the > diagram, maybe together with the other non-LMAP entities (Subscriber > Parameter Database, Results Database) so that we can have a full > picture of the whole system, with clear representation of what is in > scope for LMAP. The latter - a 'box' outside lmap scope.=20 In the next version I'll battle with asci art to add a pic showing all the = non-lmap entities. For a preview, see the framework slides (in wg meeting, = similar ones in BoF) One example, the 'box' could be an ACS which uses TR-069 to configure (ie l= map-initialise) a Broadband Forum compliant home gateway. Another example m= ight be that the Measurement Agent is an app downloaded to your tablet, wit= h pre-configuration information coded into the app.=20 >=20 > 3. Do we also need to include in the diagram (and discussion) some kind > a certificate authority to ensure that controllers and MAs, MAs and > peers, MA and collectors, are authenticated to each other, as the > security considerations section demands. I can add this - another 'box' outside lmap scope >=20 > 4. In a couple of instances OAM is wrongly expanded as Operations, > Administrations and Management, and not as per BCP 161. Thanks. (I actually did try to identify the correct ietf phrase and abbrevi= ation - but failed!) >=20 > 5. In Section 5.1 there are a few instances where some procedures or > blocks are being defined as 'out (or beyond) the scope of the IETF'. It > would be more correct to say 'out of scope for LMAP' as we can make > statements about what we do (or do not do) in LMAP but not for the IETF > as a whole. Ok Thanks phil From paitken@cisco.com Thu Jul 25 06:40:42 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 030B321F9AB4 for ; Thu, 25 Jul 2013 06:40:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A7MHBH1zlytx for ; Thu, 25 Jul 2013 06:40:36 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id 10CD821F91BF for ; Thu, 25 Jul 2013 06:40:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2642; q=dns/txt; s=iport; t=1374759636; x=1375969236; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=DKsUHY3vnqONk9vaHyh7O2r0gGOboiofVgfNvActaRg=; b=btp6MSBpbsWUy+BuFcxtNGK1WTpKv+CsPYA1T+GvTMavQ6xb60XaJJdK nWjc8bq6BvWrzSv9yIFB+y1/z+ZW07/eMyD4Ap6EDQ3LO9MjqPgiGb6Wl Q7CkXZCoALu6roEyGrVAmc8acKK8p9rDYASXSqGt7uIxYCPgO8OCXt8JK 8=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ah4FAM4p8VGQ/khM/2dsb2JhbABagwY2vi2BFxZ0giQBAQEDATIBBUABBQsLDgoJFg8JAwIBAgFFBg0BBwEBiAYGuSmPfQeEAAOXX4YjiyqDFQ X-IronPort-AV: E=Sophos;i="4.89,729,1367971200"; d="scan'208";a="15986483" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-3.cisco.com with ESMTP; 25 Jul 2013 13:40:31 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r6PDeTZc032535 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Jul 2013 13:40:29 GMT Received: from [10.61.164.215] ([10.61.164.215]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6PDeRkP001388; Thu, 25 Jul 2013 14:40:28 +0100 (BST) Message-ID: <51F12ACC.1040702@cisco.com> Date: Thu, 25 Jul 2013 14:40:28 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: Juergen Schoenwaelder References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <51EFEC2A.9010701@cisco.com> <51F0297A.7040407@it.uc3m.es> <51F0367F.1060905@cisco.com> <20130724204924.GA40227@elstar.local> <51F041FD.4050408@cisco.com> <20130725091606.GB41645@elstar.local> In-Reply-To: <20130725091606.GB41645@elstar.local> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 13:40:42 -0000 Juergen, > On Wed, Jul 24, 2013 at 10:07:09PM +0100, Paul Aitken wrote: > > [...] > > You I think you are proposing that instructions received from > different Controllers must be kept separate and the MA must act > towards a Controller as if there are no other controllers. To some extent, yes. > A potential > problem with this is that there are likely instructions and tests that > do impact each other through side effects. > > If C1 schedules a test to be started every 5 min past the hour that > requires no cross traffic while C2 schedules a test to measure > video streaming capabilities during the interval 3-7 minutes past > the hour, then C1 will be surprised that the scheduled test never > executes or produces wrong results. Forget about multiple controllers: this situation could arise from a single controller. So a rejection mechanism will surely be necessary regardless of whether there are one or multiple controllers: - whether the command protocol is two-way, so the command is immediately rejected. - or, the report protocol includes a way of reporting that the requested test could not be done. - or both. > Do you require that the MA has the logic to reject one of the > instructions in such a case? Definitely, even for the single controller case. > Or do you expect that the Controllers > will simply accept that MAs may not do what they think they should be > doing? Ideally the MA would report that a) it cannot perform the requested test, and b) why that is so. Revealing too much information in (b) is usually a security risk. However it's not an issue in this case since we're assuming that the system is under control of a single organisation. > Or do you expect to have at least read access to all > instructions? Will we end up with an access control model to handle > things properly in a configurable manner (like we did with SNMP and > NETCONF)? No, not at all. > If the assumption is complete separation then I believe this is quite > a big one - in particular if I consider devices like home routers or > hardware probes of some bigger existing measurement platforms as > potential targets for this work (usuall small embedded Linux systems > running on cheap hardware). If an MA may be commanded to perform multiple tests¹ by a single controller, then the potential conflicts are the same as multiple tests from multiple controllers. ¹ This is necessary if we are to build complex tests out of simpler test primitives, eg DNS lookup followed by file download. P. From philip.eardley@bt.com Thu Jul 25 06:52:20 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B699721F99D0 for ; Thu, 25 Jul 2013 06:52:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.399 X-Spam-Level: X-Spam-Status: No, score=-103.399 tagged_above=-999 required=5 tests=[AWL=0.200, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PsP7hCgFnpsP for ; Thu, 25 Jul 2013 06:52:15 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id 37D0A21F99C5 for ; Thu, 25 Jul 2013 06:52:15 -0700 (PDT) Received: from EVMHT68-UKRD.domain1.systemhost.net (10.36.3.105) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.298.1; Thu, 25 Jul 2013 14:52:14 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.253]) by EVMHT68-UKRD.domain1.systemhost.net ([10.36.3.105]) with mapi; Thu, 25 Jul 2013 14:52:13 +0100 From: To: , Date: Thu, 25 Jul 2013 14:52:12 +0100 Thread-Topic: [lmap] my comments on draft-akhter-lmap-framework-00 Thread-Index: Ac6IYOM5ZXCAYUqURvC81oEySuxFpAA2xzWA Message-ID: <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 13:52:20 -0000 > 5. The second significant difference in the two diagrams is the > communication between the Controller and the Collector. Later in the > draft this is described as optional and motivated by the need to > optimize for Collector Test Configuration synchronization (so that each > MA needs not report its configuration to controllers). This addition to > the basic architecture needs to be carefully discussed by the WG. Expanding on this point. There are ways we can reduce the overhead. Firstly, a Measurement Agent will send an occasional report to its Collecto= r (say once a day). So this report may include lots of 'repeat measurements= ' - the same Measurement Method made with the same parameters, every 10 min= utes (say). When these results are reported, you only need to list the Meas= urement Method and parameters once - plus the individual results and their = times. So the overhead isn't huge. This makes an assumption about encoding of information in the report (eg js= on could do this). If I remember, this issue crops up in the http protocol = draft. =20 Secondly, if this isn't enough, the measurement system could have a registr= y - so the report would just include a reference to a registry entry. This= is somewhat similar to your idea of a controller-collector interface. Pers= onally I prefer the former (as the latter needs to worry about syncronisati= on). But, more important, I don't think the wg should think about either id= ea in the initial work. Best wishes phil=20 From philip.eardley@bt.com Thu Jul 25 07:02:19 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 53BC621F9546 for ; Thu, 25 Jul 2013 07:02:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.422 X-Spam-Level: X-Spam-Status: No, score=-103.422 tagged_above=-999 required=5 tests=[AWL=0.177, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XrKnnaz4LLPR for ; Thu, 25 Jul 2013 07:02:14 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp61.intersmtp.com [62.239.224.234]) by ietfa.amsl.com (Postfix) with ESMTP id 37C2D21F9A17 for ; Thu, 25 Jul 2013 07:02:11 -0700 (PDT) Received: from EVMHT64-UKRD.domain1.systemhost.net (10.36.3.101) by RDW083A005ED61.smtp-e1.hygiene.service (10.187.98.10) with Microsoft SMTP Server (TLS) id 8.3.298.1; Thu, 25 Jul 2013 15:02:09 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.253]) by EVMHT64-UKRD.domain1.systemhost.net ([10.36.3.101]) with mapi; Thu, 25 Jul 2013 15:02:08 +0100 From: To: Date: Thu, 25 Jul 2013 15:02:08 +0100 Thread-Topic: comment on draft-akhter-lmap-framework-00 Thread-Index: Ac6JPjpGY+MCHWnARpa8vcuZGWla5A== Message-ID: <9510D26531EF184D9017DF24659BB87F35B80337F8@EMV65-UKRD.domain1.systemhost.net> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] comment on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 14:02:19 -0000 << One proposal for making use of both active and passive measurement is to allow the Measurement Agent to make local decisions on which technique to use to deliver a particular metric-- as long as the specific method is included in the report.>> interesting idea of allowing the MA to choose what Measurement Method it do= es. However, I think it is considerably more complex than the Controller tellin= g the MA what to do, and think we should exclude it (at least in the initia= l work).=20 For example, in terms of non-technical issues, if there's some problem with= a third party (perhaps the Measurement Peer), then who's gets the blame? = =20 It would also make the MA more complex (I'm hoping MA functionality is simp= le, embedded and widespread) I'd be quite against a MA having a free choice between an active and passiv= e measurement - since passive measurements raise much harder Data Protectio= n (privacy) issues.=20 Best wishes phil From paitken@cisco.com Thu Jul 25 07:08:13 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 141B121F8C72 for ; Thu, 25 Jul 2013 07:08:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gKSPgBcpShWx for ; Thu, 25 Jul 2013 07:08:08 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 2A1DB21F8CB4 for ; Thu, 25 Jul 2013 07:08:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2257; q=dns/txt; s=iport; t=1374761283; x=1375970883; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=6eDmIiX17FyTNJERA/+/hTA0TaHgRea8zm4nnnP5kqw=; b=GpNWf8rUwETJks8XnmLcbYRV7BxcwXja3DUHT5+q3GJYlL8dFDNLpznT L5ETezHAeEqBbrLpT9h6jEsNGxcWFar8FYnJBzkdDzpUHhGbZ7iQhEjWB 7TVXSftHqZFuue9bkpd/Gk2HsmF0NqWeDs0pcBIQwF5PyfObn3eFKhGKm w=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ah8FAJww8VGQ/khN/2dsb2JhbABagwY1Ab4tgRYWdIIkAQEBAwEBAQE1NgoBBQsLEg8WDwkDAgECARUiDhMBBQIBAYgGBgy5KQSPfQcWg2oDl1+GI4sqgxU X-IronPort-AV: E=Sophos;i="4.89,743,1367971200"; d="scan'208";a="85112141" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-2.cisco.com with ESMTP; 25 Jul 2013 14:07:59 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id r6PE7vZ4011979 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Jul 2013 14:07:57 GMT Received: from [10.61.164.215] ([10.61.164.215]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6PE7rtg003373; Thu, 25 Jul 2013 15:07:54 +0100 (BST) Message-ID: <51F1313A.9060800@cisco.com> Date: Thu, 25 Jul 2013 15:07:54 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: philip.eardley@bt.com References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: dromasca@avaya.com, lmap@ietf.org Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 14:08:13 -0000 Philip, >> 5. The second significant difference in the two diagrams is the >> communication between the Controller and the Collector. Later in the >> draft this is described as optional and motivated by the need to >> optimize for Collector Test Configuration synchronization (so that each >> MA needs not report its configuration to controllers). This addition to >> the basic architecture needs to be carefully discussed by the WG. > Expanding on this point. > > There are ways we can reduce the overhead. > > Firstly, a Measurement Agent will send an occasional report to its Collector (say once a day). So this report may include lots of 'repeat measurements' - the same Measurement Method made with the same parameters, every 10 minutes (say). When these results are reported, you only need to list the Measurement Method and parameters once - plus the individual results and their times. So the overhead isn't huge. Isn't huge wrt a single MA. However, if 1,000's of MAs have been commanded to perform the same test, the impact on the collector is much higher: it has to examine all the methods and parameters to determine that they all performed the same test. > This makes an assumption about encoding of information in the report (eg json could do this). If I remember, this issue crops up in the http protocol draft. > > Secondly, if this isn't enough, the measurement system could have a registry - so the report would just include a reference to a registry entry. This is somewhat similar to your idea of a controller-collector interface. What we wrote in draft-aakhter-lmap framework: as an optimisation, the controller treats the collector as an additional MA, thereby informing the collector once of the method and params to be used, presumably using some ID. Then the MA's need only report that same ID, so the collector overhead is much reduced. P. > Personally I prefer the former (as the latter needs to worry about syncronisation). But, more important, I don't think the wg should think about either idea in the initial work. > > Best wishes > phil > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From acmorton@att.com Thu Jul 25 07:24:13 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5A84B21F996A for ; Thu, 25 Jul 2013 07:24:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.524 X-Spam-Level: X-Spam-Status: No, score=-106.524 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6gok60+X34Ys for ; Thu, 25 Jul 2013 07:24:08 -0700 (PDT) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 2DB6521F9967 for ; Thu, 25 Jul 2013 07:24:08 -0700 (PDT) Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id 3BF1D120626; Thu, 25 Jul 2013 10:24:05 -0400 (EDT) Received: from njfpsrvexg1.research.att.com (njfpsrvexg1.research.att.com [135.207.177.20]) by mail-green.research.att.com (Postfix) with ESMTP id 6D669E0190; Thu, 25 Jul 2013 10:23:43 -0400 (EDT) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg1.research.att.com ([fe80::58ce:ca01:5d18:db01%13]) with mapi; Thu, 25 Jul 2013 10:24:07 -0400 From: "MORTON JR., ALFRED C (AL)" To: Paul Aitken , "philip.eardley@bt.com" Date: Thu, 25 Jul 2013 10:24:06 -0400 Thread-Topic: [lmap] my comments on draft-akhter-lmap-framework-00 Thread-Index: Ac6JQG+1qrTTNLmqTZiGwzZ0cn0TYgAAKj8g Message-ID: References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> In-Reply-To: <51F1313A.9060800@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dromasca@avaya.com" , "lmap@ietf.org" Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 14:24:13 -0000 Paul, still reading this draft, but as the messages fly by, you said: > What we wrote in draft-aakhter-lmap framework: as an optimisation, the > controller treats the collector as an additional MA, thereby informing > the collector once of the method and params to be used, presumably using > some ID. Then the MA's need only report that same ID, so the collector > overhead is much reduced. how does this optimization match the flexibility to use different measureme= nt=20 methods on the fly, as Phil highlighted: << One proposal for making use of both active and passive measurement is to allow the Measurement Agent to make local decisions on which technique to use to deliver a particular metric-- as long as the specific method is included in the report.>> I think we agree that passive and active methods are sufficiently different that they require categorization, possibly as far as summarizing=20 results from each method separately. MA local decisions certainly=20 require indicating the measurement method with each value,=20 hopefully through some shorthand like the proposed registry. It seems we can have either the optimizations with static test Instructions= , or MA's making autonomous decisions. Al > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Paul Aitken > Sent: Thursday, July 25, 2013 10:08 AM > To: philip.eardley@bt.com > Cc: dromasca@avaya.com; lmap@ietf.org > Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 >=20 > Philip, >=20 > >> 5. The second significant difference in the two diagrams is the > >> communication between the Controller and the Collector. Later in the > >> draft this is described as optional and motivated by the need to > >> optimize for Collector Test Configuration synchronization (so that eac= h > >> MA needs not report its configuration to controllers). This addition t= o > >> the basic architecture needs to be carefully discussed by the WG. > > Expanding on this point. > > > > There are ways we can reduce the overhead. > > > > Firstly, a Measurement Agent will send an occasional report to its > Collector (say once a day). So this report may include lots of 'repeat > measurements' - the same Measurement Method made with the same parameters= , > every 10 minutes (say). When these results are reported, you only need to > list the Measurement Method and parameters once - plus the individual > results and their times. So the overhead isn't huge. >=20 > Isn't huge wrt a single MA. However, if 1,000's of MAs have been > commanded to perform the same test, the impact on the collector is much > higher: it has to examine all the methods and parameters to determine > that they all performed the same test. >=20 >=20 > > This makes an assumption about encoding of information in the report (e= g > json could do this). If I remember, this issue crops up in the http > protocol draft. > > > > Secondly, if this isn't enough, the measurement system could have a > registry - so the report would just include a reference to a registry > entry. This is somewhat similar to your idea of a controller-collector > interface. >=20 > What we wrote in draft-aakhter-lmap framework: as an optimisation, the > controller treats the collector as an additional MA, thereby informing > the collector once of the method and params to be used, presumably using > some ID. Then the MA's need only report that same ID, so the collector > overhead is much reduced. >=20 > P. >=20 > > Personally I prefer the former (as the latter needs to worry about > syncronisation). But, more important, I don't think the wg should think > about either idea in the initial work. > > > > Best wishes > > phil > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From philip.eardley@bt.com Thu Jul 25 07:51:54 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5517321F9AF8 for ; Thu, 25 Jul 2013 07:51:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.439 X-Spam-Level: X-Spam-Status: No, score=-103.439 tagged_above=-999 required=5 tests=[AWL=0.159, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JiS-oOvfkLvq for ; Thu, 25 Jul 2013 07:51:48 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id 42B9F21F9AED for ; Thu, 25 Jul 2013 07:51:48 -0700 (PDT) Received: from EVMHT67-UKRD.domain1.systemhost.net (10.36.3.104) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.298.1; Thu, 25 Jul 2013 15:51:47 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.253]) by EVMHT67-UKRD.domain1.systemhost.net ([10.36.3.104]) with mapi; Thu, 25 Jul 2013 15:51:47 +0100 From: To: , Date: Thu, 25 Jul 2013 15:51:46 +0100 Thread-Topic: Feedback on draft-eardley-lmap-framework-01 Thread-Index: Ac6Iue+JkB9mX20GQROv2rsQmPcjnQAjEElg Message-ID: <9510D26531EF184D9017DF24659BB87F35B8033851@EMV65-UKRD.domain1.systemhost.net> References: In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F35B8033851EMV65UKRDdoma_" MIME-Version: 1.0 Subject: Re: [lmap] Feedback on draft-eardley-lmap-framework-01 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 14:51:54 -0000 --_000_9510D26531EF184D9017DF24659BB87F35B8033851EMV65UKRDdoma_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Thanks! In-line belwo phil From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Wei= l, Jason Sent: 24 July 2013 23:06 To: lmap@ietf.org Subject: [lmap] Feedback on draft-eardley-lmap-framework-01 Here are my comments and questions on the draft: Section 1. [use-cases] Regulators: I would drop the word 'several' from this description. It reads= as if regulators are targeting certain providers purposefully out of a gro= up which is not typically the case. --- The example given in Diversity seems to be another example supporting the i= nteroperability benefits that come with the earlier bullet on Standardizati= on. A MA on a CPE router and another MA on a WiFi device behind the router= or a MA running as a software client on a laptop/desktop might be a bette= r example? --- Section 2. The MA functions are implemented either in specialised hardware or as code on general purpose devices like a PC, tablet or smartphone. The Measurement Peer may be an LMAP device or a normal, non-LMAP device (for example if the MA measures the time for a DNS response or a webpage download from www.example.com). A Measurement Peer is an LMAP functional element. It would seem to me that = as long as an entity performs in the role of a Measurement Peer it would be= considered an LMAP device whether or not it was purpose built for a single= test or simply performing as a Measurement Peer based on its normal operat= ional role such as a recursive resolver. As such I don't believe a Measurem= ent Peer would ever be considered a non-LMAP device if it was being used fo= r a test by the MA. [phil] so I'll re-phrase something like "The Measurement Peer may be a spec= ial-purpose device or a device performing its normal operational role (for = example.... -- Section 3. 3.2. Each MA may only have a single Controller at any point in time - The '= may' in this sentence makes me wonder where the constraint exists. Is the c= onstraint that it must only take direction from a single Controller at any = given time? [phil] yes -- The topic on NAT an firewalls suggesting a pull model may be better as a se= parate constraint topic by itself. [phil] seems a good suggestion to me Thanks, Jason ________________________________ This E-mail and any of its attachments may contain Time Warner Cable propri= etary information, which is privileged, confidential, or subject to copyrig= ht belonging to Time Warner Cable. This E-mail is intended solely for the u= se of the individual or entity to which it is addressed. If you are not the= intended recipient of this E-mail, you are hereby notified that any dissem= ination, distribution, copying, or action taken in relation to the contents= of and attachments to this E-mail is strictly prohibited and may be unlawf= ul. If you have received this E-mail in error, please notify the sender imm= ediately and permanently delete the original and any copy of this E-mail an= d any printout. --_000_9510D26531EF184D9017DF24659BB87F35B8033851EMV65UKRDdoma_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Thanks!

In-line belwo

phil

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf= .org] On Behalf Of Weil, Jason
Sent: 24 July 2013 23:06To: lmap@ietf.org
Subject: [lmap] Feedback on draft-eardl= ey-lmap-framework-01

=  

Here are my comments an= d questions on the draft:

 

S= ection 1. [use-cases]

= Regulators: I would drop the word 'several' from this description. It = reads as if regulators are targeting certain providers purposefully out of = a group which is not typically the case. 

<= div>

---

The example given in Diversity seems to be another exampl= e supporting the interoperability benefits that come with the earlier bulle= t on Standardization.  A MA on a CPE router and another MA on a WiFi d= evice behind the router or a MA running as a software client on a laptop/de= sktop might be  a better example?

---

Section 2. 

 

 &=
nbsp; The MA functions are implemented either in specialised hardware or as=

   code on general purpose devices lik=
e a PC, tablet or smartphone.  The
 &=
nbsp; Measurement Peer may be an LMAP device or a normal, non-LMAP device
   (for example if the MA measures the t=
ime for a DNS response or a
   webpage =
download from www.example.com).=

 
A Measurement Peer is an LMAP functional element. It would se=
em to me that as long as an entity performs in the role of a Measurement Pe=
er it would be considered an LMAP device whether or not it was purpose buil=
t for a single test or simply performing as a Measurement Peer based on its=
 normal operational role such as a recursive resolver. As such I don’=
t believe a Measurement Peer would ever be considered a non-LMAP device if =
it was being used for a test by the MA. 
 
[phil] so I’ll re-phrase something like “<=
/span>The Measurement Peer may =
be a special-purpose device or a device performing its normal operational r=
ole (for example….
 
--=

Section 3.



3.2. Each MA may only have a single Controller= at any point in time – The 'may' in this sentence makes me wonder wh= ere the constraint exists. Is the constraint that it must only take directi= on from a single Controller at any given time?

=

 

[phil] yes

 

--

The topic on NAT an firewalls su= ggesting a pull model may be better as a separate constraint topic by itsel= f.

 

[p= hil] seems a good suggestion to me





Thanks,<= /p>



Jason

 

This E-mail and any of its attachments may contain Time Warne= r Cable proprietary information, which is privileged, confidential, or subj= ect to copyright belonging to Time Warner Cable. This E-mail is intended so= lely for the use of the individual or entity to which it is addressed. If y= ou are not the intended recipient of this E-mail, you are hereby notified t= hat any dissemination, distribution, copying, or action taken in relation t= o the contents of and attachments to this E-mail is strictly prohibited and= may be unlawful. If you have received this E-mail in error, please notify = the sender immediately and permanently delete the original and any copy of = this E-mail and any printout.

= = --_000_9510D26531EF184D9017DF24659BB87F35B8033851EMV65UKRDdoma_-- From nagami@wide.ad.jp Thu Jul 25 10:32:19 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5B0DE21F9964 for ; Thu, 25 Jul 2013 10:32:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.978 X-Spam-Level: X-Spam-Status: No, score=-1.978 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 69VnDzdExEP7 for ; Thu, 25 Jul 2013 10:32:18 -0700 (PDT) Received: from sh.wide.ad.jp (sh.wide.ad.jp [IPv6:2001:200:0:1001::6]) by ietfa.amsl.com (Postfix) with ESMTP id C3F7B21F995E for ; Thu, 25 Jul 2013 10:32:18 -0700 (PDT) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (authenticated (0 bits)) by mail.wide.ad.jp (8.14.1+3.5Wbeta/8.14.1/smtpfeed 1.21) with ESMTP id r6PHWF1W005111 (using TLSv1/SSLv3 with cipher RC4-SHA (128 bits) verified FAIL) for ; Fri, 26 Jul 2013 02:32:16 +0900 (JST) Received: by mail-pa0-f52.google.com with SMTP id kq13so2251736pab.11 for ; Thu, 25 Jul 2013 10:32:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=45JrnRLU3H1/YxR1CKlE3SIp1Qho3WwnRisY1yLoUkk=; b=L/abreYKGw8NwC1qLSwbxVJKkEVUB8mJOglLpcQdnwVU/cw+rcryq6e2kTW4tJserx g8n914St9wH+Ad/dNpGhWeZoiFpUDxW51zQETd+x8sFZpi8klYVsvRQ2q+xYSicJbKW5 l9xw3ksmJ9ybvv/gpMUKKoZzIJ0wftFEjoicszMyHb8+2pLqsRv0bovKWwFtzAUxqYZf nwHhNT3TxON2dvVfWGgT0LnDXSY2KA9EUPPYPjwOViyZeeG+cRE4bSoKLWNMdoHtUqsE eylXjVTJq/2ErlMfVDzmK7aAHn7iYfRcwugm458JqWpxMzRaUS2EgpFzBSPGFrBhlNli VbsA== MIME-Version: 1.0 X-Received: by 10.68.211.194 with SMTP id ne2mr47906612pbc.40.1374773530636; Thu, 25 Jul 2013 10:32:10 -0700 (PDT) Received: by 10.70.102.176 with HTTP; Thu, 25 Jul 2013 10:32:10 -0700 (PDT) In-Reply-To: <51ED4150.6070907@cisco.com> References: <51ED4150.6070907@cisco.com> Date: Fri, 26 Jul 2013 02:32:10 +0900 Message-ID: From: Nagami Kenichi To: Benoit Claise Content-Type: text/plain; charset=ISO-8859-1 Cc: "lmap@ietf.org" Subject: Re: [lmap] Feedback on draft-nagami-lmap-use-case-measurement-provider X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 17:32:19 -0000 Hi Benoit, Thank you for comment. > Next to the points I made regarding to draft-linsner-lmap-use-cases, which > are applicable here as well, I have one extra point: I like the fact that > the draft covers the smartphone. This use case is completely different. I > might not care if a MA generates some packets from my home, but I care if > this is from my smartphone and my bill is pay per volume. The roaming price > are still expensive. I think it depends on the mobile service environment. If the fee is determined by the amount of traffic as you wrote, it is difficult that a smartphone becomes MA. If the fee is flat rate, a smartphone becomes MA. In our experiment, 2000 smartphones participated as MA because the fee is flat rate. Regards, Ken Nagami From j.schoenwaelder@jacobs-university.de Thu Jul 25 11:20:20 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5AB7D21F969F for ; Thu, 25 Jul 2013 11:20:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EEVRZyOqoDNI for ; Thu, 25 Jul 2013 11:20:15 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 1A76A21F8427 for ; Thu, 25 Jul 2013 11:20:14 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id AD0B220DA5; Thu, 25 Jul 2013 20:20:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id tf5Yd5AeO4li; Thu, 25 Jul 2013 20:20:13 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 6799820D93; Thu, 25 Jul 2013 20:20:12 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id C8E0C2779E1F; Thu, 25 Jul 2013 20:20:07 +0200 (CEST) Date: Thu, 25 Jul 2013 20:20:07 +0200 From: Juergen Schoenwaelder To: Paul Aitken Message-ID: <20130725182007.GA42773@elstar.local> Mail-Followup-To: Paul Aitken , lmap@ietf.org References: <51ED59B3.3040701@cisco.com> <9904FB1B0159DA42B0B887B7FA8119CA1287FC5D@AZ-FFEXMB04.global.avaya.com> <51EFEC2A.9010701@cisco.com> <51F0297A.7040407@it.uc3m.es> <51F0367F.1060905@cisco.com> <20130724204924.GA40227@elstar.local> <51F041FD.4050408@cisco.com> <20130725091606.GB41645@elstar.local> <51F12ACC.1040702@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <51F12ACC.1040702@cisco.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: lmap@ietf.org Subject: Re: [lmap] Feedback on draft-eardley-lmap-terminology X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 18:20:20 -0000 On Thu, Jul 25, 2013 at 02:40:28PM +0100, Paul Aitken wrote: > >A potential > >problem with this is that there are likely instructions and tests that > >do impact each other through side effects. > > > > If C1 schedules a test to be started every 5 min past the hour that > > requires no cross traffic while C2 schedules a test to measure > > video streaming capabilities during the interval 3-7 minutes past > > the hour, then C1 will be surprised that the scheduled test never > > executes or produces wrong results. > > Forget about multiple controllers: this situation could arise from a > single controller. > > So a rejection mechanism will surely be necessary regardless of > whether there are one or multiple controllers: > - whether the command protocol is two-way, so the command is > immediately rejected. > - or, the report protocol includes a way of reporting that the > requested test could not be done. > - or both. [...] > If an MA may be commanded to perform multiple tests¹ by a single > controller, then the potential conflicts are the same as multiple > tests from multiple controllers. > > ¹ This is necessary if we are to build complex tests out of simpler > test primitives, eg DNS lookup followed by file download. Yes, but in this case, it is a single active controller making a mistake and thus this is likely easier to debug and deal with compared to multiple controllers 'fighting' over the resources of a MA. If we design for multiple active controllers, we need to build the necessary coordination primitives right into the MA for version 1.0, something I would prefer to avoid since I have the feeling that this adds complexity and will likely delay things. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From paitken@cisco.com Thu Jul 25 13:49:34 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2630821F9339 for ; Thu, 25 Jul 2013 13:49:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sCuY4PBO-M8w for ; Thu, 25 Jul 2013 13:49:29 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id 686F521F8FDC for ; Thu, 25 Jul 2013 13:49:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3337; q=dns/txt; s=iport; t=1374785366; x=1375994966; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=4wOwQdQV6c7UizNiM8N/G01FnGM0F/CoSBrgztekeVU=; b=R1f/+Ob3fUe5lecoJF3hjknA+bBOOOrLjD5nM5D3zdqxdBD3ivaaAMJf Krpwuxw9qPmGi33RwQO1FHAKR0c8qkbcDWR1N1uzJrjAj7akEj7mcMwpx Cu1Gs/r4cIqkcZp5uoW8cKWVMFqHMOTmc7q7dyGIMMY5Ie6VA3Jr4mJfN 4=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ah4FAO6N8VGQ/khM/2dsb2JhbABagwY1Ab4wgRcWdIIkAQEBAwE4QAEFCwshFg8JAwIBAgFFBg0BBwEBiAYGuTKPfQeEAAOXX4EphHqLKoMV X-IronPort-AV: E=Sophos;i="4.89,729,1367971200"; d="scan'208";a="16001038" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-3.cisco.com with ESMTP; 25 Jul 2013 20:49:25 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r6PKnMPd002042 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Jul 2013 20:49:22 GMT Received: from [10.61.92.114] (ams3-vpn-dhcp7283.cisco.com [10.61.92.114]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6PKnFgL028983; Thu, 25 Jul 2013 21:49:18 +0100 (BST) Message-ID: <51F18F4B.8050203@cisco.com> Date: Thu, 25 Jul 2013 21:49:15 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: "MORTON JR., ALFRED C (AL)" References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "dromasca@avaya.com" , "philip.eardley@bt.com" , "lmap@ietf.org" Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 20:49:34 -0000 Al, > still reading this draft, but as the messages fly by, you said: >> What we wrote in draft-aakhter-lmap framework: as an optimisation, the >> controller treats the collector as an additional MA, thereby informing >> the collector once of the method and params to be used, presumably using >> some ID. Then the MA's need only report that same ID, so the collector >> overhead is much reduced. > how does this optimization match the flexibility to use different measurement > methods on the fly, as Phil highlighted: > << One proposal for making use of both active and passive measurement is > to allow the Measurement Agent to make local decisions on which > technique to use to deliver a particular metric-- as long as the > specific method is included in the report.>> If it's necessary for the MA to report the measurement techniques used, then these should be reported in the test results. It's like another test primitive: select an appropriate technique, and include it in the test report. > I think we agree that passive and active methods are sufficiently different > that they require categorization, possibly as far as summarizing > results from each method separately. MA local decisions certainly > require indicating the measurement method with each value, > hopefully through some shorthand like the proposed registry. > > It seems we can have either the optimizations with static test Instructions, > or MA's making autonomous decisions. So, the proposal is that each MA reports: { which tests were run, which test parameters were used, what the test results were }. The selected technique could be reported as a test parameter, or as a test result (ie, as if the technique selection was a test in its own right). If the selected technique is reported as a test parameter, then there may be many variations in the { tests, parameters, results } tuple which reduce optimisation opportunities since the only part which is certainly constant is the { tests }. Whereas, if the technique is reported as a test result in its own right then the { tests, parameters } part of the tuple is a constant across all the MAs, which provides a great optimisation opportunity: each MA does not have to report this information over and over again, and the collector does not have to store N unique instances of this tuple, just one. So I propose that the test parameters reports what the MA was commanded to do by the controller, rather than MA-specific details. eg, what I'm proposing looks like: Controller to MA: id = 1239 test = perform a DNS lookup param = lookup "test.net" param = use whatever DNS server you think is best; include it in the report. param = report the lookup time MA to collector: result = 1239: 5ms from dns.near.me Since the (test+params) is constant, each MA need only report the ID and results. The collector understands the ID-to-(tests+params) mapping because a copy of the original controller message is also sent to the collector as well as to each MA in the test. Therefore the amount of duplication is easily reduced, while still allowing each MA flexibility to choose techniques and report local settings. P. From paitken@cisco.com Thu Jul 25 14:24:38 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8138B21F8468 for ; Thu, 25 Jul 2013 14:24:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QrcG+KKN7IsP for ; Thu, 25 Jul 2013 14:24:33 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id 64FB121F84B6 for ; Thu, 25 Jul 2013 14:24:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=5013; q=dns/txt; s=iport; t=1374787472; x=1375997072; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=zjfS9kZD3ZDhkNPdyPt5wOFNmdJ1BGZ97tD1nzY/BNs=; b=RoWmVREiHEDizU6z1P0TfUop9QghkHseuh1+7ho1Ggom7Cd/RDZbw1Nc GLq2ci0PLwnsGfHrHwHE7Tw29ocRYEPBFp8YPhA7Q5cThQt+sbkmnMmrx zQ97toCbyyFuVQ6WIyL5zcK3ZMrHBpLOT4MBQIe6XsXslkRm8PYSTAOmD 0=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ah4FAF+W8VGQ/khM/2dsb2JhbABagwY2uzOCfYEXFnSCJAEBAQMBODoGAQULCxIGCQwKDwkDAgECATcOBg0BBwEBF4dvBrkWj30HCoN2A5QIg1eGI4sqgxU X-IronPort-AV: E=Sophos;i="4.89,729,1367971200"; d="scan'208";a="16001957" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-3.cisco.com with ESMTP; 25 Jul 2013 21:24:31 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r6PLOT8A012285 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Jul 2013 21:24:29 GMT Received: from [10.61.92.114] (ams3-vpn-dhcp7283.cisco.com [10.61.92.114]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6PLOEqo000456; Thu, 25 Jul 2013 22:24:15 +0100 (BST) Message-ID: <51F1977E.1040108@cisco.com> Date: Thu, 25 Jul 2013 22:24:14 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: "Romascanu, Dan (Dan)" References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "lmap@ietf.org" Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 21:24:38 -0000 Dan, Thanks for your feedback. Please find some replies inline: On 24/07/13 12:28, Romascanu, Dan (Dan) wrote: > Please find below a set of comments after my first reading of draft-akhter-lmap-framework-00. These are made from the perspective of contributor, and take into account the fact that this is a 00 individual submission, so idnits was not run, spelling and references were not checked, etc. > > 1. The name of the I-D speaks about 'Inventory' which is a little bit confusing or mis-reading. Actually the authors rather seem to have described some applications and protocols that fit into the LMAP architecture, so it's rather an incomplete 'taxonomy'. See more when talking about the (lack of) requirements. We started by making an inventory of the existing/missing IETF blocks, which evolved towards a framework. > 2. The previous contributions on LMAP framework included in draft-eardley-lmap-framework were completely ignored, some material is duplicated (in content, not in text), but the eardley I-D is not even included in the list of references. As the two I-Ds (and other relevant contributions) will need to be consolidated into one single WG I-D pretty soon I will refer in the following to some of the similarities and some of the differences. Sorry for not including draft-eardley-lmap-framework. We were debating whether to include it or not; we're happy to add it, though we'll be working with the other authors to merge the drafts anyway. > 3. The order of Sections 1.2 and 1.3 should be reversed. OK. > 4. Figure 1 is almost identical to Figure 1 in draft-eardley with two differences. The first is that the entity named Measurement Peer in draft-eardley is called here Remote Measurement Test Target and seems to have a much richer functionality and variants as illustrated in Sections 3.2 and Figure 2. The option of one measurement agent communicating with multiple Remote Measurement Agents needs to bear a warning concerning scalability. Yes, we agree about scalability. We'll add some use cases. > 5. The second significant difference in the two diagrams is the communication between the Controller and the Collector. Later in the draft this is described as optional and motivated by the need to optimize for Collector Test Configuration synchronization (so that each MA needs not report its configuration to controllers). This addition to the basic architecture needs to be carefully discussed by the WG. Controller to Collector communication is optional; we see this as an optimisation for large-scale tests. it could be up to the controller based on the number of MAs involved. If the majority of the tests are single MAs, then this optimization works against. However, if there are more than 2 or 3 MAs, the optimisation works. We can discuss. > 6. One crisp limitation that is being introduced by draft-eardley is about each MA being configured only by one Controller (motivated among other by the need of simplifying authentication of Controllers with the MA). This limitation is not mentioned in this I-D, at least not explicitly. Is this an editorial miss, or the authors take a different stand on this issue? Multiple controllers are generally only for the case of redundancy failover; usually an MA has a relationship with only one controller at any time. However, since all controllers are part of the same administrative domain which is centrally controlled, each can be understood as a different controller instance. So an MA with multiple controllers is simply communicating with multiple controller instances - so allowing multiple controllers should not be problematic, since they should not issue contradictory instructions. Multiple distinct controllers under different administrative control aren't within charter. > 7. Sections 3,4,5 are to some extent written with one set of protocols already in mind to answer LMAP architectural requirements (OWAMP and TWAMP for active monitoring, PSAMP for passive monitoring, IPFIX for reporting protocol, IPFIX mediators to solve the scalability issues). Although this may be a reasonable set of solutions, it seems a little bit too early and too detailed at this phase. Missing here are the basic requirements for the Control protocol, Report protocol, and associated data models. I would prefer to see this being written first. We started by talking about firewalls, NAT etc. We can add the basic requirements; there's already an analysis of OWAMP and TWAMP. We'll separate this out so there's clear separation between the requirements of those protocols and the analysis of the individual protocols. > 8. The Security Consideration section needs to include not only information about authentication, but also about the risks of modification of information (data integrity). We'll note that it's insecure in this particular way. Note that the charter says that gaming is outside the scope. Aamer + Paul. From rachel.huang@huawei.com Thu Jul 25 20:17:58 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8705F21F8456 for ; Thu, 25 Jul 2013 20:17:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.348 X-Spam-Level: X-Spam-Status: No, score=-6.348 tagged_above=-999 required=5 tests=[AWL=-0.249, BAYES_00=-2.599, GB_ABOUTYOU=0.5, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R4uhNtnILAJp for ; Thu, 25 Jul 2013 20:17:54 -0700 (PDT) Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) by ietfa.amsl.com (Postfix) with ESMTP id 9932521F844E for ; Thu, 25 Jul 2013 20:17:52 -0700 (PDT) Received: from 172.18.7.190 (EHLO lhreml204-edg.china.huawei.com) ([172.18.7.190]) by lhrrg02-dlp.huawei.com (MOS 4.3.5-GA FastPath queued) with ESMTP id ATU47378; Fri, 26 Jul 2013 03:17:49 +0000 (GMT) Received: from LHREML403-HUB.china.huawei.com (10.201.5.217) by lhreml204-edg.china.huawei.com (172.18.7.223) with Microsoft SMTP Server (TLS) id 14.1.323.7; Fri, 26 Jul 2013 04:16:46 +0100 Received: from NKGEML404-HUB.china.huawei.com (10.98.56.35) by lhreml403-hub.china.huawei.com (10.201.5.217) with Microsoft SMTP Server (TLS) id 14.1.323.7; Fri, 26 Jul 2013 04:17:46 +0100 Received: from NKGEML501-MBS.china.huawei.com ([169.254.2.43]) by nkgeml404-hub.china.huawei.com ([10.98.56.35]) with mapi id 14.01.0323.007; Fri, 26 Jul 2013 11:17:40 +0800 From: "Huangyihong (Rachel)" To: "philip.eardley@bt.com" , "dromasca@avaya.com" , "lmap@ietf.org" Thread-Topic: on use cases and requirements Thread-Index: Ac6JNA/7jjUgkfhDSx2RandUUF8HhwAAwTCwABqq9IA= Date: Fri, 26 Jul 2013 03:17:40 +0000 Message-ID: <51E6A56BD6A85142B9D172C87FC3ABBB458595EA@nkgeml501-mbs.china.huawei.com> References: <9904FB1B0159DA42B0B887B7FA8119CA12881146@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337A5@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F35B80337A5@EMV65-UKRD.domain1.systemhost.net> Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.138.41.104] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Subject: Re: [lmap] on use cases and requirements X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 03:17:58 -0000 Hi Philip, Yes, I think it could be regarded as one usage of ISP use case.=20 It will be great to work with you and ken on this. I will attend this meeti= ng. Let's meet in the LMAP breakfast. Best regards, Rachel -----Original Message----- From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of phi= lip.eardley@bt.com Sent: Thursday, July 25, 2013 9:17 PM To: dromasca@avaya.com; lmap@ietf.org Subject: Re: [lmap] on use cases and requirements I had a look at the new drafts, couple of questions Ken, http://tools.ietf.org/html/draft-nagami-lmap-use-case-measurement-provider-= 00 you talk about a "measurement provider" - if I get it right, this is someon= e who provides information about different ISPs. this seems to me very simi= lar to the regulator use case - but it's a public interest group, or a comp= any providing the information instead of a regulator. It's probably worth p= ointing this out in http://tools.ietf.org/html/draft-linsner-lmap-use-cases= - would this then cover your use case? You also mention a smartphone, I think it's worth adding this explicitly to= draft-linsner You also have a lot of interesting details about your deployment. My feelin= g is that it would take a lot of work to summarise all the state-of-the-art= , so we shouldn't try (perhaps we could add a few references in one of the = docs) Rachel, http://tools.ietf.org/id/draft-huang-lmap-data-collection-use-case-00.txt I take this as a slight expansion of the ISP use case described in http://t= ools.ietf.org/html/draft-linsner-lmap-use-cases=20 Could we work with you on adding a little text about simulations to our dra= ft? - ie the ISP could use measurements to feed its simulation tools (assum= ing it uses simulations as part of the way it works out where a fault is, o= r where to add capacity, etc).=20 Will you be in Berlin? Marc, Ken and I are meeting in the lmap breakfast to= discuss use cases.=20 Dan, Personally I'm happy to drop as much of Section 3 as people want (even all = of it). I agree we should think carefully in terms of what the output of lmap needs= to do - what are the most important aspects of which use cases. Reading between the lines, you think the use cases doc should only cover th= ose aspects that we're going to (try to) make sure we solve during the init= ial lmap charter. And not be a general use case doc that would need future = lmap capability. Is that right?=20 (I'm ok if the answer is 'yes') Thanks phil > -----Original Message----- > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of > Romascanu, Dan (Dan) > Sent: 25 July 2013 13:40 > To: lmap@ietf.org > Subject: [lmap] on use cases and requirements >=20 > If you have a look at the updated agenda at > https://datatracker.ietf.org/meeting/87/agenda/lmap/ you will realized > that Jason and me slightly updated the section dedicated to use cases > and added ten minutes of discussion. Let me try to explain the > motivation and what we would try to achieve. According to the charter > in a couple of months the WG will need to submit the first WG Internet- > Drafts on Use Cases and Framework. These two documents should provide > clear enough requirements for the following phases where we need to > proceed to the creation of the Information Model, and discuss about the > selection or development of protocols for Control and Reporting and > their associated data models. At this moment in time we have one draft- > linsner-lmap-use-cases-03 as principal document on use cases, and two > new contributions, which is good for this phase. What we need to start > doing is to sort out of the use cases the subset that matches the > current WG charter - and we may need to drop or prioritize some of the > aspects of the use cases for this purpose. >=20 > (To be more specific and take this just as an example, I am not sure > that all the characteristics described in Sections 3.2, 3.3, 3.5 in > draft-linsner-lmap-use-cases-03 are LMAP problems or problems that > belong to the requirements of the first phase of LMAP.) >=20 > Regards, >=20 > Dan >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap From philip.eardley@bt.com Fri Jul 26 02:20:26 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D6F7721F8613 for ; Fri, 26 Jul 2013 02:20:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.154 X-Spam-Level: X-Spam-Status: No, score=-103.154 tagged_above=-999 required=5 tests=[AWL=-0.155, BAYES_00=-2.599, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q2HOAgleStnz for ; Fri, 26 Jul 2013 02:20:22 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp62.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id E68D121F852D for ; Fri, 26 Jul 2013 02:20:21 -0700 (PDT) Received: from EVMHT66-UKRD.domain1.systemhost.net (10.36.3.103) by RDW083A006ED62.smtp-e2.hygiene.service (10.187.98.11) with Microsoft SMTP Server (TLS) id 8.3.298.1; Fri, 26 Jul 2013 10:20:12 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.2.130]) by EVMHT66-UKRD.domain1.systemhost.net ([10.36.3.103]) with mapi; Fri, 26 Jul 2013 10:20:12 +0100 From: To: , Date: Fri, 26 Jul 2013 10:20:11 +0100 Thread-Topic: [lmap] my comments on draft-akhter-lmap-framework-00 Thread-Index: Ac6JeHOc4Rz8BnzWRZSQJn2MpooKdwAaBj2g Message-ID: <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> In-Reply-To: <51F18F4B.8050203@cisco.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: dromasca@avaya.com, lmap@ietf.org Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 09:20:27 -0000 There'd be some complexities to handle, for instance when the test details = change to get the results synchronised with the information you're sending = from the controller.=20 Anyway, I like the idea of keeping the reporting stateless.=20 I'm not convinced the gain in de-duplication is worthwhile (is the overhead= really that big?), it would be good to have some numbers here on typical b= ytes of a report. > -----Original Message----- > From: Paul Aitken [mailto:paitken@cisco.com] > Sent: 25 July 2013 21:49 > To: MORTON JR., ALFRED C (AL) > Cc: Eardley,PL,Philip,TUB8 R; dromasca@avaya.com; lmap@ietf.org > Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 >=20 > Al, >=20 > > still reading this draft, but as the messages fly by, you said: > >> What we wrote in draft-aakhter-lmap framework: as an optimisation, > >> the controller treats the collector as an additional MA, thereby > >> informing the collector once of the method and params to be used, > >> presumably using some ID. Then the MA's need only report that same > >> ID, so the collector overhead is much reduced. > > how does this optimization match the flexibility to use different > > measurement methods on the fly, as Phil highlighted: > > << One proposal for making use of both active and passive measurement > is > > to allow the Measurement Agent to make local decisions on which > > technique to use to deliver a particular metric-- as long as the > > specific method is included in the report.>> >=20 > If it's necessary for the MA to report the measurement techniques used, > then these should be reported in the test results. >=20 > It's like another test primitive: select an appropriate technique, and > include it in the test report. >=20 >=20 > > I think we agree that passive and active methods are sufficiently > different > > that they require categorization, possibly as far as summarizing > > results from each method separately. MA local decisions certainly > > require indicating the measurement method with each value, > > hopefully through some shorthand like the proposed registry. > > > > It seems we can have either the optimizations with static test > Instructions, > > or MA's making autonomous decisions. >=20 > So, the proposal is that each MA reports: { which tests were run, which > test parameters were used, what the test results were }. >=20 > The selected technique could be reported as a test parameter, or as a > test result (ie, as if the technique selection was a test in its own > right). >=20 > If the selected technique is reported as a test parameter, then there > may be many variations in the { tests, parameters, results } tuple > which > reduce optimisation opportunities since the only part which is > certainly > constant is the { tests }. >=20 > Whereas, if the technique is reported as a test result in its own right > then the { tests, parameters } part of the tuple is a constant across > all the MAs, which provides a great optimisation opportunity: each MA > does not have to report this information over and over again, and the > collector does not have to store N unique instances of this tuple, just > one. >=20 > So I propose that the test parameters reports what the MA was commanded > to do by the controller, rather than MA-specific details. >=20 > eg, what I'm proposing looks like: >=20 > Controller to MA: >=20 > id =3D 1239 > test =3D perform a DNS lookup > param =3D lookup "test.net" > param =3D use whatever DNS server you think is best; include it > in the report. > param =3D report the lookup time >=20 > MA to collector: >=20 > result =3D 1239: 5ms from dns.near.me >=20 >=20 > Since the (test+params) is constant, each MA need only report the ID > and > results. >=20 > The collector understands the ID-to-(tests+params) mapping because a > copy of the original controller message is also sent to the collector > as > well as to each MA in the test. >=20 > Therefore the amount of duplication is easily reduced, while still > allowing each MA flexibility to choose techniques and report local > settings. >=20 > P. From sam@samknows.com Fri Jul 26 02:47:38 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E7DC721F8EEA for ; Fri, 26 Jul 2013 02:47:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.221 X-Spam-Level: X-Spam-Status: No, score=-0.221 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, FRT_LITTLE=1.555, HTML_MESSAGE=0.001, J_CHICKENPOX_21=0.6, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N-9+A5A00m9g for ; Fri, 26 Jul 2013 02:47:34 -0700 (PDT) Received: from mail-oa0-f51.google.com (mail-oa0-f51.google.com [209.85.219.51]) by ietfa.amsl.com (Postfix) with ESMTP id E8C0E21F8CB4 for ; Fri, 26 Jul 2013 02:47:33 -0700 (PDT) Received: by mail-oa0-f51.google.com with SMTP id i4so6906549oah.38 for ; Fri, 26 Jul 2013 02:47:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=ubMwpdMedP0uDZa72S71VZNeof3KO/Tzs/5mRpuXUXo=; b=TJmF2QdltVQsCZhoj90rR4ObRrKtxS4b7ZRYgnwbS0y7hLLNMeiDK9L7JNQgK4999A pcaesusnc6f7RswQpJbY1Y0VXY1epQooAhik2AXisUQjRBGUzBWLAGarmVwjn1Q6RSsm bKbPYBUiilOfaLZFeYRugzJU9AOCT8r63PIwDur42mxxjrZHw971qJeFhrWwcucUWb+r au4XcEYu1t9YNGWC+YSPHS9vZGFoj4YS6JX2pjB+KzkMYVwgrcAuNLSjM70UqazrgTOR E97x/PYoB2qvpqaMSCB70ug85pSIvDv+WnFk0s0gYqN/msHsFgPl1MeAfAVcKNJfeBcM 7j3A== MIME-Version: 1.0 X-Received: by 10.60.56.166 with SMTP id b6mr8980745oeq.38.1374832053420; Fri, 26 Jul 2013 02:47:33 -0700 (PDT) Received: by 10.76.169.72 with HTTP; Fri, 26 Jul 2013 02:47:33 -0700 (PDT) In-Reply-To: <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> Date: Fri, 26 Jul 2013 10:47:33 +0100 Message-ID: From: Sam Crawford To: "EARDLEY, Phil" Content-Type: multipart/alternative; boundary=089e0149d17a2b939a04e26708f4 X-Gm-Message-State: ALoCoQmIWRPDuutp6p+3cm+E9h9Rb3UgZLGxtMQJq+jiFFE3Uiugu6HrlJEkLZ37Wz1ao9PGtXh0 Cc: dromasca@avaya.com, lmap@ietf.org, paitken@cisco.com, acmorton Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 09:47:38 -0000 --089e0149d17a2b939a04e26708f4 Content-Type: text/plain; charset=ISO-8859-1 I also very much like the idea of keeping reporting stateless. It removes the need for the client to know about the internal state of the collector when reporting data. It's how we do it at the moment, and its simplicity has been very beneficial a few times. Regarding overhead: To give you a very rough idea, each run of one of our metrics on our probes generates about 70-100 bytes of report data. Around 30-50% of that details the input parameters. These numbers vary by metric of course, and are without any compression or de-duplication. It's a small amount and the added overhead has never caused us concern. Thanks, Sam On 26 July 2013 10:20, wrote: > There'd be some complexities to handle, for instance when the test details > change to get the results synchronised with the information you're sending > from the controller. > Anyway, I like the idea of keeping the reporting stateless. > > I'm not convinced the gain in de-duplication is worthwhile (is the > overhead really that big?), it would be good to have some numbers here on > typical bytes of a report. > > > > > -----Original Message----- > > From: Paul Aitken [mailto:paitken@cisco.com] > > Sent: 25 July 2013 21:49 > > To: MORTON JR., ALFRED C (AL) > > Cc: Eardley,PL,Philip,TUB8 R; dromasca@avaya.com; lmap@ietf.org > > Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 > > > > Al, > > > > > still reading this draft, but as the messages fly by, you said: > > >> What we wrote in draft-aakhter-lmap framework: as an optimisation, > > >> the controller treats the collector as an additional MA, thereby > > >> informing the collector once of the method and params to be used, > > >> presumably using some ID. Then the MA's need only report that same > > >> ID, so the collector overhead is much reduced. > > > how does this optimization match the flexibility to use different > > > measurement methods on the fly, as Phil highlighted: > > > << One proposal for making use of both active and passive measurement > > is > > > to allow the Measurement Agent to make local decisions on which > > > technique to use to deliver a particular metric-- as long as the > > > specific method is included in the report.>> > > > > If it's necessary for the MA to report the measurement techniques used, > > then these should be reported in the test results. > > > > It's like another test primitive: select an appropriate technique, and > > include it in the test report. > > > > > > > I think we agree that passive and active methods are sufficiently > > different > > > that they require categorization, possibly as far as summarizing > > > results from each method separately. MA local decisions certainly > > > require indicating the measurement method with each value, > > > hopefully through some shorthand like the proposed registry. > > > > > > It seems we can have either the optimizations with static test > > Instructions, > > > or MA's making autonomous decisions. > > > > So, the proposal is that each MA reports: { which tests were run, which > > test parameters were used, what the test results were }. > > > > The selected technique could be reported as a test parameter, or as a > > test result (ie, as if the technique selection was a test in its own > > right). > > > > If the selected technique is reported as a test parameter, then there > > may be many variations in the { tests, parameters, results } tuple > > which > > reduce optimisation opportunities since the only part which is > > certainly > > constant is the { tests }. > > > > Whereas, if the technique is reported as a test result in its own right > > then the { tests, parameters } part of the tuple is a constant across > > all the MAs, which provides a great optimisation opportunity: each MA > > does not have to report this information over and over again, and the > > collector does not have to store N unique instances of this tuple, just > > one. > > > > So I propose that the test parameters reports what the MA was commanded > > to do by the controller, rather than MA-specific details. > > > > eg, what I'm proposing looks like: > > > > Controller to MA: > > > > id = 1239 > > test = perform a DNS lookup > > param = lookup "test.net" > > param = use whatever DNS server you think is best; include it > > in the report. > > param = report the lookup time > > > > MA to collector: > > > > result = 1239: 5ms from dns.near.me > > > > > > Since the (test+params) is constant, each MA need only report the ID > > and > > results. > > > > The collector understands the ID-to-(tests+params) mapping because a > > copy of the original controller message is also sent to the collector > > as > > well as to each MA in the test. > > > > Therefore the amount of duplication is easily reduced, while still > > allowing each MA flexibility to choose techniques and report local > > settings. > > > > P. > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > -- ---- Sam Crawford SamKnows Limited Switchboard: +44 (0) 20 3111 4330 DDI: +44 (0) 20 3111 4332 This email is sent for and on behalf of SamKnows Limited. This email and any attachments are confidential, legally privileged and protected by copyright. If you are not the intended recipient dissemination or copying of this email is prohibited. If you have received this in error, please notify the sender by replying by email and then delete the email completely from your system. SamKnows Limited, Registered Number: 06510477, Registered Office: Hill House, 1 Little New Street, London, EC4A 3TR. Registered in England and Wales. Trade Mark 2507103 --089e0149d17a2b939a04e26708f4 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I also very much like the idea of keeping reporting s= tateless. It removes the need for the client to know about the internal sta= te of the collector when reporting data. It's how we do it at the momen= t, and its simplicity has been very beneficial a few times.

Regarding overhead: To give you a very rough idea, each= run of one of our metrics on our probes generates about 70-100 bytes of re= port data. Around 30-50% of that details the input parameters. These number= s vary by metric of course, and are without any compression or de-duplicati= on. It's a small amount and the added overhead has never caused us conc= ern.

Thanks,

Sam



On 26= July 2013 10:20, <philip.eardley@bt.com> wrote:
There'd be some complexities to handle, = for instance when the test details change to get the results synchronised w= ith the information you're sending from the controller.
Anyway, I like the idea of keeping the reporting stateless.

I'm not convinced the gain in de-duplication is worthwhile (is the over= head really that big?), it would be good to have some numbers here on typic= al bytes of a report.



> -----Original Message-----
> From: Paul Aitken [mailto:paitken= @cisco.com]
> Sent: 25 July 2013 21:49
> To: MORTON JR., ALFRED C (AL)
> Cc: Eardley,PL,Philip,TUB8 R; dr= omasca@avaya.com; lmap@ietf.org > Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00
>
> Al,
>
> > still reading this draft, but as the messages fly by, you said: > >> What we wrote in draft-aakhter-lmap framework: as an optimisa= tion,
> >> the controller treats the collector as an additional MA, ther= eby
> >> informing the collector once of the method and params to be u= sed,
> >> presumably using some ID. Then the MA's need only report = that same
> >> ID, so the collector overhead is much reduced.
> > how does this optimization match the flexibility to use different=
> > measurement methods on the fly, as Phil highlighted:
> > << One proposal for making use of both active and passive m= easurement
> is
> > =A0 =A0 to allow the Measurement Agent to make local decisions on= which
> > =A0 =A0 technique to use to deliver a particular metric-- as long= as the
> > =A0 =A0 specific method is included in the report.>>
>
> If it's necessary for the MA to report the measurement techniques = used,
> then these should be reported in the test results.
>
> It's like another test primitive: select an appropriate technique,= and
> include it in the test report.
>
>
> > I think we agree that passive and active methods are sufficiently=
> different
> > that they require categorization, possibly as far as summarizing<= br> > > results from each method separately. MA local decisions certainly=
> > require indicating the measurement method with each value,
> > hopefully through some shorthand like the proposed registry.
> >
> > It seems we can have either the optimizations with static test > Instructions,
> > or MA's making autonomous decisions.
>
> So, the proposal is that each MA reports: { which tests were run, whic= h
> test parameters were used, what the test results were }.
>
> The selected technique could be reported as a test parameter, or as a<= br> > test result (ie, as if the technique selection was a test in its own > right).
>
> If the selected technique is reported as a test parameter, then there<= br> > may be many variations in the { tests, parameters, results } tuple
> which
> reduce optimisation opportunities since the only part which is
> certainly
> constant is the { tests }.
>
> Whereas, if the technique is reported as a test result in its own righ= t
> then the { tests, parameters } part of the tuple is a constant across<= br> > all the MAs, which provides a great optimisation opportunity: each MA<= br> > does not have to report this information over and over again, and the<= br> > collector does not have to store N unique instances of this tuple, jus= t
> one.
>
> So I propose that the test parameters reports what the MA was commande= d
> to do by the controller, rather than MA-specific details.
>
> eg, what I'm proposing looks like:
>
> =A0 =A0 =A0Controller to MA:
>
> =A0 =A0 =A0 =A0 =A0id =3D 1239
> =A0 =A0 =A0 =A0 =A0test =3D perform a DNS lookup
> =A0 =A0 =A0 =A0 =A0param =3D lookup "test.net"
> =A0 =A0 =A0 =A0 =A0param =3D use whatever DNS server you think is best= ; include it
> in the report.
> =A0 =A0 =A0 =A0 =A0param =3D report the lookup time
>
> =A0 =A0 =A0MA to collector:
>
> =A0 =A0 =A0 =A0 =A0result =3D 1239: 5ms from dns.near.me
>
>
> Since the (test+params) is constant, each MA need only report the ID > and
> results.
>
> The collector understands the ID-to-(tests+params) mapping because a > copy of the original controller message is also sent to the collector<= br> > as
> well as to each MA in the test.
>
> Therefore the amount of duplication is easily reduced, while still
> allowing each MA flexibility to choose techniques and report local
> settings.
>
> P.

_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap



--
=
----
Sam Crawford
SamKnows Limited

Switchboar= d: +44 (0) 20 3111 4330
DDI: +44 (0) 20 3111 4332

This email is = sent for and on behalf of SamKnows Limited.

This email and any attachments are confidential, legally privileged and= protected by copyright. If you are not the intended recipient disseminatio= n or copying of this email is prohibited. If you have received this in erro= r, please notify the sender by replying by email and then delete the email = completely from your system.

SamKnows Limited, Registered Number: 06510477, Registered Office:=A0Hil= l House, 1 Little New Street, London, EC4A 3TR.=A0 Registered in England an= d Wales. Trade Mark 2507103
--089e0149d17a2b939a04e26708f4-- From trammell@tik.ee.ethz.ch Fri Jul 26 04:51:44 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E3D9021F9377 for ; Fri, 26 Jul 2013 04:51:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.844 X-Spam-Level: X-Spam-Status: No, score=-3.844 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FRT_LITTLE=1.555, J_CHICKENPOX_21=0.6, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gK4+TjDtboAR for ; Fri, 26 Jul 2013 04:51:37 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 649E721F9339 for ; Fri, 26 Jul 2013 04:51:35 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 8A654D9314; Fri, 26 Jul 2013 13:51:29 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id x5rJUOe9pY+M; Fri, 26 Jul 2013 13:51:29 +0200 (MEST) Received: from [172.27.10.188] (unknown [195.37.142.72]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 11C55D9309; Fri, 26 Jul 2013 13:51:29 +0200 (MEST) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) From: Brian Trammell In-Reply-To: Date: Fri, 26 Jul 2013 13:51:30 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> To: Sam Crawford X-Mailer: Apple Mail (2.1508) Cc: dromasca@avaya.com, "EARDLEY, Phil" , acmorton , paitken@cisco.com, lmap@ietf.org Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 11:51:44 -0000 Hi, all, Trying to catch up with these threads before Tuesday morning; but wanted = to chime in here: +1 to stateless reporting; I'd go so far as to say that control should = be "as stateless as possible" -- it should be possible to determine = whether a task completed at an MA or not, but this doesn't necessarily = need to be handled as an error condition at the control level, = especially in the case that the MA is on a device where the customer = controls the power and/or whether the app is running: it's simply = something that the collector needs to know about to support analysis of = the results. MAs may "lose sync" for any number of reasons, so = collectors and controllers will also have to be robust against MAs that = are running out-of-date instructions; this is also not necessarily an = error condition. More later, best regards, Brian On 26 Jul 2013, at 11:47, Sam Crawford wrote: > I also very much like the idea of keeping reporting stateless. It = removes the need for the client to know about the internal state of the = collector when reporting data. It's how we do it at the moment, and its = simplicity has been very beneficial a few times. >=20 > Regarding overhead: To give you a very rough idea, each run of one of = our metrics on our probes generates about 70-100 bytes of report data. = Around 30-50% of that details the input parameters. These numbers vary = by metric of course, and are without any compression or de-duplication. = It's a small amount and the added overhead has never caused us concern. >=20 > Thanks, >=20 > Sam >=20 >=20 >=20 > On 26 July 2013 10:20, wrote: > There'd be some complexities to handle, for instance when the test = details change to get the results synchronised with the information = you're sending from the controller. > Anyway, I like the idea of keeping the reporting stateless. >=20 > I'm not convinced the gain in de-duplication is worthwhile (is the = overhead really that big?), it would be good to have some numbers here = on typical bytes of a report. >=20 >=20 >=20 > > -----Original Message----- > > From: Paul Aitken [mailto:paitken@cisco.com] > > Sent: 25 July 2013 21:49 > > To: MORTON JR., ALFRED C (AL) > > Cc: Eardley,PL,Philip,TUB8 R; dromasca@avaya.com; lmap@ietf.org > > Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 > > > > Al, > > > > > still reading this draft, but as the messages fly by, you said: > > >> What we wrote in draft-aakhter-lmap framework: as an = optimisation, > > >> the controller treats the collector as an additional MA, thereby > > >> informing the collector once of the method and params to be used, > > >> presumably using some ID. Then the MA's need only report that = same > > >> ID, so the collector overhead is much reduced. > > > how does this optimization match the flexibility to use different > > > measurement methods on the fly, as Phil highlighted: > > > << One proposal for making use of both active and passive = measurement > > is > > > to allow the Measurement Agent to make local decisions on = which > > > technique to use to deliver a particular metric-- as long as = the > > > specific method is included in the report.>> > > > > If it's necessary for the MA to report the measurement techniques = used, > > then these should be reported in the test results. > > > > It's like another test primitive: select an appropriate technique, = and > > include it in the test report. > > > > > > > I think we agree that passive and active methods are sufficiently > > different > > > that they require categorization, possibly as far as summarizing > > > results from each method separately. MA local decisions certainly > > > require indicating the measurement method with each value, > > > hopefully through some shorthand like the proposed registry. > > > > > > It seems we can have either the optimizations with static test > > Instructions, > > > or MA's making autonomous decisions. > > > > So, the proposal is that each MA reports: { which tests were run, = which > > test parameters were used, what the test results were }. > > > > The selected technique could be reported as a test parameter, or as = a > > test result (ie, as if the technique selection was a test in its own > > right). > > > > If the selected technique is reported as a test parameter, then = there > > may be many variations in the { tests, parameters, results } tuple > > which > > reduce optimisation opportunities since the only part which is > > certainly > > constant is the { tests }. > > > > Whereas, if the technique is reported as a test result in its own = right > > then the { tests, parameters } part of the tuple is a constant = across > > all the MAs, which provides a great optimisation opportunity: each = MA > > does not have to report this information over and over again, and = the > > collector does not have to store N unique instances of this tuple, = just > > one. > > > > So I propose that the test parameters reports what the MA was = commanded > > to do by the controller, rather than MA-specific details. > > > > eg, what I'm proposing looks like: > > > > Controller to MA: > > > > id =3D 1239 > > test =3D perform a DNS lookup > > param =3D lookup "test.net" > > param =3D use whatever DNS server you think is best; = include it > > in the report. > > param =3D report the lookup time > > > > MA to collector: > > > > result =3D 1239: 5ms from dns.near.me > > > > > > Since the (test+params) is constant, each MA need only report the ID > > and > > results. > > > > The collector understands the ID-to-(tests+params) mapping because a > > copy of the original controller message is also sent to the = collector > > as > > well as to each MA in the test. > > > > Therefore the amount of duplication is easily reduced, while still > > allowing each MA flexibility to choose techniques and report local > > settings. > > > > P. >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap >=20 >=20 >=20 > --=20 > ---- > Sam Crawford > SamKnows Limited >=20 > Switchboard: +44 (0) 20 3111 4330=20 > DDI: +44 (0) 20 3111 4332 >=20 > This email is sent for and on behalf of SamKnows Limited.=20 >=20 > This email and any attachments are confidential, legally privileged = and protected by copyright. If you are not the intended recipient = dissemination or copying of this email is prohibited. If you have = received this in error, please notify the sender by replying by email = and then delete the email completely from your system.=20 >=20 > SamKnows Limited, Registered Number: 06510477, Registered Office: Hill = House, 1 Little New Street, London, EC4A 3TR. Registered in England and = Wales. Trade Mark 2507103 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From acmorton@att.com Fri Jul 26 05:44:09 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ECBBB21F922A for ; Fri, 26 Jul 2013 05:44:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.161 X-Spam-Level: X-Spam-Status: No, score=-105.161 tagged_above=-999 required=5 tests=[AWL=-1.317, BAYES_00=-2.599, FRT_LITTLE=1.555, J_CHICKENPOX_21=0.6, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AdP+Tb3HVasb for ; Fri, 26 Jul 2013 05:44:04 -0700 (PDT) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 11BE521F8EE6 for ; Fri, 26 Jul 2013 05:44:04 -0700 (PDT) Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id 40B5E120683; Fri, 26 Jul 2013 08:43:58 -0400 (EDT) Received: from njfpsrvexg5.research.att.com (njfpsrvexg5.research.att.com [135.207.177.27]) by mail-blue.research.att.com (Postfix) with ESMTP id B4478F037B; Fri, 26 Jul 2013 08:44:00 -0400 (EDT) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg5.research.att.com ([fe80::a501:da3:2345:4587%10]) with mapi; Fri, 26 Jul 2013 08:44:00 -0400 From: "MORTON JR., ALFRED C (AL)" To: Brian Trammell , Sam Crawford Date: Fri, 26 Jul 2013 08:43:59 -0400 Thread-Topic: [lmap] my comments on draft-akhter-lmap-framework-00 Thread-Index: Ac6J9oSH/Fjj5ZGvRzi8ZDSgSLvnIgABSQPQ Message-ID: References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> In-Reply-To: <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dromasca@avaya.com" , "EARDLEY, Phil" , "paitken@cisco.com" , "lmap@ietf.org" Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 12:44:09 -0000 If the collector has state that is essential to interpreting the results, then the state must be appended when moving the results to the analysis pha= se (and the information compression is lost) or the analysis system must also have the same state, if I've understood th= e proposal. If we compress the information using IDs or handles with definitions that a= re widely accessible, then every entity that gets the data can use and expand the results with co= nfidence, even those outside the LMAP system functions. We had some related discussio= n of handles several months back: http://www.ietf.org/mail-archive/web/lmap/current/msg= 00266.html so, leaning toward stateless, Al > -----Original Message----- > From: Brian Trammell [mailto:trammell@tik.ee.ethz.ch] > Sent: Friday, July 26, 2013 7:52 AM > To: Sam Crawford > Cc: EARDLEY, Phil; dromasca@avaya.com; lmap@ietf.org; paitken@cisco.com; > MORTON JR., ALFRED C (AL) > Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 >=20 > Hi, all, >=20 > Trying to catch up with these threads before Tuesday morning; but wanted > to chime in here: >=20 > +1 to stateless reporting; I'd go so far as to say that control should be > "as stateless as possible" -- it should be possible to determine whether = a > task completed at an MA or not, but this doesn't necessarily need to be > handled as an error condition at the control level, especially in the cas= e > that the MA is on a device where the customer controls the power and/or > whether the app is running: it's simply something that the collector need= s > to know about to support analysis of the results. MAs may "lose sync" for > any number of reasons, so collectors and controllers will also have to be > robust against MAs that are running out-of-date instructions; this is als= o > not necessarily an error condition. >=20 > More later, best regards, >=20 > Brian >=20 > On 26 Jul 2013, at 11:47, Sam Crawford wrote: >=20 > > I also very much like the idea of keeping reporting stateless. It > removes the need for the client to know about the internal state of the > collector when reporting data. It's how we do it at the moment, and its > simplicity has been very beneficial a few times. > > > > Regarding overhead: To give you a very rough idea, each run of one of > our metrics on our probes generates about 70-100 bytes of report data. > Around 30-50% of that details the input parameters. These numbers vary by > metric of course, and are without any compression or de-duplication. It's > a small amount and the added overhead has never caused us concern. > > > > Thanks, > > > > Sam > > > > > > > > On 26 July 2013 10:20, wrote: > > There'd be some complexities to handle, for instance when the test > details change to get the results synchronised with the information you'r= e > sending from the controller. > > Anyway, I like the idea of keeping the reporting stateless. > > > > I'm not convinced the gain in de-duplication is worthwhile (is the > overhead really that big?), it would be good to have some numbers here on > typical bytes of a report. > > > > > > > > > -----Original Message----- > > > From: Paul Aitken [mailto:paitken@cisco.com] > > > Sent: 25 July 2013 21:49 > > > To: MORTON JR., ALFRED C (AL) > > > Cc: Eardley,PL,Philip,TUB8 R; dromasca@avaya.com; lmap@ietf.org > > > Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 > > > > > > Al, > > > > > > > still reading this draft, but as the messages fly by, you said: > > > >> What we wrote in draft-aakhter-lmap framework: as an optimisation, > > > >> the controller treats the collector as an additional MA, thereby > > > >> informing the collector once of the method and params to be used, > > > >> presumably using some ID. Then the MA's need only report that same > > > >> ID, so the collector overhead is much reduced. > > > > how does this optimization match the flexibility to use different > > > > measurement methods on the fly, as Phil highlighted: > > > > << One proposal for making use of both active and passive > measurement > > > is > > > > to allow the Measurement Agent to make local decisions on which > > > > technique to use to deliver a particular metric-- as long as th= e > > > > specific method is included in the report.>> > > > > > > If it's necessary for the MA to report the measurement techniques > used, > > > then these should be reported in the test results. > > > > > > It's like another test primitive: select an appropriate technique, an= d > > > include it in the test report. > > > > > > > > > > I think we agree that passive and active methods are sufficiently > > > different > > > > that they require categorization, possibly as far as summarizing > > > > results from each method separately. MA local decisions certainly > > > > require indicating the measurement method with each value, > > > > hopefully through some shorthand like the proposed registry. > > > > > > > > It seems we can have either the optimizations with static test > > > Instructions, > > > > or MA's making autonomous decisions. > > > > > > So, the proposal is that each MA reports: { which tests were run, > which > > > test parameters were used, what the test results were }. > > > > > > The selected technique could be reported as a test parameter, or as a > > > test result (ie, as if the technique selection was a test in its own > > > right). > > > > > > If the selected technique is reported as a test parameter, then there > > > may be many variations in the { tests, parameters, results } tuple > > > which > > > reduce optimisation opportunities since the only part which is > > > certainly > > > constant is the { tests }. > > > > > > Whereas, if the technique is reported as a test result in its own > right > > > then the { tests, parameters } part of the tuple is a constant across > > > all the MAs, which provides a great optimisation opportunity: each MA > > > does not have to report this information over and over again, and the > > > collector does not have to store N unique instances of this tuple, > just > > > one. > > > > > > So I propose that the test parameters reports what the MA was > commanded > > > to do by the controller, rather than MA-specific details. > > > > > > eg, what I'm proposing looks like: > > > > > > Controller to MA: > > > > > > id =3D 1239 > > > test =3D perform a DNS lookup > > > param =3D lookup "test.net" > > > param =3D use whatever DNS server you think is best; include= it > > > in the report. > > > param =3D report the lookup time > > > > > > MA to collector: > > > > > > result =3D 1239: 5ms from dns.near.me > > > > > > > > > Since the (test+params) is constant, each MA need only report the ID > > > and > > > results. > > > > > > The collector understands the ID-to-(tests+params) mapping because a > > > copy of the original controller message is also sent to the collector > > > as > > > well as to each MA in the test. > > > > > > Therefore the amount of duplication is easily reduced, while still > > > allowing each MA flexibility to choose techniques and report local > > > settings. > > > > > > P. > > > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap > > > > > > > > -- > > ---- > > Sam Crawford > > SamKnows Limited > > > > Switchboard: +44 (0) 20 3111 4330 > > DDI: +44 (0) 20 3111 4332 > > > > This email is sent for and on behalf of SamKnows Limited. > > > > This email and any attachments are confidential, legally privileged and > protected by copyright. If you are not the intended recipient > dissemination or copying of this email is prohibited. If you have receive= d > this in error, please notify the sender by replying by email and then > delete the email completely from your system. > > > > SamKnows Limited, Registered Number: 06510477, Registered Office: Hill > House, 1 Little New Street, London, EC4A 3TR. Registered in England and > Wales. Trade Mark 2507103 > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap From paitken@cisco.com Fri Jul 26 06:03:05 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1602921F852D for ; Fri, 26 Jul 2013 06:03:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.221 X-Spam-Level: X-Spam-Status: No, score=-9.221 tagged_above=-999 required=5 tests=[AWL=-1.377, BAYES_00=-2.599, FRT_LITTLE=1.555, J_CHICKENPOX_21=0.6, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k9wi+KRfboJK for ; Fri, 26 Jul 2013 06:03:00 -0700 (PDT) Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id D685521F84F6 for ; Fri, 26 Jul 2013 06:02:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=8134; q=dns/txt; s=iport; t=1374843779; x=1376053379; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=aKnx85IDiLPnDZdNx77+SyNzjpvb4bkQw+VjBll95lQ=; b=C6gLtL2aNivAbeyZpnSgYujABGp1Qveuk694XFpwS5/mHfqyNnq456aG TjJ2FDzPzw7gVYkEV+ITJTOE/38Fr1kgnCAb/jbm46B1pv4mwL2oM1sgr S0VT/IxP5WWkK0jhd4pbTWYXF8FNL1TauaafKntCkjbMQrFTyDes511L3 o=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgkFAJ9y8lGQ/khL/2dsb2JhbABagwY1vhOBGBZ0giQBAQEEAQEBNQ8lAggDDAQLEQQBAQEJHgcPAhYfCQgGDQEFAgEBiAwMqzONYI4+CgQDgSkFBwaDfwOXX4EphHqLKYMVgWcJFw X-IronPort-AV: E=Sophos;i="4.89,751,1367971200"; d="scan'208";a="16503670" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-4.cisco.com with ESMTP; 26 Jul 2013 13:02:57 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id r6QD2tEW020084 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Jul 2013 13:02:55 GMT Received: from [10.61.92.114] (ams3-vpn-dhcp7283.cisco.com [10.61.92.114]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6QD2pFf015994; Fri, 26 Jul 2013 14:02:52 +0100 (BST) Message-ID: <51F2737C.40905@cisco.com> Date: Fri, 26 Jul 2013 14:02:52 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: Brian Trammell References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> In-Reply-To: <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: dromasca@avaya.com, "EARDLEY, Phil" , acmorton , Sam Crawford , lmap@ietf.org Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 13:03:05 -0000 Brian, > +1 to stateless reporting; I'd go so far as to say that control should be "as stateless as possible" -- it should be possible to determine whether a task completed at an MA or not, but this doesn't necessarily need to be handled as an error condition at the control level Aamer and I were discussing this yesterday: 1. if a controller commands an MA to perform a test which it cannot do, then the MA could immediately indicate this (eg, by some NACK in the control protocol) so the controller knows and is enabled to take appropriate action, whatever that may be (report error, revise the test, command another test, ...; it's out of scope). 2. if the MA accepts the command but is later unable to perform the test as requested (eg, lack of resource, insufficient traffic, backed off for too long, conflict), then there's no longer any back-channel to report this to the controller. All the MA can do is indicate the failure to the collector. 3. If the MA is unable to perform the test (or at least, is unable to report the results) then the controller receives nothing and doesn't even know that the MA had been requested to perform some test, but failed to do so. If we don't close this loop then the controller will be blind to the fact that the requested test didn't happen. So how do we close the loop? Does the collector inform the controller, or does the controller check on the collector to ensure that the results arrived? P. > , especially in the case that the MA is on a device where the customer controls the power and/or whether the app is running: it's simply something that the collector needs to know about to support analysis of the results. MAs may "lose sync" for any number of reasons, so collectors and controllers will also have to be robust against MAs that are running out-of-date instructions; this is also not necessarily an error condition. > > More later, best regards, > > Brian > > On 26 Jul 2013, at 11:47, Sam Crawford wrote: > >> I also very much like the idea of keeping reporting stateless. It removes the need for the client to know about the internal state of the collector when reporting data. It's how we do it at the moment, and its simplicity has been very beneficial a few times. >> >> Regarding overhead: To give you a very rough idea, each run of one of our metrics on our probes generates about 70-100 bytes of report data. Around 30-50% of that details the input parameters. These numbers vary by metric of course, and are without any compression or de-duplication. It's a small amount and the added overhead has never caused us concern. >> >> Thanks, >> >> Sam >> >> >> >> On 26 July 2013 10:20, wrote: >> There'd be some complexities to handle, for instance when the test details change to get the results synchronised with the information you're sending from the controller. >> Anyway, I like the idea of keeping the reporting stateless. >> >> I'm not convinced the gain in de-duplication is worthwhile (is the overhead really that big?), it would be good to have some numbers here on typical bytes of a report. >> >> >> >>> -----Original Message----- >>> From: Paul Aitken [mailto:paitken@cisco.com] >>> Sent: 25 July 2013 21:49 >>> To: MORTON JR., ALFRED C (AL) >>> Cc: Eardley,PL,Philip,TUB8 R; dromasca@avaya.com; lmap@ietf.org >>> Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 >>> >>> Al, >>> >>>> still reading this draft, but as the messages fly by, you said: >>>>> What we wrote in draft-aakhter-lmap framework: as an optimisation, >>>>> the controller treats the collector as an additional MA, thereby >>>>> informing the collector once of the method and params to be used, >>>>> presumably using some ID. Then the MA's need only report that same >>>>> ID, so the collector overhead is much reduced. >>>> how does this optimization match the flexibility to use different >>>> measurement methods on the fly, as Phil highlighted: >>>> << One proposal for making use of both active and passive measurement >>> is >>>> to allow the Measurement Agent to make local decisions on which >>>> technique to use to deliver a particular metric-- as long as the >>>> specific method is included in the report.>> >>> If it's necessary for the MA to report the measurement techniques used, >>> then these should be reported in the test results. >>> >>> It's like another test primitive: select an appropriate technique, and >>> include it in the test report. >>> >>> >>>> I think we agree that passive and active methods are sufficiently >>> different >>>> that they require categorization, possibly as far as summarizing >>>> results from each method separately. MA local decisions certainly >>>> require indicating the measurement method with each value, >>>> hopefully through some shorthand like the proposed registry. >>>> >>>> It seems we can have either the optimizations with static test >>> Instructions, >>>> or MA's making autonomous decisions. >>> So, the proposal is that each MA reports: { which tests were run, which >>> test parameters were used, what the test results were }. >>> >>> The selected technique could be reported as a test parameter, or as a >>> test result (ie, as if the technique selection was a test in its own >>> right). >>> >>> If the selected technique is reported as a test parameter, then there >>> may be many variations in the { tests, parameters, results } tuple >>> which >>> reduce optimisation opportunities since the only part which is >>> certainly >>> constant is the { tests }. >>> >>> Whereas, if the technique is reported as a test result in its own right >>> then the { tests, parameters } part of the tuple is a constant across >>> all the MAs, which provides a great optimisation opportunity: each MA >>> does not have to report this information over and over again, and the >>> collector does not have to store N unique instances of this tuple, just >>> one. >>> >>> So I propose that the test parameters reports what the MA was commanded >>> to do by the controller, rather than MA-specific details. >>> >>> eg, what I'm proposing looks like: >>> >>> Controller to MA: >>> >>> id = 1239 >>> test = perform a DNS lookup >>> param = lookup "test.net" >>> param = use whatever DNS server you think is best; include it >>> in the report. >>> param = report the lookup time >>> >>> MA to collector: >>> >>> result = 1239: 5ms from dns.near.me >>> >>> >>> Since the (test+params) is constant, each MA need only report the ID >>> and >>> results. >>> >>> The collector understands the ID-to-(tests+params) mapping because a >>> copy of the original controller message is also sent to the collector >>> as >>> well as to each MA in the test. >>> >>> Therefore the amount of duplication is easily reduced, while still >>> allowing each MA flexibility to choose techniques and report local >>> settings. >>> >>> P. >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> >> >> >> -- >> ---- >> Sam Crawford >> SamKnows Limited >> >> Switchboard: +44 (0) 20 3111 4330 >> DDI: +44 (0) 20 3111 4332 >> >> This email is sent for and on behalf of SamKnows Limited. >> >> This email and any attachments are confidential, legally privileged and protected by copyright. If you are not the intended recipient dissemination or copying of this email is prohibited. If you have received this in error, please notify the sender by replying by email and then delete the email completely from your system. >> >> SamKnows Limited, Registered Number: 06510477, Registered Office: Hill House, 1 Little New Street, London, EC4A 3TR. Registered in England and Wales. Trade Mark 2507103 >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap From philip.eardley@bt.com Fri Jul 26 06:16:17 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1EF7521F9966 for ; Fri, 26 Jul 2013 06:16:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.441 X-Spam-Level: X-Spam-Status: No, score=-103.441 tagged_above=-999 required=5 tests=[AWL=0.158, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e0ExpwJ-XxGY for ; Fri, 26 Jul 2013 06:16:12 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id B70A521F996F for ; Fri, 26 Jul 2013 06:16:11 -0700 (PDT) Received: from EVMHT69-UKRD.domain1.systemhost.net (10.36.3.129) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.298.1; Fri, 26 Jul 2013 14:16:09 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.2.130]) by EVMHT69-UKRD.domain1.systemhost.net ([10.36.3.129]) with mapi; Fri, 26 Jul 2013 14:16:09 +0100 From: To: , Date: Fri, 26 Jul 2013 14:16:07 +0100 Thread-Topic: [lmap] my comments on draft-akhter-lmap-framework-00 Thread-Index: Ac6JfWCz7QLEQv61THmxpZekIxMNawAg/Pdw Message-ID: <9510D26531EF184D9017DF24659BB87F35CA37F461@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <51F1977E.1040108@cisco.com> In-Reply-To: <51F1977E.1040108@cisco.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: lmap@ietf.org Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 13:16:17 -0000 > > 6. One crisp limitation that is being introduced by draft-eardley is > about each MA being configured only by one Controller (motivated among > other by the need of simplifying authentication of Controllers with the > MA). This limitation is not mentioned in this I-D, at least not > explicitly. Is this an editorial miss, or the authors take a different > stand on this issue? >=20 > Multiple controllers are generally only for the case of redundancy > failover; usually an MA has a relationship with only one controller at > any time. > However, since all controllers are part of the same administrative > domain which is centrally controlled, each can be understood as a > different controller instance. > So an MA with multiple controllers is simply communicating with > multiple > controller instances - so allowing multiple controllers should not be > problematic, since they should not issue contradictory instructions. >=20 > Multiple distinct controllers under different administrative control > aren't within charter. >=20 The constraint proposed in eardley-lmap-framework is that a MA gets instruc= tions from one Controller at any point in time. If the operator of the measurement chooses to implement the Controller func= tion in a distributed way then it can. But I think this should be left as i= nternals to how the Controller is implemented - ie we shouldn't worry about= it when designing the Control Protocol.=20 At some point we do have to think about what happens if the Controller fail= s, and to what extent the WG solves the issue. I don't think that point is = now.=20 Best wishes phil From trammell@tik.ee.ethz.ch Fri Jul 26 06:25:04 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AB5F121F9956 for ; Fri, 26 Jul 2013 06:25:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.844 X-Spam-Level: X-Spam-Status: No, score=-3.844 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FRT_LITTLE=1.555, J_CHICKENPOX_21=0.6, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3W4TxT4EXXIq for ; Fri, 26 Jul 2013 06:25:00 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id C426F21F86AE for ; Fri, 26 Jul 2013 06:24:59 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id CC512D93A2; Fri, 26 Jul 2013 15:24:57 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id M+0M7AwJTnY7; Fri, 26 Jul 2013 15:24:57 +0200 (MEST) Received: from [172.27.10.188] (unknown [195.37.142.72]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 52E2DD9493; Fri, 26 Jul 2013 15:24:57 +0200 (MEST) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) From: Brian Trammell In-Reply-To: <51F2737C.40905@cisco.com> Date: Fri, 26 Jul 2013 15:24:59 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> <51F2737C.40905@cisco.com> To: Paul Aitken X-Mailer: Apple Mail (2.1508) Cc: dromasca@avaya.com, "EARDLEY, Phil" , acmorton , Sam Crawford , lmap@ietf.org Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 13:25:04 -0000 hi Paul, all, comments and questions inline, On 26 Jul 2013, at 15:02, Paul Aitken wrote: > Brian, >=20 >> +1 to stateless reporting; I'd go so far as to say that control = should be "as stateless as possible" -- it should be possible to = determine whether a task completed at an MA or not, but this doesn't = necessarily need to be handled as an error condition at the control = level >=20 > Aamer and I were discussing this yesterday: >=20 > 1. if a controller commands an MA to perform a test which it cannot = do, then the MA could immediately indicate this (eg, by some NACK in the = control protocol) so the controller knows and is enabled to take = appropriate action, whatever that may be (report error, revise the test, = command another test, ...; it's out of scope). Yep, but that's a different case than out-of-sync. Again, depending on = the measurement being run, this may not be a real problem, and how this = gets presented to the controller administrator is situation dependent, = but it makes sense to tell the controller this in the control protocol. > 2. if the MA accepts the command but is later unable to perform the = test as requested (eg, lack of resource, insufficient traffic, backed = off for too long, conflict), then there's no longer any back-channel to = report this to the controller. All the MA can do is indicate the failure = to the collector. >=20 > 3. If the MA is unable to perform the test (or at least, is unable to = report the results) then the controller receives nothing and doesn't = even know that the MA had been requested to perform some test, but = failed to do so. >=20 > If we don't close this loop then the controller will be blind to the = fact that the requested test didn't happen. Is this a problem? Why does the controller need to know about = measurement failures in cases 2 or 3? If an MA is still up and knows = that it won't run a test, it can certainly send a no-report message to = the collector (so it knows there was an instruction sent that was not = executed) so that no-report can be factored into post-facto analysis. = But once the controller sends an instruction, why is it involved in the = further fate of the measurement task? > So how do we close the loop? Does the collector inform the controller, = or does the controller check on the collector to ensure that the results = arrived? >=20 >=20 > P. >=20 >> , especially in the case that the MA is on a device where the = customer controls the power and/or whether the app is running: it's = simply something that the collector needs to know about to support = analysis of the results. MAs may "lose sync" for any number of reasons, = so collectors and controllers will also have to be robust against MAs = that are running out-of-date instructions; this is also not necessarily = an error condition. >>=20 >> More later, best regards, >>=20 >> Brian >>=20 >> On 26 Jul 2013, at 11:47, Sam Crawford wrote: >>=20 >>> I also very much like the idea of keeping reporting stateless. It = removes the need for the client to know about the internal state of the = collector when reporting data. It's how we do it at the moment, and its = simplicity has been very beneficial a few times. >>>=20 >>> Regarding overhead: To give you a very rough idea, each run of one = of our metrics on our probes generates about 70-100 bytes of report = data. Around 30-50% of that details the input parameters. These numbers = vary by metric of course, and are without any compression or = de-duplication. It's a small amount and the added overhead has never = caused us concern. >>>=20 >>> Thanks, >>>=20 >>> Sam >>>=20 >>>=20 >>>=20 >>> On 26 July 2013 10:20, wrote: >>> There'd be some complexities to handle, for instance when the test = details change to get the results synchronised with the information = you're sending from the controller. >>> Anyway, I like the idea of keeping the reporting stateless. >>>=20 >>> I'm not convinced the gain in de-duplication is worthwhile (is the = overhead really that big?), it would be good to have some numbers here = on typical bytes of a report. >>>=20 >>>=20 >>>=20 >>>> -----Original Message----- >>>> From: Paul Aitken [mailto:paitken@cisco.com] >>>> Sent: 25 July 2013 21:49 >>>> To: MORTON JR., ALFRED C (AL) >>>> Cc: Eardley,PL,Philip,TUB8 R; dromasca@avaya.com; lmap@ietf.org >>>> Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 >>>>=20 >>>> Al, >>>>=20 >>>>> still reading this draft, but as the messages fly by, you said: >>>>>> What we wrote in draft-aakhter-lmap framework: as an = optimisation, >>>>>> the controller treats the collector as an additional MA, thereby >>>>>> informing the collector once of the method and params to be used, >>>>>> presumably using some ID. Then the MA's need only report that = same >>>>>> ID, so the collector overhead is much reduced. >>>>> how does this optimization match the flexibility to use different >>>>> measurement methods on the fly, as Phil highlighted: >>>>> << One proposal for making use of both active and passive = measurement >>>> is >>>>> to allow the Measurement Agent to make local decisions on = which >>>>> technique to use to deliver a particular metric-- as long as = the >>>>> specific method is included in the report.>> >>>> If it's necessary for the MA to report the measurement techniques = used, >>>> then these should be reported in the test results. >>>>=20 >>>> It's like another test primitive: select an appropriate technique, = and >>>> include it in the test report. >>>>=20 >>>>=20 >>>>> I think we agree that passive and active methods are sufficiently >>>> different >>>>> that they require categorization, possibly as far as summarizing >>>>> results from each method separately. MA local decisions certainly >>>>> require indicating the measurement method with each value, >>>>> hopefully through some shorthand like the proposed registry. >>>>>=20 >>>>> It seems we can have either the optimizations with static test >>>> Instructions, >>>>> or MA's making autonomous decisions. >>>> So, the proposal is that each MA reports: { which tests were run, = which >>>> test parameters were used, what the test results were }. >>>>=20 >>>> The selected technique could be reported as a test parameter, or as = a >>>> test result (ie, as if the technique selection was a test in its = own >>>> right). >>>>=20 >>>> If the selected technique is reported as a test parameter, then = there >>>> may be many variations in the { tests, parameters, results } tuple >>>> which >>>> reduce optimisation opportunities since the only part which is >>>> certainly >>>> constant is the { tests }. >>>>=20 >>>> Whereas, if the technique is reported as a test result in its own = right >>>> then the { tests, parameters } part of the tuple is a constant = across >>>> all the MAs, which provides a great optimisation opportunity: each = MA >>>> does not have to report this information over and over again, and = the >>>> collector does not have to store N unique instances of this tuple, = just >>>> one. >>>>=20 >>>> So I propose that the test parameters reports what the MA was = commanded >>>> to do by the controller, rather than MA-specific details. >>>>=20 >>>> eg, what I'm proposing looks like: >>>>=20 >>>> Controller to MA: >>>>=20 >>>> id =3D 1239 >>>> test =3D perform a DNS lookup >>>> param =3D lookup "test.net" >>>> param =3D use whatever DNS server you think is best; = include it >>>> in the report. >>>> param =3D report the lookup time >>>>=20 >>>> MA to collector: >>>>=20 >>>> result =3D 1239: 5ms from dns.near.me >>>>=20 >>>>=20 >>>> Since the (test+params) is constant, each MA need only report the = ID >>>> and >>>> results. >>>>=20 >>>> The collector understands the ID-to-(tests+params) mapping because = a >>>> copy of the original controller message is also sent to the = collector >>>> as >>>> well as to each MA in the test. >>>>=20 >>>> Therefore the amount of duplication is easily reduced, while still >>>> allowing each MA flexibility to choose techniques and report local >>>> settings. >>>>=20 >>>> P. >>> _______________________________________________ >>> lmap mailing list >>> lmap@ietf.org >>> https://www.ietf.org/mailman/listinfo/lmap >>>=20 >>>=20 >>>=20 >>> --=20 >>> ---- >>> Sam Crawford >>> SamKnows Limited >>>=20 >>> Switchboard: +44 (0) 20 3111 4330 >>> DDI: +44 (0) 20 3111 4332 >>>=20 >>> This email is sent for and on behalf of SamKnows Limited. >>>=20 >>> This email and any attachments are confidential, legally privileged = and protected by copyright. If you are not the intended recipient = dissemination or copying of this email is prohibited. If you have = received this in error, please notify the sender by replying by email = and then delete the email completely from your system. >>>=20 >>> SamKnows Limited, Registered Number: 06510477, Registered Office: = Hill House, 1 Little New Street, London, EC4A 3TR. Registered in = England and Wales. Trade Mark 2507103 >>> _______________________________________________ >>> lmap mailing list >>> lmap@ietf.org >>> https://www.ietf.org/mailman/listinfo/lmap From paitken@cisco.com Fri Jul 26 07:27:30 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1BDC821F99CD for ; Fri, 26 Jul 2013 07:27:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.049 X-Spam-Level: X-Spam-Status: No, score=-9.049 tagged_above=-999 required=5 tests=[AWL=-1.205, BAYES_00=-2.599, FRT_LITTLE=1.555, J_CHICKENPOX_21=0.6, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 62L+QW44wuQd for ; Fri, 26 Jul 2013 07:27:25 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 7EB2821F95EF for ; Fri, 26 Jul 2013 07:27:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=10676; q=dns/txt; s=iport; t=1374848844; x=1376058444; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=zFA2Z2+3Tz0C8x6cRKe2PljmQj+l/DIrkivwp64H1qs=; b=md2LvvaJ3CKx9sPn2EKbFaKzS6Gokrm/muQ+efRlh52vYxrv0DQl+y6C DF9rmt+LYFNXQgY2Xxgcz5melF74J69X9p1lBt5X6MxLAYjTYjDZdxHGK UxOiPuNT2YFg+X9jcgTxTqjvTAbh5D4yMPRwHQJgD7wxicwn51jb9Dzh8 I=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgkFAMeG8lGQ/khN/2dsb2JhbABbgwY1vhSBFxZ0giQBAQEDAQEBATUPJQIIAgEFBwQLEQQBAQEJFggHCQMCAQIBFR8JCAYNAQUCAQGIBgYMqyiNV44+CgQDgSkFBwaDfwOXX4EphHqLKYMVgWcFAgIX X-IronPort-AV: E=Sophos;i="4.89,751,1367971200"; d="scan'208";a="85156272" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-2.cisco.com with ESMTP; 26 Jul 2013 14:27:20 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id r6QERIF7013591 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Jul 2013 14:27:18 GMT Received: from [10.61.92.114] (ams3-vpn-dhcp7283.cisco.com [10.61.92.114]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6QERFIY021803; Fri, 26 Jul 2013 15:27:15 +0100 (BST) Message-ID: <51F28744.5090200@cisco.com> Date: Fri, 26 Jul 2013 15:27:16 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: Brian Trammell References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> <51F2737C.40905@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: dromasca@avaya.com, "EARDLEY, Phil" , acmorton , Sam Crawford , lmap@ietf.org Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 14:27:30 -0000 Brian, > hi Paul, all, > > comments and questions inline, > > On 26 Jul 2013, at 15:02, Paul Aitken wrote: > >> Brian, >> >>> +1 to stateless reporting; I'd go so far as to say that control should be "as stateless as possible" -- it should be possible to determine whether a task completed at an MA or not, but this doesn't necessarily need to be handled as an error condition at the control level >> Aamer and I were discussing this yesterday: >> >> 1. if a controller commands an MA to perform a test which it cannot do, then the MA could immediately indicate this (eg, by some NACK in the control protocol) so the controller knows and is enabled to take appropriate action, whatever that may be (report error, revise the test, command another test, ...; it's out of scope). > Yep, but that's a different case than out-of-sync. Again, depending on the measurement being run, this may not be a real problem, and how this gets presented to the controller administrator is situation dependent, but it makes sense to tell the controller this in the control protocol. > >> 2. if the MA accepts the command but is later unable to perform the test as requested (eg, lack of resource, insufficient traffic, backed off for too long, conflict), then there's no longer any back-channel to report this to the controller. All the MA can do is indicate the failure to the collector. >> >> 3. If the MA is unable to perform the test (or at least, is unable to report the results) then the controller receives nothing and doesn't even know that the MA had been requested to perform some test, but failed to do so. >> >> If we don't close this loop then the controller will be blind to the fact that the requested test didn't happen. > Is this a problem? Why does the controller need to know about measurement failures in cases 2 or 3? I'm thinking of it like ICMP messages: if nobody tells the controller that the requested test didn't work, then it might continue to request the same. The administrator might go to the collector looking to analyse the results and be surprised not to find any. > If an MA is still up and knows that it won't run a test, it can certainly send a no-report message to the collector (so it knows there was an instruction sent that was not executed) so that no-report can be factored into post-facto analysis. Right, that's case 2. What do you propose for case 3? > But once the controller sends an instruction, why is it involved in the further fate of the measurement task? Not so much in the fate of that specific task per se, but of the overall measurement. eg, depending on the failure reason, it could be that the test should be modified or a different test run. Or perhaps an action raised to upgrade the failing set of MAs. Suppose Benoit tells me to write a report and send it to you. Normally I say "OK" and do it. But I might say "No", in which case we negotiate a different report or timeframe. Or I might say "OK" then just not do it, so you receive nothing. You don't even know that you didn't receive the report, because nobody told you to expect it - so when Benoit comes to discuss it with you, you're clueless. Not good. So normally Benoit would tell you, "I've asked Paul to send you the report" so you'd be expecting it. Then you can check back with either of us if you didn't receive it. Or, Benoit would periodically check with us to see whether I finished the report and whether you received it. LMAP should work like this too. P. >> So how do we close the loop? Does the collector inform the controller, or does the controller check on the collector to ensure that the results arrived? >> >> >> P. >> >>> , especially in the case that the MA is on a device where the customer controls the power and/or whether the app is running: it's simply something that the collector needs to know about to support analysis of the results. MAs may "lose sync" for any number of reasons, so collectors and controllers will also have to be robust against MAs that are running out-of-date instructions; this is also not necessarily an error condition. >>> >>> More later, best regards, >>> >>> Brian >>> >>> On 26 Jul 2013, at 11:47, Sam Crawford wrote: >>> >>>> I also very much like the idea of keeping reporting stateless. It removes the need for the client to know about the internal state of the collector when reporting data. It's how we do it at the moment, and its simplicity has been very beneficial a few times. >>>> >>>> Regarding overhead: To give you a very rough idea, each run of one of our metrics on our probes generates about 70-100 bytes of report data. Around 30-50% of that details the input parameters. These numbers vary by metric of course, and are without any compression or de-duplication. It's a small amount and the added overhead has never caused us concern. >>>> >>>> Thanks, >>>> >>>> Sam >>>> >>>> >>>> >>>> On 26 July 2013 10:20, wrote: >>>> There'd be some complexities to handle, for instance when the test details change to get the results synchronised with the information you're sending from the controller. >>>> Anyway, I like the idea of keeping the reporting stateless. >>>> >>>> I'm not convinced the gain in de-duplication is worthwhile (is the overhead really that big?), it would be good to have some numbers here on typical bytes of a report. >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: Paul Aitken [mailto:paitken@cisco.com] >>>>> Sent: 25 July 2013 21:49 >>>>> To: MORTON JR., ALFRED C (AL) >>>>> Cc: Eardley,PL,Philip,TUB8 R; dromasca@avaya.com; lmap@ietf.org >>>>> Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 >>>>> >>>>> Al, >>>>> >>>>>> still reading this draft, but as the messages fly by, you said: >>>>>>> What we wrote in draft-aakhter-lmap framework: as an optimisation, >>>>>>> the controller treats the collector as an additional MA, thereby >>>>>>> informing the collector once of the method and params to be used, >>>>>>> presumably using some ID. Then the MA's need only report that same >>>>>>> ID, so the collector overhead is much reduced. >>>>>> how does this optimization match the flexibility to use different >>>>>> measurement methods on the fly, as Phil highlighted: >>>>>> << One proposal for making use of both active and passive measurement >>>>> is >>>>>> to allow the Measurement Agent to make local decisions on which >>>>>> technique to use to deliver a particular metric-- as long as the >>>>>> specific method is included in the report.>> >>>>> If it's necessary for the MA to report the measurement techniques used, >>>>> then these should be reported in the test results. >>>>> >>>>> It's like another test primitive: select an appropriate technique, and >>>>> include it in the test report. >>>>> >>>>> >>>>>> I think we agree that passive and active methods are sufficiently >>>>> different >>>>>> that they require categorization, possibly as far as summarizing >>>>>> results from each method separately. MA local decisions certainly >>>>>> require indicating the measurement method with each value, >>>>>> hopefully through some shorthand like the proposed registry. >>>>>> >>>>>> It seems we can have either the optimizations with static test >>>>> Instructions, >>>>>> or MA's making autonomous decisions. >>>>> So, the proposal is that each MA reports: { which tests were run, which >>>>> test parameters were used, what the test results were }. >>>>> >>>>> The selected technique could be reported as a test parameter, or as a >>>>> test result (ie, as if the technique selection was a test in its own >>>>> right). >>>>> >>>>> If the selected technique is reported as a test parameter, then there >>>>> may be many variations in the { tests, parameters, results } tuple >>>>> which >>>>> reduce optimisation opportunities since the only part which is >>>>> certainly >>>>> constant is the { tests }. >>>>> >>>>> Whereas, if the technique is reported as a test result in its own right >>>>> then the { tests, parameters } part of the tuple is a constant across >>>>> all the MAs, which provides a great optimisation opportunity: each MA >>>>> does not have to report this information over and over again, and the >>>>> collector does not have to store N unique instances of this tuple, just >>>>> one. >>>>> >>>>> So I propose that the test parameters reports what the MA was commanded >>>>> to do by the controller, rather than MA-specific details. >>>>> >>>>> eg, what I'm proposing looks like: >>>>> >>>>> Controller to MA: >>>>> >>>>> id = 1239 >>>>> test = perform a DNS lookup >>>>> param = lookup "test.net" >>>>> param = use whatever DNS server you think is best; include it >>>>> in the report. >>>>> param = report the lookup time >>>>> >>>>> MA to collector: >>>>> >>>>> result = 1239: 5ms from dns.near.me >>>>> >>>>> >>>>> Since the (test+params) is constant, each MA need only report the ID >>>>> and >>>>> results. >>>>> >>>>> The collector understands the ID-to-(tests+params) mapping because a >>>>> copy of the original controller message is also sent to the collector >>>>> as >>>>> well as to each MA in the test. >>>>> >>>>> Therefore the amount of duplication is easily reduced, while still >>>>> allowing each MA flexibility to choose techniques and report local >>>>> settings. >>>>> >>>>> P. >>>> _______________________________________________ >>>> lmap mailing list >>>> lmap@ietf.org >>>> https://www.ietf.org/mailman/listinfo/lmap >>>> >>>> >>>> >>>> -- >>>> ---- >>>> Sam Crawford >>>> SamKnows Limited >>>> >>>> Switchboard: +44 (0) 20 3111 4330 >>>> DDI: +44 (0) 20 3111 4332 >>>> >>>> This email is sent for and on behalf of SamKnows Limited. >>>> >>>> This email and any attachments are confidential, legally privileged and protected by copyright. If you are not the intended recipient dissemination or copying of this email is prohibited. If you have received this in error, please notify the sender by replying by email and then delete the email completely from your system. >>>> >>>> SamKnows Limited, Registered Number: 06510477, Registered Office: Hill House, 1 Little New Street, London, EC4A 3TR. Registered in England and Wales. Trade Mark 2507103 >>>> _______________________________________________ >>>> lmap mailing list >>>> lmap@ietf.org >>>> https://www.ietf.org/mailman/listinfo/lmap From paitken@cisco.com Fri Jul 26 07:38:42 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7C84C21F94DC for ; Fri, 26 Jul 2013 07:38:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.293 X-Spam-Level: X-Spam-Status: No, score=-10.293 tagged_above=-999 required=5 tests=[AWL=0.306, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0udQlD3LyIVb for ; Fri, 26 Jul 2013 07:38:34 -0700 (PDT) Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id A937A21F90FD for ; Fri, 26 Jul 2013 07:38:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1201; q=dns/txt; s=iport; t=1374849513; x=1376059113; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=Yarr5w7gHCrvxJ32S0Vo3/gjLygGupMxb1aIzKooO8A=; b=Cv0pARFp/6cBhwCxbDVB2/bB4NfOxvrvJESnGivseCGapq1YBrGT1FGk BBYv//g6pIZImWHdw7ugMJx5AgtyqhWhqlR0jIp4ZGlMRoQWVymJvmWFQ R2UhZU+nDE6fxWBlFcLdKVSc0Fb4F2BkBzlGOiOjbXUrhTp+Ttsobq1vQ g=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgcFAOuI8lGQ/khL/2dsb2JhbABbgwa+SIEXFnSCJAEBAQMBOEABBQsLIRYPCQMCAQIBRRMBBwEBiAYGuQqPfQcWg28Dl1+GI4spgxU X-IronPort-AV: E=Sophos;i="4.89,751,1367971200"; d="scan'208";a="16506507" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-4.cisco.com with ESMTP; 26 Jul 2013 14:38:32 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id r6QEcU5C013223 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Jul 2013 14:38:30 GMT Received: from [10.61.92.114] (ams3-vpn-dhcp7283.cisco.com [10.61.92.114]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6QEcSNW022374; Fri, 26 Jul 2013 15:38:29 +0100 (BST) Message-ID: <51F289E5.2060309@cisco.com> Date: Fri, 26 Jul 2013 15:38:29 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: philip.eardley@bt.com References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <51F1977E.1040108@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F461@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F35CA37F461@EMV65-UKRD.domain1.systemhost.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: dromasca@avaya.com, lmap@ietf.org Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 14:38:42 -0000 Philip, > The constraint proposed in eardley-lmap-framework is that a MA gets instructions from one Controller at any point in time. While I know what you mean, if it's a single CPU (or at least, a single MA process) then it's necessarily only getting instructions from one Controller at a time - although there could be different controllers every few milliseconds... Why do we fear multiple controllers? By the LMAP charter, they're under control of a single organisation, so they should not be issuing contradictory instructions. > If the operator of the measurement chooses to implement the Controller function in a distributed way then it can. But I think this should be left as internals to how the Controller is implemented - ie we shouldn't worry about it when designing the Control Protocol. > > At some point we do have to think about what happens if the Controller fails, and to what extent the WG solves the issue. I don't think that point is now. If it's not mentioned early on, then either the accepted design will prevent it from being added later, or it'll later be seen as a new requirement and rejected. Surely we've all seen this many times? P. From trevor.burbridge@bt.com Fri Jul 26 07:43:44 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 84C8421F9978 for ; Fri, 26 Jul 2013 07:43:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.999 X-Spam-Level: X-Spam-Status: No, score=-2.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 71ciVkCZT6cI for ; Fri, 26 Jul 2013 07:43:38 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp61.intersmtp.com [62.239.224.234]) by ietfa.amsl.com (Postfix) with ESMTP id E77B121F9976 for ; Fri, 26 Jul 2013 07:43:37 -0700 (PDT) Received: from EVMHT62-UKRD.domain1.systemhost.net (10.36.3.128) by RDW083A005ED61.smtp-e1.hygiene.service (10.187.98.10) with Microsoft SMTP Server (TLS) id 8.3.298.1; Fri, 26 Jul 2013 15:43:36 +0100 Received: from EMV64-UKRD.domain1.systemhost.net ([169.254.2.51]) by EVMHT62-UKRD.domain1.systemhost.net ([10.36.3.128]) with mapi; Fri, 26 Jul 2013 15:43:36 +0100 From: To: , Date: Fri, 26 Jul 2013 15:43:34 +0100 Thread-Topic: [lmap] my comments on draft-akhter-lmap-framework-00 Thread-Index: Ac6KDEUI+YHbORkURYaJDttjxnkczgAASpAw Message-ID: References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> <51F2737C.40905@cisco.com> <51F28744.5090200@cisco.com> In-Reply-To: <51F28744.5090200@cisco.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: dromasca@avaya.com, philip.eardley@bt.com, lmap@ietf.org, acmorton@att.com, sam@samknows.com Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 14:43:44 -0000 My couple of contributions inline... >-----Original Message----- >From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of >Paul Aitken >Sent: 26 July 2013 15:27 >To: Brian Trammell >Cc: dromasca@avaya.com; Eardley,PL,Philip,TUB8 R; acmorton; Sam >Crawford; lmap@ietf.org >Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 > >Brian, > >> hi Paul, all, >> >> comments and questions inline, >> >> On 26 Jul 2013, at 15:02, Paul Aitken wrote: >> >>> Brian, >>> >>>> +1 to stateless reporting; I'd go so far as to say that control >>>> +should be "as stateless as possible" -- it should be possible to >>>> +determine whether a task completed at an MA or not, but this >>>> +doesn't necessarily need to be handled as an error condition at the >>>> +control level >>> Aamer and I were discussing this yesterday: >>> >>> 1. if a controller commands an MA to perform a test which it cannot do, >then the MA could immediately indicate this (eg, by some NACK in the >control protocol) so the controller knows and is enabled to take appropria= te >action, whatever that may be (report error, revise the test, command >another test, ...; it's out of scope). >> Yep, but that's a different case than out-of-sync. Again, depending on t= he >measurement being run, this may not be a real problem, and how this gets >presented to the controller administrator is situation dependent, but it >makes sense to tell the controller this in the control protocol. >> >>> 2. if the MA accepts the command but is later unable to perform the tes= t >as requested (eg, lack of resource, insufficient traffic, backed off for t= oo long, >conflict), then there's no longer any back-channel to report this to the >controller. All the MA can do is indicate the failure to the collector. >>> >>> 3. If the MA is unable to perform the test (or at least, is unable to r= eport >the results) then the controller receives nothing and doesn't even know th= at >the MA had been requested to perform some test, but failed to do so. >>> >>> If we don't close this loop then the controller will be blind to the fa= ct that >the requested test didn't happen. >> Is this a problem? Why does the controller need to know about >measurement failures in cases 2 or 3? > >I'm thinking of it like ICMP messages: if nobody tells the controller that= the >requested test didn't work, then it might continue to request the same. Th= e >administrator might go to the collector looking to analyse the results and= be >surprised not to find any. If nobody tells the controller, then it definitely WON'T keep going to the = MA as it will assume the test is running fine. In the information model we'= ve talked about reporting back to the controller but imagined that most of = what we want to do could be done asynchronously via some sort of log messag= es so we can easily use HTTP Rest type approaches. The measurement itself can decide how to report failed tests. This could s= imply be a failed flag, or more sophisticated data columns. This has to be = left to the test since failure modes for different tests and how that shoul= d be reported will be different. Someone going to the data should see test failure messages. >> If an MA is still up and knows that it won't run a test, it can certainl= y send >a no-report message to the collector (so it knows there was an instruction >sent that was not executed) so that no-report can be factored into post-fa= cto >analysis. > >Right, that's case 2. > >What do you propose for case 3? > > >> But once the controller sends an instruction, why is it involved in the >further fate of the measurement task? > >Not so much in the fate of that specific task per se, but of the overall >measurement. eg, depending on the failure reason, it could be that the tes= t >should be modified or a different test run. Or perhaps an action raised to >upgrade the failing set of MAs. > > >Suppose Benoit tells me to write a report and send it to you. Normally I s= ay >"OK" and do it. But I might say "No", in which case we negotiate a differe= nt >report or timeframe. Or I might say "OK" then just not do it, so you recei= ve >nothing. You don't even know that you didn't receive the report, because >nobody told you to expect it - so when Benoit comes to discuss it with you= , >you're clueless. Not good. > >So normally Benoit would tell you, "I've asked Paul to send you the report= " >so you'd be expecting it. Then you can check back with either of us if you >didn't receive it. > >Or, Benoit would periodically check with us to see whether I finished the >report and whether you received it. > >LMAP should work like this too. It's a big database with lots of data. I don't see the Collector should be = trying to police the data it is receiving. It would have to know all of the= schedules of different MAs and implement timeouts etc. We want to keep the= Collector simple. Another motivation is if the Collector is a 3rd party wh= o we simply want to ship data to. From philip.eardley@bt.com Fri Jul 26 07:46:47 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3F5A621F9A33 for ; Fri, 26 Jul 2013 07:46:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.453 X-Spam-Level: X-Spam-Status: No, score=-103.453 tagged_above=-999 required=5 tests=[AWL=0.145, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DVQkM1sx8Z2a for ; Fri, 26 Jul 2013 07:46:42 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 8412E21F9A2A for ; Fri, 26 Jul 2013 07:45:40 -0700 (PDT) Received: from EVMHT68-UKRD.domain1.systemhost.net (10.36.3.105) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.298.1; Fri, 26 Jul 2013 15:45:39 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.2.130]) by EVMHT68-UKRD.domain1.systemhost.net ([10.36.3.105]) with mapi; Fri, 26 Jul 2013 15:45:39 +0100 From: To: Date: Fri, 26 Jul 2013 15:45:37 +0100 Thread-Topic: more comments on draft-akhter-lmap-framework Thread-Index: Ac6KAlnFRWmvDRXbSs6U8NaTz3KU5g== Message-ID: <9510D26531EF184D9017DF24659BB87F35CA37F523@EMV65-UKRD.domain1.systemhost.net> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F35CA37F523EMV65UKRDdoma_" MIME-Version: 1.0 Subject: [lmap] more comments on draft-akhter-lmap-framework X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 14:46:47 -0000 --_000_9510D26531EF184D9017DF24659BB87F35CA37F523EMV65UKRDdoma_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable My final batch of comments First, I think some of the material in Section 3 could be helpful expansion= (on what was rather briefly mentioned in draft-eardley-lmap-framework). Se= ctions 4 & 5 have lots of interesting stuff - question is whether this is a= general framework for large-scale measurements (where we need to talk in d= etail about the tests) or whether it's a framework for LMAP WG (which only = needs a brief mention of areas outside the scope of lmap wg, and so wouldn'= t have much about the tests, as this is ippm). Secondly, there were several comments about multiple measurement agents (& = peers?) acting together. (Section 3.2 & 6.2.1). you mention several differe= nt cases, I'm not sure I understood properly or got them all. (1) I'm sure there's a case where one test triggers another (say, the dns r= esult followed by request to website) - this is ok. (2) I think you have a 'latency under load' test (S3.2), where the latency = is measured for a pkt/file being sent between MA & Measurement Peer, but an= other Peer is generating other traffic so that a bottleneck queue is loaded= . I haven't thought about this much - do you think there are other tests in= volving a second Peer? or even 3rd, 4th...? Or is it just 'latency under lo= ad'? I'm wondering whether it would be best to think about the specifics o= f how to do the 'latency under load' test, rather than a general capability= for coordinating multiple Peers in one test. (3) you mention (S6.2.1) a case where multiple Measurement Agents submit re= sults all with the same Measurement Task ID. I guess this is a test where l= ots of coordinated measurements are made. You also mention a shared key bet= ween multiple MAs, which I think must be related. I don't understand this c= ase, please explain more. Am interested to understand this whole area better, regardless of where the= WG decides to draw the line between in scope & out. Third, you have some stuff about task scheduling where it seems important t= o have highly synchronised time between the controller, MAs and collector (= and measurement peer?). why would this be needed? You mention about NTP and= the issue that NTP doesn't give very accurate time sync to a device on the= end of an access line (because you can't triangulate to multiple time serv= ers). I suppose the error might be a few millisecs - the sorts of cases I h= ad in mind it would easily be accurate enough. Incidentally I don't see why= your solution (NTP clock sent from controller) helps, since you still have= the lack of triangulation on the end of an access line. Thanks Best wishes phil --_000_9510D26531EF184D9017DF24659BB87F35CA37F523EMV65UKRDdoma_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

My final batch of comment= s

 

= First, I think some of the material in Section 3 could be helpful expansion= (on what was rather briefly mentioned in draft-eardley-lmap-framework). Se= ctions 4 & 5 have lots of interesting stuff – question is whether= this is a general framework for large-scale measurements (where we need to= talk in detail about the tests) or whether it’s a framework for LMAP= WG (which only needs a brief mention of areas outside the scope of lmap wg= , and so wouldn’t have much about the tests, as this is ippm).  =

 

S= econdly, there were several comments about multiple measurement agents (&am= p; peers?) acting together. (Section 3.2 & 6.2.1). you mention several = different cases, I’m not sure I understood properly or got them all. =

(1) I’m sure there’s a case w= here one test triggers another (say, the dns result followed by request to = website) – this is ok.

(2) I think = you have a ‘latency under load’ test (S3.2), where the latency = is measured for a pkt/file being sent between MA & Measurement Peer, bu= t another Peer is generating other traffic so that a bottleneck queue is lo= aded. I haven’t thought about this much – do you think there ar= e other tests involving a second Peer? or even 3rd, 4th…? Or is it just ‘latency under load’?  I’m = wondering whether it would be best to think about the specifics of how to d= o the ‘latency under load’ test, rather than a general capabili= ty for coordinating multiple Peers in one test.

(3) you mention (S6.2.1) a case where multiple Measurement Agents sub= mit results all with the same Measurement Task ID. I guess this is a test w= here lots of coordinated measurements are made. You also mention a shared k= ey between multiple MAs, which I think must be related. I don’t under= stand this case, please explain more.

Am = interested to understand this whole area better, regardless of where the WG= decides to draw the line between in scope & out.

=

 

Third, you have some st= uff about task scheduling where it seems important to have highly synchroni= sed time between the controller, MAs and collector (and measurement peer?).= why would this be needed? You mention about NTP and the issue that NTP doe= sn’t give very accurate time sync to a device on the end of an access= line (because you can’t triangulate to multiple time servers). I sup= pose the error might be a few millisecs - the sorts of cases I had in mind = it would easily be accurate enough. Incidentally I don’t see why your= solution (NTP clock sent from controller) helps, since you still have the = lack of triangulation on the end of an access line.

<= p class=3DMsoNormal> 

Thanks=

Best wishes

phil

 

= --_000_9510D26531EF184D9017DF24659BB87F35CA37F523EMV65UKRDdoma_-- From paitken@cisco.com Fri Jul 26 07:53:10 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 207AC21F85D1 for ; Fri, 26 Jul 2013 07:53:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.024 X-Spam-Level: X-Spam-Status: No, score=-10.024 tagged_above=-999 required=5 tests=[AWL=-0.024, BAYES_00=-2.599, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QqdXaOya-+-7 for ; Fri, 26 Jul 2013 07:53:03 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 74F8821F844D for ; Fri, 26 Jul 2013 07:53:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=5974; q=dns/txt; s=iport; t=1374850382; x=1376059982; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=6wTxrQ+8rFuaZUWFj/Jt0sM//W65MM/Ld2obWSlNnPY=; b=IPtrzgq/W5l0iNDZG1u5LL5Wd+RNbewzyhEFYhG991mMcCOJgcbnKB/z IA0NHyyXB4q3tsW7tVPjoGIYs9ijGd+3ILPrvSpQBpXrDl7Tj9ZVODo3P YNalgC0dyN77lqDZmyq8XCM4DmSPhEKZSyGUdm5MZU8uHJZ4ScYOfPGtU c=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AggFAGeM8lGQ/khM/2dsb2JhbABbgwY1vhOBFxZ0giQBAQEDATgPMgUHBAsRBAEBAQkeBw8CNQkIEwEFAgEBiAYGqy6NV45PgS4HBoN/A5dfhiOLKYMVgWcFAg X-IronPort-AV: E=Sophos;i="4.89,752,1367971200"; d="scan'208";a="85157226" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-2.cisco.com with ESMTP; 26 Jul 2013 14:52:58 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r6QEquPT019507 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Jul 2013 14:52:56 GMT Received: from [10.61.92.114] (ams3-vpn-dhcp7283.cisco.com [10.61.92.114]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6QEqrVR023194; Fri, 26 Jul 2013 15:52:53 +0100 (BST) Message-ID: <51F28D46.70206@cisco.com> Date: Fri, 26 Jul 2013 15:52:54 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: trevor.burbridge@bt.com References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> <51F2737C.40905@cisco.com> <51F28744.5090200@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: acmorton@att.com, lmap@ietf.org, philip.eardley@bt.com, dromasca@avaya.com, sam@samknows.com, trammell@tik.ee.ethz.ch Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 14:53:10 -0000 Trevor, > My couple of contributions inline... > >> -----Original Message----- >> From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of >> Paul Aitken >> Sent: 26 July 2013 15:27 >> To: Brian Trammell >> Cc: dromasca@avaya.com; Eardley,PL,Philip,TUB8 R; acmorton; Sam >> Crawford; lmap@ietf.org >> Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 >> >> Brian, >> >>> hi Paul, all, >>> >>> comments and questions inline, >>> >>> On 26 Jul 2013, at 15:02, Paul Aitken wrote: >>> >>>> Brian, >>>> >>>>> +1 to stateless reporting; I'd go so far as to say that control >>>>> +should be "as stateless as possible" -- it should be possible to >>>>> +determine whether a task completed at an MA or not, but this >>>>> +doesn't necessarily need to be handled as an error condition at the >>>>> +control level >>>> Aamer and I were discussing this yesterday: >>>> >>>> 1. if a controller commands an MA to perform a test which it cannot do, >> then the MA could immediately indicate this (eg, by some NACK in the >> control protocol) so the controller knows and is enabled to take appropriate >> action, whatever that may be (report error, revise the test, command >> another test, ...; it's out of scope). >>> Yep, but that's a different case than out-of-sync. Again, depending on the >> measurement being run, this may not be a real problem, and how this gets >> presented to the controller administrator is situation dependent, but it >> makes sense to tell the controller this in the control protocol. >>>> 2. if the MA accepts the command but is later unable to perform the test >> as requested (eg, lack of resource, insufficient traffic, backed off for too long, >> conflict), then there's no longer any back-channel to report this to the >> controller. All the MA can do is indicate the failure to the collector. >>>> 3. If the MA is unable to perform the test (or at least, is unable to report >> the results) then the controller receives nothing and doesn't even know that >> the MA had been requested to perform some test, but failed to do so. >>>> If we don't close this loop then the controller will be blind to the fact that >> the requested test didn't happen. >>> Is this a problem? Why does the controller need to know about >> measurement failures in cases 2 or 3? >> >> I'm thinking of it like ICMP messages: if nobody tells the controller that the >> requested test didn't work, then it might continue to request the same. The >> administrator might go to the collector looking to analyse the results and be >> surprised not to find any. > > If nobody tells the controller, then it definitely WON'T keep going to the MA as it will assume the test is running fine. In the information model we've talked about reporting back to the controller but imagined that most of what we want to do could be done asynchronously via some sort of log messages so we can easily use HTTP Rest type approaches. > > The measurement itself can decide how to report failed tests. This could simply be a failed flag, or more sophisticated data columns. This has to be left to the test since failure modes for different tests and how that should be reported will be different. Oh please, no. This was an oversight in IPFIX: there's no official method of reporting that certain metrics weren't seen or couldn't be measured. So now there are a variety of field-specific values and implementation-specific methods such that collectors need a lot of specific knowledge. There should be a single standard mechanism which everyone uses. I have a draft with some proposals. Similarly for LMAP: if the collector is to be kept simple, then there should be a common method of reporting test failure. > Someone going to the data should see test failure messages. Unless case (3) where the MA isn't able to send the test report. eg, the report message is lost or the MA loses power. >>> If an MA is still up and knows that it won't run a test, it can certainly send >> a no-report message to the collector (so it knows there was an instruction >> sent that was not executed) so that no-report can be factored into post-facto >> analysis. >> >> Right, that's case 2. >> >> What do you propose for case 3? >> >> >>> But once the controller sends an instruction, why is it involved in the >> further fate of the measurement task? >> >> Not so much in the fate of that specific task per se, but of the overall >> measurement. eg, depending on the failure reason, it could be that the test >> should be modified or a different test run. Or perhaps an action raised to >> upgrade the failing set of MAs. >> >> >> Suppose Benoit tells me to write a report and send it to you. Normally I say >> "OK" and do it. But I might say "No", in which case we negotiate a different >> report or timeframe. Or I might say "OK" then just not do it, so you receive >> nothing. You don't even know that you didn't receive the report, because >> nobody told you to expect it - so when Benoit comes to discuss it with you, >> you're clueless. Not good. >> >> So normally Benoit would tell you, "I've asked Paul to send you the report" >> so you'd be expecting it. Then you can check back with either of us if you >> didn't receive it. >> >> Or, Benoit would periodically check with us to see whether I finished the >> report and whether you received it. >> >> LMAP should work like this too. > > It's a big database with lots of data. I don't see the Collector should be trying to police the data it is receiving. It would have to know all of the schedules of different MAs and implement timeouts etc. We want to keep the Collector simple. Another motivation is if the Collector is a 3rd party who we simply want to ship data to. The shipped data doesn't arrive. Who's responsible for a) detecting, and b) addressing that? P. From trevor.burbridge@bt.com Fri Jul 26 08:23:41 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0734A11E8101 for ; Fri, 26 Jul 2013 08:23:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c2aU4am0DR2S for ; Fri, 26 Jul 2013 08:23:36 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp62.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id B1D4B11E80F4 for ; Fri, 26 Jul 2013 08:23:32 -0700 (PDT) Received: from EVMHT66-UKRD.domain1.systemhost.net (10.36.3.103) by RDW083A006ED62.smtp-e2.hygiene.service (10.187.98.11) with Microsoft SMTP Server (TLS) id 8.3.298.1; Fri, 26 Jul 2013 16:23:30 +0100 Received: from EMV64-UKRD.domain1.systemhost.net ([169.254.2.51]) by EVMHT66-UKRD.domain1.systemhost.net ([10.36.3.103]) with mapi; Fri, 26 Jul 2013 16:23:30 +0100 From: To: Date: Fri, 26 Jul 2013 16:23:29 +0100 Thread-Topic: [lmap] my comments on draft-akhter-lmap-framework-00 Thread-Index: Ac6KD9IB971dfyZUSY+mF6MQTKd9IgAA/NTw Message-ID: References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337D1@EMV65-UKRD.domain1.systemhost.net> <51F1313A.9060800@cisco.com> <51F18F4B.8050203@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F21E@EMV65-UKRD.domain1.systemhost.net> <739E12CC-B1FE-4DA5-8059-4776F0418D26@tik.ee.ethz.ch> <51F2737C.40905@cisco.com> <51F28744.5090200@cisco.com> <51F28D46.70206@cisco.com> In-Reply-To: <51F28D46.70206@cisco.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: acmorton@att.com, lmap@ietf.org, philip.eardley@bt.com, dromasca@avaya.com, sam@samknows.com, trammell@tik.ee.ethz.ch Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 15:23:41 -0000 >-----Original Message----- >From: Paul Aitken [mailto:paitken@cisco.com] >Sent: 26 July 2013 15:53 >To: Burbridge,T,Trevor,TUB8 R >Cc: trammell@tik.ee.ethz.ch; dromasca@avaya.com; Eardley,PL,Philip,TUB8 >R; acmorton@att.com; sam@samknows.com; lmap@ietf.org >Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 > >Trevor, > > >> My couple of contributions inline... >> >>> -----Original Message----- >>> From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf >>> Of Paul Aitken >>> Sent: 26 July 2013 15:27 >>> To: Brian Trammell >>> Cc: dromasca@avaya.com; Eardley,PL,Philip,TUB8 R; acmorton; Sam >>> Crawford; lmap@ietf.org >>> Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 >>> >>> Brian, >>> >>>> hi Paul, all, >>>> >>>> comments and questions inline, >>>> >>>> On 26 Jul 2013, at 15:02, Paul Aitken wrote: >>>> >>>>> Brian, >>>>> >>>>>> +1 to stateless reporting; I'd go so far as to say that control >>>>>> +should be "as stateless as possible" -- it should be possible to >>>>>> +determine whether a task completed at an MA or not, but this >>>>>> +doesn't necessarily need to be handled as an error condition at >>>>>> +the control level >>>>> Aamer and I were discussing this yesterday: >>>>> >>>>> 1. if a controller commands an MA to perform a test which it cannot >>>>> do, >>> then the MA could immediately indicate this (eg, by some NACK in the >>> control protocol) so the controller knows and is enabled to take >>> appropriate action, whatever that may be (report error, revise the >>> test, command another test, ...; it's out of scope). >>>> Yep, but that's a different case than out-of-sync. Again, depending >>>> on the >>> measurement being run, this may not be a real problem, and how this >>> gets presented to the controller administrator is situation >>> dependent, but it makes sense to tell the controller this in the contro= l >protocol. >>>>> 2. if the MA accepts the command but is later unable to perform the >>>>> test >>> as requested (eg, lack of resource, insufficient traffic, backed off >>> for too long, conflict), then there's no longer any back-channel to >>> report this to the controller. All the MA can do is indicate the failur= e to >the collector. >>>>> 3. If the MA is unable to perform the test (or at least, is unable >>>>> to report >>> the results) then the controller receives nothing and doesn't even >>> know that the MA had been requested to perform some test, but failed >to do so. >>>>> If we don't close this loop then the controller will be blind to >>>>> the fact that >>> the requested test didn't happen. >>>> Is this a problem? Why does the controller need to know about >>> measurement failures in cases 2 or 3? >>> >>> I'm thinking of it like ICMP messages: if nobody tells the controller >>> that the requested test didn't work, then it might continue to >>> request the same. The administrator might go to the collector looking >>> to analyse the results and be surprised not to find any. >> >> If nobody tells the controller, then it definitely WON'T keep going to t= he >MA as it will assume the test is running fine. In the information model we= 've >talked about reporting back to the controller but imagined that most of >what we want to do could be done asynchronously via some sort of log >messages so we can easily use HTTP Rest type approaches. >> >> The measurement itself can decide how to report failed tests. This coul= d >simply be a failed flag, or more sophisticated data columns. This has to b= e >left to the test since failure modes for different tests and how that shou= ld be >reported will be different. > >Oh please, no. > >This was an oversight in IPFIX: there's no official method of reporting th= at >certain metrics weren't seen or couldn't be measured. So now there are a >variety of field-specific values and implementation-specific methods such >that collectors need a lot of specific knowledge. There should be a single >standard mechanism which everyone uses. I have a draft with some >proposals. > >Similarly for LMAP: if the collector is to be kept simple, then there shou= ld be >a common method of reporting test failure. OK, so we mandate a success/fail column in the measurement reporting inform= ation model. I think the only think I have mandated at the moment is a time= stamp. >> Someone going to the data should see test failure messages. > >Unless case (3) where the MA isn't able to send the test report. eg, the >report message is lost or the MA loses power. > > >>>> If an MA is still up and knows that it won't run a test, it can certai= nly >send >>> a no-report message to the collector (so it knows there was an instruct= ion >>> sent that was not executed) so that no-report can be factored into post= - >facto >>> analysis. >>> >>> Right, that's case 2. >>> >>> What do you propose for case 3? >>> >>> >>>> But once the controller sends an instruction, why is it involved in th= e >>> further fate of the measurement task? >>> >>> Not so much in the fate of that specific task per se, but of the overal= l >>> measurement. eg, depending on the failure reason, it could be that the >test >>> should be modified or a different test run. Or perhaps an action raised= to >>> upgrade the failing set of MAs. >>> >>> >>> Suppose Benoit tells me to write a report and send it to you. Normally = I >say >>> "OK" and do it. But I might say "No", in which case we negotiate a >different >>> report or timeframe. Or I might say "OK" then just not do it, so you >receive >>> nothing. You don't even know that you didn't receive the report, becaus= e >>> nobody told you to expect it - so when Benoit comes to discuss it with >you, >>> you're clueless. Not good. >>> >>> So normally Benoit would tell you, "I've asked Paul to send you the >report" >>> so you'd be expecting it. Then you can check back with either of us if = you >>> didn't receive it. >>> >>> Or, Benoit would periodically check with us to see whether I finished t= he >>> report and whether you received it. >>> >>> LMAP should work like this too. >> >> It's a big database with lots of data. I don't see the Collector should = be >trying to police the data it is receiving. It would have to know all of th= e >schedules of different MAs and implement timeouts etc. We want to keep >the Collector simple. Another motivation is if the Collector is a 3rd part= y who >we simply want to ship data to. > >The shipped data doesn't arrive. Who's responsible for a) detecting, and >b) addressing that? It's shipped using HTTP for example. The MA can have a retry/failover strat= egy. Trevor. From philip.eardley@bt.com Fri Jul 26 08:52:39 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0728621F9A1E for ; Fri, 26 Jul 2013 08:52:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.163 X-Spam-Level: X-Spam-Status: No, score=-103.163 tagged_above=-999 required=5 tests=[AWL=-0.164, BAYES_00=-2.599, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kUTT96rfN1vR for ; Fri, 26 Jul 2013 08:52:31 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id 0577221F9994 for ; Fri, 26 Jul 2013 08:52:28 -0700 (PDT) Received: from EVMHT62-UKRD.domain1.systemhost.net (10.36.3.128) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.298.1; Fri, 26 Jul 2013 16:52:28 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.2.130]) by EVMHT62-UKRD.domain1.systemhost.net ([10.36.3.128]) with mapi; Fri, 26 Jul 2013 16:52:27 +0100 From: To: Date: Fri, 26 Jul 2013 16:52:26 +0100 Thread-Topic: [lmap] my comments on draft-akhter-lmap-framework-00 Thread-Index: Ac6KDdJTRZoN1FyJSs2nrmSs9IEFWQACRrGQ Message-ID: <9510D26531EF184D9017DF24659BB87F35CA37F59F@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA1287FAD3@AZ-FFEXMB04.global.avaya.com> <51F1977E.1040108@cisco.com> <9510D26531EF184D9017DF24659BB87F35CA37F461@EMV65-UKRD.domain1.systemhost.net> <51F289E5.2060309@cisco.com> In-Reply-To: <51F289E5.2060309@cisco.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: dromasca@avaya.com, lmap@ietf.org Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 15:52:39 -0000 > -----Original Message----- > From: Paul Aitken [mailto:paitken@cisco.com] > Sent: 26 July 2013 15:38 > To: Eardley,PL,Philip,TUB8 R > Cc: dromasca@avaya.com; lmap@ietf.org > Subject: Re: [lmap] my comments on draft-akhter-lmap-framework-00 >=20 > Philip, >=20 > > The constraint proposed in eardley-lmap-framework is that a MA gets > instructions from one Controller at any point in time. >=20 > While I know what you mean, if it's a single CPU (or at least, a single > MA process) then it's necessarily only getting instructions from one > Controller at a time - although there could be different controllers > every few milliseconds... >=20 > Why do we fear multiple controllers? By the LMAP charter, they're under > control of a single organisation, so they should not be issuing > contradictory instructions. I think there's less chance of going wrong=20 It's more obvious what to do in nasty cases like juergen's examples It keeps the MA simpler - easier for the MA to handle if it does (reply to = instruction with reject code) I don't see that multiple controllers gains you anything. >=20 >=20 > > If the operator of the measurement chooses to implement the > Controller function in a distributed way then it can. But I think this > should be left as internals to how the Controller is implemented - ie > we shouldn't worry about it when designing the Control Protocol. > > > > At some point we do have to think about what happens if the > Controller fails, and to what extent the WG solves the issue. I don't > think that point is now. >=20 > If it's not mentioned early on, then either the accepted design will > prevent it from being added later, or it'll later be seen as a new > requirement and rejected. Surely we've all seen this many times? As trevor points out to me, the information model i-d already provides resi= lience. The bootstrap process informs the MA about its Controller, this inf= ormation is a FQDN, therefore we get resilience from dns. Similarly for col= lector. Not sure we need to do anything more complicated. Maybe something l= ike, if MA finishes all its instruction, then check in with controller?? >=20 > P. From nagami@wide.ad.jp Fri Jul 26 11:26:04 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3443F21F9A49 for ; Fri, 26 Jul 2013 11:26:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.728 X-Spam-Level: X-Spam-Status: No, score=-1.728 tagged_above=-999 required=5 tests=[AWL=-0.250, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, GB_ABOUTYOU=0.5, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hOOCsljKpAcx for ; Fri, 26 Jul 2013 11:26:03 -0700 (PDT) Received: from sh.wide.ad.jp (sh.wide.ad.jp [IPv6:2001:200:0:1001::6]) by ietfa.amsl.com (Postfix) with ESMTP id 2F69D21F9A15 for ; Fri, 26 Jul 2013 11:26:02 -0700 (PDT) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (authenticated (0 bits)) by mail.wide.ad.jp (8.14.1+3.5Wbeta/8.14.1/smtpfeed 1.21) with ESMTP id r6QIQ0pF009549 (using TLSv1/SSLv3 with cipher RC4-SHA (128 bits) verified FAIL) for ; Sat, 27 Jul 2013 03:26:01 +0900 (JST) Received: by mail-pd0-f170.google.com with SMTP id x11so3217679pdj.1 for ; Fri, 26 Jul 2013 11:25:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=actiSfmMWcQOxxl4VupLxuD697AwbYfCK7pSvQfB82k=; b=WZRRxZ115N8T8ENHfvLhpsVqb44e4wk4aZVfhU23Mf6Wbhoj2+jUfV191FndbgElge G0PwNVfXrZlkxvxyzJgXPu513ysee3J+MkFYRFJGeIb4iphh0FuV+tZIt7DIUFeTltCo N7ftQeQ59kyC6NwNM7LeEGUhf8nvRE/ryIsbm/TNcFEyuHziG5rG/6LvSlUSwlrFO1on drY406B/Qu7AAlP1L8IJ5YlZjeevL8w8JrYmKnxwfsksW5/y6Vx2lYioIAGs8uQepfc/ 1hhoWrDvqq+r5eELOenAk8fHYd5F3ZJ88g3nT1Jq2VXeqlW3mU2q/qxu9BRjhEe0Rf+x c1Bw== MIME-Version: 1.0 X-Received: by 10.66.232.101 with SMTP id tn5mr51086789pac.132.1374863154932; Fri, 26 Jul 2013 11:25:54 -0700 (PDT) Received: by 10.70.102.176 with HTTP; Fri, 26 Jul 2013 11:25:54 -0700 (PDT) In-Reply-To: <9510D26531EF184D9017DF24659BB87F35B80337A5@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA12881146@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337A5@EMV65-UKRD.domain1.systemhost.net> Date: Sat, 27 Jul 2013 03:25:54 +0900 Message-ID: From: Nagami Kenichi To: "philip.eardley@bt.com" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "Romascanu, Dan \(Dan\)" , lmap WG Subject: Re: [lmap] on use cases and requirements X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 18:26:04 -0000 Phil, > http://tools.ietf.org/html/draft-nagami-lmap-use-case-measurement-provide= r-00 > you talk about a "measurement provider" - if I get it right, this is some= one who provides information about different ISPs. this seems to me very si= milar to the regulator use case - but it's a public interest group, or a co= mpany providing the information instead of a regulator. It's probably worth= pointing this out in http://tools.ietf.org/html/draft-linsner-lmap-use-cas= es - would this then cover your use case? I think a use case for a regulator is similar to a measurement provider. We were thinking the following case. Multiple measurement providers can measure a performance from MAs in the users to MAs in the multiple content providers. The measurement provider publishes the results that are easy to understand for users. It is necessary to show the measurement parameters in order to allow comparison with the result of the other. For example, the parameters are how to measure a performance, measure a performance from where to where, etc. If a use case for a regulator is same as this, I would like to add "a public interest group, or a company providing the information instead of a regulator" for clarity. And it is necessary to show measurement parameters as well as the measurement result. Regards, Ken Nagami > > You also mention a smartphone, I think it's worth adding this explicitly = to draft-linsner > > You also have a lot of interesting details about your deployment. My feel= ing is that it would take a lot of work to summarise all the state-of-the-a= rt, so we shouldn't try (perhaps we could add a few references in one of th= e docs) > > > Rachel, > http://tools.ietf.org/id/draft-huang-lmap-data-collection-use-case-00.txt > I take this as a slight expansion of the ISP use case described in http:/= /tools.ietf.org/html/draft-linsner-lmap-use-cases > Could we work with you on adding a little text about simulations to our d= raft? - ie the ISP could use measurements to feed its simulation tools (ass= uming it uses simulations as part of the way it works out where a fault is,= or where to add capacity, etc). > Will you be in Berlin? Marc, Ken and I are meeting in the lmap breakfast = to discuss use cases. > > > Dan, > Personally I'm happy to drop as much of Section 3 as people want (even al= l of it). > I agree we should think carefully in terms of what the output of lmap nee= ds to do - what are the most important aspects of which use cases. > Reading between the lines, you think the use cases doc should only cover = those aspects that we're going to (try to) make sure we solve during the in= itial lmap charter. And not be a general use case doc that would need futur= e lmap capability. > Is that right? > (I'm ok if the answer is 'yes') > > Thanks > phil > >> -----Original Message----- >> From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of >> Romascanu, Dan (Dan) >> Sent: 25 July 2013 13:40 >> To: lmap@ietf.org >> Subject: [lmap] on use cases and requirements >> >> If you have a look at the updated agenda at >> https://datatracker.ietf.org/meeting/87/agenda/lmap/ you will realized >> that Jason and me slightly updated the section dedicated to use cases >> and added ten minutes of discussion. Let me try to explain the >> motivation and what we would try to achieve. According to the charter >> in a couple of months the WG will need to submit the first WG Internet- >> Drafts on Use Cases and Framework. These two documents should provide >> clear enough requirements for the following phases where we need to >> proceed to the creation of the Information Model, and discuss about the >> selection or development of protocols for Control and Reporting and >> their associated data models. At this moment in time we have one draft- >> linsner-lmap-use-cases-03 as principal document on use cases, and two >> new contributions, which is good for this phase. What we need to start >> doing is to sort out of the use cases the subset that matches the >> current WG charter - and we may need to drop or prioritize some of the >> aspects of the use cases for this purpose. >> >> (To be more specific and take this just as an example, I am not sure >> that all the characteristics described in Sections 3.2, 3.3, 3.5 in >> draft-linsner-lmap-use-cases-03 are LMAP problems or problems that >> belong to the requirements of the first phase of LMAP.) >> >> Regards, >> >> Dan >> >> >> >> >> >> >> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > From aakhter@cisco.com Fri Jul 26 15:58:19 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BA5CA11E8178 for ; Fri, 26 Jul 2013 15:58:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.598 X-Spam-Level: X-Spam-Status: No, score=-110.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NBV7ksyN5nrf for ; Fri, 26 Jul 2013 15:58:13 -0700 (PDT) Received: from rcdn-iport-2.cisco.com (rcdn-iport-2.cisco.com [173.37.86.73]) by ietfa.amsl.com (Postfix) with ESMTP id 793D211E8162 for ; Fri, 26 Jul 2013 15:58:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=19515; q=dns/txt; s=iport; t=1374879493; x=1376089093; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=uf4ZUqbdcUsp3skzfhTQWi81784PSn5/DzCSkihp4Gw=; b=JeEbiKGDo2EgZTay4alnZhq2eFKM6oAQlGGvLqy+6vkbsM5FB7BtNd8g /10+t592oHI/tKdFKYEAGEdmUYRp8alH0RoygFoSO7gks2UAbR3MU3PXf 9PT1oo2QSU3qGRwZCZHNuQloeKDgY4imX6zetuz/K6PUnGtwgMfNyzOLj w=; X-IronPort-AV: E=Sophos;i="4.89,754,1367971200"; d="scan'208,217";a="240094812" Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by rcdn-iport-2.cisco.com with ESMTP; 26 Jul 2013 22:58:12 +0000 Received: from xhc-aln-x15.cisco.com (xhc-aln-x15.cisco.com [173.36.12.89]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id r6QMwCbr007405 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 26 Jul 2013 22:58:12 GMT Received: from xmb-rcd-x15.cisco.com ([169.254.5.80]) by xhc-aln-x15.cisco.com ([173.36.12.89]) with mapi id 14.02.0318.004; Fri, 26 Jul 2013 17:58:11 -0500 From: "Aamer Akhter (aakhter)" To: "philip.eardley@bt.com" , "lmap@ietf.org" Thread-Topic: more comments on draft-akhter-lmap-framework Thread-Index: Ac6KAlnFRWmvDRXbSs6U8NaTz3KU5gATHF6Q Date: Fri, 26 Jul 2013 22:58:11 +0000 Message-ID: <75C0E47A1889264493A2DCB2869AC0963337932E@xmb-rcd-x15.cisco.com> References: <9510D26531EF184D9017DF24659BB87F35CA37F523@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F35CA37F523@EMV65-UKRD.domain1.systemhost.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [64.102.40.39] Content-Type: multipart/alternative; boundary="_000_75C0E47A1889264493A2DCB2869AC0963337932Exmbrcdx15ciscoc_" MIME-Version: 1.0 Cc: "Paul Aitken \(paitken\)" Subject: Re: [lmap] more comments on draft-akhter-lmap-framework X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 22:58:19 -0000 --_000_75C0E47A1889264493A2DCB2869AC0963337932Exmbrcdx15ciscoc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Phil, Thanks for the comments. Inline with AA> From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of phi= lip.eardley@bt.com Sent: Friday, July 26, 2013 10:46 AM To: lmap@ietf.org Subject: [lmap] more comments on draft-akhter-lmap-framework My final batch of comments First, I think some of the material in Section 3 could be helpful expansion= (on what was rather briefly mentioned in draft-eardley-lmap-framework). Se= ctions 4 & 5 have lots of interesting stuff - question is whether this is a= general framework for large-scale measurements (where we need to talk in d= etail about the tests) or whether it's a framework for LMAP WG (which only = needs a brief mention of areas outside the scope of lmap wg, and so wouldn'= t have much about the tests, as this is ippm). AA> Thanks and do not disagree completely. However, when it comes to the me= thods used for controlling MA to MA we do need to be mindful of what the op= tions are as well as the tradeoffs. I do like the earlier suggestion (I thi= nk from Dan) that suggests separating out the properties of the active test= control protocol that we (in the LMAP case) would need to worry about. Thi= s might even suggest if there is even a direct MA to MA test control channe= l or if it all needs to go via the controller. Secondly, there were several comments about multiple measurement agents (& = peers?) acting together. (Section 3.2 & 6.2.1). you mention several differe= nt cases, I'm not sure I understood properly or got them all. AA> The multiple measurement agent case is when there is no anchor measurem= ent agent (or the anchor measurement agent does not have enough capacity to= properly saturate the path-in the case of a path capacity measurement). Th= e lack of an anchor measurement agent may come about when the regulator is = trying to make an access link test (within the ISP) but as it's a non-ISP e= ntity it would not have test resources that are well bandwidth connected wi= thin the ISP. Also, the regulator does not want to stress the ISP upstream = interface. If we were to pool the resources of multiple ISP connected MAs (= sitting in other broadband sites) we would be able create the saturation w/= o stressing the other broadband sites links. I put a diagram in the present= ation that Paul will be presenting that might be more helpful. (1) I'm sure there's a case where one test triggers another (say, the dns r= esult followed by request to website) - this is ok. (2) I think you have a 'latency under load' test (S3.2), where the latency = is measured for a pkt/file being sent between MA & Measurement Peer, but an= other Peer is generating other traffic so that a bottleneck queue is loaded= . I haven't thought about this much - do you think there are other tests in= volving a second Peer? or even 3rd, 4th...? Or is it just 'latency under lo= ad'? I'm wondering whether it would be best to think about the specifics o= f how to do the 'latency under load' test, rather than a general capability= for coordinating multiple Peers in one test. AA> it was meant to just be an aggregate path capacity measurement. Sorry i= t was not clear-I think this section needs more verbiage etc. (3) you mention (S6.2.1) a case where multiple Measurement Agents submit re= sults all with the same Measurement Task ID. I guess this is a test where l= ots of coordinated measurements are made. You also mention a shared key bet= ween multiple MAs, which I think must be related. I don't understand this c= ase, please explain more. Am interested to understand this whole area better, regardless of where the= WG decides to draw the line between in scope & out. AA> The measurement task ID is simply the index that is used to collate the= individual results submitted by the MAs post-collection. This would be whe= n the MAs are all given the same instruction etc.. I did a search in the do= cument for 'shared key' and this is only in S4.2.2. This shared key is comp= letely different and is a security device used to secure test control proto= col (eg. OWAMP) communications between the MAs. Third, you have some stuff about task scheduling where it seems important t= o have highly synchronised time between the controller, MAs and collector (= and measurement peer?). why would this be needed? You mention about NTP and= the issue that NTP doesn't give very accurate time sync to a device on the= end of an access line (because you can't triangulate to multiple time serv= ers). I suppose the error might be a few millisecs - the sorts of cases I h= ad in mind it would easily be accurate enough. Incidentally I don't see why= your solution (NTP clock sent from controller) helps, since you still have= the lack of triangulation on the end of an access line. AA> The context of the time synchronization was when (for example) a one-wa= y-latency test is run between two MAs. In the case of a round-trip-time tes= t such time synchronization would not be needed, and of course we simply ca= n't do RTT/2. The time concerns are really just bubbling up from the OWAMP = spec, but I believe them to be valid. AA> regarding the NTP source as the controller - this was simply an attempt= to get the MAs on to the same time hierarchy rather than differing unknown= ones. It doesn't really have to be the controller itself, just known time = source that everybody agrees on. Will clarify. AA> that said ( I don't believe this is in the draft-aakhter-lmap-framework= ) I do see some value in having the collector being loosely (vs tightly) sy= nchronised with the MAs. This could be used as a way to detect a MA that is= completely out of time sync when it submits its reports. Perhaps useful...= . Thanks again, aa Thanks Best wishes phil --_000_75C0E47A1889264493A2DCB2869AC0963337932Exmbrcdx15ciscoc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Phil,

 

Thanks for the comment= s.

 

Inline with AA>

 

From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of philip.eardley@bt.com
Sent: Friday, July 26, 2013 10:46 AM
To: lmap@ietf.org
Subject: [lmap] more comments on draft-akhter-lmap-framework

 

My final batch of comments=

 

First, I think some of the= material in Section 3 could be helpful expansion (on what was rather brief= ly mentioned in draft-eardley-lmap-framework). Sections 4 & 5 have lots of interesting stuff – question is whether this is= a general framework for large-scale measurements (where we need to talk in= detail about the tests) or whether it’s a framework for LMAP WG (whi= ch only needs a brief mention of areas outside the scope of lmap wg, and so wouldn’t have much about the tests, as this= is ippm).  

 <= /o:p>

AA> = Thanks and do not disagree completely. However, when it comes to the method= s used for controlling MA to MA we do need to be mindful of what the option= s are as well as the tradeoffs. I do like the earlier suggestion (I think from Dan) that suggests separating out the= properties of the active test control protocol that we (in the LMAP case) = would need to worry about. This might even suggest if there is even a direc= t MA to MA test control channel or if it all needs to go via the controller.

&n= bsp;

&n= bsp;

Secondly, there were sever= al comments about multiple measurement agents (& peers?) acting togethe= r. (Section 3.2 & 6.2.1). you mention several different cases, I’m not sure I understood properly or got them all.

 

AA> The multiple measurement= agent case is when there is no anchor measurement agent (or the anchor mea= surement agent does not have enough capacity to properly saturate the path&= #8212;in the case of a path capacity measurement). The lack of an anchor measurement agent may come about when the regulator = is trying to make an access link test (within the ISP) but as it’s a = non-ISP entity it would not have test resources that are well bandwidth con= nected within the ISP. Also, the regulator does not want to stress the ISP upstream interface. If we were to pool the= resources of multiple ISP connected MAs (sitting in other broadband sites)= we would be able create the saturation w/o stressing the other broadband s= ites links. I put a diagram in the presentation that Paul will be presenting that might be more helpful.

 

(1) I’m sure there&#= 8217;s a case where one test triggers another (say, the dns result followed= by request to website) – this is ok.

(2) I think you have a = 216;latency under load’ test (S3.2), where the latency is measured fo= r a pkt/file being sent between MA & Measurement Peer, but another Peer is generating other traffic so that a bottleneck queue is loaded. I haven&= #8217;t thought about this much – do you think there are other tests = involving a second Peer? or even 3rd, 4th…? Or = is it just ‘latency under load’?  I’m wondering whet= her it would be best to think about the specifics of how to do the ‘late= ncy under load’ test, rather than a general capability for coordinati= ng multiple Peers in one test.

 

AA> it was meant to just be = an aggregate path capacity measurement. Sorry it was not clear—I thin= k this section needs more verbiage etc.

 

(3) you mention (S6.2.1) a= case where multiple Measurement Agents submit results all with the same Me= asurement Task ID. I guess this is a test where lots of coordinated measurements are made. You also mention a shared key between multiple MAs,= which I think must be related. I don’t understand this case, please = explain more.

Am interested to understan= d this whole area better, regardless of where the WG decides to draw the li= ne between in scope & out.

 

AA> The measurement task ID = is simply the index that is used to collate the individual results submitte= d by the MAs post-collection. This would be when the MAs are all given the = same instruction etc.. I did a search in the document for ‘shared key’ and this is only in S4.2.2. This= shared key is completely different and is a security device used to secure= test control protocol (eg. OWAMP) communications between the MAs.

 

 

Third, you have some stuff= about task scheduling where it seems important to have highly synchronised= time between the controller, MAs and collector (and measurement peer?). why would this be needed? You mention about NTP and the issue that= NTP doesn’t give very accurate time sync to a device on the end of a= n access line (because you can’t triangulate to multiple time servers= ). I suppose the error might be a few millisecs - the sorts of cases I had in mind it would easily be accurate enough. Inc= identally I don’t see why your solution (NTP clock sent from controll= er) helps, since you still have the lack of triangulation on the end of an = access line.

 

AA> The context of the time = synchronization was when (for example) a one-way-latency test is run betwee= n two MAs. In the case of a round-trip-time test such time synchronization = would not be needed, and of course we simply can’t do RTT/2. The time concerns are really just bubbling up from t= he OWAMP spec, but I believe them to be valid.

 

AA> regarding the NTP source= as the controller – this was simply an attempt to get the MAs on to = the same time hierarchy rather than differing unknown ones. It doesn’= t really have to be the controller itself, just known time source that everybody agrees on. Will clarify.

 

AA> that said ( I don’= t believe this is in the draft-aakhter-lmap-framework) I do see some value = in having the collector being loosely (vs tightly) synchronised with the MA= s. This could be used as a way to detect a MA that is completely out of time sync when it submits its reports. Perhaps u= seful….

 

Thanks again,=

aa

 

 

Thanks

Best wishes

phil

 

--_000_75C0E47A1889264493A2DCB2869AC0963337932Exmbrcdx15ciscoc_-- From philip.eardley@bt.com Sun Jul 28 10:10:10 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 249F621F9C3D for ; Sun, 28 Jul 2013 10:10:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.152 X-Spam-Level: X-Spam-Status: No, score=-103.152 tagged_above=-999 required=5 tests=[AWL=-0.154, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K9RkMVwNp72Y for ; Sun, 28 Jul 2013 10:10:04 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id 273D921F9B98 for ; Sun, 28 Jul 2013 10:10:01 -0700 (PDT) Received: from EVMHT61-UKRD.domain1.systemhost.net (10.36.3.127) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.298.1; Sun, 28 Jul 2013 18:10:00 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.32]) by EVMHT61-UKRD.domain1.systemhost.net ([10.36.3.127]) with mapi; Sun, 28 Jul 2013 18:09:59 +0100 From: To: , Date: Sun, 28 Jul 2013 18:09:59 +0100 Thread-Topic: more comments on draft-akhter-lmap-framework Thread-Index: Ac6KAlnFRWmvDRXbSs6U8NaTz3KU5gATHF6QAFi3DJw= Message-ID: <9510D26531EF184D9017DF24659BB87F35CB3F1EA2@EMV65-UKRD.domain1.systemhost.net> References: <9510D26531EF184D9017DF24659BB87F35CA37F523@EMV65-UKRD.domain1.systemhost.net>, <75C0E47A1889264493A2DCB2869AC0963337932E@xmb-rcd-x15.cisco.com> In-Reply-To: <75C0E47A1889264493A2DCB2869AC0963337932E@xmb-rcd-x15.cisco.com> Accept-Language: en-US, en-GB Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F35CB3F1EA2EMV65UKRDdoma_" MIME-Version: 1.0 Cc: paitken@cisco.com Subject: Re: [lmap] more comments on draft-akhter-lmap-framework X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Jul 2013 17:10:10 -0000 --_000_9510D26531EF184D9017DF24659BB87F35CB3F1EA2EMV65UKRDdoma_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable thanks, a few more follow-ups! In the terminology draft we tried to distinguish the "Measurement Agent" an= d the "Measurement Peer". The MA interacts with the Controller and Collecto= r, whilst the MP only interacts with the MA. below you talk about MA to MA comms and control channel. i'm not sure if th= is is simply a terminology difference (ie what I'd call MA-MP comms - OWAMP= , IPPM stuff etc) or whether you also want comms between (what I term) two = MAs? you mention a test that 'saturates the path'. how to achieve this depends w= here the bottleneck is expected to be? if the access link, then the MA has = got to be involved? thanks for the picture in your sldes for the meeting - that will be very he= lpful i think your Measurement Task ID is similar to the Cycle ID in draft-eardle= y-terminology? Cycle-ID: A tag that is sent by the Controller in an Instruction and echoed by the MA in its Report; Measurement Results with the same Cycle-ID are expected to be comparable. The purpose of the Cycle-ID is to allow the data analysis tools to identify easily Measurement Results that are expected to be comparable, typically because the associated Measurement Tasks all operate the same Measurement Method with the same values for its parameters. This set of Measurement Tasks could be termed the Measurement Cycle. [I'd been thinking that a specific Cycle-ID would apply only to a particula= r MA, but I guess there's no reason why a Controller shouldn't send the sam= e Cycle-ID to more than one MA, if it knows that the Measurement Result wil= l be comparable] thanks phil ________________________________ From: Aamer Akhter (aakhter) [aakhter@cisco.com] Sent: 26 July 2013 23:58 To: Eardley,PL,Philip,TUB8 R; lmap@ietf.org Cc: Paul Aitken (paitken) Subject: RE: more comments on draft-akhter-lmap-framework Hi Phil, Thanks for the comments. Inline with AA> From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of phi= lip.eardley@bt.com Sent: Friday, July 26, 2013 10:46 AM To: lmap@ietf.org Subject: [lmap] more comments on draft-akhter-lmap-framework My final batch of comments First, I think some of the material in Section 3 could be helpful expansion= (on what was rather briefly mentioned in draft-eardley-lmap-framework). Se= ctions 4 & 5 have lots of interesting stuff =96 question is whether this is= a general framework for large-scale measurements (where we need to talk in= detail about the tests) or whether it=92s a framework for LMAP WG (which o= nly needs a brief mention of areas outside the scope of lmap wg, and so wou= ldn=92t have much about the tests, as this is ippm). AA> Thanks and do not disagree completely. However, when it comes to the me= thods used for controlling MA to MA we do need to be mindful of what the op= tions are as well as the tradeoffs. I do like the earlier suggestion (I thi= nk from Dan) that suggests separating out the properties of the active test= control protocol that we (in the LMAP case) would need to worry about. Thi= s might even suggest if there is even a direct MA to MA test control channe= l or if it all needs to go via the controller. Secondly, there were several comments about multiple measurement agents (& = peers?) acting together. (Section 3.2 & 6.2.1). you mention several differe= nt cases, I=92m not sure I understood properly or got them all. AA> The multiple measurement agent case is when there is no anchor measurem= ent agent (or the anchor measurement agent does not have enough capacity to= properly saturate the path=97in the case of a path capacity measurement). = The lack of an anchor measurement agent may come about when the regulator i= s trying to make an access link test (within the ISP) but as it=92s a non-I= SP entity it would not have test resources that are well bandwidth connecte= d within the ISP. Also, the regulator does not want to stress the ISP upstr= eam interface. If we were to pool the resources of multiple ISP connected M= As (sitting in other broadband sites) we would be able create the saturatio= n w/o stressing the other broadband sites links. I put a diagram in the pre= sentation that Paul will be presenting that might be more helpful. (1) I=92m sure there=92s a case where one test triggers another (say, the d= ns result followed by request to website) =96 this is ok. (2) I think you have a =91latency under load=92 test (S3.2), where the late= ncy is measured for a pkt/file being sent between MA & Measurement Peer, bu= t another Peer is generating other traffic so that a bottleneck queue is lo= aded. I haven=92t thought about this much =96 do you think there are other = tests involving a second Peer? or even 3rd, 4th=85? Or is it just =91latenc= y under load=92? I=92m wondering whether it would be best to think about t= he specifics of how to do the =91latency under load=92 test, rather than a = general capability for coordinating multiple Peers in one test. AA> it was meant to just be an aggregate path capacity measurement. Sorry i= t was not clear=97I think this section needs more verbiage etc. (3) you mention (S6.2.1) a case where multiple Measurement Agents submit re= sults all with the same Measurement Task ID. I guess this is a test where l= ots of coordinated measurements are made. You also mention a shared key bet= ween multiple MAs, which I think must be related. I don=92t understand this= case, please explain more. Am interested to understand this whole area better, regardless of where the= WG decides to draw the line between in scope & out. AA> The measurement task ID is simply the index that is used to collate the= individual results submitted by the MAs post-collection. This would be whe= n the MAs are all given the same instruction etc.. I did a search in the do= cument for =91shared key=92 and this is only in S4.2.2. This shared key is = completely different and is a security device used to secure test control p= rotocol (eg. OWAMP) communications between the MAs. Third, you have some stuff about task scheduling where it seems important t= o have highly synchronised time between the controller, MAs and collector (= and measurement peer?). why would this be needed? You mention about NTP and= the issue that NTP doesn=92t give very accurate time sync to a device on t= he end of an access line (because you can=92t triangulate to multiple time = servers). I suppose the error might be a few millisecs - the sorts of cases= I had in mind it would easily be accurate enough. Incidentally I don=92t s= ee why your solution (NTP clock sent from controller) helps, since you stil= l have the lack of triangulation on the end of an access line. AA> The context of the time synchronization was when (for example) a one-wa= y-latency test is run between two MAs. In the case of a round-trip-time tes= t such time synchronization would not be needed, and of course we simply ca= n=92t do RTT/2. The time concerns are really just bubbling up from the OWAM= P spec, but I believe them to be valid. AA> regarding the NTP source as the controller =96 this was simply an attem= pt to get the MAs on to the same time hierarchy rather than differing unkno= wn ones. It doesn=92t really have to be the controller itself, just known t= ime source that everybody agrees on. Will clarify. AA> that said ( I don=92t believe this is in the draft-aakhter-lmap-framewo= rk) I do see some value in having the collector being loosely (vs tightly) = synchronised with the MAs. This could be used as a way to detect a MA that = is completely out of time sync when it submits its reports. Perhaps useful= =85. Thanks again, aa Thanks Best wishes phil --_000_9510D26531EF184D9017DF24659BB87F35CB3F1EA2EMV65UKRDdoma_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
thanks, a few more follow-ups!
 
In the terminology draft we tried to distinguish= the "Measurement Agent" and the "Measurement Peer". Th= e MA interacts with the Controller and Collector, whilst the MP only intera= cts with the MA.
below you talk about MA to MA comms and control = channel. i'm not sure if this is simply a terminology difference (ie what I= 'd call MA-MP comms - OWAMP, IPPM stuff etc) or whether you also want comms= between (what I term) two MAs?
 
you mention a test that 'saturates the path'. ho= w to achieve this depends where the bottleneck is expected to be? if the ac= cess link, then the MA has got to be involved?
thanks for the picture in your sldes for the mee= ting - that will be very helpful
 
i think your Measurement Task ID is similar= to the Cycle ID in draft-eardley-terminology?
Cycle-ID: A tag that is sent by the Controller in an Instruction and    echoed by the MA in its Report; Measurement Results with the s= ame
   Cycle-ID are expected to be comparable.

   The purpose of the Cycle-ID is to allow the data analysis= tools to
   identify easily Measurement Results that are expected to be    comparable, typically because the associated Measurement Tasks= all
   operate the same Measurement Method with the same values for i= ts
   parameters.  This set of Measurement Tasks could be terme= d the
   Measurement Cycle.

 

[I'd been thinking that a specific Cycle-ID would = apply only to a particular MA, but I guess there's no reason why a Controll= er shouldn't send the same Cycle-ID to more than one MA, if it knows that t= he Measurement Result will be comparable]

 

thanks

phil


From: Aamer = Akhter (aakhter) [aakhter@cisco.com]
Sent: 26 July 2013 23:58
To: Eardley,PL,Philip,TUB8 R; lmap@ietf.org
Cc: Paul Aitken (paitken)
Subject: RE: more comments on draft-akhter-lmap-framework

Hi Phil,

 

Thanks for the commen= ts.

 

Inline with AA>

 

From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of philip.eardley@bt.com
Sent: Friday, July 26, 2013 10:46 AM
To: lmap@ietf.org
Subject: [lmap] more comments on draft-akhter-lmap-framework

 

My final batch of comments

 

First, I think some of the material in Sectio= n 3 could be helpful expansion (on what was rather briefly mentioned in dra= ft-eardley-lmap-framework). Sections 4 & 5 have lots of interesting stuff =96 question is whether this is a g= eneral framework for large-scale measurements (where we need to talk in det= ail about the tests) or whether it=92s a framework for LMAP WG (which only = needs a brief mention of areas outside the scope of lmap wg, and so wouldn=92t have much about the tests, as this is = ippm).  

 

AA>= Thanks and do not disagree completely. However, when it comes to the metho= ds used for controlling MA to MA we do need to be mindful of what the optio= ns are as well as the tradeoffs. I do like the earlier suggestion (I think from Dan) that suggests separating out the= properties of the active test control protocol that we (in the LMAP case) = would need to worry about. This might even suggest if there is even a direc= t MA to MA test control channel or if it all needs to go via the controller.

 

 

Secondly, there were several comments about m= ultiple measurement agents (& peers?) acting together. (Section 3.2 &am= p; 6.2.1). you mention several different cases, I=92m not sure I understood properly or got them all.

 

AA> The multiple measurement= agent case is when there is no anchor measurement agent (or the anchor mea= surement agent does not have enough capacity to properly saturate the path= =97in the case of a path capacity measurement). The lack of an anchor measurement agent may come about when the regulator = is trying to make an access link test (within the ISP) but as it=92s a non-= ISP entity it would not have test resources that are well bandwidth connect= ed within the ISP. Also, the regulator does not want to stress the ISP upstream interface. If we were to pool the= resources of multiple ISP connected MAs (sitting in other broadband sites)= we would be able create the saturation w/o stressing the other broadband s= ites links. I put a diagram in the presentation that Paul will be presenting that might be more helpful.

 

(1) I=92m sure there=92s a case where one tes= t triggers another (say, the dns result followed by request to website) =96= this is ok.

(2) I think you have a =91latency under load= =92 test (S3.2), where the latency is measured for a pkt/file being sent be= tween MA & Measurement Peer, but another Peer is generating other traffic so that a bottleneck queue is loaded. I haven= =92t thought about this much =96 do you think there are other tests involvi= ng a second Peer? or even 3rd, 4th=85? Or is it just = =91latency under load=92?  I=92m wondering whether it would be best to think about the specifics of how to do the =91latency = under load=92 test, rather than a general capability for coordinating multi= ple Peers in one test.

 

AA> it was meant to just be = an aggregate path capacity measurement. Sorry it was not clear=97I think th= is section needs more verbiage etc.

 

(3) you mention (S6.2.1) a case where multipl= e Measurement Agents submit results all with the same Measurement Task ID. = I guess this is a test where lots of coordinated measurements are made. You also mention a shared key between multiple MAs,= which I think must be related. I don=92t understand this case, please expl= ain more.

Am interested to understand this whole area b= etter, regardless of where the WG decides to draw the line between in scope= & out.

 

AA> The measurement task ID = is simply the index that is used to collate the individual results submitte= d by the MAs post-collection. This would be when the MAs are all given the = same instruction etc.. I did a search in the document for =91shared key=92 and this is only in S4.2.2. This shared = key is completely different and is a security device used to secure test co= ntrol protocol (eg. OWAMP) communications between the MAs.

 

 

Third, you have some stuff about task schedul= ing where it seems important to have highly synchronised time between the c= ontroller, MAs and collector (and measurement peer?). why would this be needed? You mention about NTP and the issue that= NTP doesn=92t give very accurate time sync to a device on the end of an ac= cess line (because you can=92t triangulate to multiple time servers). I sup= pose the error might be a few millisecs - the sorts of cases I had in mind it would easily be accurate enough. Inc= identally I don=92t see why your solution (NTP clock sent from controller) = helps, since you still have the lack of triangulation on the end of an acce= ss line.

 

AA> The context of the time = synchronization was when (for example) a one-way-latency test is run betwee= n two MAs. In the case of a round-trip-time test such time synchronization = would not be needed, and of course we simply can=92t do RTT/2. The time concerns are really just bubbling up from the O= WAMP spec, but I believe them to be valid.

 

AA> regarding the NTP source= as the controller =96 this was simply an attempt to get the MAs on to the = same time hierarchy rather than differing unknown ones. It doesn=92t really= have to be the controller itself, just known time source that everybody agrees on. Will clarify.

 

AA> that said ( I don=92t be= lieve this is in the draft-aakhter-lmap-framework) I do see some value in h= aving the collector being loosely (vs tightly) synchronised with the MAs. T= his could be used as a way to detect a MA that is completely out of time sync when it submits its reports. Perhaps u= seful=85.

 

Thanks again,

aa

 

 

Thanks

Best wishes

phil

 

--_000_9510D26531EF184D9017DF24659BB87F35CB3F1EA2EMV65UKRDdoma_-- From aakhter@cisco.com Sun Jul 28 16:21:29 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 412A021F9EC4 for ; Sun, 28 Jul 2013 16:21:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -109.998 X-Spam-Level: X-Spam-Status: No, score=-109.998 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ni3etZoga3O9 for ; Sun, 28 Jul 2013 16:21:23 -0700 (PDT) Received: from rcdn-iport-6.cisco.com (rcdn-iport-6.cisco.com [173.37.86.77]) by ietfa.amsl.com (Postfix) with ESMTP id 1C2ED21F9EAA for ; Sun, 28 Jul 2013 16:21:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=84026; q=dns/txt; s=iport; t=1375053682; x=1376263282; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=7RTXtsksB+z5TsdbTtMupz9Z62xizUweB/DppSqTRWU=; b=mBXGPP77CgxOeJv+ewoAvS1gPEfC142z1XcZFbDU38syscVXde+tt2n7 etYGEyifg6xwTKbp1GZtdInWrAK0Q1naemAm5p+w8hQ4B+ODltEsUh/Sv etZ+lWbUIfH+g7YoEibS9NvGtPByUS/rrXFR/8nDGRkDCMmvA5H10/785 c=; X-IronPort-AV: E=Sophos;i="4.89,765,1367971200"; d="scan'208,217";a="240597832" Received: from rcdn-core-6.cisco.com ([173.37.93.157]) by rcdn-iport-6.cisco.com with ESMTP; 28 Jul 2013 23:21:19 +0000 Received: from xhc-rcd-x03.cisco.com (xhc-rcd-x03.cisco.com [173.37.183.77]) by rcdn-core-6.cisco.com (8.14.5/8.14.5) with ESMTP id r6SNLJY3006100 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 28 Jul 2013 23:21:19 GMT Received: from xmb-rcd-x15.cisco.com ([169.254.5.80]) by xhc-rcd-x03.cisco.com ([173.37.183.77]) with mapi id 14.02.0318.004; Sun, 28 Jul 2013 18:21:18 -0500 From: "Aamer Akhter (aakhter)" To: "philip.eardley@bt.com" , "lmap@ietf.org" Thread-Topic: more comments on draft-akhter-lmap-framework Thread-Index: Ac6KAlnFRWmvDRXbSs6U8NaTz3KU5gATHF6QAFi3DJwADUXjIA== Date: Sun, 28 Jul 2013 23:21:18 +0000 Message-ID: <75C0E47A1889264493A2DCB2869AC09633383423@xmb-rcd-x15.cisco.com> References: <9510D26531EF184D9017DF24659BB87F35CA37F523@EMV65-UKRD.domain1.systemhost.net>, <75C0E47A1889264493A2DCB2869AC0963337932E@xmb-rcd-x15.cisco.com> <9510D26531EF184D9017DF24659BB87F35CB3F1EA2@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F35CB3F1EA2@EMV65-UKRD.domain1.systemhost.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.116.25.28] Content-Type: multipart/alternative; boundary="_000_75C0E47A1889264493A2DCB2869AC09633383423xmbrcdx15ciscoc_" MIME-Version: 1.0 Cc: "Paul Aitken \(paitken\)" Subject: Re: [lmap] more comments on draft-akhter-lmap-framework X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Jul 2013 23:21:29 -0000 --_000_75C0E47A1889264493A2DCB2869AC09633383423xmbrcdx15ciscoc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Phil, Please see inline at AA> From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of phi= lip.eardley@bt.com Sent: Sunday, July 28, 2013 1:10 PM To: Aamer Akhter (aakhter); lmap@ietf.org Cc: Paul Aitken (paitken) Subject: Re: [lmap] more comments on draft-akhter-lmap-framework thanks, a few more follow-ups! In the terminology draft we tried to distinguish the "Measurement Agent" an= d the "Measurement Peer". The MA interacts with the Controller and Collecto= r, whilst the MP only interacts with the MA. below you talk about MA to MA comms and control channel. i'm not sure if th= is is simply a terminology difference (ie what I'd call MA-MP comms - OWAMP= , IPPM stuff etc) or whether you also want comms between (what I term) two = MAs? AA> Indeed, draft-akhter-lmap-arch has a similar distinction between the MA= (the one initiating the test) and Remote Measurement Test Target (RMTT). T= here can be various kinds of RMTTs (public websites, other remote MAs, othe= r anchor MAs etc.). When the draft mentions MA-remote MA (MA-MA), both the = MAs are part of the LMAP domain however one is the initiator and the other = the test collector/responder. There is still a question regarding the contr= ol channel in my mind due to the restrictions imposed by NATs and firewalls= . Based on this, I think we still need to figure out if the control channe= l needs to go via the controller or directly between the MA-remote MA. you mention a test that 'saturates the path'. how to achieve this depends w= here the bottleneck is expected to be? if the access link, then the MA has = got to be involved? thanks for the picture in your sldes for the meeting - that will be very he= lpful AA> please find the graphic at http://db.tt/CuUsrNy6 AA> the bottleneck is expected to be the access link and yes, the MA defini= tely needs to be involved. i think your Measurement Task ID is similar to the Cycle ID in draft-eardle= y-terminology? Cycle-ID: A tag that is sent by the Controller in an Instruction and echoed by the MA in its Report; Measurement Results with the same Cycle-ID are expected to be comparable. The purpose of the Cycle-ID is to allow the data analysis tools to identify easily Measurement Results that are expected to be comparable, typically because the associated Measurement Tasks all operate the same Measurement Method with the same values for its parameters. This set of Measurement Tasks could be termed the Measurement Cycle. [I'd been thinking that a specific Cycle-ID would apply only to a particula= r MA, but I guess there's no reason why a Controller shouldn't send the sam= e Cycle-ID to more than one MA, if it knows that the Measurement Result wil= l be comparable] AA> yes-I think it's similar, our intention was to provide an index across = multi-MA-results for a mass test that would have comparable results. This w= ould hopefully make it easier for the collector-analyzer to collate the inc= oming results. Task-ID is the same index that allows the controller-collect= or optimization. AA> I think it might still be helpful to have a cycle-id (would have called= it a 'MA job ID' if it's specifically that) on a per-MA case. thanks phil ________________________________ From: Aamer Akhter (aakhter) [aakhter@cisco.com] Sent: 26 July 2013 23:58 To: Eardley,PL,Philip,TUB8 R; lmap@ietf.org Cc: Paul Aitken (paitken) Subject: RE: more comments on draft-akhter-lmap-framework Hi Phil, Thanks for the comments. Inline with AA> From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of philip.eardley@bt.com Sent: Friday, July 26, 2013 10:46 AM To: lmap@ietf.org Subject: [lmap] more comments on draft-akhter-lmap-framework My final batch of comments First, I think some of the material in Section 3 could be helpful expansion= (on what was rather briefly mentioned in draft-eardley-lmap-framework). Se= ctions 4 & 5 have lots of interesting stuff - question is whether this is a= general framework for large-scale measurements (where we need to talk in d= etail about the tests) or whether it's a framework for LMAP WG (which only = needs a brief mention of areas outside the scope of lmap wg, and so wouldn'= t have much about the tests, as this is ippm). AA> Thanks and do not disagree completely. However, when it comes to the me= thods used for controlling MA to MA we do need to be mindful of what the op= tions are as well as the tradeoffs. I do like the earlier suggestion (I thi= nk from Dan) that suggests separating out the properties of the active test= control protocol that we (in the LMAP case) would need to worry about. Thi= s might even suggest if there is even a direct MA to MA test control channe= l or if it all needs to go via the controller. Secondly, there were several comments about multiple measurement agents (& = peers?) acting together. (Section 3.2 & 6.2.1). you mention several differe= nt cases, I'm not sure I understood properly or got them all. AA> The multiple measurement agent case is when there is no anchor measurem= ent agent (or the anchor measurement agent does not have enough capacity to= properly saturate the path-in the case of a path capacity measurement). Th= e lack of an anchor measurement agent may come about when the regulator is = trying to make an access link test (within the ISP) but as it's a non-ISP e= ntity it would not have test resources that are well bandwidth connected wi= thin the ISP. Also, the regulator does not want to stress the ISP upstream = interface. If we were to pool the resources of multiple ISP connected MAs (= sitting in other broadband sites) we would be able create the saturation w/= o stressing the other broadband sites links. I put a diagram in the present= ation that Paul will be presenting that might be more helpful. (1) I'm sure there's a case where one test triggers another (say, the dns r= esult followed by request to website) - this is ok. (2) I think you have a 'latency under load' test (S3.2), where the latency = is measured for a pkt/file being sent between MA & Measurement Peer, but an= other Peer is generating other traffic so that a bottleneck queue is loaded= . I haven't thought about this much - do you think there are other tests in= volving a second Peer? or even 3rd, 4th...? Or is it just 'latency under lo= ad'? I'm wondering whether it would be best to think about the specifics o= f how to do the 'latency under load' test, rather than a general capability= for coordinating multiple Peers in one test. AA> it was meant to just be an aggregate path capacity measurement. Sorry i= t was not clear-I think this section needs more verbiage etc. (3) you mention (S6.2.1) a case where multiple Measurement Agents submit re= sults all with the same Measurement Task ID. I guess this is a test where l= ots of coordinated measurements are made. You also mention a shared key bet= ween multiple MAs, which I think must be related. I don't understand this c= ase, please explain more. Am interested to understand this whole area better, regardless of where the= WG decides to draw the line between in scope & out. AA> The measurement task ID is simply the index that is used to collate the= individual results submitted by the MAs post-collection. This would be whe= n the MAs are all given the same instruction etc.. I did a search in the do= cument for 'shared key' and this is only in S4.2.2. This shared key is comp= letely different and is a security device used to secure test control proto= col (eg. OWAMP) communications between the MAs. Third, you have some stuff about task scheduling where it seems important t= o have highly synchronised time between the controller, MAs and collector (= and measurement peer?). why would this be needed? You mention about NTP and= the issue that NTP doesn't give very accurate time sync to a device on the= end of an access line (because you can't triangulate to multiple time serv= ers). I suppose the error might be a few millisecs - the sorts of cases I h= ad in mind it would easily be accurate enough. Incidentally I don't see why= your solution (NTP clock sent from controller) helps, since you still have= the lack of triangulation on the end of an access line. AA> The context of the time synchronization was when (for example) a one-wa= y-latency test is run between two MAs. In the case of a round-trip-time tes= t such time synchronization would not be needed, and of course we simply ca= n't do RTT/2. The time concerns are really just bubbling up from the OWAMP = spec, but I believe them to be valid. AA> regarding the NTP source as the controller - this was simply an attempt= to get the MAs on to the same time hierarchy rather than differing unknown= ones. It doesn't really have to be the controller itself, just known time = source that everybody agrees on. Will clarify. AA> that said ( I don't believe this is in the draft-aakhter-lmap-framework= ) I do see some value in having the collector being loosely (vs tightly) sy= nchronised with the MAs. This could be used as a way to detect a MA that is= completely out of time sync when it submits its reports. Perhaps useful...= . Thanks again, aa Thanks Best wishes phil --_000_75C0E47A1889264493A2DCB2869AC09633383423xmbrcdx15ciscoc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Phil,

 

Please see inline at AA>

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.= org] On Behalf Of philip.eardley@bt.com
Sent: Sunday, July 28, 2013 1:10 PM
To: Aamer Akhter (aakhter); lmap@ietf.org
Cc: Paul Aitken (paitken)
Subject: Re: [lmap] more comments on draft-akhter-lmap-framework

&nbs= p;

thanks, = a few more follow-ups!

 

In the t= erminology draft we tried to distinguish the "Measurement Agent" = and the "Measurement Peer". The MA interacts with the Controller and= Collector, whilst the MP only interacts with the MA.

below yo= u talk about MA to MA comms and control channel. i'm not sure if this is simply a terminology difference (ie what I'd call MA-MP comms - OWAMP, = IPPM stuff etc) or whether you also want comms between (what I term) two MA= s?

 

 I = think we still need to figure out if the control channel needs to go via th= e controller or directly between the MA-remote MA.=

you ment= ion a test that 'saturates the path'. how to achieve this depends where the bottleneck is expected to be? if the access link, then the MA has got = to be involved?

thanks f= or the picture in your sldes for the meeting - that will be very helpf= ul

http://db.tt/CuUsrNy6

 

i think&= nbsp;your Measurement Task ID is similar to the Cycle ID in draft-eardley-t= erminology?

Cycle-ID: A tag that is sent by the Controller in an Instruction and
   echoed by the MA in its Report; Measurement Results with the s= ame
   Cycle-ID are expected to be comparable.

 &n= bsp; The purpose of the Cycle-ID is to allow the data analysis tools to
   identify easily Measurement Results that are expected to be    comparable, typically because the associated Measurement Tasks= all
   operate the same Measurement Method with the same values for i= ts
   parameters.  This set of Measurement Tasks could be terme= d the
   Measurement Cycle.

 = ;

[I'd = been thinking that a specific Cycle-ID would apply only to a particular MA,= but I guess there's no reason why a Controller shouldn't send the same Cycle-ID to more than one MA, if it knows that the Measureme= nt Result will be comparable]

 

AA> yes—I think it’s similar, our intention was to provide an inde= x across multi-MA-results for a mass test that would have comparable result= s. This would hopefully make it easier for the collector-analyzer to collat= e the incoming results. Task-ID is the same index that allows the controller-collector optimization.

&= nbsp;

AA> I think it might still be helpful to have a cycle-id (would have called it= a ‘MA job ID’ if it’s specifically that) on a per-MA cas= e.

&= nbsp;

 = ;

thank= s

phil<= o:p>

&nb= sp;

<= span style=3D"mso-bookmark:_MailOriginal">From: Aamer Akhter (aakhter) [aakhter@cisco.com]
Sent: 26 July 2013 23:58
To: Eardley,PL,Philip,TUB8 R; lmap@ietf.org
Cc: Paul Aitken (paitken)
Subject: RE: more comments on draft-akhter-lmap-framework=

Hi Phil,=

 

Thanks for the comments.

 

Inline with AA>

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf= .org] On Behalf Of = philip.eardley@bt.com
Sent: Friday, July 26, 2013 10:46 AM
To: lmap@ietf.org
Subject: [lmap] more comments on draft-akhter-lmap-framework

 

My final batch of comm= ents

 

First, I think some of= the material in Section 3 could be helpful expansion (on what was rather briefly mentioned in draft-eardley-lmap-framework). Sections 4 = & 5 have lots of interesting stuff – question is whether this is = a general framework for large-scale measurements (where we need to talk in = detail about the tests) or whether it’s a framework for LMAP WG (which only needs a brief mention of areas outside t= he scope of lmap wg, and so wouldn’t have much about the tests, as th= is is ippm).  

 

AA> Thanks a= nd do not disagree completely. However, when it comes to the methods used f= or controlling MA to MA we do need to be mindful of what the options are as well as the tradeoffs. I do like the earlier su= ggestion (I think from Dan) that suggests separating out the properties of = the active test control protocol that we (in the LMAP case) would need to w= orry about. This might even suggest if there is even a direct MA to MA test control channel or if it all needs= to go via the controller.

 

 

Secondly, there were s= everal comments about multiple measurement agents (& peers?) acting together. (Section 3.2 & 6.2.1). you mention several different = cases, I’m not sure I understood properly or got them all.

 

AA> The multip= le measurement agent case is when there is no anchor measurement agent (or = the anchor measurement agent does not have enough capacity to properly saturate the path—in the case of a path = capacity measurement). The lack of an anchor measurement agent may come abo= ut when the regulator is trying to make an access link test (within the ISP= ) but as it’s a non-ISP entity it would not have test resources that are well bandwidth connected within the ISP. = Also, the regulator does not want to stress the ISP upstream interface. If = we were to pool the resources of multiple ISP connected MAs (sitting in oth= er broadband sites) we would be able create the saturation w/o stressing the other broadband sites links. = I put a diagram in the presentation that Paul will be presenting that might= be more helpful.

 

(1) I’m sure the= re’s a case where one test triggers another (say, the dns result foll= owed by request to website) – this is ok.

(2) I think you have a= ‘latency under load’ test (S3.2), where the latency is measure= d for a pkt/file being sent between MA & Measurement Peer, but another P= eer is generating other traffic so that a bottleneck queue is loaded. I hav= en’t thought about this much – do you think there are other tes= ts involving a second Peer? or even 3rd, 4th…? Or is it just ‘latency under load’?&nbs= p; I’m wondering whether it would be best to think about the specific= s of how to do the ‘latency under load’ test, rather than a gen= eral capability for coordinating multiple Peers in one test.

 

AA> it was mea= nt to just be an aggregate path capacity measurement. Sorry it was not clea= r—I think this section needs more verbiage etc.

 

(3) you mention (S6.2.= 1) a case where multiple Measurement Agents submit results all with the same Measurement Task ID. I guess this is a test where lots of co= ordinated measurements are made. You also mention a shared key between mult= iple MAs, which I think must be related. I don’t understand this case= , please explain more.

Am interested to under= stand this whole area better, regardless of where the WG decides to draw the line between in scope & out.

 

AA> The measur= ement task ID is simply the index that is used to collate the individual re= sults submitted by the MAs post-collection. This would be when the MAs are all given the same instruction etc.. I did = a search in the document for ‘shared key’ and this is only in S= 4.2.2. This shared key is completely different and is a security device use= d to secure test control protocol (eg. OWAMP) communications between the MAs.

 

 

Third, you have some s= tuff about task scheduling where it seems important to have highly synchronised time between the controller, MAs and collector (and me= asurement peer?). why would this be needed? You mention about NTP and the i= ssue that NTP doesn’t give very accurate time sync to a device on the= end of an access line (because you can’t triangulate to multiple time servers). I suppose the error might be a few = millisecs - the sorts of cases I had in mind it would easily be accurate en= ough. Incidentally I don’t see why your solution (NTP clock sent from= controller) helps, since you still have the lack of triangulation on the end of an access line.

 

AA> The contex= t of the time synchronization was when (for example) a one-way-latency test= is run between two MAs. In the case of a round-trip-time test such time synchronization would not be needed, and of course we simpl= y can’t do RTT/2. The time concerns are really just bubbling up from = the OWAMP spec, but I believe them to be valid.

 

AA> regarding = the NTP source as the controller – this was simply an attempt to get = the MAs on to the same time hierarchy rather than differing unknown ones. It doesn’t really have to be the controller = itself, just known time source that everybody agrees on. Will clarify.

 

AA> that said = ( I don’t believe this is in the draft-aakhter-lmap-framework) I do s= ee some value in having the collector being loosely (vs tightly) synchronised with the MAs. This could be used as a way to det= ect a MA that is completely out of time sync when it submits its reports. P= erhaps useful….<= /span>

 

Thanks again,

aa

 

 

Thanks

Best wishes

phil

 

--_000_75C0E47A1889264493A2DCB2869AC09633383423xmbrcdx15ciscoc_-- From dromasca@avaya.com Mon Jul 29 00:59:15 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 761ED21F9929 for ; Mon, 29 Jul 2013 00:59:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.972 X-Spam-Level: X-Spam-Status: No, score=-102.972 tagged_above=-999 required=5 tests=[AWL=-0.373, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D9yyAUV6iS+h for ; Mon, 29 Jul 2013 00:59:09 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id E261821F9963 for ; Mon, 29 Jul 2013 00:59:08 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AggFAGQg9lGHCzI1/2dsb2JhbABbgmUhgQW9VYEVFnSCJgEBAxJ5ARUVViYBBBsah24Bl1iERJsuF49Mg1BvA4hylTOLBoMUgio X-IronPort-AV: E=Sophos;i="4.89,767,1367985600"; d="scan'208";a="21922522" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 29 Jul 2013 03:59:08 -0400 Received: from unknown (HELO AZ-FFEXHC02.global.avaya.com) ([135.64.58.12]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 29 Jul 2013 03:52:40 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC02.global.avaya.com ([135.64.58.12]) with mapi id 14.03.0146.000; Mon, 29 Jul 2013 09:59:06 +0200 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: LMAP at IETF-87 Thread-Index: Ac6MMX833cYP2nHLQWi1FyRV4HNloQ== Date: Mon, 29 Jul 2013 07:59:06 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1288BC8D@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] LMAP at IETF-87 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 07:59:15 -0000 This is a reminder that:=20 - the LMAP meeting is tomorrow - we need slides in advance from all presenters who own items in the agenda - we need volunteers for notes taking and jabber - we shall meet tomorrow morning at breakfast for a welcome session for new= comers. Meeting place at 8AM at the entry of the breakfast place of the Int= erContinental hotel (LA Caf=E9) Regards, Jason and Dan From yaojk@cnnic.cn Mon Jul 29 01:01:17 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD06321F9D01 for ; Mon, 29 Jul 2013 01:01:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -100.846 X-Spam-Level: X-Spam-Status: No, score=-100.846 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, MIME_BASE64_TEXT=1.753, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZLso+6t2cFPs for ; Mon, 29 Jul 2013 01:01:13 -0700 (PDT) Received: from cnnic.cn (smtp.cnnic.cn [218.241.118.7]) by ietfa.amsl.com (Postfix) with SMTP id 9C81721F9BC2 for ; Mon, 29 Jul 2013 01:01:12 -0700 (PDT) X-EYOUMAIL-SMTPAUTH: yaojk@cnnic.cn Received: from unknown127.0.0.1 (HELO healthyao-think) (127.0.0.1) by 127.0.0.1 with SMTP; Mon, 29 Jul 2013 16:01:00 +0800 Date: Mon, 29 Jul 2013 16:00:57 +0800 From: "Jiankang Yao" To: "Paul Aitken" , lmap-chairs References: <51DD4E33.5000101@cisco.com> X-Priority: 3 X-Has-Attach: no X-Mailer: Foxmail 7.0.1.92[cn] Mime-Version: 1.0 Message-ID: <2013072916000102252734@cnnic.cn> Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: base64 Cc: Aamer Akhter , "lmap@ietf.org" Subject: Re: [lmap] LMAP framework draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: yaojk List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 08:01:17 -0000 DQoNClRoZXJlIGFyZSAyIGZyYW1ld29yayBkb2N1bWVudHMNCkEgZnJhbWV3b3JrIGZvciBsYXJn ZS1zY2FsZSBtZWFzdXJlbWVudHMgICAgICAgICAgICAgICAgICANCmRyYWZ0LWVhcmRsZXktbG1h cC1mcmFtZXdvcmstMDENCiAgIA0KQSBGcmFtZXdvcmsgYW5kIEludmVudG9yeSBmb3IgYSBMYXJn ZSBTY2FsZSBNZWFzdXJlbWVudCBTeXN0ZW0NCmRyYWZ0LWFraHRlci1sbWFwLWZyYW1ld29yay0w MC50eHQNCg0KSWYgYm90aCBkb2N1bWVudHMgbmVlZCB0byBiZSBXRyBkb2N1bWVudHMsDQoNCkkg YW0gd29uZGVyaW5nIHdoZXRoZXIgYm90aCBkb2N1bWVudHMgY2FuIG1lcmdlIGludG8gMSBmcmFt ZXdvcmsgZG9jdW1lbnQgc2luY2UgdHdvIGZyYW1ld29yayBkb2N1bWVudHMgaW4gYSBXRyBzZWVt cyB0byBoYXZlIGEgbGl0dGxlIGNvbmZ1c2luZyBiYXNlZCBvbiB0aGUgbmFtZS4NCg0KDQpKaWFu a2FuZyBZYW8NCg0KRnJvbTogUGF1bCBBaXRrZW4NCkRhdGU6IDIwMTMtMDctMTAgMjA6MDYNClRv OiBsbWFwLWNoYWlycw0KQ0M6IEFhbWVyIEFraHRlcjsgbG1hcA0KU3ViamVjdDogW2xtYXBdIExN QVAgZnJhbWV3b3JrIGRyYWZ0DQpMTUFQIGNoYWlycywNCg0KV2UndmUgYmVlbiB3cml0aW5nIGFu IExNQVAgZnJhbWV3b3JrIGRvY3VtZW50IHdoaWNoIGRpc2N1c3NlcyB0aGUgDQpidWlsZGluZyBi bG9ja3MsIGxvb2tpbmcgYXQgd2hhdCB3ZSBhbHJlYWR5IGhhdmUgaW4gdGhlIElFVEYgYW5kIHdo YXQncyANCm1pc3NpbmcsIHdoZXJlIHRoZSBnYXBzIGFyZS4NCkl0IHdpbGwgYmUgcHVibGlzaGVk IGJ5IE1vbmRheSdzIGN1dC1vZmYuDQoNCldlJ2Qgd2VsY29tZSB0aGUgb3Bwb3J0dW5pdHkgdG8g cHJlc2VudCB0aGUgZHJhZnQgaW4gQmVybGluLg0KDQpUaGFua3MsDQpQLg0KX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCmxtYXAgbWFpbGluZyBsaXN0DQps bWFwQGlldGYub3JnDQpodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xtYXA= From bs7652@att.com Mon Jul 29 05:25:59 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 098C421F9E24 for ; Mon, 29 Jul 2013 05:25:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cKQ-OrmL6y3R for ; Mon, 29 Jul 2013 05:25:42 -0700 (PDT) Received: from nbfkord-smmo07.seg.att.com (nbfkord-smmo07.seg.att.com [209.65.160.93]) by ietfa.amsl.com (Postfix) with ESMTP id 7BFC421F9AA9 for ; Mon, 29 Jul 2013 05:25:32 -0700 (PDT) Received: from unknown [144.160.20.145] (EHLO nbfkord-smmo07.seg.att.com) by nbfkord-smmo07.seg.att.com(mxl_mta-6.15.0-1) with ESMTP id c3f56f15.2aab0765e940.3541817.00-522.9738312.nbfkord-smmo07.seg.att.com (envelope-from ); Mon, 29 Jul 2013 12:25:32 +0000 (UTC) X-MXL-Hash: 51f65f3c3b637059-c7516821e494c00c7b3b6ad170729a4e9b4f6dfc Received: from unknown [144.160.20.145] (EHLO mlpd192.enaf.sfdc.sbc.com) by nbfkord-smmo07.seg.att.com(mxl_mta-6.15.0-1) over TLS secured channel with ESMTP id 63f56f15.0.3541776.00-394.9738185.nbfkord-smmo07.seg.att.com (envelope-from ); Mon, 29 Jul 2013 12:25:26 +0000 (UTC) X-MXL-Hash: 51f65f3610c2d90c-1eaff2801626fce70092e5ca7af492a35e941297 Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r6TCPQqd003360; Mon, 29 Jul 2013 08:25:26 -0400 Received: from alpi131.aldc.att.com (alpi131.aldc.att.com [130.8.218.69]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r6TCPIEr003293 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 29 Jul 2013 08:25:19 -0400 Received: from GAALPA1MSGHUB9E.ITServices.sbc.com (gaalpa1msghub9e.itservices.sbc.com [130.8.36.91]) by alpi131.aldc.att.com (RSA Interceptor); Mon, 29 Jul 2013 12:25:10 GMT Received: from GAALPA1MSGUSR9L.ITServices.sbc.com ([130.8.36.69]) by GAALPA1MSGHUB9E.ITServices.sbc.com ([130.8.36.91]) with mapi id 14.02.0342.003; Mon, 29 Jul 2013 08:25:10 -0400 From: "STARK, BARBARA H" To: "Romascanu, Dan (Dan)" , "lmap@ietf.org" Thread-Topic: LMAP at IETF-87 Thread-Index: Ac6MMX833cYP2nHLQWi1FyRV4HNloQAJGDRw Date: Mon, 29 Jul 2013 12:25:09 +0000 Message-ID: <2D09D61DDFA73D4C884805CC7865E61130314F31@GAALPA1MSGUSR9L.ITServices.sbc.com> References: <9904FB1B0159DA42B0B887B7FA8119CA1288BC8D@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1288BC8D@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.70.29.155] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-RSA-Inspected: yes X-RSA-Classifications: public X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [144.160.20.145] X-AnalysisOut: [v=2.0 cv=YP8oP26x c=1 sm=0 a=ZRNLZ4dFUbCvG8UMqPvVAA==:17 a] X-AnalysisOut: [=PVZk7qdKzRAA:10 a=ofMgfj31e3cA:10 a=BLceEmwcHowA:10 a=8nJ] X-AnalysisOut: [EP1OIZ-IA:10 a=zQP7CpKOAAAA:8 a=XIqpo32RAAAA:8 a=2XtplGO3q] X-AnalysisOut: [kUA:10 a=8pxcAgufGuT2FIVxfGgA:9 a=wPNLvfGTeEIA:10 a=zn0-HG] X-AnalysisOut: [sCQOQA:10 a=p46FGmh5YFEA:10] Subject: Re: [lmap] LMAP at IETF-87 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 12:25:59 -0000 > - we shall meet tomorrow morning at breakfast for a welcome session for > newcomers. Meeting place at 8AM at the entry of the breakfast place of th= e > InterContinental hotel (LA Caf=E9) Since breakfast in LA Caf=E9 is only for people staying at the InterContine= ntal Hotel, I'm curious if there are other people like me who are not stayi= ng at the InterContinental, and won't be able to access LA Caf=E9, but who = might be interested in meeting. Barbara From philip.eardley@bt.com Mon Jul 29 07:47:12 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 06B7211E80E6 for ; Mon, 29 Jul 2013 07:47:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.46 X-Spam-Level: X-Spam-Status: No, score=-103.46 tagged_above=-999 required=5 tests=[AWL=0.139, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2Ivlq5cEEY6X for ; Mon, 29 Jul 2013 07:47:07 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id D879121F9A49 for ; Mon, 29 Jul 2013 07:47:06 -0700 (PDT) Received: from EVMHT62-UKRD.domain1.systemhost.net (10.36.3.128) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.298.1; Mon, 29 Jul 2013 15:46:54 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.32]) by EVMHT62-UKRD.domain1.systemhost.net ([10.36.3.128]) with mapi; Mon, 29 Jul 2013 15:46:54 +0100 From: To: , , Date: Mon, 29 Jul 2013 15:43:51 +0100 Thread-Topic: [lmap] LMAP framework draft Thread-Index: Ac6MMdBbtpaPiGvtRFCYfYLCuorvAgAODpRT Message-ID: <9510D26531EF184D9017DF24659BB87F35CB3F1EBE@EMV65-UKRD.domain1.systemhost.net> References: <51DD4E33.5000101@cisco.com>,<2013072916000102252734@cnnic.cn> In-Reply-To: <2013072916000102252734@cnnic.cn> Accept-Language: en-US, en-GB Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: aakhter@cisco.com, lmap@ietf.org Subject: Re: [lmap] LMAP framework draft X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 14:47:12 -0000 absolutely. We (Al, Paul & I) have just had a very good meeting. We discussed a bit how= we could do a first cut merger - will suggest this during the wg meeting, = for comments and improvements thanks phil ________________________________________ From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of Jiankang Y= ao [yaojk@cnnic.cn] Sent: 29 July 2013 09:00 To: Paul Aitken; lmap-chairs Cc: Aamer Akhter; lmap@ietf.org Subject: Re: [lmap] LMAP framework draft There are 2 framework documents A framework for large-scale measurements draft-eardley-lmap-framework-01 A Framework and Inventory for a Large Scale Measurement System draft-akhter-lmap-framework-00.txt If both documents need to be WG documents, I am wondering whether both documents can merge into 1 framework document s= ince two framework documents in a WG seems to have a little confusing based= on the name. Jiankang Yao From: Paul Aitken Date: 2013-07-10 20:06 To: lmap-chairs CC: Aamer Akhter; lmap Subject: [lmap] LMAP framework draft LMAP chairs, We've been writing an LMAP framework document which discusses the building blocks, looking at what we already have in the IETF and what's missing, where the gaps are. It will be published by Monday's cut-off. We'd welcome the opportunity to present the draft in Berlin. Thanks, P. _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap= From philip.eardley@bt.com Mon Jul 29 07:47:34 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 613DF21E8082 for ; Mon, 29 Jul 2013 07:47:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.468 X-Spam-Level: X-Spam-Status: No, score=-103.468 tagged_above=-999 required=5 tests=[AWL=0.132, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U+LMefJ5XWK3 for ; Mon, 29 Jul 2013 07:47:30 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id 9C89C11E80ED for ; Mon, 29 Jul 2013 07:47:24 -0700 (PDT) Received: from EVMHT67-UKRD.domain1.systemhost.net (10.36.3.104) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.298.1; Mon, 29 Jul 2013 15:47:22 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.32]) by EVMHT67-UKRD.domain1.systemhost.net ([10.36.3.104]) with mapi; Mon, 29 Jul 2013 15:47:22 +0100 From: To: , , Date: Mon, 29 Jul 2013 15:47:22 +0100 Thread-Topic: LMAP at IETF-87 Thread-Index: Ac6MMX833cYP2nHLQWi1FyRV4HNloQAJGDRwAAT2GAo= Message-ID: <9510D26531EF184D9017DF24659BB87F35CB3F1EBD@EMV65-UKRD.domain1.systemhost.net> References: <9904FB1B0159DA42B0B887B7FA8119CA1288BC8D@AZ-FFEXMB04.global.avaya.com>, <2D09D61DDFA73D4C884805CC7865E61130314F31@GAALPA1MSGUSR9L.ITServices.sbc.com> In-Reply-To: <2D09D61DDFA73D4C884805CC7865E61130314F31@GAALPA1MSGUSR9L.ITServices.sbc.com> Accept-Language: en-US, en-GB Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] LMAP at IETF-87 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 14:47:34 -0000 i'm not staying in the Intercontinental either. i'm hoping we can sneak in as having breakfast with someone staying at IC..= . or is this impossible? ________________________________________ From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of STARK, BAR= BARA H [bs7652@att.com] Sent: 29 July 2013 13:25 To: Romascanu, Dan (Dan); lmap@ietf.org Subject: Re: [lmap] LMAP at IETF-87 > - we shall meet tomorrow morning at breakfast for a welcome session for > newcomers. Meeting place at 8AM at the entry of the breakfast place of th= e > InterContinental hotel (LA Caf=E9) Since breakfast in LA Caf=E9 is only for people staying at the InterContine= ntal Hotel, I'm curious if there are other people like me who are not stayi= ng at the InterContinental, and won't be able to access LA Caf=E9, but who = might be interested in meeting. Barbara _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap= From j.schoenwaelder@jacobs-university.de Mon Jul 29 08:03:25 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 67FB621F9A4C for ; Mon, 29 Jul 2013 08:03:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CMEA6s9BdTUl for ; Mon, 29 Jul 2013 08:03:20 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 3547F11E80D2 for ; Mon, 29 Jul 2013 08:03:18 -0700 (PDT) Received: from localhost (demetrius2.jacobs-university.de [212.201.44.47]) by hermes.jacobs-university.de (Postfix) with ESMTP id 6908120BE5; Mon, 29 Jul 2013 17:03:17 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius2.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id Js4R77xavtPp; Mon, 29 Jul 2013 17:03:17 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id CA243208C6; Mon, 29 Jul 2013 17:03:16 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 6CDDC279B904; Mon, 29 Jul 2013 17:03:13 +0200 (CEST) Date: Mon, 29 Jul 2013 17:03:12 +0200 From: Juergen Schoenwaelder To: philip.eardley@bt.com Message-ID: <20130729150312.GA13136@elstar.local> Mail-Followup-To: philip.eardley@bt.com, bs7652@att.com, dromasca@avaya.com, lmap@ietf.org References: <9904FB1B0159DA42B0B887B7FA8119CA1288BC8D@AZ-FFEXMB04.global.avaya.com> <2D09D61DDFA73D4C884805CC7865E61130314F31@GAALPA1MSGUSR9L.ITServices.sbc.com> <9510D26531EF184D9017DF24659BB87F35CB3F1EBD@EMV65-UKRD.domain1.systemhost.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <9510D26531EF184D9017DF24659BB87F35CB3F1EBD@EMV65-UKRD.domain1.systemhost.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: dromasca@avaya.com, bs7652@att.com, lmap@ietf.org Subject: Re: [lmap] LMAP at IETF-87 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 15:03:25 -0000 I am not at the IC either - one option is we meat at 08:20 at a table outside the cafe - this way, IC people can have their breakfast and the others have some additional time to pickup breakfast on their way to the IC. /js On Mon, Jul 29, 2013 at 03:47:22PM +0100, philip.eardley@bt.com wrote: > i'm not staying in the Intercontinental either. > i'm hoping we can sneak in as having breakfast with someone staying at IC... or is this impossible? > > ________________________________________ > From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of STARK, BARBARA H [bs7652@att.com] > Sent: 29 July 2013 13:25 > To: Romascanu, Dan (Dan); lmap@ietf.org > Subject: Re: [lmap] LMAP at IETF-87 > > > - we shall meet tomorrow morning at breakfast for a welcome session for > > newcomers. Meeting place at 8AM at the entry of the breakfast place of the > > InterContinental hotel (LA Café) > > Since breakfast in LA Café is only for people staying at the InterContinental Hotel, I'm curious if there are other people like me who are not staying at the InterContinental, and won't be able to access LA Café, but who might be interested in meeting. > Barbara > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From jbustos@niclabs.cl Mon Jul 29 08:08:54 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1CDFA11E8106 for ; Mon, 29 Jul 2013 08:08:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.299 X-Spam-Level: X-Spam-Status: No, score=-2.299 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QpEUVBSuSS-H for ; Mon, 29 Jul 2013 08:08:03 -0700 (PDT) Received: from mail-pd0-f175.google.com (mail-pd0-f175.google.com [209.85.192.175]) by ietfa.amsl.com (Postfix) with ESMTP id E180721E805A for ; Mon, 29 Jul 2013 08:07:52 -0700 (PDT) Received: by mail-pd0-f175.google.com with SMTP id 5so254561pdd.6 for ; Mon, 29 Jul 2013 08:07:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=R6mmeQFZXoEYaKTVW7giqBX5XPqw60015/9vRLQcX2A=; b=pVl7UGI8/B2/4OrM0OHH4+C5gaD0yDWhoF9pxjORklEVsBGgr3KqLIXV6T07HS0WCg KIOTItggVKqL5AGR6btKT+ZPI14Uiafgpejyh4yzkXaGM+wGxCwIjsdPYWeSKHrh2c/r E23DrpfluhArclgVwqOp+BkYb52VMI6eaNpHOpIIkS1nlaMZCM+2Td66vqJUpVnuYvjz dEGfv3KVeMaDzqYEZQ2fR3gk9VBi4CIuCw0kh4/5OiDIU+cIos7r0Z3EUbimyPr0y1c1 DWUIjiCAg+5OgByUlmpE1d1BVTJKDBKfJDdrX2VElsZxXBoCBuLm8fFcA9K/4K2Gllzc MUMg== X-Received: by 10.66.37.43 with SMTP id v11mr31433517paj.108.1375110464880; Mon, 29 Jul 2013 08:07:44 -0700 (PDT) Received: from ?IPv6:2001:df8::16:4c56:9ae3:33a:fe5b? ([2001:df8:0:16:4c56:9ae3:33a:fe5b]) by mx.google.com with ESMTPSA id bb1sm67627715pbc.10.2013.07.29.08.07.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 29 Jul 2013 08:07:43 -0700 (PDT) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) From: =?iso-8859-1?Q?Javier_Bustos_Jim=E9nez?= In-Reply-To: <20130729150312.GA13136@elstar.local> Date: Mon, 29 Jul 2013 17:07:39 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <9904FB1B0159DA42B0B887B7FA8119CA1288BC8D@AZ-FFEXMB04.global.avaya.com> <2D09D61DDFA73D4C884805CC7865E61130314F31@GAALPA1MSGUSR9L.ITServices.sbc.com> <9510D26531EF184D9017DF24659BB87F35CB3F1EBD@EMV65-UKRD.domain1.systemhost.net> <20130729150312.GA13136@elstar.local> To: Juergen Schoenwaelder X-Mailer: Apple Mail (2.1508) X-Gm-Message-State: ALoCoQk1MwoGKvVVz9Y5OsYnqOq9zwWEvudTTC2LzlKXaiTywUhFv00ARz+N2VXK5VwIx2IZdRD2 Cc: dromasca@avaya.com, philip.eardley@bt.com, bs7652@att.com, lmap@ietf.org Subject: Re: [lmap] LMAP at IETF-87 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 15:08:54 -0000 Hello I'm not at the IC either, but I'll do my best effort to arrive around = 8:30 here Best regards Javier El 29-07-2013, a las 17:03, Juergen Schoenwaelder = escribi=F3: > I am not at the IC either - one option is we meat at 08:20 at a table > outside the cafe - this way, IC people can have their breakfast and > the others have some additional time to pickup breakfast on their way > to the IC. >=20 > /js >=20 > On Mon, Jul 29, 2013 at 03:47:22PM +0100, philip.eardley@bt.com wrote: >> i'm not staying in the Intercontinental either. >> i'm hoping we can sneak in as having breakfast with someone staying = at IC... or is this impossible? >>=20 >> ________________________________________ >> From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of = STARK, BARBARA H [bs7652@att.com] >> Sent: 29 July 2013 13:25 >> To: Romascanu, Dan (Dan); lmap@ietf.org >> Subject: Re: [lmap] LMAP at IETF-87 >>=20 >>> - we shall meet tomorrow morning at breakfast for a welcome session = for >>> newcomers. Meeting place at 8AM at the entry of the breakfast place = of the >>> InterContinental hotel (LA Caf=E9) >>=20 >> Since breakfast in LA Caf=E9 is only for people staying at the = InterContinental Hotel, I'm curious if there are other people like me = who are not staying at the InterContinental, and won't be able to access = LA Caf=E9, but who might be interested in meeting. >> Barbara >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >=20 > --=20 > Juergen Schoenwaelder Jacobs University Bremen gGmbH > Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany > Fax: +49 421 200 3103 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From dromasca@avaya.com Mon Jul 29 08:51:18 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F2D5521F9F80 for ; Mon, 29 Jul 2013 08:51:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.447 X-Spam-Level: X-Spam-Status: No, score=-103.447 tagged_above=-999 required=5 tests=[AWL=0.152, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mwlM19-CLgmU for ; Mon, 29 Jul 2013 08:51:10 -0700 (PDT) Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id 38E7821F9A05 for ; Mon, 29 Jul 2013 08:51:04 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgsFAPeO9lGHCzI1/2dsb2JhbABbgmUhNVC9V4EYFnSCJAEBAQEDAQEBD1wXBAIBCA0EBAEBCx0HJwsUCQgBAQQBEggah24BC5xfm2gTBI9MOAaDEm8DiHKVM4sGgxSCKg X-IronPort-AV: E=Sophos;i="4.89,770,1367985600"; d="scan'208";a="18148462" Received: from unknown (HELO p-us1-erheast.us1.avaya.com) ([135.11.50.53]) by de307622-de-outbound.net.avaya.com with ESMTP; 29 Jul 2013 11:50:57 -0400 Received: from unknown (HELO AZ-FFEXHC04.global.avaya.com) ([135.64.58.14]) by p-us1-erheast-out.us1.avaya.com with ESMTP; 29 Jul 2013 11:44:28 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC04.global.avaya.com ([135.64.58.14]) with mapi id 14.03.0146.000; Mon, 29 Jul 2013 11:50:55 -0400 From: "Romascanu, Dan (Dan)" To: "philip.eardley@bt.com" , "bs7652@att.com" , "lmap@ietf.org" Thread-Topic: LMAP at IETF-87 Thread-Index: Ac6MMX833cYP2nHLQWi1FyRV4HNloQAJGDRwAAT2GAoAAk9IUA== Date: Mon, 29 Jul 2013 15:50:54 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1288C408@AZ-FFEXMB04.global.avaya.com> References: <9904FB1B0159DA42B0B887B7FA8119CA1288BC8D@AZ-FFEXMB04.global.avaya.com>, <2D09D61DDFA73D4C884805CC7865E61130314F31@GAALPA1MSGUSR9L.ITServices.sbc.com> <9510D26531EF184D9017DF24659BB87F35CB3F1EBD@EMV65-UKRD.domain1.systemhost.net> In-Reply-To: <9510D26531EF184D9017DF24659BB87F35CB3F1EBD@EMV65-UKRD.domain1.systemhost.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.46] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] LMAP at IETF-87 X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 15:51:18 -0000 People not staying in the Intercontinental can access LA Caf=E9 - either pa= y for the breakfast, or just enter and say that they are accompanying guest= s without eating anything.=20 Regards, Dan > -----Original Message----- > From: philip.eardley@bt.com [mailto:philip.eardley@bt.com] > Sent: Monday, July 29, 2013 5:47 PM > To: bs7652@att.com; Romascanu, Dan (Dan); lmap@ietf.org > Subject: RE: LMAP at IETF-87 >=20 > i'm not staying in the Intercontinental either. > i'm hoping we can sneak in as having breakfast with someone staying at > IC... or is this impossible? >=20 > ________________________________________ > From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of STARK, > BARBARA H [bs7652@att.com] > Sent: 29 July 2013 13:25 > To: Romascanu, Dan (Dan); lmap@ietf.org > Subject: Re: [lmap] LMAP at IETF-87 >=20 > > - we shall meet tomorrow morning at breakfast for a welcome session > > for newcomers. Meeting place at 8AM at the entry of the breakfast > > place of the InterContinental hotel (LA Caf=E9) >=20 > Since breakfast in LA Caf=E9 is only for people staying at the > InterContinental Hotel, I'm curious if there are other people like me > who are not staying at the InterContinental, and won't be able to access > LA Caf=E9, but who might be interested in meeting. > Barbara > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From jbustos@niclabs.cl Tue Jul 30 00:33:58 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1A56F21E80CC for ; Tue, 30 Jul 2013 00:33:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.799 X-Spam-Level: X-Spam-Status: No, score=-1.799 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, GB_ABOUTYOU=0.5, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vmIM1oSprgCn for ; Tue, 30 Jul 2013 00:33:53 -0700 (PDT) Received: from mail-pd0-f179.google.com (mail-pd0-f179.google.com [209.85.192.179]) by ietfa.amsl.com (Postfix) with ESMTP id 70D6C21E80C9 for ; Tue, 30 Jul 2013 00:33:53 -0700 (PDT) Received: by mail-pd0-f179.google.com with SMTP id v10so6247561pde.10 for ; Tue, 30 Jul 2013 00:33:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=ICgs+Dz8FD2HtvzgHxglRaPjzBr0dM6PaCa40nMlKtk=; b=XJgbnjphgyXy9gehAasMOmaI8Es0iy1IWAL7+k4cXT/SZeKxJQkGHYnB+/8no/hFDr iFfn9i9t2UgDoLbwdJ5o0EckRLiKctkMbhfAw4CFAJl+c/LkWL0v7aX9tz6ZE4THjPdB s1mQ8BQh69x0NjrgxtUlDkNe3eVr9joyZUi6sOkFP3G3wzQJnNvWsZ+M2fj96CLmCtKP avoxITNDQkfq5ffhmUOPsmB3MFWPBYzVcazhc9zWWA9ELYXKFcQhcZ+9gq+uHg2nZ/cS B3H8y0A40tTSXJUp8YHW0FqCTTTmZbpKfh6MGiWe2woAWiyD7fxABv4eXJT0LILnn/lw 4rCA== X-Received: by 10.66.25.70 with SMTP id a6mr63696089pag.68.1375169632377; Tue, 30 Jul 2013 00:33:52 -0700 (PDT) Received: from dhcp-17c5.meeting.ietf.org (dhcp-17c5.meeting.ietf.org. [130.129.23.197]) by mx.google.com with ESMTPSA id ot4sm25314073pac.17.2013.07.30.00.33.50 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 30 Jul 2013 00:33:51 -0700 (PDT) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) From: =?iso-8859-1?Q?Javier_Bustos_Jim=E9nez?= In-Reply-To: Date: Tue, 30 Jul 2013 03:33:47 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <451142E8-6E5D-47C6-9700-FB195FBA4BCE@niclabs.cl> References: <9904FB1B0159DA42B0B887B7FA8119CA12881146@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337A5@EMV65-UKRD.domain1.systemhost.net> To: lmap WG X-Mailer: Apple Mail (2.1508) X-Gm-Message-State: ALoCoQkMTueMMiYglHwkoH0k/QRBPeAP6LQg+jrT42j1Oocm3XSkaikJ799LjvQz5oWpp1Vz2dOR Subject: Re: [lmap] on use cases and requirements X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 07:33:58 -0000 Hello Watching the slides for draft-linsner-lmap-use-cases two comments come = to my mind: 1.- QoE is defined at ISO 9241-210: Human-centred design for interactive = systems. Quality of Experience: "a person's perceptions and responses = that result from the use or anticipated use of a product, system or = service" 2.- About a question about how much often the probing has to be made and = how many measures agents hould be distributed in order to be = statistically representative, I remember that we had that problem when = we built our national infrastructure and saldy the answer is "depends". = It depends in what is wanted to measure, the size and the distribution = of the population. Best regards Javier =20 El 26-07-2013, a las 14:25, Nagami Kenichi escribi=F3:= > Phil, >=20 >> = http://tools.ietf.org/html/draft-nagami-lmap-use-case-measurement-provider= -00 >> you talk about a "measurement provider" - if I get it right, this is = someone who provides information about different ISPs. this seems to me = very similar to the regulator use case - but it's a public interest = group, or a company providing the information instead of a regulator. = It's probably worth pointing this out in = http://tools.ietf.org/html/draft-linsner-lmap-use-cases - would this = then cover your use case? >=20 > I think a use case for a regulator is similar to a measurement = provider. >=20 > We were thinking the following case. >=20 > Multiple measurement providers can measure a performance > from MAs in the users to MAs in the multiple content providers. >=20 > The measurement provider publishes the results that are easy to > understand for users. > It is necessary to show the measurement parameters in order to allow > comparison with the result of the other. > For example, the parameters are how to measure a performance, > measure a performance from where to where, etc. >=20 > If a use case for a regulator is same as this, > I would like to add "a public interest group, or a company providing > the information instead of a regulator" for clarity. > And it is necessary to show measurement parameters as well as the > measurement result. >=20 > Regards, > Ken Nagami >=20 >>=20 >> You also mention a smartphone, I think it's worth adding this = explicitly to draft-linsner >>=20 >> You also have a lot of interesting details about your deployment. My = feeling is that it would take a lot of work to summarise all the = state-of-the-art, so we shouldn't try (perhaps we could add a few = references in one of the docs) >>=20 >>=20 >> Rachel, >> = http://tools.ietf.org/id/draft-huang-lmap-data-collection-use-case-00.txt >> I take this as a slight expansion of the ISP use case described in = http://tools.ietf.org/html/draft-linsner-lmap-use-cases >> Could we work with you on adding a little text about simulations to = our draft? - ie the ISP could use measurements to feed its simulation = tools (assuming it uses simulations as part of the way it works out = where a fault is, or where to add capacity, etc). >> Will you be in Berlin? Marc, Ken and I are meeting in the lmap = breakfast to discuss use cases. >>=20 >>=20 >> Dan, >> Personally I'm happy to drop as much of Section 3 as people want = (even all of it). >> I agree we should think carefully in terms of what the output of lmap = needs to do - what are the most important aspects of which use cases. >> Reading between the lines, you think the use cases doc should only = cover those aspects that we're going to (try to) make sure we solve = during the initial lmap charter. And not be a general use case doc that = would need future lmap capability. >> Is that right? >> (I'm ok if the answer is 'yes') >>=20 >> Thanks >> phil >>=20 >>> -----Original Message----- >>> From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf = Of >>> Romascanu, Dan (Dan) >>> Sent: 25 July 2013 13:40 >>> To: lmap@ietf.org >>> Subject: [lmap] on use cases and requirements >>>=20 >>> If you have a look at the updated agenda at >>> https://datatracker.ietf.org/meeting/87/agenda/lmap/ you will = realized >>> that Jason and me slightly updated the section dedicated to use = cases >>> and added ten minutes of discussion. Let me try to explain the >>> motivation and what we would try to achieve. According to the = charter >>> in a couple of months the WG will need to submit the first WG = Internet- >>> Drafts on Use Cases and Framework. These two documents should = provide >>> clear enough requirements for the following phases where we need to >>> proceed to the creation of the Information Model, and discuss about = the >>> selection or development of protocols for Control and Reporting and >>> their associated data models. At this moment in time we have one = draft- >>> linsner-lmap-use-cases-03 as principal document on use cases, and = two >>> new contributions, which is good for this phase. What we need to = start >>> doing is to sort out of the use cases the subset that matches the >>> current WG charter - and we may need to drop or prioritize some of = the >>> aspects of the use cases for this purpose. >>>=20 >>> (To be more specific and take this just as an example, I am not sure >>> that all the characteristics described in Sections 3.2, 3.3, 3.5 in >>> draft-linsner-lmap-use-cases-03 are LMAP problems or problems that >>> belong to the requirements of the first phase of LMAP.) >>>=20 >>> Regards, >>>=20 >>> Dan >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>> _______________________________________________ >>> lmap mailing list >>> lmap@ietf.org >>> https://www.ietf.org/mailman/listinfo/lmap >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap >>=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From acooper@cdt.org Tue Jul 30 01:02:57 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 10C9521E80D3 for ; Tue, 30 Jul 2013 01:02:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tb7+d6glZgwS for ; Tue, 30 Jul 2013 01:02:44 -0700 (PDT) Received: from mail.maclaboratory.net (mail.maclaboratory.net [209.190.215.232]) by ietfa.amsl.com (Postfix) with ESMTP id D856A21E80BF for ; Tue, 30 Jul 2013 01:02:41 -0700 (PDT) X-Footer: Y2R0Lm9yZw== Received: from localhost ([127.0.0.1]) by mail.maclaboratory.net (using TLSv1/SSLv3 with cipher AES128-SHA (128 bits)) for lmap@ietf.org; Tue, 30 Jul 2013 04:02:39 -0400 From: Alissa Cooper Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org> Date: Tue, 30 Jul 2013 10:02:37 +0200 To: "lmap@ietf.org" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) Subject: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 08:02:57 -0000 Just to clarify from the discussion at the mic: I don't think the eventual LMAP use cases document should say that = user-initiated measurements are out of scope. I think they are in scope. With that said, at this point it's not clear that user-initiated tests = impose any additional requirements beyond the ISP use case and the = regulator use case. User-initiated tests could use, at a minimum, the = same information model that will be defined to support the other two use = cases. User-initiated tests could also potentially make use of the = reporting protocol if the user wants to report the results to a = collector. Alissa= From bclaise@cisco.com Tue Jul 30 01:14:20 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EB08B21F9B91 for ; Tue, 30 Jul 2013 01:14:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.536 X-Spam-Level: X-Spam-Status: No, score=-10.536 tagged_above=-999 required=5 tests=[AWL=0.063, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dqn+eHBZY16R for ; Tue, 30 Jul 2013 01:14:15 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 6EADA21F9AFD for ; Tue, 30 Jul 2013 01:14:14 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6U8Dtt0003421; Tue, 30 Jul 2013 10:13:55 +0200 (CEST) Received: from [10.61.164.23] ([10.61.164.23]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r6U8DGTw006639; Tue, 30 Jul 2013 10:13:31 +0200 (CEST) Message-ID: <51F77576.2030401@cisco.com> Date: Tue, 30 Jul 2013 10:12:38 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: Alissa Cooper References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org> In-Reply-To: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 08:14:20 -0000 Exactly my thoughts. Thanks Alissa Regards, Benoit > Just to clarify from the discussion at the mic: > > I don't think the eventual LMAP use cases document should say that user-initiated measurements are out of scope. I think they are in scope. > > With that said, at this point it's not clear that user-initiated tests impose any additional requirements beyond the ISP use case and the regulator use case. User-initiated tests could use, at a minimum, the same information model that will be defined to support the other two use cases. User-initiated tests could also potentially make use of the reporting protocol if the user wants to report the results to a collector. > > Alissa > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > > From trammell@tik.ee.ethz.ch Tue Jul 30 01:17:06 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2D19521E80BC for ; Tue, 30 Jul 2013 01:17:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.539 X-Spam-Level: X-Spam-Status: No, score=-6.539 tagged_above=-999 required=5 tests=[AWL=0.060, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qS9qk54125KC for ; Tue, 30 Jul 2013 01:16:59 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 7262121E80E7 for ; Tue, 30 Jul 2013 01:16:53 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 462A4D9309; Tue, 30 Jul 2013 10:16:52 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id QvdWrXV3AEQ2; Tue, 30 Jul 2013 10:16:52 +0200 (MEST) Received: from dhcp-90be.meeting.ietf.org (dhcp-90be.meeting.ietf.org [130.129.8.190]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 0CF2ED9308; Tue, 30 Jul 2013 10:16:52 +0200 (MEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) From: Brian Trammell In-Reply-To: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org> Date: Tue, 30 Jul 2013 10:16:53 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org> To: Alissa Cooper X-Mailer: Apple Mail (2.1508) Cc: "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 08:17:06 -0000 hi Alissa, all, On 30 Jul 2013, at 10:02, Alissa Cooper wrote: > Just to clarify from the discussion at the mic: >=20 > I don't think the eventual LMAP use cases document should say that = user-initiated measurements are out of scope. I think they are in scope. +1 > With that said, at this point it's not clear that user-initiated tests = impose any additional requirements beyond the ISP use case and the = regulator use case. User-initiated tests could use, at a minimum, the = same information model that will be defined to support the other two use = cases. User-initiated tests could also potentially make use of the = reporting protocol if the user wants to report the results to a = collector. Indeed, and to clarify my point from the mic, we should make a = distinction between user-requested (i.e., user sends a request to the = LMAP infrastructure, which initiates a measurement on the user's behalf) = and user-run (controller runs on the user device, or measurement agent = runs on the user device without the controller's involvement). The = scalability issues that Paul and Phil were worried about only affect the = latter case (and I personally agree it's probably a bad idea to try to = accommodate it). Cheers, Brian= From jukka.manner@aalto.fi Tue Jul 30 01:41:03 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 46C9321E80D7 for ; Tue, 30 Jul 2013 01:41:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v1UZ7pzWEwDj for ; Tue, 30 Jul 2013 01:40:58 -0700 (PDT) Received: from mx03.aalto.fi (mx03.aalto.fi [130.233.222.102]) by ietfa.amsl.com (Postfix) with ESMTP id 7136C21F9F20 for ; Tue, 30 Jul 2013 01:38:33 -0700 (PDT) Received: from mx03.aalto.fi (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 863B6281CF for ; Tue, 30 Jul 2013 11:38:31 +0300 (EEST) Received: from EXHUB03.org.aalto.fi (exhub03.org.aalto.fi [130.233.222.116]) by mx03.aalto.fi (Postfix) with ESMTP id 7C638281C6 for ; Tue, 30 Jul 2013 11:38:31 +0300 (EEST) Received: from EXMDB06.org.aalto.fi ([169.254.6.231]) by EXHUB03.org.aalto.fi ([130.233.222.116]) with mapi id 14.03.0146.000; Tue, 30 Jul 2013 11:38:31 +0300 From: Manner Jukka To: "lmap@ietf.org" Thread-Topic: [lmap] user-initiated testing Thread-Index: AQHOjP0lb/VdXMz6UUWqpgY0nU8Tf5l8tBMA Date: Tue, 30 Jul 2013 08:38:30 +0000 Message-ID: References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org> <1672_1375172234_51F77689_1672_283_1_6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> In-Reply-To: <1672_1375172234_51F77689_1672_283_1_6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> Accept-Language: en-US, fi-FI Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [130.129.18.13] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 08:41:03 -0000 Hi, A comment to the latter item, i.e., scalability of user-run measurements, l= oad balancing and load control solves the issue quite well.=20 Our "Netradar" (available on iOS, WP, Android, Meego, Symbian, and data at = netradar.org) mobile network measurements service is using end-user run mea= surements. We have measurement points globally distributed on three contine= nts and control the number of users to each individual measurement server. = So far has worked like a charm, for over a year. If the load, i.e. number o= f active MAs, grows, we can power up more measurement points and balance th= e load on them. So, user-run is not a problem if one plans for it. I'm here in Berlin up to Wednesday to chat more if needed. cheers, Jukka On Jul 30, 2013, at 10:16 AM, Brian Trammell wrote: > hi Alissa, all, >=20 > On 30 Jul 2013, at 10:02, Alissa Cooper wrote: >=20 >> Just to clarify from the discussion at the mic: >>=20 >> I don't think the eventual LMAP use cases document should say that user-= initiated measurements are out of scope. I think they are in scope. >=20 > +1 >=20 >> With that said, at this point it's not clear that user-initiated tests i= mpose any additional requirements beyond the ISP use case and the regulator= use case. User-initiated tests could use, at a minimum, the same informati= on model that will be defined to support the other two use cases. User-init= iated tests could also potentially make use of the reporting protocol if th= e user wants to report the results to a collector. >=20 > Indeed, and to clarify my point from the mic, we should make a distinctio= n between user-requested (i.e., user sends a request to the LMAP infrastruc= ture, which initiates a measurement on the user's behalf) and user-run (con= troller runs on the user device, or measurement agent runs on the user devi= ce without the controller's involvement). The scalability issues that Paul = and Phil were worried about only affect the latter case (and I personally a= gree it's probably a bad idea to try to accommodate it). >=20 > Cheers, >=20 > Brian > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From Joachim.Fabini@tuwien.ac.at Tue Jul 30 02:51:54 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 00D5811E80D5 for ; Tue, 30 Jul 2013 02:51:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.43 X-Spam-Level: X-Spam-Status: No, score=-5.43 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_AT=0.424, HOST_EQ_AT=0.745, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qFinsN+oSOe0 for ; Tue, 30 Jul 2013 02:51:47 -0700 (PDT) Received: from mail1.zserv.tuwien.ac.at (mail1.zserv.tuwien.ac.at [128.130.35.37]) by ietfa.amsl.com (Postfix) with ESMTP id 3166B11E81CC for ; Tue, 30 Jul 2013 02:51:40 -0700 (PDT) Received: from [128.131.88.241] (priamos.ibk.tuwien.ac.at [128.131.88.241]) (authenticated bits=0) by mail1.zserv.tuwien.ac.at (8.13.8/8.13.8) with ESMTP id r6U9pcd7003978 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 30 Jul 2013 11:51:38 +0200 Message-ID: <51F78C91.9010001@tuwien.ac.at> Date: Tue, 30 Jul 2013 11:51:13 +0200 From: Joachim Fabini User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: lmap@ietf.org Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 8bit Subject: [lmap] Data Model, Communication, Framework: Pre-Requisites? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 09:51:54 -0000 Having followed part of the preliminary mailing list discussions, two questions (coming from the IPPM corner): 1. Has there been any prior LMAP discussion on quering/requesting/logging/enforcing measurement pre-, intermediate- and post-measurement conditions? It's a security- and privacy-sensitive matter of control, which can end up in denial-of-service. Where would this functionality fit best in the architecture? Exemplary use case from practical experience: VDSL line with dedicated, QoS-enabled VoIP profile. Whenever a VoIP call is ongoing (IPTV activated, etc.), dedicated capacity is reserved for the QoS-enabled service, which substantially decreases remainging capacity available for Internet traffic (data). Depending on the use case the Controller might be required to disable, activate or log status of such (non-)QoS-enabled services during test in the MA, leading to paranoic situations where users can not call emergency services because ISPs test his connection. I believe that an active call ending and terminating within a VDSL-data measurement without being explicitly logged will generate a lot of confusion. There are many more situations I can think of and believe that these should be considered at an early stage in an abstract manner. 2. Is the definition of a test parameter control interfaces (MA test capabilities negotiation and configuration) a matter of IPPM or LMAP (or coordinated) definition? regards Joachim -- --------------------------------------- Dr. Joachim Fabini Institute of Telecommunications Vienna University of Technology Favoritenstraße 9/389 A-1040 Vienna, Austria Tel: +43 1 58801-38813 Fax: +43 1 58801-38898 mailto: Joachim.Fabini@tuwien.ac.at --------------------------------------- From trevor.burbridge@bt.com Tue Jul 30 03:14:20 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DFC4711E8144 for ; Tue, 30 Jul 2013 03:14:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.849 X-Spam-Level: X-Spam-Status: No, score=-2.849 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, J_CHICKENPOX_21=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6Ad7b2vOB2TO for ; Tue, 30 Jul 2013 03:14:16 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id 95B4921E808A for ; Tue, 30 Jul 2013 03:14:08 -0700 (PDT) Received: from EVMHT65-UKRD.domain1.systemhost.net (10.36.3.102) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.298.1; Tue, 30 Jul 2013 11:14:06 +0100 Received: from EMV64-UKRD.domain1.systemhost.net ([169.254.1.34]) by EVMHT65-UKRD.domain1.systemhost.net ([10.36.3.102]) with mapi; Tue, 30 Jul 2013 11:14:06 +0100 From: To: , Date: Tue, 30 Jul 2013 11:14:04 +0100 Thread-Topic: [lmap] Data Model, Communication, Framework: Pre-Requisites? Thread-Index: Ac6NCoRVPBXwYJuxSmWZoRHrWvVzJwAAfZ8A Message-ID: References: <51F78C91.9010001@tuwien.ac.at> In-Reply-To: <51F78C91.9010001@tuwien.ac.at> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] Data Model, Communication, Framework: Pre-Requisites? X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 10:14:21 -0000 1. The basic approach that has been discussed is that the MA will have know= ledge of cross-traffic on the same broadband line. Therefore it can avoid t= esting during these times. The exact framework needs to be flexible enough = to define a strategy for this avoidance that is sufficient for the sort of = example you describe (e.g. not testing for X seconds after activity to allo= w the VoIP capacity to be unreserved). We are currently saying that the bac= k-off/interrupt strategy will be unique to each test and can be coded into = the test design and/or configurable Measurement Task options. Making this s= ort of back-off and rescheduling capability a generic part of the Instruct= ion information would be very complex and that is why we are trying to avoi= d that approach. 2. the parameter control is done in the LMAP instruction (see the Measureme= nt Task Configuration part of the Information Model draft). Which parameter= s are configurable is define in a registry which is an output of IPPM. Trevor. Trevor Burbridge Network Infrastructure & Innovation | BT Innovate & Design Tel: 01473 645115 Fax: 01473 640929 This email contains BT information, which may be privileged or confidential= . It's meant only for the individual(s) or entity named above. If you're no= t the intended recipient, note that disclosing, copying, distributing or us= ing this information is prohibited. If you've received this email in error,= please let me know immediately on the email address above. Thank you. We monitor our email system, and may record your emails.=20 British Telecommunications plc Registered office: 81 Newgate Street London EC1A 7AJ Registered in England no: 1800000=20 >-----Original Message----- >From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of >Joachim Fabini >Sent: 30 July 2013 10:51 >To: lmap@ietf.org >Subject: [lmap] Data Model, Communication, Framework: Pre-Requisites? > >Having followed part of the preliminary mailing list discussions, two >questions (coming from the IPPM corner): > >1. Has there been any prior LMAP discussion on >quering/requesting/logging/enforcing measurement pre-, intermediate- and >post-measurement conditions? It's a security- and privacy-sensitive >matter of control, which can end up in denial-of-service. Where would >this functionality fit best in the architecture? >Exemplary use case from practical experience: VDSL line with dedicated, >QoS-enabled VoIP profile. Whenever a VoIP call is ongoing (IPTV >activated, etc.), dedicated capacity is reserved for the QoS-enabled >service, which substantially decreases remainging capacity available for >Internet traffic (data). > >Depending on the use case the Controller might be required to disable, >activate or log status of such (non-)QoS-enabled services during test in >the MA, leading to paranoic situations where users can not call >emergency services because ISPs test his connection. I believe that an >active call ending and terminating within a VDSL-data measurement >without being explicitly logged will generate a lot of confusion. >There are many more situations I can think of and believe that these >should be considered at an early stage in an abstract manner. > >2. Is the definition of a test parameter control interfaces (MA test >capabilities negotiation and configuration) a matter of IPPM or LMAP (or >coordinated) definition? > >regards >Joachim > >-- >--------------------------------------- >Dr. Joachim Fabini >Institute of Telecommunications >Vienna University of Technology >Favoritenstra=DFe 9/389 >A-1040 Vienna, Austria > >Tel: +43 1 58801-38813 >Fax: +43 1 58801-38898 >mailto: Joachim.Fabini@tuwien.ac.at >--------------------------------------- >_______________________________________________ >lmap mailing list >lmap@ietf.org >https://www.ietf.org/mailman/listinfo/lmap From acmorton@att.com Tue Jul 30 04:10:25 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C280611E81E1 for ; Tue, 30 Jul 2013 04:10:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.479 X-Spam-Level: X-Spam-Status: No, score=-106.479 tagged_above=-999 required=5 tests=[AWL=0.120, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qFCZIU8EUyt8 for ; Tue, 30 Jul 2013 04:10:19 -0700 (PDT) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id BB07A11E81DF for ; Tue, 30 Jul 2013 04:10:14 -0700 (PDT) Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id 990EA120474; Tue, 30 Jul 2013 07:10:09 -0400 (EDT) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-blue.research.att.com (Postfix) with ESMTP id 683DDF035C; Tue, 30 Jul 2013 07:10:12 -0400 (EDT) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Tue, 30 Jul 2013 07:10:12 -0400 From: "MORTON JR., ALFRED C (AL)" To: Brian Trammell , Alissa Cooper Date: Tue, 30 Jul 2013 07:10:12 -0400 Thread-Topic: [lmap] user-initiated testing Thread-Index: Ac6M/TtWfGmyb+2iQsWx9IR0OyxZxgAFY1Pb Message-ID: References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> In-Reply-To: <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 11:10:25 -0000 On User Initiated tests: Time was short and I couldn't make this point at t= he critical time. Yesterday, we (Paul and Phil and I) summed-up the related discussion with = the term "User Requested", as-in users can initiate measurement taks from the LMAP infrastructure with= the help of=20 the infrastructure, and then other desirable features follow (such as selec= tion of the best MAs or MPs and the ability to avoid inaccurate results due to overloaded test devi= ces). User initiated measurement requests are important. The evidence shows that = almost everybody wants to run tests, but they also want accurate and meaningful results, and= we won't really achieve LMAP's intent without delivering both. Al ________________________________________ From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of Brian Tram= mell [trammell@tik.ee.ethz.ch] ... Indeed, and to clarify my point from the mic, we should make a distinction = between user-requested (i.e., user sends a request to the LMAP infrastructu= re, which initiates a measurement on the user's behalf) and user-run (contr= oller runs on the user device, or measurement agent runs on the user device= without the controller's involvement). The scalability issues that Paul an= d Phil were worried about only affect the latter case (and I personally agr= ee it's probably a bad idea to try to accommodate it). Cheers, Brian _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap= From aservin@lacnic.net Tue Jul 30 05:43:34 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 01BBC11E81DE for ; Tue, 30 Jul 2013 05:43:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zhX+DQf1T7in for ; Tue, 30 Jul 2013 05:43:33 -0700 (PDT) Received: from mail.lacnic.net.uy (mail.lacnic.net.uy [IPv6:2001:13c7:7001:4000::3]) by ietfa.amsl.com (Postfix) with ESMTP id C65A711E81CB for ; Tue, 30 Jul 2013 05:43:32 -0700 (PDT) Received: from Arturos-MacBook-Pro.local (unknown [IPv6:2001:df8:0:16:8453:df21:a48:bedf]) by mail.lacnic.net.uy (Postfix) with ESMTP id 33DE2308455 for ; Tue, 30 Jul 2013 09:43:08 -0300 (UYT) Message-ID: <51F7B4F0.5090606@lacnic.net> Date: Tue, 30 Jul 2013 14:43:28 +0200 From: Arturo Servin Organization: LACNIC User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: "lmap@ietf.org" References: <20130710053057.19859.26080.idtracker@ietfa.amsl.com> <51DCF290.20906@it.uc3m.es> In-Reply-To: <51DCF290.20906@it.uc3m.es> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-LACNIC.uy-MailScanner-Information: Please contact the ISP for more information X-LACNIC.uy-MailScanner: Found to be clean X-LACNIC.uy-MailScanner-SpamCheck: X-LACNIC.uy-MailScanner-From: aservin@lacnic.net Subject: Re: [lmap] Fwd: I-D Action: draft-bagnulo-lmap-http-00.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 12:43:34 -0000 Hi, It was a shame that we didn't have time to comment or discuss this approach during the meeting. Anyway I wanted just to comment that I like the approach to use http as transport. We have used http in one of our research projects and we found it very convenient, easy to implement and a reliable way to perform, control and collect measurement data. It would be good to see this work to move forward. I will provide more specific comments later. Regards, as On 7/10/13 7:35 AM, marcelo bagnulo braun wrote: > Hi, > > We have posted this draft exploring the possibility of using http as an > LMAP protocol. > > We would be interested in hearing you comments. > > Regards, marcelo > > > > -------- Mensaje original -------- > Asunto: I-D Action: draft-bagnulo-lmap-http-00.txt > Fecha: Tue, 09 Jul 2013 22:30:57 -0700 > De: internet-drafts@ietf.org > Responder a: internet-drafts@ietf.org > Para: i-d-announce@ietf.org > > > > A New Internet-Draft is available from the on-line Internet-Drafts directories. > > > Title : Large MeAsurement Platform Protocol > Author(s) : Marcelo Bagnulo > Trevor Burbridge > Sam Crawford > Juergen Schoenwaelder > Vaibhav Bajpai > Filename : draft-bagnulo-lmap-http-00.txt > Pages : 20 > Date : 2013-07-09 > > Abstract: > This documents specifies the LMAP protocol based on HTTP for the > Control and Report in Large Scale Measurement Platforms. > > > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-bagnulo-lmap-http > > There's also a htmlized version available at: > http://tools.ietf.org/html/draft-bagnulo-lmap-http-00 > > > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ > > _______________________________________________ > I-D-Announce mailing list > I-D-Announce@ietf.org > https://www.ietf.org/mailman/listinfo/i-d-announce > Internet-Draft directories: http://www.ietf.org/shadow.html > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt > > > > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > From coverdale@sympatico.ca Tue Jul 30 06:02:18 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D761011E81CB for ; Tue, 30 Jul 2013 06:02:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.439 X-Spam-Level: X-Spam-Status: No, score=-0.439 tagged_above=-999 required=5 tests=[AWL=-1.357, BAYES_40=-0.185, MIME_8BIT_HEADER=0.3, MSGID_FROM_MTA_HEADER=0.803] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fFz0RQQoiZWJ for ; Tue, 30 Jul 2013 06:02:06 -0700 (PDT) Received: from blu0-omc1-s22.blu0.hotmail.com (blu0-omc1-s22.blu0.hotmail.com [65.55.116.33]) by ietfa.amsl.com (Postfix) with ESMTP id D371C21F9DCA for ; Tue, 30 Jul 2013 06:02:05 -0700 (PDT) Received: from BLU0-SMTP12 ([65.55.116.8]) by blu0-omc1-s22.blu0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 30 Jul 2013 06:02:04 -0700 X-EIP: [SLDmZPH6kGKYcdyEmSjh9boWGdLXdDed] X-Originating-Email: [coverdale@sympatico.ca] Message-ID: Received: from PaulNewPC ([130.129.18.59]) by BLU0-SMTP12.phx.gbl over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Tue, 30 Jul 2013 06:02:02 -0700 From: Paul Coverdale To: =?iso-8859-1?Q?'Javier_Bustos_Jim=E9nez'?= , "'lmap WG'" References: <9904FB1B0159DA42B0B887B7FA8119CA12881146@AZ-FFEXMB04.global.avaya.com> <9510D26531EF184D9017DF24659BB87F35B80337A5@EMV65-UKRD.domain1.systemhost.net> <451142E8-6E5D-47C6-9700-FB195FBA4BCE@niclabs.cl> In-Reply-To: <451142E8-6E5D-47C6-9700-FB195FBA4BCE@niclabs.cl> Date: Tue, 30 Jul 2013 15:01:56 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Ac6M9ylqpEpSketOS9KB4UW33oxPbAALK2Zw Content-Language: en-us X-OriginalArrivalTime: 30 Jul 2013 13:02:02.0174 (UTC) FILETIME=[FBB70DE0:01CE8D24] Subject: Re: [lmap] on use cases and requirements X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 13:02:18 -0000 As I mentioned at the mike, LMAP will not "measure QoE". On the other = hand, LMAP may enable some measurements to be made which can be used to = estimate QoE. ...Paul >-----Original Message----- >From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of >Javier Bustos Jim=E9nez >Sent: Tuesday, July 30, 2013 9:34 AM >To: lmap WG >Subject: Re: [lmap] on use cases and requirements > >Hello > >Watching the slides for draft-linsner-lmap-use-cases two comments come >to my mind: > >1.- QoE is defined at ISO 9241-210: Human-centred design for = interactive >systems. Quality of Experience: "a person's perceptions and responses >that result from the use or anticipated use of a product, system or >service" > >2.- About a question about how much often the probing has to be made = and >how many measures agents hould be distributed in order to be >statistically representative, I remember that we had that problem when >we built our national infrastructure and saldy the answer is "depends". >It depends in what is wanted to measure, the size and the distribution >of the population. > >Best regards >Javier > From paitken@cisco.com Tue Jul 30 06:46:50 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EC7B211E81E4 for ; Tue, 30 Jul 2013 06:46:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.395 X-Spam-Level: X-Spam-Status: No, score=-10.395 tagged_above=-999 required=5 tests=[AWL=0.204, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eHawoDW+nl-z for ; Tue, 30 Jul 2013 06:46:44 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id D2F8A11E81E2 for ; Tue, 30 Jul 2013 06:46:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3434; q=dns/txt; s=iport; t=1375192003; x=1376401603; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=nGrWmEZW8SFNsfQA+rSx8ze/LjPkR9pEfaAP+ig89ZM=; b=E0BUwMRKXXHW39xXnZ7/VQ449vjsUM1AYKeXiBXYsoNz0hnIi6TSJn4/ kKeoaP/zqyoe3klr0ItI8Q3etTNTvI2t/OPmhitPf7yLbOSjFfFVwMHox xFIshATTb78KiolfNTuz6LlBrwFD33ZcbZddpd1RtxhZUoxy3mr0QaYiU M=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AiAFAHnC91GQ/khM/2dsb2JhbABbgwY1vmSBHBZ0giQBAQEEAQEBNTYLEAsVAwklDwIWMAYBDAEFAgEBiAwMuR0Ej34HhAkDl1+GI4spgxU X-IronPort-AV: E=Sophos;i="4.89,778,1367971200"; d="scan'208";a="157745930" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-1.cisco.com with ESMTP; 30 Jul 2013 13:46:39 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r6UDkclE020286 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 30 Jul 2013 13:46:38 GMT Received: from [10.61.87.232] (ams3-vpn-dhcp6121.cisco.com [10.61.87.232]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6UDkaLY012248; Tue, 30 Jul 2013 14:46:37 +0100 (BST) Message-ID: <51F7C3BB.50304@cisco.com> Date: Tue, 30 Jul 2013 14:46:35 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: "MORTON JR., ALFRED C (AL)" , "lmap@ietf.org" References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alissa Cooper , Brian Trammell Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 13:46:50 -0000 +1 If an end-user wants to initiate a test, they must somehow contact a controller. eg, even if the user has an MA app on their smart phone, the "test now" button must somehow indicate the test request to the cellular provider's controller, which then uses the LMAP framework to initiate the test. We do not envisage a combined controller/MA device. This is important for several reasons which include: * only the controller has a "big picture" view eg if 1000's of users request a bandwidth test about the same time, the bandwidth server could be overloaded and give the wrong results. * the controller can ensure appropriate conditions for the test to be run eg if a test requires quiescent conditions on the measurement peer, only the controller can ensure that no other tests are scheduled against that peer at the same time. * individual user's measurements may have little meaning on their own, ie there's no baseline or context to be compared against. the controller can schedule several tests from similar nodes for comparison. * the collector would not be expecting the reports the collector may not currently have resources or ability to process the test results * combined controller/MA device puts the controller under control of the end users rather than within the control of a single organisation. so it's outwith the LMAP scope. So end-users _can_ request tests using the LMAP infra in the framework, and not by some other means. However, end-users aren't under control of the organisation, so they're out of scope :-) P. On 30/07/13 12:10, MORTON JR., ALFRED C (AL) wrote: > On User Initiated tests: Time was short and I couldn't make this point at the critical time. > > Yesterday, we (Paul and Phil and I) summed-up the related discussion with the term "User Requested", > as-in users can initiate measurement taks from the LMAP infrastructure with the help of > the infrastructure, and then other desirable features follow (such as selection of the best MAs or > MPs and the ability to avoid inaccurate results due to overloaded test devices). > > User initiated measurement requests are important. The evidence shows that almost everybody > wants to run tests, but they also want accurate and meaningful results, and we won't really > achieve LMAP's intent without delivering both. > > Al > > ________________________________________ > From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of Brian Trammell [trammell@tik.ee.ethz.ch] > ... > Indeed, and to clarify my point from the mic, we should make a distinction between user-requested (i.e., user sends a request to the LMAP infrastructure, which initiates a measurement on the user's behalf) and user-run (controller runs on the user device, or measurement agent runs on the user device without the controller's involvement). The scalability issues that Paul and Phil were worried about only affect the latter case (and I personally agree it's probably a bad idea to try to accommodate it). > > Cheers, > > Brian > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From Jean-Francois.TremblayING@videotron.com Tue Jul 30 07:42:18 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D5DE421E80C2 for ; Tue, 30 Jul 2013 07:42:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cxeZX5HWYU36 for ; Tue, 30 Jul 2013 07:42:12 -0700 (PDT) Received: from mx01.videotron.com (mx01.videotron.com [24.201.243.152]) by ietfa.amsl.com (Postfix) with ESMTP id 3C65421E80F5 for ; Tue, 30 Jul 2013 07:41:57 -0700 (PDT) In-Reply-To: <51F7C3BB.50304@cisco.com> References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> <51F7C3BB.50304@cisco.com> To: "lmap@ietf.org" MIME-Version: 1.0 X-KeepSent: B242B160:6F3917E9-85257BB8:004F01D6; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 8.5.3FP1 March 08, 2012 Message-ID: From: Jean-Francois.TremblayING@videotron.com Date: Tue, 30 Jul 2013 10:41:57 -0400 X-MIMETrack: Serialize by Router on DOMMSG01/SRV/GVL(Release 8.5.3FP3|November 15, 2012) at 07/30/2013 10:41:55, Serialize complete at 07/30/2013 10:41:55 Content-Type: text/plain; charset="US-ASCII" Received-SPF: none Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 14:42:18 -0000 > De : Paul Aitken > > However, end-users aren't under control of the organisation, so they're > out of scope :-) Following this from the side lines (I am not in Berlin), I'd like to voice some support to Alissa's point of vue. I believe there is some value in having user-initiated measurements in-scope, as long as the software used (either an app or a web-based interface) is consistent and is using a different collector than the ISP-owned MAs. As mentioned in the use-case draft, reliable statistics can't be produced from such a self-selecting group. But for an ISP, there is some value in seeing measurements from end-users and gathering statistics from their point of view. This is a rather rough measurement of the user's quality of experience, but it does give an idea of how that experience measures up to an ISP's advertised services. Videotron (my employer) actually already performs such measurements by running its own speedtest service and gathering data correlated to the end-user tier. It gives us a high-level view of users "happiness" levels with regards to their services. (see http://testvitesse.videotron.ca and http://ipv6.testvitesse.videotron.ca). Such data is desirable for an ISP and I believe should be in-scope. An ISP could, for example, publish an LMAP-compliant app for its users to test their speed and gather the results in a separate collector. /JF From dengguangqing@cnnic.cn Tue Jul 30 10:28:35 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A6AA621F9A2A for ; Tue, 30 Jul 2013 10:28:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.845 X-Spam-Level: X-Spam-Status: No, score=-0.845 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_BASE64_TEXT=1.753] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1c0d4vv90v+v for ; Tue, 30 Jul 2013 10:28:31 -0700 (PDT) Received: from cnnic.cn (smtp.cnnic.cn [218.241.118.7]) by ietfa.amsl.com (Postfix) with SMTP id 0C97B21F9A13 for ; Tue, 30 Jul 2013 10:28:30 -0700 (PDT) X-EYOUMAIL-SMTPAUTH: dengguangqing@cnnic.cn Received: from unknown127.0.0.1 (HELO dgq-pc) (127.0.0.1) by 127.0.0.1 with SMTP; Wed, 31 Jul 2013 01:28:18 +0800 Date: Wed, 31 Jul 2013 01:28:20 +0800 From: "Guangqing Deng" To: Jean-Francois.TremblayING References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> <51F7C3BB.50304@cisco.com>, X-Priority: 3 X-Has-Attach: no X-Mailer: Foxmail 7.0.1.91[cn] Mime-Version: 1.0 Message-ID: <2013073101281916001716@cnnic.cn> Content-Type: multipart/alternative; boundary="----=_001_NextPart605773347778_=----" Cc: "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: dengguangqing List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 17:28:35 -0000 This is a multi-part message in MIME format. ------=_001_NextPart605773347778_=---- Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: base64 VG90YWxseSBhZ3JlZS4gV2hhdCdzIG1vcmUsIGRpZmZlcmVudCB1c2VycyBtYXkgY2FyZSBhYm91 dCBkaWZmZXJlbnQgbWV0cmljcy4gU3VjaCB1c2VycyB3aG8gbGlrZSB3YXRjaGluZyBvbmxpbmUg dmlkZW9zIG1heSBjYXJlIGFib3V0IGRvd25sb2FkIHNwZWVkIHZlcnkgbXVjaDsgd2hpbGUgdGhv c2Ugd2hvIHByZWZlciBvbmxpbmUgZ2FtZXMgbWF5IGNhcmUgbmV0d29yayBkZWxheSBtb3JlLiBT bywgZGlmZmVyZW50IHVzZXJzIG1heSBwZXJmb3JtIG1lYXN1cmVtZW50cyBmb3IgZGlmZmVyZW50 IG1ldHJpY3MsIGlmIHVzZXItaW5pdGlhdGVkIG1lYXN1cmVtZW50cyBhcmUgc3VwcG9ydGVkLiBG b3IgSVNQcywgdGhleSBjYW4gbGVhcm4gbW9yZSBhYm91dCB3aGF0IHVzZXJzIGFyZSBwYXlpbmcg YXR0ZW50aW9uIHRvIHRocm91Z2ggc3VjaCBtZWFzdXJlbWVudHMgKGluaXRpYXRlZCBieSB1c2Vy cykgYW5kIHRoZW4gYWNjb3JkaW5nbHkgYWRqdXN0IHRoZWlyIG5ldHdvcmtzIHRvIG1lZXQgdGhl IGRlbWFuZCBvZiBkaWZmZXJlbnQgdXNlcnMuIFNvIGl0IHNlZW1zIHRoYXQgaGF2aW5nIHVzZXIt aW5pdGlhdGVkIG1lYXN1cmVtZW50IGluLXNjb3BlIGlzIGJlbmVmaWNpYWwgdG8gYm90aCB1c2Vy cyBhbmQgSVNQcy4gDQoNCg0KDQoNCkd1YW5ncWluZyBEZW5nDQpjbm5pYyANCg0KRnJvbTogSmVh bi1GcmFuY29pcy5UcmVtYmxheUlORw0KRGF0ZTogMjAxMy0wNy0zMCAyMjo0MQ0KVG86IGxtYXBA aWV0Zi5vcmcNClN1YmplY3Q6IFJlOiBbbG1hcF0gdXNlci1pbml0aWF0ZWQgdGVzdGluZw0KPiBE ZSA6IFBhdWwgQWl0a2VuIDxwYWl0a2VuQGNpc2NvLmNvbT4NCj4NCj4gSG93ZXZlciwgZW5kLXVz ZXJzIGFyZW4ndCB1bmRlciBjb250cm9sIG9mIHRoZSBvcmdhbmlzYXRpb24sIHNvIHRoZXkncmUg DQo+IG91dCBvZiBzY29wZSA6LSkNCg0KRm9sbG93aW5nIHRoaXMgZnJvbSB0aGUgc2lkZSBsaW5l cyAoSSBhbSBub3QgaW4gQmVybGluKSwgSSdkIGxpa2UgdG8gdm9pY2UgDQoNCnNvbWUgc3VwcG9y dCB0byBBbGlzc2EncyBwb2ludCBvZiB2dWUuIEkgYmVsaWV2ZSB0aGVyZSBpcyBzb21lIHZhbHVl IGluIA0KaGF2aW5nIHVzZXItaW5pdGlhdGVkIG1lYXN1cmVtZW50cyBpbi1zY29wZSwgYXMgbG9u ZyBhcyB0aGUgc29mdHdhcmUgdXNlZA0KKGVpdGhlciBhbiBhcHAgb3IgYSB3ZWItYmFzZWQgaW50 ZXJmYWNlKSBpcyBjb25zaXN0ZW50IGFuZCBpcyB1c2luZyBhIA0KZGlmZmVyZW50IGNvbGxlY3Rv ciB0aGFuIHRoZSBJU1Atb3duZWQgTUFzLiANCg0KQXMgbWVudGlvbmVkIGluIHRoZSB1c2UtY2Fz ZSBkcmFmdCwgcmVsaWFibGUgc3RhdGlzdGljcyBjYW4ndCBiZSBwcm9kdWNlZA0KZnJvbSBzdWNo IGEgc2VsZi1zZWxlY3RpbmcgZ3JvdXAuIEJ1dCBmb3IgYW4gSVNQLCB0aGVyZSBpcyBzb21lIHZh bHVlIGluIA0Kc2VlaW5nIG1lYXN1cmVtZW50cyBmcm9tIGVuZC11c2VycyBhbmQgZ2F0aGVyaW5n IHN0YXRpc3RpY3MgZnJvbSB0aGVpciANCnBvaW50IG9mIHZpZXcuIFRoaXMgaXMgYSByYXRoZXIg cm91Z2ggbWVhc3VyZW1lbnQgb2YgdGhlIHVzZXIncyBxdWFsaXR5DQpvZiBleHBlcmllbmNlLCBi dXQgaXQgZG9lcyBnaXZlIGFuIGlkZWEgb2YgaG93IHRoYXQgZXhwZXJpZW5jZSBtZWFzdXJlcw0K dXAgdG8gYW4gSVNQJ3MgYWR2ZXJ0aXNlZCBzZXJ2aWNlcy4gDQoNClZpZGVvdHJvbiAobXkgZW1w bG95ZXIpIGFjdHVhbGx5IGFscmVhZHkgcGVyZm9ybXMgc3VjaCBtZWFzdXJlbWVudHMgYnkgDQpy dW5uaW5nIGl0cyBvd24gc3BlZWR0ZXN0IHNlcnZpY2UgYW5kIGdhdGhlcmluZyBkYXRhIGNvcnJl bGF0ZWQgdG8gdGhlIA0KZW5kLXVzZXIgdGllci4gSXQgZ2l2ZXMgdXMgYSBoaWdoLWxldmVsIHZp ZXcgb2YgdXNlcnMgImhhcHBpbmVzcyIgbGV2ZWxzIA0Kd2l0aCByZWdhcmRzIHRvIHRoZWlyIHNl cnZpY2VzLiAoc2VlIGh0dHA6Ly90ZXN0dml0ZXNzZS52aWRlb3Ryb24uY2EgYW5kDQpodHRwOi8v aXB2Ni50ZXN0dml0ZXNzZS52aWRlb3Ryb24uY2EpLiBTdWNoIGRhdGEgaXMgZGVzaXJhYmxlIGZv ciBhbiANCklTUCBhbmQgSSBiZWxpZXZlIHNob3VsZCBiZSBpbi1zY29wZS4gDQoNCkFuIElTUCBj b3VsZCwgZm9yIGV4YW1wbGUsIHB1Ymxpc2ggYW4gTE1BUC1jb21wbGlhbnQgYXBwIGZvciBpdHMg dXNlcnMgDQp0byB0ZXN0IHRoZWlyIHNwZWVkIGFuZCBnYXRoZXIgdGhlIHJlc3VsdHMgaW4gYSBz ZXBhcmF0ZSBjb2xsZWN0b3IuIA0KDQovSkYNCg0KDQpfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXw0KbG1hcCBtYWlsaW5nIGxpc3QNCmxtYXBAaWV0Zi5vcmcN Cmh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vbG1hcA== ------=_001_NextPart605773347778_=---- Content-Type: text/html; charset="gb2312" Content-Transfer-Encoding: quoted-printable
Totally agree. What's more, different users may care about different=20 metrics. Such users who like watching online videos may care about downloa= d=20 speed very much; while those who prefer online games may care network dela= y=20 more. So, different users may perform measurements for different metrics,=20 if user-initiated measurements are supported. For ISPs, they can lear= n more=20 about what users are paying attention to through such measurements (initia= ted by=20 users) and then accordingly adjust their networks to meet the demand of=20 different users. So it seems that having user-initiated measurement=20 in-scope is beneficial to both users and ISPs. 
 
Gua= ngqing=20 Deng
cnnic 
 
From: Jean-Francois.Trem= blayING
Date: 2013-07-30 22:41
To: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing
> De : Paul Aitken <paitken@cisco.com&= gt;
>
> However, end-users aren't under control=  of the organisation, so they're 
> out of scope :-)
 
Following this from the side lines (I&n= bsp;am not in Berlin), I'd like to voic= e 
 
some support to Alissa's point of vue.&= nbsp;I believe there is some value in <= /DIV>
having user-initiated measurements in-scope, as&n= bsp;long as the software used
(either an app or a web-based interface= ) is consistent and is using a 
different collector than the ISP-owned MAs.&= nbsp;
 
As mentioned in the use-case draft, rel= iable statistics can't be produced
from such a self-selecting group. But f= or an ISP, there is some value in =
seeing measurements from end-users and gathe= ring statistics from their 
point of view. This is a rather ro= ugh measurement of the user's quality
of experience, but it does give an = ;idea of how that experience measures
up to an ISP's advertised services. 
 
Videotron (my employer) actually already per= forms such measurements by 
running its own speedtest service and g= athering data correlated to the 
end-user tier. It gives us a high-level=  view of users "happiness" levels 
with regards to their services. (see ht= tp://testvitesse.videotron.ca and
http://ipv6.testvitesse.videotron.ca). Such data is&nb= sp;desirable for an 
ISP and I believe should be in-scope.&n= bsp;
 
An ISP could, for example, publish an&n= bsp;LMAP-compliant app for its users 
to test their speed and gather the = ;results in a separate collector. 
 
/JF
 
 
_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap
------=_001_NextPart605773347778_=------ From sharam.hakimi@exfo.com Tue Jul 30 11:18:50 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6A76311E8213 for ; Tue, 30 Jul 2013 11:18:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v3MG3yXpCA47 for ; Tue, 30 Jul 2013 11:18:46 -0700 (PDT) Received: from smtpinqc.exfo.com (smtpinqc.exfo.com [206.162.164.97]) by ietfa.amsl.com (Postfix) with ESMTP id AB65D11E81DF for ; Tue, 30 Jul 2013 11:18:45 -0700 (PDT) Received: from spqcexc04.exfo.com ([172.16.48.171]) by smtpinqc.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 30 Jul 2013 14:18:44 -0400 Received: from spboexc01.exfo.com ([10.10.10.16]) by spqcexc04.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 30 Jul 2013 14:18:43 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CE8D50.072710F7" Date: Tue, 30 Jul 2013 14:10:44 -0400 Message-ID: <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [lmap] user-initiated testing Thread-Index: Ac6NSjtekpbBz+p4Qu6rftpkTnatOAABK19A References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch><51F7C3BB.50304@cisco.com>, <2013073101281916001716@cnnic.cn> From: "Sharam Hakimi" To: "dengguangqing" , "Jean-Francois.TremblayING" X-OriginalArrivalTime: 30 Jul 2013 18:18:43.0757 (UTC) FILETIME=[398A85D0:01CE8D51] Cc: lmap@ietf.org Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 18:18:50 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01CE8D50.072710F7 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Agree with capability for some user initiated tests but it does not have to involve the Controller necessarily. A Resource Authorization function that would authorize a test being done can exist in a Controller but it could also be a function outside the controller. With large numbers of possible users a distributed test authorization function would work better in my opinion. Sharam Hakimi, =20 EXFO =20 From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Guangqing Deng Sent: Tuesday, July 30, 2013 1:28 PM To: Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: Re: [lmap] user-initiated testing =20 Totally agree. What's more, different users may care about different metrics. Such users who like watching online videos may care about download speed very much; while those who prefer online games may care network delay more. So, different users may perform measurements for different metrics, if user-initiated measurements are supported. For ISPs, they can learn more about what users are paying attention to through such measurements (initiated by users) and then accordingly adjust their networks to meet the demand of different users. So it seems that having user-initiated measurement in-scope is beneficial to both users and ISPs.=20 =20 ________________________________ Guangqing Deng cnnic=20 =20 From: Jean-Francois.TremblayING =20 Date: 2013-07-30 22:41 To: lmap@ietf.org Subject: Re: [lmap] user-initiated testing > De : Paul Aitken >=20 > However, end-users aren't under control of the organisation, so they're=20 > out of scope :-) =20 Following this from the side lines (I am not in Berlin), I'd like to voice=20 =20 some support to Alissa's point of vue. I believe there is some value in=20 having user-initiated measurements in-scope, as long as the software used (either an app or a web-based interface) is consistent and is using a=20 different collector than the ISP-owned MAs.=20 =20 As mentioned in the use-case draft, reliable statistics can't be produced from such a self-selecting group. But for an ISP, there is some value in seeing measurements from end-users and gathering statistics from their=20 point of view. This is a rather rough measurement of the user's quality of experience, but it does give an idea of how that experience measures up to an ISP's advertised services.=20 =20 Videotron (my employer) actually already performs such measurements by=20 running its own speedtest service and gathering data correlated to the=20 end-user tier. It gives us a high-level view of users "happiness" levels with regards to their services. (see http://testvitesse.videotron.ca and http://ipv6.testvitesse.videotron.ca). Such data is desirable for an=20 ISP and I believe should be in-scope.=20 =20 An ISP could, for example, publish an LMAP-compliant app for its users=20 to test their speed and gather the results in a separate collector.=20 =20 /JF =20 =20 _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap ------_=_NextPart_001_01CE8D50.072710F7 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Agree with capability for some user initiated tests but = it does not have to involve the Controller necessarily. A Resource Authorization function that would authorize a test being done can exist in a = Controller but it could also be a function outside the controller. With large numbers = of possible users a distributed test authorization function would work = better in my opinion.

Sharam Hakimi, 

EXFO

 

From:= lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of = Guangqing Deng
Sent: Tuesday, July 30, 2013 1:28 PM
To: Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing

 

Totally agree. What's more, different users may care about different metrics. = Such users who like watching online videos may care about download speed very = much; while those who prefer online games may care network delay more. So, = different users may perform measurements for different metrics, = if user-initiated measurements are supported. For ISPs, they can learn more about what = users are paying attention to through such measurements (initiated by users) and = then accordingly adjust their networks to meet the demand of = different users. So it seems that having user-initiated measurement in-scope is = beneficial to both users and ISPs. 

 

Guangqing Deng
cnnic 

 

Date: 2013-07-30 22:41=

Subject: Re: [lmap] user-initiated testing

> De : Paul Aitken = ;<paitken@cisco.com>

> 

> However, end-users aren't&= nbsp;under control of the organisation, so = they're 

> out of scope :-)=

 

Following this from the si= de lines (I am not in Berlin), I'd&nbs= p;like to voice 

 

some support to Alissa's p= oint of vue. I believe there is some&n= bsp;value in 

having user-initiated measurements&n= bsp;in-scope, as long as the software used<= o:p>

(either an app or a w= eb-based interface) is consistent and is us= ing a 

different collector than the&nb= sp;ISP-owned MAs. 

 

As mentioned in the use-ca= se draft, reliable statistics can't be prod= uced

from such a self-selecting = ;group. But for an ISP, there is some&= nbsp;value in 

seeing measurements from end-us= ers and gathering statistics from their 

point of view. This is&nbs= p;a rather rough measurement of the user's&= nbsp;quality

of experience, but it does=  give an idea of how that experience&n= bsp;measures

up to an ISP's advertised&= nbsp;services. 

 

Videotron (my employer) actuall= y already performs such measurements by 

running its own speedtest = service and gathering data correlated to th= e 

end-user tier. It gives us=  a high-level view of users "happiness= " levels 

with regards to their serv= ices. (see http://testvitesse.videotron.ca and<= /span>

http://ipv6.testvitesse.videotron.ca). Su= ch data is desirable for an 

ISP and I believe should&n= bsp;be in-scope. 

 

An ISP could, for example,=  publish an LMAP-compliant app for its = ;users 

to test their speed and&nb= sp;gather the results in a separate collect= or. 

 

/JF

 

 

______________________________________________= _

lmap mailing list<= /p>

lmap@ietf.org

https://www.ietf.org/mailman/listinfo/lmap

------_=_NextPart_001_01CE8D50.072710F7-- From Michael.K.Bugenhagen@centurylink.com Tue Jul 30 11:32:02 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A0AF511E822E for ; Tue, 30 Jul 2013 11:32:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7p56PW2Qd3YF for ; Tue, 30 Jul 2013 11:31:56 -0700 (PDT) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id B999911E8226 for ; Tue, 30 Jul 2013 11:31:49 -0700 (PDT) Received: from lxdenvmpc030.qintra.com (emailout.qintra.com [10.1.51.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id r6UIURml016181 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 30 Jul 2013 13:30:27 -0500 (CDT) Received: from lxdenvmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id B79191E006C; Tue, 30 Jul 2013 12:30:21 -0600 (MDT) Received: from sudnp797.qintra.com (unknown [151.119.91.93]) by lxdenvmpc030.qintra.com (Postfix) with ESMTP id 95F741E006B; Tue, 30 Jul 2013 12:30:21 -0600 (MDT) Received: from sudnp797.qintra.com (localhost [127.0.0.1]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r6UIULGk021858; Tue, 30 Jul 2013 12:30:21 -0600 (MDT) Received: from vodcwhubex501.ctl.intranet (vodcwhubex501.qintra.com [151.117.206.27]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r6UIUK8q021825 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 30 Jul 2013 12:30:20 -0600 (MDT) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex501.ctl.intranet ([151.117.206.27]) with mapi id 14.02.0318.001; Tue, 30 Jul 2013 13:30:20 -0500 From: "Bugenhagen, Michael K" To: "'Sharam Hakimi'" , dengguangqing , "Jean-Francois.TremblayING" Thread-Topic: [lmap] user-initiated testing Thread-Index: AQHOjPs2+A7wEvhTU0SLWsc0HC741wABK19AmX2BqWA= Date: Tue, 30 Jul 2013 18:30:18 +0000 Message-ID: References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch><51F7C3BB.50304@cisco.com>, <2013073101281916001716@cnnic.cn> <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> In-Reply-To: <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.7] Content-Type: multipart/alternative; boundary="_000_A68F3CAC468B2E48BB775ACE2DD99B5E04778ECBpodcwmbxex505ct_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 18:32:02 -0000 --_000_A68F3CAC468B2E48BB775ACE2DD99B5E04778ECBpodcwmbxex505ct_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable With user "initiated" we really need to contextualize if we mean 1) User testing their own Home network Vs. 2) User hooks into an ISP test system that executes a test for them. Of course the key issue is who ownes the Measurement point at the User inte= rface (DSL or Cable modem).. These are often multi-interface, so the only = aggregated interface / access test point is generally located on the ISP si= de of the NID vs. the customers side. Regards, Mike From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Sha= ram Hakimi Sent: Tuesday, July 30, 2013 1:11 PM To: dengguangqing; Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: Re: [lmap] user-initiated testing Agree with capability for some user initiated tests but it does not have to= involve the Controller necessarily. A Resource Authorization function that= would authorize a test being done can exist in a Controller but it could a= lso be a function outside the controller. With large numbers of possible us= ers a distributed test authorization function would work better in my opini= on. Sharam Hakimi, EXFO From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of Guangqing Deng Sent: Tuesday, July 30, 2013 1:28 PM To: Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: Re: [lmap] user-initiated testing Totally agree. What's more, different users may care about different metric= s. Such users who like watching online videos may care about download speed= very much; while those who prefer online games may care network delay more= . So, different users may perform measurements for different metrics, if us= er-initiated measurements are supported. For ISPs, they can learn more abou= t what users are paying attention to through such measurements (initiated b= y users) and then accordingly adjust their networks to meet the demand of d= ifferent users. So it seems that having user-initiated measurement in-scope= is beneficial to both users and ISPs. ________________________________ Guangqing Deng cnnic From: Jean-Francois.TremblayING Date: 2013-07-30 22:41 To: lmap@ietf.org Subject: Re: [lmap] user-initiated testing > De : Paul Aitken > > > However, end-users aren't under control of the organisation, so they're > out of scope :-) Following this from the side lines (I am not in Berlin), I'd like to voice some support to Alissa's point of vue. I believe there is some value in having user-initiated measurements in-scope, as long as the software used (either an app or a web-based interface) is consistent and is using a different collector than the ISP-owned MAs. As mentioned in the use-case draft, reliable statistics can't be produced from such a self-selecting group. But for an ISP, there is some value in seeing measurements from end-users and gathering statistics from their point of view. This is a rather rough measurement of the user's quality of experience, but it does give an idea of how that experience measures up to an ISP's advertised services. Videotron (my employer) actually already performs such measurements by running its own speedtest service and gathering data correlated to the end-user tier. It gives us a high-level view of users "happiness" levels with regards to their services. (see http://testvitesse.videotron.ca and http://ipv6.testvitesse.videotron.ca). Such data is desirable for an ISP and I believe should be in-scope. An ISP could, for example, publish an LMAP-compliant app for its users to test their speed and gather the results in a separate collector. /JF _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_A68F3CAC468B2E48BB775ACE2DD99B5E04778ECBpodcwmbxex505ct_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

With user “initiate= d” we really need to contextualize if we mean

1)  &= nbsp;   User testing thei= r own Home network

Vs.

2)  &= nbsp;   User hooks into a= n ISP test system that executes a test for them.

Of course the key issue is who ownes the Measurement point at the User= interface (DSL or Cable modem)..  These are often multi-interface, so the only aggregated interface / access test point is generally located = on the ISP side of the NID vs. the customers side.

Regards,

Mike

 <= /p>

From: lmap-bounces@ietf.org [mailto:lmap-bounc= es@ietf.org] On Behalf Of Sharam Hakimi
Sent: Tuesday, July 30, 2013 1:11 PM
To: dengguangqing; Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing

 

Agree with capability for= some user initiated tests but it does not have to involve the Controller n= ecessarily. A Resource Authorization function that would authorize a test being done can exist in a Controller but it could also be= a function outside the controller. With large numbers of possible users a = distributed test authorization function would work better in my opinion.

Sharam Hakimi, 

EXFO

 <= /p>

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Guangqing Deng
Sent: Tuesday, July 30, 2013 1:28 PM
To: Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing

 

= Totally agree. What's more, different users may care about different metric= s. Such users who like watching online videos may care about download speed very much; while those who prefer online games m= ay care network delay more. So, different users may perform measurements fo= r different metrics, if user-initiated measurements are supported. For= ISPs, they can learn more about what users are paying attention to through such measurements (initiated by user= s) and then accordingly adjust their networks to meet the demand of differe= nt users. So it seems that having user-initiated measurement in-scope = is beneficial to both users and ISPs. 

=  

Guangqing Deng
cnnic 

=  

Date: 2013-07-30 22:41=

Subject: Re: [lmap] user-initiated tes= ting

= > De : Paul Aitken <paitken@cisco.com>

= > 

= > However, end-users aren't under control = of the organisation, so they're =

= > out of scope :-)

=  

= Following this from the side lines (I am=  not in Berlin), I'd like to voice =

=  

= some support to Alissa's point of vue. I=  believe there is some value in 

= having user-initiated measurements in-scope, as lo= ng as the software used

= (either an app or a web-based interface) = ;is consistent and is using a 

= different collector than the ISP-owned MAs. <= o:p>

=  

= As mentioned in the use-case draft, reliable&= nbsp;statistics can't be produced

= from such a self-selecting group. But for&nbs= p;an ISP, there is some value in 

= seeing measurements from end-users and gathering&n= bsp;statistics from their 

= point of view. This is a rather rough&nb= sp;measurement of the user's quality<= /p>

= of experience, but it does give an idea&= nbsp;of how that experience measures<= /p>

= up to an ISP's advertised services. 

=  

= Videotron (my employer) actually already performs&= nbsp;such measurements by 

= running its own speedtest service and gatheri= ng data correlated to the 

= end-user tier. It gives us a high-level = view of users "happiness" levels 

= with regards to their services. (see http://testvitesse.videotron.ca&nb= sp;and

= http://ipv6.testvitesse.vi= deotron.ca). Such data is desirable for a= n 

= ISP and I believe should be in-scope. 

=  

= An ISP could, for example, publish an LM= AP-compliant app for its users <= /p>

= to test their speed and gather the resul= ts in a separate collector. 

=  

= /JF

=  

=  

= _______________________________________________

= lmap mailing list

--_000_A68F3CAC468B2E48BB775ACE2DD99B5E04778ECBpodcwmbxex505ct_-- From sharam.hakimi@exfo.com Tue Jul 30 11:50:52 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6893021E8093 for ; Tue, 30 Jul 2013 11:50:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U2Eb7MXMf4h7 for ; Tue, 30 Jul 2013 11:50:44 -0700 (PDT) Received: from smtpinqc.exfo.com (smtpinqc.exfo.com [206.162.164.97]) by ietfa.amsl.com (Postfix) with ESMTP id 2ECE021F96DA for ; Tue, 30 Jul 2013 11:50:43 -0700 (PDT) Received: from spqcexc04.exfo.com ([172.16.48.171]) by smtpinqc.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 30 Jul 2013 14:50:41 -0400 Received: from spboexc01.exfo.com ([10.10.10.16]) by spqcexc04.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 30 Jul 2013 14:50:40 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CE8D55.AC08D1BF" Date: Tue, 30 Jul 2013 14:51:08 -0400 Message-ID: <084CDC75FEC1E640B60338273BEACDFA02732F25@spboexc01.exfo.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [lmap] user-initiated testing Thread-Index: AQHOjPs2+A7wEvhTU0SLWsc0HC741wABK19AmX2BqWCAAAWjAA== References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch><51F7C3BB.50304@cisco.com>, <2013073101281916001716@cnnic.cn> <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> From: "Sharam Hakimi" To: "Bugenhagen, Michael K" , "dengguangqing" , "Jean-Francois.TremblayING" X-OriginalArrivalTime: 30 Jul 2013 18:50:40.0823 (UTC) FILETIME=[B0339470:01CE8D55] Cc: lmap@ietf.org Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 18:50:52 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01CE8D55.AC08D1BF Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Mike , I think there is one more case: 3) User uses an ISP test system to perform a test for the user itself Regards, Sharam =20 From: Bugenhagen, Michael K [mailto:Michael.K.Bugenhagen@centurylink.com]=20 Sent: Tuesday, July 30, 2013 2:30 PM To: Sharam Hakimi; dengguangqing; Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: RE: [lmap] user-initiated testing =20 With user "initiated" we really need to contextualize if we mean 1) User testing their own Home network=20 Vs. 2) User hooks into an ISP test system that executes a test for them. Of course the key issue is who ownes the Measurement point at the User interface (DSL or Cable modem).. These are often multi-interface, so the only aggregated interface / access test point is generally located on the ISP side of the NID vs. the customers side. Regards, Mike =20 From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Sharam Hakimi Sent: Tuesday, July 30, 2013 1:11 PM To: dengguangqing; Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: Re: [lmap] user-initiated testing =20 Agree with capability for some user initiated tests but it does not have to involve the Controller necessarily. A Resource Authorization function that would authorize a test being done can exist in a Controller but it could also be a function outside the controller. With large numbers of possible users a distributed test authorization function would work better in my opinion. Sharam Hakimi, =20 EXFO =20 From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Guangqing Deng Sent: Tuesday, July 30, 2013 1:28 PM To: Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: Re: [lmap] user-initiated testing =20 Totally agree. What's more, different users may care about different metrics. Such users who like watching online videos may care about download speed very much; while those who prefer online games may care network delay more. So, different users may perform measurements for different metrics, if user-initiated measurements are supported. For ISPs, they can learn more about what users are paying attention to through such measurements (initiated by users) and then accordingly adjust their networks to meet the demand of different users. So it seems that having user-initiated measurement in-scope is beneficial to both users and ISPs.=20 =20 ________________________________ Guangqing Deng cnnic=20 =20 From: Jean-Francois.TremblayING =20 Date: 2013-07-30 22:41 To: lmap@ietf.org Subject: Re: [lmap] user-initiated testing > De : Paul Aitken >=20 > However, end-users aren't under control of the organisation, so they're=20 > out of scope :-) =20 Following this from the side lines (I am not in Berlin), I'd like to voice=20 =20 some support to Alissa's point of vue. I believe there is some value in=20 having user-initiated measurements in-scope, as long as the software used (either an app or a web-based interface) is consistent and is using a=20 different collector than the ISP-owned MAs.=20 =20 As mentioned in the use-case draft, reliable statistics can't be produced from such a self-selecting group. But for an ISP, there is some value in seeing measurements from end-users and gathering statistics from their=20 point of view. This is a rather rough measurement of the user's quality of experience, but it does give an idea of how that experience measures up to an ISP's advertised services.=20 =20 Videotron (my employer) actually already performs such measurements by=20 running its own speedtest service and gathering data correlated to the=20 end-user tier. It gives us a high-level view of users "happiness" levels with regards to their services. (see http://testvitesse.videotron.ca and http://ipv6.testvitesse.videotron.ca). Such data is desirable for an=20 ISP and I believe should be in-scope.=20 =20 An ISP could, for example, publish an LMAP-compliant app for its users=20 to test their speed and gather the results in a separate collector.=20 =20 /JF =20 =20 _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap ------_=_NextPart_001_01CE8D55.AC08D1BF Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Mike ,

I think there is one more case:

3)    User uses an ISP test = system to perform a test for the user itself

Regards,

Sharam

 

From:= = Bugenhagen, Michael K [mailto:Michael.K.Bugenhagen@centurylink.com]
Sent: Tuesday, July 30, 2013 2:30 PM
To: Sharam Hakimi; dengguangqing; Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: RE: [lmap] user-initiated testing

 

With user “initiated” we really need to contextualize if we mean

1)      = User testing their own Home network

Vs.

2)      = User hooks into an ISP test system that executes a test for = them.

Of course the key = issue is who ownes the Measurement point at the User interface (DSL or Cable modem)..  These are often multi-interface, so the only aggregated interface / access test point is generally located on the ISP side of = the NID vs. the customers side.

Regards,

Mike<= /p>

 

From:= lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of = Sharam Hakimi
Sent: Tuesday, July 30, 2013 1:11 PM
To: dengguangqing; Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing

 

Agree with capability for some user initiated tests but = it does not have to involve the Controller necessarily. A Resource Authorization function that would authorize a test being done can exist in a = Controller but it could also be a function outside the controller. With large numbers = of possible users a distributed test authorization function would work = better in my opinion.

Sharam Hakimi, 

EXFO

 

From:= lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] = On Behalf Of Guangqing Deng
Sent: Tuesday, July 30, 2013 1:28 PM
To: Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing

 

Totally agree. What's more, different users may care about different metrics. = Such users who like watching online videos may care about download speed very = much; while those who prefer online games may care network delay more. So, = different users may perform measurements for different metrics, = if user-initiated measurements are supported. For ISPs, they can learn more about what = users are paying attention to through such measurements (initiated by users) and = then accordingly adjust their networks to meet the demand of different users. So it = seems that having user-initiated measurement in-scope is beneficial to both = users and ISPs. 

 

Guangqing Deng
cnnic 

 

Date: 2013-07-30 22:41=

Subject: Re: [lmap] user-initiated testing

> De : Paul Aitken = ;<paitken@cisco.com>

> 

> However, end-users aren't&= nbsp;under control of the organisation, so = they're 

> out of scope :-)=

 

Following this from the si= de lines (I am not in Berlin), I'd&nbs= p;like to voice 

 

some support to Alissa's p= oint of vue. I believe there is some&n= bsp;value in 

having user-initiated measurements&n= bsp;in-scope, as long as the software used<= o:p>

(either an app or a w= eb-based interface) is consistent and is us= ing a 

different collector than the&nb= sp;ISP-owned MAs. 

 

As mentioned in the use-ca= se draft, reliable statistics can't be prod= uced

from such a self-selecting = ;group. But for an ISP, there is some&= nbsp;value in 

seeing measurements from end-us= ers and gathering statistics from their 

point of view. This is&nbs= p;a rather rough measurement of the user's&= nbsp;quality

of experience, but it does=  give an idea of how that experience&n= bsp;measures

up to an ISP's advertised&= nbsp;services. 

 

Videotron (my employer) actuall= y already performs such measurements by 

running its own speedtest = service and gathering data correlated to th= e 

end-user tier. It gives us=  a high-level view of users "happiness= " levels 

with regards to their serv= ices. (see http://testvitesse.videotron.ca<= /a> and

http://ipv6.testvitesse.vid= eotron.ca). Such data is desirable for = an 

ISP and I believe should&n= bsp;be in-scope. 

 

An ISP could, for example,=  publish an LMAP-compliant app for its = ;users 

to test their speed and&nb= sp;gather the results in a separate collect= or. 

 

/JF

 

 

______________________________________________= _

lmap mailing list<= /p>

------_=_NextPart_001_01CE8D55.AC08D1BF-- From Michael.K.Bugenhagen@centurylink.com Tue Jul 30 11:53:19 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9167221E80A3 for ; Tue, 30 Jul 2013 11:53:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cWF0N+Mi9skZ for ; Tue, 30 Jul 2013 11:53:14 -0700 (PDT) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id 2699121E8089 for ; Tue, 30 Jul 2013 11:53:14 -0700 (PDT) Received: from lxdenvmpc030.qintra.com (emailout.qintra.com [10.1.51.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id r6UIqvZM006297 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 30 Jul 2013 13:52:57 -0500 (CDT) Received: from lxdenvmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id BE1441E0069; Tue, 30 Jul 2013 12:52:51 -0600 (MDT) Received: from suomp60i.qintra.com (unknown [151.119.91.93]) by lxdenvmpc030.qintra.com (Postfix) with ESMTP id 729151E0049; Tue, 30 Jul 2013 12:52:51 -0600 (MDT) Received: from suomp60i.qintra.com (localhost [127.0.0.1]) by suomp60i.qintra.com (8.14.4/8.14.4) with ESMTP id r6UIqoR7009784; Tue, 30 Jul 2013 13:52:50 -0500 (CDT) Received: from vodcwhubex501.ctl.intranet (vodcwhubex501.qintra.com [151.117.206.27]) by suomp60i.qintra.com (8.14.4/8.14.4) with ESMTP id r6UIqoVV009750 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 30 Jul 2013 13:52:50 -0500 (CDT) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex501.ctl.intranet ([151.117.206.27]) with mapi id 14.02.0318.001; Tue, 30 Jul 2013 13:52:49 -0500 From: "Bugenhagen, Michael K" To: "'Sharam Hakimi'" , dengguangqing , "Jean-Francois.TremblayING" Thread-Topic: [lmap] user-initiated testing Thread-Index: AQHOjPs2+A7wEvhTU0SLWsc0HC741wABK19AmX2BqWCAAAWjAIAAAROA Date: Tue, 30 Jul 2013 18:52:49 +0000 Message-ID: References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch><51F7C3BB.50304@cisco.com>, <2013073101281916001716@cnnic.cn> <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> <084CDC75FEC1E640B60338273BEACDFA02732F25@spboexc01.exfo.com> In-Reply-To: <084CDC75FEC1E640B60338273BEACDFA02732F25@spboexc01.exfo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.7] Content-Type: multipart/alternative; boundary="_000_A68F3CAC468B2E48BB775ACE2DD99B5E04779154podcwmbxex505ct_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 18:53:19 -0000 --_000_A68F3CAC468B2E48BB775ACE2DD99B5E04779154podcwmbxex505ct_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Good catch.... I'd call in a "2b" user case,,, but yes the ISP system test from the B= roadband Access NID aggregation device to the Internet, or to the customer.= .. From: Sharam Hakimi [mailto:sharam.hakimi@exfo.com] Sent: Tuesday, July 30, 2013 1:51 PM To: Bugenhagen, Michael K; dengguangqing; Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: RE: [lmap] user-initiated testing Mike , I think there is one more case: 3) User uses an ISP test system to perform a test for the user itself Regards, Sharam From: Bugenhagen, Michael K [mailto:Michael.K.Bugenhagen@centurylink.com] Sent: Tuesday, July 30, 2013 2:30 PM To: Sharam Hakimi; dengguangqing; Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: RE: [lmap] user-initiated testing With user "initiated" we really need to contextualize if we mean 1) User testing their own Home network Vs. 2) User hooks into an ISP test system that executes a test for them. Of course the key issue is who ownes the Measurement point at the User inte= rface (DSL or Cable modem).. These are often multi-interface, so the only = aggregated interface / access test point is generally located on the ISP si= de of the NID vs. the customers side. Regards, Mike From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of Sharam Hakimi Sent: Tuesday, July 30, 2013 1:11 PM To: dengguangqing; Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: Re: [lmap] user-initiated testing Agree with capability for some user initiated tests but it does not have to= involve the Controller necessarily. A Resource Authorization function that= would authorize a test being done can exist in a Controller but it could a= lso be a function outside the controller. With large numbers of possible us= ers a distributed test authorization function would work better in my opini= on. Sharam Hakimi, EXFO From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of Guangqing Deng Sent: Tuesday, July 30, 2013 1:28 PM To: Jean-Francois.TremblayING Cc: lmap@ietf.org Subject: Re: [lmap] user-initiated testing Totally agree. What's more, different users may care about different metric= s. Such users who like watching online videos may care about download speed= very much; while those who prefer online games may care network delay more= . So, different users may perform measurements for different metrics, if us= er-initiated measurements are supported. For ISPs, they can learn more abou= t what users are paying attention to through such measurements (initiated b= y users) and then accordingly adjust their networks to meet the demand of d= ifferent users. So it seems that having user-initiated measurement in-scope= is beneficial to both users and ISPs. ________________________________ Guangqing Deng cnnic From: Jean-Francois.TremblayING Date: 2013-07-30 22:41 To: lmap@ietf.org Subject: Re: [lmap] user-initiated testing > De : Paul Aitken > > > However, end-users aren't under control of the organisation, so they're > out of scope :-) Following this from the side lines (I am not in Berlin), I'd like to voice some support to Alissa's point of vue. I believe there is some value in having user-initiated measurements in-scope, as long as the software used (either an app or a web-based interface) is consistent and is using a different collector than the ISP-owned MAs. As mentioned in the use-case draft, reliable statistics can't be produced from such a self-selecting group. But for an ISP, there is some value in seeing measurements from end-users and gathering statistics from their point of view. This is a rather rough measurement of the user's quality of experience, but it does give an idea of how that experience measures up to an ISP's advertised services. Videotron (my employer) actually already performs such measurements by running its own speedtest service and gathering data correlated to the end-user tier. It gives us a high-level view of users "happiness" levels with regards to their services. (see http://testvitesse.videotron.ca and http://ipv6.testvitesse.videotron.ca). Such data is desirable for an ISP and I believe should be in-scope. An ISP could, for example, publish an LMAP-compliant app for its users to test their speed and gather the results in a separate collector. /JF _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap --_000_A68F3CAC468B2E48BB775ACE2DD99B5E04779154podcwmbxex505ct_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Good catch…. 

   I’= ;d call in a “2b” user case,,, but yes the  ISP system tes= t from the  Broadband Access NID aggregation device to the Internet, o= r to the customer…

 <= /p>

 <= /p>

 <= /p>

From: Sharam Hakimi [mailto:sharam.hakimi@exfo= .com]
Sent: Tuesday, July 30, 2013 1:51 PM
To: Bugenhagen, Michael K; dengguangqing; Jean-Francois.TremblayING<= br> Cc: lmap@ietf.org
Subject: RE: [lmap] user-initiated testing

 

Mike ,<= /p>

I think there is one more= case:

3)   &nb= sp;User uses an ISP test system to perform a test for the user itself<= /o:p>

Regards,

Sharam<= /span>

 <= /p>

From: Bugenhagen, Michael K [mailto:Michael.K.Bugenhagen@centuryl= ink.com]
Sent: Tuesday, July 30, 2013 2:30 PM
To: Sharam Hakimi; dengguangqing; Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: RE: [lmap] user-initiated testing

 

With user “initiate= d” we really need to contextualize if we mean

1)  &= nbsp;   User testing thei= r own Home network

Vs.

2)  &= nbsp;   User hooks into a= n ISP test system that executes a test for them.

Of course the key issue is who ownes the Measurement point at the User= interface (DSL or Cable modem)..  These are often multi-interface, so the only aggregated interface / access test point is generally located = on the ISP side of the NID vs. the customers side.

Regards,

Mike

 <= /p>

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Sharam Hakimi
Sent: Tuesday, July 30, 2013 1:11 PM
To: dengguangqing; Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing

 

Agree with capability for= some user initiated tests but it does not have to involve the Controller n= ecessarily. A Resource Authorization function that would authorize a test being done can exist in a Controller but it could also be= a function outside the controller. With large numbers of possible users a = distributed test authorization function would work better in my opinion.

Sharam Hakimi, 

EXFO

 <= /p>

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Guangqing Deng
Sent: Tuesday, July 30, 2013 1:28 PM
To: Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing

 

= Totally agree. What's more, different users may care about different metric= s. Such users who like watching online videos may care about download speed very much; while those who prefer online games m= ay care network delay more. So, different users may perform measurements fo= r different metrics, if user-initiated measurements are supported. For= ISPs, they can learn more about what users are paying attention to through such measurements (initiated by user= s) and then accordingly adjust their networks to meet the demand of differe= nt users. So it seems that having user-initiated measurement in-scope = is beneficial to both users and ISPs. 

=  

Guangqing Deng
cnnic 

=  

Date: 2013-07-30 22:41=

Subject: Re: [lmap] user-initiated tes= ting

= > De : Paul Aitken <paitken@cisco.com>

= > 

= > However, end-users aren't under control = of the organisation, so they're =

= > out of scope :-)

=  

= Following this from the side lines (I am=  not in Berlin), I'd like to voice =

=  

= some support to Alissa's point of vue. I=  believe there is some value in 

= having user-initiated measurements in-scope, as lo= ng as the software used

= (either an app or a web-based interface) = ;is consistent and is using a 

= different collector than the ISP-owned MAs. <= o:p>

=  

= As mentioned in the use-case draft, reliable&= nbsp;statistics can't be produced

= from such a self-selecting group. But for&nbs= p;an ISP, there is some value in 

= seeing measurements from end-users and gathering&n= bsp;statistics from their 

= point of view. This is a rather rough&nb= sp;measurement of the user's quality<= /p>

= of experience, but it does give an idea&= nbsp;of how that experience measures<= /p>

= up to an ISP's advertised services. 

=  

= Videotron (my employer) actually already performs&= nbsp;such measurements by 

= running its own speedtest service and gatheri= ng data correlated to the 

= end-user tier. It gives us a high-level = view of users "happiness" levels 

= with regards to their services. (see http://testvitesse.videotron.ca&nb= sp;and

= http://ipv6.testvitesse.vi= deotron.ca). Such data is desirable for a= n 

= ISP and I believe should be in-scope. 

=  

= An ISP could, for example, publish an LM= AP-compliant app for its users <= /p>

= to test their speed and gather the resul= ts in a separate collector. 

=  

= /JF

=  

=  

= _______________________________________________

= lmap mailing list

--_000_A68F3CAC468B2E48BB775ACE2DD99B5E04779154podcwmbxex505ct_-- From dengguangqing@cnnic.cn Tue Jul 30 18:18:16 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 298EB11E80E7 for ; Tue, 30 Jul 2013 18:18:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.845 X-Spam-Level: X-Spam-Status: No, score=-0.845 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_BASE64_TEXT=1.753] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yMAToLZr8aGe for ; Tue, 30 Jul 2013 18:18:12 -0700 (PDT) Received: from cnnic.cn (smtp.cnnic.cn [218.241.118.7]) by ietfa.amsl.com (Postfix) with SMTP id 28CF911E80EF for ; Tue, 30 Jul 2013 18:18:10 -0700 (PDT) X-EYOUMAIL-SMTPAUTH: dengguangqing@cnnic.cn Received: from unknown127.0.0.1 (HELO dgq-pc) (127.0.0.1) by 127.0.0.1 with SMTP; Wed, 31 Jul 2013 09:18:04 +0800 Date: Wed, 31 Jul 2013 09:18:06 +0800 From: "Guangqing Deng" To: "Sharam Hakimi" , =?gb2312?B?QnVnZW5oYWdlbiwgTWljaGFlbCBL?= , Jean-Francois.TremblayING References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch><51F7C3BB.50304@cisco.com>, <2013073101281916001716@cnnic.cn> <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> , <084CDC75FEC1E640B60338273BEACDFA02732F25@spboexc01.exfo.com> X-Priority: 3 X-Has-Attach: no X-Mailer: Foxmail 7.0.1.91[cn] Mime-Version: 1.0 Message-ID: <201307310918056784263@cnnic.cn> Content-Type: multipart/alternative; boundary="----=_001_NextPart660286820285_=----" Cc: "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: dengguangqing List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 01:18:16 -0000 This is a multi-part message in MIME format. ------=_001_NextPart660286820285_=---- Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: base64 SSB0aGluayB0aGUgSVNQcycgTE1BUCBtZWFzdXJlbWVudCBpbmZyYXN0cnVjdHVyZSBzaG91bGQg aGF2ZSB0aGUgYWJpbGl0eSB0byBwZXJmb3JtIHRoZSB1c2VyLWluaXRpYXRlZCBtZWFzdXJlbWVu dCBzaW5jZSB0aGUgdXNlcnMgYXJlIHRoZSBsYXJnZXN0IGdyb3VwIG9mIHBhcnRpY2lwYW50cyBv ZiBhbGwga2luZHMgb2YgbmV0d29ya3MgYW5kIHRoZXkgaGF2ZSB0aGUgcmVxdWlyZW1lbnQgdGhh dCB0aGUgdXNlci1pbml0aWF0ZWQgbWVhc3VyZW1lbnQgY2FuIGJlIHBlcmZvcm1lZCBhdCBhbnkg dGltZSB3aGVuIHVzZXJzIGhhdmUgYSBtZWFzdXJlbWVudCByZXF1aXJlbWVudCAoZm9yIGluc3Rh bmNlLCB3aGVuIHRoZXJlIGlzIHNvbWV0aGluZyB3cm9uZyB3aXRoIHVzZXJzJyBob21lIG5ldHdv cmspLiANCg0KDQoNCg0KR3VhbmdxaW5nIERlbmcNCmNubmljIA0KDQpGcm9tOiBTaGFyYW0gSGFr aW1pDQpEYXRlOiAyMDEzLTA3LTMxIDAyOjUxDQpUbzogQnVnZW5oYWdlbiwgTWljaGFlbCBLOyBk ZW5nZ3VhbmdxaW5nOyBKZWFuLUZyYW5jb2lzLlRyZW1ibGF5SU5HDQpDQzogbG1hcA0KU3ViamVj dDogUmU6IFtsbWFwXSB1c2VyLWluaXRpYXRlZCB0ZXN0aW5nDQpNaWtlICwNCkkgdGhpbmsgdGhl cmUgaXMgb25lIG1vcmUgY2FzZToNCjMpICAgIFVzZXIgdXNlcyBhbiBJU1AgdGVzdCBzeXN0ZW0g dG8gcGVyZm9ybSBhIHRlc3QgZm9yIHRoZSB1c2VyIGl0c2VsZg0KUmVnYXJkcywNClNoYXJhbQ0K IA0KRnJvbTogQnVnZW5oYWdlbiwgTWljaGFlbCBLIFttYWlsdG86TWljaGFlbC5LLkJ1Z2VuaGFn ZW5AY2VudHVyeWxpbmsuY29tXSANClNlbnQ6IFR1ZXNkYXksIEp1bHkgMzAsIDIwMTMgMjozMCBQ TQ0KVG86IFNoYXJhbSBIYWtpbWk7IGRlbmdndWFuZ3Fpbmc7IEplYW4tRnJhbmNvaXMuVHJlbWJs YXlJTkcNCkNjOiBsbWFwQGlldGYub3JnDQpTdWJqZWN0OiBSRTogW2xtYXBdIHVzZXItaW5pdGlh dGVkIHRlc3RpbmcNCiANCldpdGggdXNlciChsGluaXRpYXRlZKGxIHdlIHJlYWxseSBuZWVkIHRv IGNvbnRleHR1YWxpemUgaWYgd2UgbWVhbg0KMSkgICAgICBVc2VyIHRlc3RpbmcgdGhlaXIgb3du IEhvbWUgbmV0d29yayANClZzLg0KMikgICAgICBVc2VyIGhvb2tzIGludG8gYW4gSVNQIHRlc3Qg c3lzdGVtIHRoYXQgZXhlY3V0ZXMgYSB0ZXN0IGZvciB0aGVtLg0KT2YgY291cnNlIHRoZSBrZXkg aXNzdWUgaXMgd2hvIG93bmVzIHRoZSBNZWFzdXJlbWVudCBwb2ludCBhdCB0aGUgVXNlciBpbnRl cmZhY2UgKERTTCBvciBDYWJsZSBtb2RlbSkuLiAgVGhlc2UgYXJlIG9mdGVuIG11bHRpLWludGVy ZmFjZSwgc28gdGhlIG9ubHkgYWdncmVnYXRlZCBpbnRlcmZhY2UgLyBhY2Nlc3MgdGVzdCBwb2lu dCBpcyBnZW5lcmFsbHkgbG9jYXRlZCBvbiB0aGUgSVNQIHNpZGUgb2YgdGhlIE5JRCB2cy4gdGhl IGN1c3RvbWVycyBzaWRlLg0KUmVnYXJkcywNCk1pa2UNCiANCkZyb206IGxtYXAtYm91bmNlc0Bp ZXRmLm9yZyBbbWFpbHRvOmxtYXAtYm91bmNlc0BpZXRmLm9yZ10gT24gQmVoYWxmIE9mIFNoYXJh bSBIYWtpbWkNClNlbnQ6IFR1ZXNkYXksIEp1bHkgMzAsIDIwMTMgMToxMSBQTQ0KVG86IGRlbmdn dWFuZ3Fpbmc7IEplYW4tRnJhbmNvaXMuVHJlbWJsYXlJTkcNCkNjOiBsbWFwQGlldGYub3JnDQpT dWJqZWN0OiBSZTogW2xtYXBdIHVzZXItaW5pdGlhdGVkIHRlc3RpbmcNCiANCkFncmVlIHdpdGgg Y2FwYWJpbGl0eSBmb3Igc29tZSB1c2VyIGluaXRpYXRlZCB0ZXN0cyBidXQgaXQgZG9lcyBub3Qg aGF2ZSB0byBpbnZvbHZlIHRoZSBDb250cm9sbGVyIG5lY2Vzc2FyaWx5LiBBIFJlc291cmNlIEF1 dGhvcml6YXRpb24gZnVuY3Rpb24gdGhhdCB3b3VsZCBhdXRob3JpemUgYSB0ZXN0IGJlaW5nIGRv bmUgY2FuIGV4aXN0IGluIGEgQ29udHJvbGxlciBidXQgaXQgY291bGQgYWxzbyBiZSBhIGZ1bmN0 aW9uIG91dHNpZGUgdGhlIGNvbnRyb2xsZXIuIFdpdGggbGFyZ2UgbnVtYmVycyBvZiBwb3NzaWJs ZSB1c2VycyBhIGRpc3RyaWJ1dGVkIHRlc3QgYXV0aG9yaXphdGlvbiBmdW5jdGlvbiB3b3VsZCB3 b3JrIGJldHRlciBpbiBteSBvcGluaW9uLg0KU2hhcmFtIEhha2ltaSwgIA0KRVhGTw0KIA0KRnJv bTogbG1hcC1ib3VuY2VzQGlldGYub3JnIFttYWlsdG86bG1hcC1ib3VuY2VzQGlldGYub3JnXSBP biBCZWhhbGYgT2YgR3VhbmdxaW5nIERlbmcNClNlbnQ6IFR1ZXNkYXksIEp1bHkgMzAsIDIwMTMg MToyOCBQTQ0KVG86IEplYW4tRnJhbmNvaXMuVHJlbWJsYXlJTkcNCkNjOiBsbWFwQGlldGYub3Jn DQpTdWJqZWN0OiBSZTogW2xtYXBdIHVzZXItaW5pdGlhdGVkIHRlc3RpbmcNCiANClRvdGFsbHkg YWdyZWUuIFdoYXQncyBtb3JlLCBkaWZmZXJlbnQgdXNlcnMgbWF5IGNhcmUgYWJvdXQgZGlmZmVy ZW50IG1ldHJpY3MuIFN1Y2ggdXNlcnMgd2hvIGxpa2Ugd2F0Y2hpbmcgb25saW5lIHZpZGVvcyBt YXkgY2FyZSBhYm91dCBkb3dubG9hZCBzcGVlZCB2ZXJ5IG11Y2g7IHdoaWxlIHRob3NlIHdobyBw cmVmZXIgb25saW5lIGdhbWVzIG1heSBjYXJlIG5ldHdvcmsgZGVsYXkgbW9yZS4gU28sIGRpZmZl cmVudCB1c2VycyBtYXkgcGVyZm9ybSBtZWFzdXJlbWVudHMgZm9yIGRpZmZlcmVudCBtZXRyaWNz LCBpZiB1c2VyLWluaXRpYXRlZCBtZWFzdXJlbWVudHMgYXJlIHN1cHBvcnRlZC4gRm9yIElTUHMs IHRoZXkgY2FuIGxlYXJuIG1vcmUgYWJvdXQgd2hhdCB1c2VycyBhcmUgcGF5aW5nIGF0dGVudGlv biB0byB0aHJvdWdoIHN1Y2ggbWVhc3VyZW1lbnRzIChpbml0aWF0ZWQgYnkgdXNlcnMpIGFuZCB0 aGVuIGFjY29yZGluZ2x5IGFkanVzdCB0aGVpciBuZXR3b3JrcyB0byBtZWV0IHRoZSBkZW1hbmQg b2YgZGlmZmVyZW50IHVzZXJzLiBTbyBpdCBzZWVtcyB0aGF0IGhhdmluZyB1c2VyLWluaXRpYXRl ZCBtZWFzdXJlbWVudCBpbi1zY29wZSBpcyBiZW5lZmljaWFsIHRvIGJvdGggdXNlcnMgYW5kIElT UHMuIA0KIA0KDQoNCg0KR3VhbmdxaW5nIERlbmcNCmNubmljIA0KIA0KRnJvbTogSmVhbi1GcmFu Y29pcy5UcmVtYmxheUlORw0KRGF0ZTogMjAxMy0wNy0zMCAyMjo0MQ0KVG86IGxtYXBAaWV0Zi5v cmcNClN1YmplY3Q6IFJlOiBbbG1hcF0gdXNlci1pbml0aWF0ZWQgdGVzdGluZw0KPiBEZSA6IFBh dWwgQWl0a2VuIDxwYWl0a2VuQGNpc2NvLmNvbT4NCj4gDQo+IEhvd2V2ZXIsIGVuZC11c2VycyBh cmVuJ3QgdW5kZXIgY29udHJvbCBvZiB0aGUgb3JnYW5pc2F0aW9uLCBzbyB0aGV5J3JlIA0KPiBv dXQgb2Ygc2NvcGUgOi0pDQogDQpGb2xsb3dpbmcgdGhpcyBmcm9tIHRoZSBzaWRlIGxpbmVzIChJ IGFtIG5vdCBpbiBCZXJsaW4pLCBJJ2QgbGlrZSB0byB2b2ljZSANCiANCnNvbWUgc3VwcG9ydCB0 byBBbGlzc2EncyBwb2ludCBvZiB2dWUuIEkgYmVsaWV2ZSB0aGVyZSBpcyBzb21lIHZhbHVlIGlu IA0KaGF2aW5nIHVzZXItaW5pdGlhdGVkIG1lYXN1cmVtZW50cyBpbi1zY29wZSwgYXMgbG9uZyBh cyB0aGUgc29mdHdhcmUgdXNlZA0KKGVpdGhlciBhbiBhcHAgb3IgYSB3ZWItYmFzZWQgaW50ZXJm YWNlKSBpcyBjb25zaXN0ZW50IGFuZCBpcyB1c2luZyBhIA0KZGlmZmVyZW50IGNvbGxlY3RvciB0 aGFuIHRoZSBJU1Atb3duZWQgTUFzLiANCiANCkFzIG1lbnRpb25lZCBpbiB0aGUgdXNlLWNhc2Ug ZHJhZnQsIHJlbGlhYmxlIHN0YXRpc3RpY3MgY2FuJ3QgYmUgcHJvZHVjZWQNCmZyb20gc3VjaCBh IHNlbGYtc2VsZWN0aW5nIGdyb3VwLiBCdXQgZm9yIGFuIElTUCwgdGhlcmUgaXMgc29tZSB2YWx1 ZSBpbiANCnNlZWluZyBtZWFzdXJlbWVudHMgZnJvbSBlbmQtdXNlcnMgYW5kIGdhdGhlcmluZyBz dGF0aXN0aWNzIGZyb20gdGhlaXIgDQpwb2ludCBvZiB2aWV3LiBUaGlzIGlzIGEgcmF0aGVyIHJv dWdoIG1lYXN1cmVtZW50IG9mIHRoZSB1c2VyJ3MgcXVhbGl0eQ0Kb2YgZXhwZXJpZW5jZSwgYnV0 IGl0IGRvZXMgZ2l2ZSBhbiBpZGVhIG9mIGhvdyB0aGF0IGV4cGVyaWVuY2UgbWVhc3VyZXMNCnVw IHRvIGFuIElTUCdzIGFkdmVydGlzZWQgc2VydmljZXMuIA0KIA0KVmlkZW90cm9uIChteSBlbXBs b3llcikgYWN0dWFsbHkgYWxyZWFkeSBwZXJmb3JtcyBzdWNoIG1lYXN1cmVtZW50cyBieSANCnJ1 bm5pbmcgaXRzIG93biBzcGVlZHRlc3Qgc2VydmljZSBhbmQgZ2F0aGVyaW5nIGRhdGEgY29ycmVs YXRlZCB0byB0aGUgDQplbmQtdXNlciB0aWVyLiBJdCBnaXZlcyB1cyBhIGhpZ2gtbGV2ZWwgdmll dyBvZiB1c2VycyAiaGFwcGluZXNzIiBsZXZlbHMgDQp3aXRoIHJlZ2FyZHMgdG8gdGhlaXIgc2Vy dmljZXMuIChzZWUgaHR0cDovL3Rlc3R2aXRlc3NlLnZpZGVvdHJvbi5jYSBhbmQNCmh0dHA6Ly9p cHY2LnRlc3R2aXRlc3NlLnZpZGVvdHJvbi5jYSkuIFN1Y2ggZGF0YSBpcyBkZXNpcmFibGUgZm9y IGFuIA0KSVNQIGFuZCBJIGJlbGlldmUgc2hvdWxkIGJlIGluLXNjb3BlLiANCiANCkFuIElTUCBj b3VsZCwgZm9yIGV4YW1wbGUsIHB1Ymxpc2ggYW4gTE1BUC1jb21wbGlhbnQgYXBwIGZvciBpdHMg dXNlcnMgDQp0byB0ZXN0IHRoZWlyIHNwZWVkIGFuZCBnYXRoZXIgdGhlIHJlc3VsdHMgaW4gYSBz ZXBhcmF0ZSBjb2xsZWN0b3IuIA0KIA0KL0pGDQogDQogDQpfX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fXw0KbG1hcCBtYWlsaW5nIGxpc3QNCmxtYXBAaWV0Zi5v cmcNCmh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vbG1hcA== ------=_001_NextPart660286820285_=---- Content-Type: text/html; charset="gb2312" Content-Transfer-Encoding: quoted-printable
I think the ISPs' LMAP measurement infrastructure should have the abi= lity=20 to perform the user-initiated measurement since the users are the largest = group=20 of participants of all kinds of networks and they have the requirement tha= t the=20 user-initiated measurement can be performed at any time when users ha= ve a=20 measurement requirement (for instance, when there is something wrong with = users'=20 home network). 
 
Gua= ngqing=20 Deng
cnnic 
 
Date: 2013-07-31 02:51
To: Bugenhagen, Michael K= ; dengguangqing; Jean-Francois.Trem= blayING
CC: lmap
Subject: Re: [lmap] user-initiated testing

Mike=20 ,

I=20 think there is one more case:

3)=20    User=20 uses an ISP test system to perform a test for the user=20 itself

Regards,

Sharam

 

From:= Bugenhagen,= Michael=20 K [mailto:Michael.K.Bugenhagen@centurylink.com]
Sent: Tuesday, = July=20 30, 2013 2:30 PM
To: Sharam Hakimi; dengguangqing;=20 Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: R= E:=20 [lmap] user-initiated testing

 

With=20 user =A1=B0initiated=A1=B1 we really need to contextualize if we=20 mean

1)     =20 User=20 testing their own Home network

Vs.

2)     =20 User=20 hooks into an ISP test system that executes a test for=20 them.

Of=20 course the key issue is who ownes the Measurement point at the User interf= ace=20 (DSL or Cable modem)..  These are often multi-interface, so the only=20 aggregated interface / access test point is generally located on the ISP s= ide of=20 the NID vs. the customers side.

Regards,

Mike

 

From:= =20 lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of S= haram=20 Hakimi
Sent: Tuesday, July 30, 2013 1:11 PM
To:=20 dengguangqing; Jean-Francois.TremblayING
Cc:=20 lmap@ietf.org
Subject: Re: [lmap] user-initiated=20 testing

 

Agree=20 with capability for some user initiated tests but it does not have to invo= lve=20 the Controller necessarily. A Resource Authorization function that would=20 authorize a test being done can exist in a Controller but it could also be= a=20 function outside the controller. With large numbers of possible users a=20 distributed test authorization function would work better in my=20 opinion.

Sharam=20 Hakimi, 

EXFO

 

From:= lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On=20 Behalf Of Guangqing Deng
Sent: Tuesday, July 30, 2013 1:28=20 PM
To: Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lm= ap]=20 user-initiated testing

 

Totally=20 agree. What's more, different users may care about different metrics. Such= users=20 who like watching online videos may care about download speed very much; w= hile=20 those who prefer online games may care network delay more. So, different u= sers=20 may perform measurements for different metrics, if user-initiated=20 measurements are supported. For ISPs, they can learn more about what users= are=20 paying attention to through such measurements (initiated by users) and the= n=20 accordingly adjust their networks to meet the demand of different use= rs. So=20 it seems that having user-initiated measurement in-scope is beneficial to = both=20 users and ISPs. 

 

Guangqing=20 Deng
cnnic 

 

From: Jean-Francois.Trem= blayING

Date: 2013-07-30 22:41

To: lmap@ietf.org

Subject: Re:=20 [lmap] user-initiated testing

> De : Paul Aitken <paitken@cisco.com>

> 

> However, end-users aren't under contr= ol of the organisation, so they're 

> out of scope :-)

 

Following this from the side lines (I=  am not in Berlin), I'd like to vo= ice 

 

some support to Alissa's point of vue= . I believe there is some value in = ;

having user-initiated measurements in-scope, as=  long as the software used

<= /DIV>

(either an app or a web-based interfa= ce) is consistent and is using a <= /o:p>

different collector than the ISP-owned MAs= . 

 

As mentioned in the use-case draft, r= eliable statistics can't be produced=

from such a self-selecting group. But = ;for an ISP, there is some value in&nbs= p;

seeing measurements from end-users and gat= hering statistics from their 

point of view. This is a rather = rough measurement of the user's quality

of experience, but it does give an&nb= sp;idea of how that experience measures

up to an ISP's advertised services. <= o:p>

 

Videotron (my employer) actually already p= erforms such measurements by 

running its own speedtest service and = ;gathering data correlated to the 

end-user tier. It gives us a high-lev= el view of users "happiness" levels 

with regards to their services. (see = http://testvitesse.videotron.ca and

http://ipv6.testvitesse.vide= otron.ca). Such data is desirable for an=  

ISP and I believe should be in-scope.=  

 

An ISP could, for example, publish an=  LMAP-compliant app for its users 

to test their speed and gather the&nb= sp;results in a separate collector. 

 

/JF

 

 

_______________________________________________

lmap mailing list

lmap@ietf.org

https://www.ietf.org/m= ailman/listinfo/lmap

------=_001_NextPart660286820285_=------ From j.schoenwaelder@jacobs-university.de Tue Jul 30 23:16:37 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5E53E21F9DB2 for ; Tue, 30 Jul 2013 23:16:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bscsUzrDor4G for ; Tue, 30 Jul 2013 23:16:27 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 2107021F9DAA for ; Tue, 30 Jul 2013 23:16:26 -0700 (PDT) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 4D73A20BEA; Wed, 31 Jul 2013 08:16:25 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id ZQeNXxNyXo41; Wed, 31 Jul 2013 08:16:25 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 71BCC20BE1; Wed, 31 Jul 2013 08:16:23 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id BD654279F56F; Wed, 31 Jul 2013 08:16:19 +0200 (CEST) Date: Wed, 31 Jul 2013 08:16:19 +0200 From: Juergen Schoenwaelder To: "Bugenhagen, Michael K" Message-ID: <20130731061619.GB21038@elstar.local> Mail-Followup-To: "Bugenhagen, Michael K" , 'Sharam Hakimi' , dengguangqing , "Jean-Francois.TremblayING" , "lmap@ietf.org" References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org> <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> <51F7C3BB.50304@cisco.com> <2013073101281916001716@cnnic.cn> <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: 'Sharam Hakimi' , dengguangqing , "Jean-Francois.TremblayING" , "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 06:16:37 -0000 On Tue, Jul 30, 2013 at 06:30:18PM +0000, Bugenhagen, Michael K wrote: > With user "initiated" we really need to contextualize if we mean > > 1) User testing their own Home network > > Vs. > > 2) User hooks into an ISP test system that executes a test for them. > Of course the key issue is who ownes the Measurement point at the User interface (DSL or Cable modem).. These are often multi-interface, so the only aggregated interface / access test point is generally located on the ISP side of the NID vs. the customers side. I think there is no reason to restrict 1) to the user's home network. I think the cases are: A) A user runs his own LMAP system (e.g. she owns a home router including an MA and he runs his own controller/collector and data analysis application). B) A user hooks into a 3rd party provided LMAP measurement system and then uses that system to run tests and gain insights into the network. With the assumption that the LMAP measurement system is under the control of a single organization, I do not yet see that user-initiated measurements lead to any additional requirements. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From bs7652@att.com Wed Jul 31 01:19:00 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CF80321F997B for ; Wed, 31 Jul 2013 01:18:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8nlAAxvb4cP0 for ; Wed, 31 Jul 2013 01:18:49 -0700 (PDT) Received: from nbfkord-smmo05.seg.att.com (nbfkord-smmo05.seg.att.com [209.65.160.92]) by ietfa.amsl.com (Postfix) with ESMTP id CF47D21F941F for ; Wed, 31 Jul 2013 01:18:37 -0700 (PDT) Received: from unknown [144.160.20.145] (EHLO mlpd192.enaf.sfdc.sbc.com) by nbfkord-smmo05.seg.att.com(mxl_mta-6.15.0-1) over TLS secured channel with ESMTP id c58c8f15.0.6170424.00-410.17026281.nbfkord-smmo05.seg.att.com (envelope-from ); Wed, 31 Jul 2013 08:18:37 +0000 (UTC) X-MXL-Hash: 51f8c85d75d7ce5b-00aa859134775739e80b9e392779c6f0a124835d Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r6V8IaDj012930 for ; Wed, 31 Jul 2013 04:18:36 -0400 Received: from alpi131.aldc.att.com (alpi131.aldc.att.com [130.8.218.69]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r6V8IP03012900 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 31 Jul 2013 04:18:28 -0400 Received: from GAALPA1MSGHUB9B.ITServices.sbc.com (gaalpa1msghub9b.itservices.sbc.com [130.8.36.88]) by alpi131.aldc.att.com (RSA Interceptor) for ; Wed, 31 Jul 2013 08:18:14 GMT Received: from GAALPA1MSGUSR9L.ITServices.sbc.com ([130.8.36.69]) by GAALPA1MSGHUB9B.ITServices.sbc.com ([130.8.36.88]) with mapi id 14.02.0342.003; Wed, 31 Jul 2013 04:18:14 -0400 From: "STARK, BARBARA H" To: "lmap@ietf.org" Thread-Topic: [lmap] user-initiated testing Thread-Index: AQHOjPs+xWW3u4xEfkeovwXB1A6NGgABK19AmX3FawCAAMVCgP//0BzA Date: Wed, 31 Jul 2013 08:18:14 +0000 Message-ID: <2D09D61DDFA73D4C884805CC7865E6113031715C@GAALPA1MSGUSR9L.ITServices.sbc.com> References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org> <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> <51F7C3BB.50304@cisco.com> <2013073101281916001716@cnnic.cn> <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> <20130731061619.GB21038@elstar.local> In-Reply-To: <20130731061619.GB21038@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [130.10.107.167] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-RSA-Inspected: yes X-RSA-Classifications: public X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [144.160.20.145] X-AnalysisOut: [v=2.0 cv=Sa5AgItu c=1 sm=0 a=ZRNLZ4dFUbCvG8UMqPvVAA==:17 a] X-AnalysisOut: [=tkbh6-Dvpn0A:10 a=wuPkfGMddxUA:10 a=ofMgfj31e3cA:10 a=BLc] X-AnalysisOut: [eEmwcHowA:10 a=kj9zAlcOel0A:10 a=zQP7CpKOAAAA:8 a=XIqpo32R] X-AnalysisOut: [AAAA:8 a=pcQB44yNe20A:10 a=zm7zxRMdsmKuIieTAOIA:9 a=CjuIK1] X-AnalysisOut: [q_8ugA:10] Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 08:19:00 -0000 I agree with recent comments that suggest the phrase "user-initiated" has n= o assumptions attached to it as to (a) domain that controls MA/controller a= nd (b) whether collected data gets reported to anyone other than the user. = Collected data does not necessarily have to end up in the hands of regulato= rs or ISPs or be used for regulatory purposes, even when it is reported to = someone other than the user. Summary of my opinion: MA and controller in ISP/measurement provider/regula= tor domain creates no new work items or scope expansion. If we want to cons= ider MA and controller in the user domain, the work not currently included = in the scope would be to figure out how to allow user-controlled MA access = (authentication/authorization) to desired Measurement Peers, how to supply = service attributes to the user domain, and how to include service attribute= s in reported data. Details of my opinion: (1) For (a) Domain =3D ISP, (b) don't care whether collected data reported = to anyone other than user Where domain of control is the ISP, I think posters have correctly suggeste= d that the functionality needed to make this happen, that are above and bey= ond the work elements described in the current charter, should probably rem= ain out of scope at this point. That would include elements like how to int= erface with the user, how the user interface gets to the controller for ini= tiation, and how the resulting data gets to the user interface. Whether res= ulting data gets used/stored/collected by the service provider or reported = as part of a regulatory program I believe is "don't care". The chartered wo= rk will allow for data to be reported anywhere, but doesn't insist that dat= a be reported to regulators or stored by ISPs. There's nothing I would like= to see added to currently-defined scope of work to more completely support= this case. I also don't see impacts to the framework or information model. (2) For (a) Domain =3D regulator or other measurement provider, (b) don't c= are I think my comments for domain =3D ISP apply here as well. (3a) For (a) Domain =3D user, (b) no data reported to anyone other than use= r I think that for this case the controller is likely also to be inside the h= ome network and may even be collocated with the MA. If user wants to test a= gainst the same Measurement Peers as get used by ISP or regulator, then I t= hink there may be a problem with credentials, depending on whether any of t= he Measurement Peers require some form of authentication prior to participa= ting in a test with the MA. There is also no means for the user to get serv= ice attributes. Currently neither of these is in scope. Personally, I consi= der this to be an important use case, and would be happy if these were in s= cope. I've floated proposals for getting service attributes to the user, an= d think this would be incredibly easy to do. (3b) For (a) Domain =3D user, (b) user supplies data to another entity Same issues as for (3a), but I think it would be necessary to better identi= fy the recommended reporting protocol used to supply the data in this case,= since this is a cross-domain transfer of info (which requires greater stan= dardization than intra-domain protocols). To me this suggests that this cas= e should be considered when selecting a reporting protocol. Also, this case= implies the service attributes need to be supplied in the reporting inform= ation model. I think that including service attributes in the report inform= ation model in support of this case would be a good idea. Note that there i= s nothing implied in this case as to whether the other entity receiving the= data has anything to do with a regulator, or that the supplied data will b= e used for regulatory purposes. Personally, I would object to assuming that= this data would be used for regulatory purposes. The only assumption I'm m= aking is that the receiving entity is not the end user. Barbara From paitken@cisco.com Wed Jul 31 03:30:15 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3120B21F9F9D for ; Wed, 31 Jul 2013 03:30:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.408 X-Spam-Level: X-Spam-Status: No, score=-10.408 tagged_above=-999 required=5 tests=[AWL=0.191, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ivOotEPeDPb9 for ; Wed, 31 Jul 2013 03:30:09 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id 5FD8311E80EC for ; Wed, 31 Jul 2013 03:30:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1177; q=dns/txt; s=iport; t=1375266609; x=1376476209; h=message-id:date:from:mime-version:to:cc:subject: content-transfer-encoding; bh=EoPqiGRdkuseEbcd7U1t55cWH1fXv+yZoJPpkK+HAKM=; b=gP1aFHCpfQh16LH6X/HJWp8QgfUD/STr/zU/jpFcol3O784B26BsyYe/ QyeEKd6uiuu4E8Rez2t+LjdAB4nnjWFJt7wmgE2FI/phlzW8CBFhAOnQK u6Rpgg1AmEX56njJCg1h2CJqIu+1N0SoCsp1MLFCqYaBJINWg7M/8P7nC s=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhMFAEDm+FGQ/khL/2dsb2JhbABbgwa/IIEYFnSCY0ABGxIPFhgDAgECATcUAQwBBwEBiAy4Zo9+hBADl1+GI4spgxU X-IronPort-AV: E=Sophos;i="4.89,786,1367971200"; d="scan'208";a="157788267" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-1.cisco.com with ESMTP; 31 Jul 2013 10:30:07 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id r6VAU6ix029224 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 10:30:06 GMT Received: from [10.61.88.11] (ams3-vpn-dhcp6156.cisco.com [10.61.88.11]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6VAU0Xh003770; Wed, 31 Jul 2013 11:30:00 +0100 (BST) Message-ID: <51F8E728.8070403@cisco.com> Date: Wed, 31 Jul 2013 11:30:00 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: Jean-Francois.TremblayING@videotron.com, sharam.hakimi@exfo.com, Michael.K.Bugenhagen@centurylink.com, Juergen Schoenwaelder Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 10:30:15 -0000 X-List-Received-Date: Wed, 31 Jul 2013 10:30:15 -0000 Jean-Francois, Sharam, Mike, Juergen, All, The user-initiated test scenario which you describe looks like this: ispController ispCollector \ / \ / MA--{test peers) / \ / \ userController userCollector This architecture is contradictory to the WG charter which states, "the measurement system is under the control of a single organization", because two organisations (ISP, user) now share resources (MAs). If the WG wants to address this, we should acknowledge that and address it _later_, once we have the basic LMAP blocks in place. If we don't get the basic blocks in place then the WG will never get anything done. This architecture is also quite different to the proposed framework and the assumption that the MA is under a single Controller at any point in time. Last week some people argued strongly against multiple controllers... This architecture presents multiple issues as I indicated in my email yesterday - which I saw nobody disagreed with. Should we now add that the MA may be conflicted by multiple Controllers? P. From j.schoenwaelder@jacobs-university.de Wed Jul 31 03:38:48 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 513A911E8123 for ; Wed, 31 Jul 2013 03:38:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nfovhDzSuhhe for ; Wed, 31 Jul 2013 03:38:43 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 17E4411E80E4 for ; Wed, 31 Jul 2013 03:38:39 -0700 (PDT) Received: from localhost (demetrius1.jacobs-university.de [212.201.44.46]) by hermes.jacobs-university.de (Postfix) with ESMTP id 9AF9A20BE9; Wed, 31 Jul 2013 12:38:38 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius1.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id jOsz0GXUuKUT; Wed, 31 Jul 2013 12:38:38 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id B969E20BDE; Wed, 31 Jul 2013 12:38:37 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 7164F27A0010; Wed, 31 Jul 2013 12:38:34 +0200 (CEST) Date: Wed, 31 Jul 2013 12:38:34 +0200 From: Juergen Schoenwaelder To: Paul Aitken Message-ID: <20130731103834.GA22261@elstar.local> Mail-Followup-To: Paul Aitken , Jean-Francois.TremblayING@videotron.com, sharam.hakimi@exfo.com, Michael.K.Bugenhagen@centurylink.com, "lmap@ietf.org" References: <51F8E728.8070403@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51F8E728.8070403@cisco.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: sharam.hakimi@exfo.com, Jean-Francois.TremblayING@videotron.com, Michael.K.Bugenhagen@centurylink.com, "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 10:38:48 -0000 On Wed, Jul 31, 2013 at 11:30:00AM +0100, Paul Aitken wrote: > Jean-Francois, Sharam, Mike, Juergen, All, > > The user-initiated test scenario which you describe looks like this: > > > ispController ispCollector > \ / > \ / > MA--{test peers) > / \ > / \ > userController userCollector > > > This architecture is contradictory to the WG charter which states, > "the measurement system is under the control of a single > organization", because two organisations (ISP, user) now share > resources (MAs). It seems you misread my message. Sorry for not having been clear enough. The picture I drew was this: ispController ispCollector \ / \ / MA ------------------------------------ MA / \ / \ userController userCollector Both are simply comlete distinct LMAP systems, inline with the assumption stated in the charter. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From paitken@cisco.com Wed Jul 31 03:39:52 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5633321E8082 for ; Wed, 31 Jul 2013 03:39:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.419 X-Spam-Level: X-Spam-Status: No, score=-10.419 tagged_above=-999 required=5 tests=[AWL=0.180, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pd2CshpgARiE for ; Wed, 31 Jul 2013 03:39:46 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id 6778521E80B3 for ; Wed, 31 Jul 2013 03:39:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2065; q=dns/txt; s=iport; t=1375267180; x=1376476780; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=f3VnR/XRKVfF1l4Skv53RDhX8xI8Owl7AxsGot6FNP4=; b=iW09sbvth5Xfr13e8yw9lhD/HQ+u3CfmIKGwHBpuky/BQWr+3As8enKP yeae6QoV960ieQDgicxwXG1d1D5r0BHCByeuZ/vHlm1NbyZuaoruPEHyG OXojiVfCzAVtBdemBHRw3Slm0QB0Serwb9C2HNI6KVgGwJMulexnhR5bm 4=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AigFAOHo+FGQ/khM/2dsb2JhbABbgwY2vmuBGBZ0giQBAQEDATg0CgIBBQsLEgYJFg8JAwIBAgE3DgYBDAEHAQGIBga4aI4/gUgHhAsDl1+GI4sqgxWBcA X-IronPort-AV: E=Sophos;i="4.89,729,1367971200"; d="scan'208";a="16180446" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-3.cisco.com with ESMTP; 31 Jul 2013 10:39:39 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r6VAdbFC003694 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 10:39:37 GMT Received: from [10.61.88.11] (ams3-vpn-dhcp6156.cisco.com [10.61.88.11]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6VAdXWh004469; Wed, 31 Jul 2013 11:39:33 +0100 (BST) Message-ID: <51F8E965.5020806@cisco.com> Date: Wed, 31 Jul 2013 11:39:33 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: "lmap@ietf.org" , Juergen Schoenwaelder References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org> <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch> <51F7C3BB.50304@cisco.com> <2013073101281916001716@cnnic.cn> <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> <20130731061619.GB21038@elstar.local> In-Reply-To: <20130731061619.GB21038@elstar.local> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: 'Sharam Hakimi' , dengguangqing , "Jean-Francois.TremblayING" , "Bugenhagen, Michael K" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 10:39:52 -0000 Juergen, > On Tue, Jul 30, 2013 at 06:30:18PM +0000, Bugenhagen, Michael K wrote: >> With user "initiated" we really need to contextualize if we mean >> >> 1) User testing their own Home network >> >> Vs. >> >> 2) User hooks into an ISP test system that executes a test for them. >> Of course the key issue is who ownes the Measurement point at the User interface (DSL or Cable modem).. These are often multi-interface, so the only aggregated interface / access test point is generally located on the ISP side of the NID vs. the customers side. > I think there is no reason to restrict 1) to the user's home network. I > think the cases are: > > A) A user runs his own LMAP system (e.g. she owns a home router > including an MA and he runs his own controller/collector and data > analysis application). > > B) A user hooks into a 3rd party provided LMAP measurement system and > then uses that system to run tests and gain insights into the > network. Scenario (B) could work, since it looks something like this: isp . ispCollector \ . / \ . / controller ---- MA -- {test peers} / . \ / . \ user . userCollector LMAP boundary This is subtly, though crucially, different from the scenario in my previous email, because the system now falls under a single Controller. The ISP could instruct the controller to initiate some tests reporting to the ispCollector, while the user instructs the controller to initiate tests reporting to the userCollector. With this architecture, everything within the LMAP boundary is in accordance with the WG charter and the proposed framework. P. > > With the assumption that the LMAP measurement system is under the > control of a single organization, I do not yet see that user-initiated > measurements lead to any additional requirements. > > /js > From bs7652@att.com Wed Jul 31 03:43:49 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C806511E80E3 for ; Wed, 31 Jul 2013 03:43:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mBB91BLy1-Hb for ; Wed, 31 Jul 2013 03:43:38 -0700 (PDT) Received: from nbfkord-smmo05.seg.att.com (nbfkord-smmo05.seg.att.com [209.65.160.92]) by ietfa.amsl.com (Postfix) with ESMTP id 1113411E80D1 for ; Wed, 31 Jul 2013 03:43:37 -0700 (PDT) Received: from unknown [144.160.20.145] (EHLO mlpd192.enaf.sfdc.sbc.com) by nbfkord-smmo05.seg.att.com(mxl_mta-6.15.0-1) over TLS secured channel with ESMTP id 95ae8f15.0.6206159.00-466.17124113.nbfkord-smmo05.seg.att.com (envelope-from ); Wed, 31 Jul 2013 10:43:38 +0000 (UTC) X-MXL-Hash: 51f8ea5a4d152e96-9eaf219240c16b9643e8b8e9b71e0ec7bbe3cf2a Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r6VAhbIP028879 for ; Wed, 31 Jul 2013 06:43:37 -0400 Received: from alpi132.aldc.att.com (alpi132.aldc.att.com [130.8.217.2]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r6VAhZXP028875 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 31 Jul 2013 06:43:36 -0400 Received: from GAALPA1MSGHUB9A.ITServices.sbc.com (gaalpa1msghub9a.itservices.sbc.com [130.8.36.87]) by alpi132.aldc.att.com (RSA Interceptor) for ; Wed, 31 Jul 2013 10:43:21 GMT Received: from GAALPA1MSGUSR9L.ITServices.sbc.com ([130.8.36.69]) by GAALPA1MSGHUB9A.ITServices.sbc.com ([130.8.36.87]) with mapi id 14.02.0342.003; Wed, 31 Jul 2013 06:43:21 -0400 From: "STARK, BARBARA H" To: "lmap@ietf.org" Thread-Topic: draft-eardley-lmap-framework: Extensibility Thread-Index: Ac6N2sXu4qyjuxcAQV6Ll8U30MAFUA== Date: Wed, 31 Jul 2013 10:43:19 +0000 Message-ID: <2D09D61DDFA73D4C884805CC7865E611303172BC@GAALPA1MSGUSR9L.ITServices.sbc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [130.10.236.91] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-RSA-Inspected: yes X-RSA-Classifications: public X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [144.160.20.145] X-AnalysisOut: [v=2.0 cv=Sa5AgItu c=1 sm=0 a=ZRNLZ4dFUbCvG8UMqPvVAA==:17 a] X-AnalysisOut: [=BSMUUEfofzoA:10 a=JRTX2GwzQmcA:10 a=ofMgfj31e3cA:10 a=BLc] X-AnalysisOut: [eEmwcHowA:10 a=kj9zAlcOel0A:10 a=zQP7CpKOAAAA:8 a=XIqpo32R] X-AnalysisOut: [AAAA:8 a=1okb_UxIFqYA:10 a=PGVh3UWVStKljCeCMikA:9 a=CjuIK1] X-AnalysisOut: [q_8ugA:10 a=zcmxt3HbQ9Dp-Nea:21 a=fJr7CPkJxjuvy6Lw:21] Subject: [lmap] draft-eardley-lmap-framework: Extensibility X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 10:43:50 -0000 I read through the current draft and had a comment on the inclusion of the = "Extensible" bullet in the Introduction. I can't find anything in the rest of the framework or the charter + deliver= ables that really has anything to do with extensibility; but if it is to be= called out as a desirable feature in the Introduction, I would expect some= discussion of how to achieve it or what's being done to work towards achie= ving it. Or don't call it out so strongly. Or call it out and state that wh= ile it is desirable, the current proposed framework does nothing to address= it.=20 Or state that extensibility of Measurement Methods is being provided by the= ippm-defined registry (which will be a "living" registry?)?; and the lmap = Information Model and protocols/data models will ensure that this extensibi= lity of Measurement Methods is supported so MAs. I think that there's also = the intention for MAs to be able to express to a controller which measureme= nts they support / what they're capable of -- which helps in the area of ex= tensibility. This would need to be in the information model.=20 I prefer the last of these options, if that is what we're planning. BTW, extensibility areas that aren't currently in scope of lmap are being a= ble to upgrade Measurement Methods in a MA, or upgrade the MA software vers= ion. These would be functions of bootstrap/initialization, which aren't in = scope. From paitken@cisco.com Wed Jul 31 03:50:43 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 394A421F99DC for ; Wed, 31 Jul 2013 03:50:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.429 X-Spam-Level: X-Spam-Status: No, score=-10.429 tagged_above=-999 required=5 tests=[AWL=0.170, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GZH90V-5LD1O for ; Wed, 31 Jul 2013 03:50:34 -0700 (PDT) Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id 5621E21F99AB for ; Wed, 31 Jul 2013 03:50:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2234; q=dns/txt; s=iport; t=1375267833; x=1376477433; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=3tiNXZXspVWQB44ZWlfcmhiz29AJoN5p0CQydUmngZE=; b=Mk+Ydeaja0yl0UuKpkFGa269iIaYG4X9KURbrqx2bPxRl/S7ocE/41hI GZMHsdVfzbY0n0YMQGty66Wy2jvor9qK8SJccFgXn2BkB4TWD4LK2Hrsl +ayNr4WlgO1ulWqCrF9DQ9AWHDQVNNCrPAiKNnhYTu5xlzmBX/NwuUl47 c=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AicFACnr+FGQ/khR/2dsb2JhbABbgwY2vmuBGBZ0giQBAQEDAThAAQULCw4EBgkWDwkDAgECATcOBg0BBwEBiAYGuG6QBweECwOXX4YjiyqDFQ X-IronPort-AV: E=Sophos;i="4.89,786,1367971200"; d="scan'208";a="16655582" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-4.cisco.com with ESMTP; 31 Jul 2013 10:50:27 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id r6VAoPWn019226 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 10:50:25 GMT Received: from [10.61.88.11] (ams3-vpn-dhcp6156.cisco.com [10.61.88.11]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6VAoLIR005735; Wed, 31 Jul 2013 11:50:22 +0100 (BST) Message-ID: <51F8EBED.7040907@cisco.com> Date: Wed, 31 Jul 2013 11:50:21 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: Juergen Schoenwaelder References: <51F8E728.8070403@cisco.com> <20130731103834.GA22261@elstar.local> In-Reply-To: <20130731103834.GA22261@elstar.local> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: sharam.hakimi@exfo.com, Jean-Francois.TremblayING@videotron.com, Michael.K.Bugenhagen@centurylink.com, "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 10:50:43 -0000 Juergen, > On Wed, Jul 31, 2013 at 11:30:00AM +0100, Paul Aitken wrote: >> Jean-Francois, Sharam, Mike, Juergen, All, >> >> The user-initiated test scenario which you describe looks like this: >> >> >> ispController ispCollector >> \ / >> \ / >> MA--{test peers) >> / \ >> / \ >> userController userCollector >> >> >> This architecture is contradictory to the WG charter which states, >> "the measurement system is under the control of a single >> organization", because two organisations (ISP, user) now share >> resources (MAs). > It seems you misread my message. Sorry for not having been clear > enough. The picture I drew was this: > > ispController ispCollector > \ / > \ / > MA > > ------------------------------------ > > MA > / \ > / \ > userController userCollector > > Both are simply comlete distinct LMAP systems, inline with the > assumption stated in the charter. If these are distinct MAs, then I completely agree; there is no issue here at all. However, if these are simply different logical sub-divisions of a single MA, then the division is not so straightforward. eg, if the ISP is running a CPU-loading test which the user is unaware of, the user-side results may be very poor (eg, user bandwidth seems very low). Whereas a single controller would know that a 100% CPU test is scheduled, and therefore not schedule other tests at the same time. Should we move conflict resolution into each MA? ie, should the MA refuse the user initiated test because it conflicts with the ISP test? What should it report? Ideally it should return as much information as possible as to why the test could not be run, so that the controller can potentially schedule an appropriately modified test. However, if it returns too much information, it could be possible for the user to determine a lot of information about the ISP's testing. In a different scenario, this could be two different ISPs so privacy is an issue... P. From bs7652@att.com Wed Jul 31 03:52:30 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4DD9521E8082 for ; Wed, 31 Jul 2013 03:52:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id auyBUi2zWac7 for ; Wed, 31 Jul 2013 03:52:15 -0700 (PDT) Received: from nbfkord-smmo07.seg.att.com (nbfkord-smmo07.seg.att.com [209.65.160.93]) by ietfa.amsl.com (Postfix) with ESMTP id A2EF621F9A4C for ; Wed, 31 Jul 2013 03:52:02 -0700 (PDT) Received: from unknown [144.160.20.145] (EHLO mlpd192.enaf.sfdc.sbc.com) by nbfkord-smmo07.seg.att.com(mxl_mta-6.15.0-1) over TLS secured channel with ESMTP id 25ce8f15.0.4854700.00-249.13401470.nbfkord-smmo07.seg.att.com (envelope-from ); Wed, 31 Jul 2013 10:52:12 +0000 (UTC) X-MXL-Hash: 51f8ec5c0630450e-c9abbbe7d09723af96029bff270f421db2ea51df Received: from enaf.sfdc.sbc.com (localhost.localdomain [127.0.0.1]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r6VAq2IT031293 for ; Wed, 31 Jul 2013 06:52:02 -0400 Received: from alpi131.aldc.att.com (alpi131.aldc.att.com [130.8.218.69]) by mlpd192.enaf.sfdc.sbc.com (8.14.5/8.14.5) with ESMTP id r6VApuMm031271 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 31 Jul 2013 06:51:56 -0400 Received: from GAALPA1MSGHUB9A.ITServices.sbc.com (gaalpa1msghub9a.itservices.sbc.com [130.8.36.87]) by alpi131.aldc.att.com (RSA Interceptor) for ; Wed, 31 Jul 2013 10:51:34 GMT Received: from GAALPA1MSGUSR9L.ITServices.sbc.com ([130.8.36.69]) by GAALPA1MSGHUB9A.ITServices.sbc.com ([130.8.36.87]) with mapi id 14.02.0342.003; Wed, 31 Jul 2013 06:51:34 -0400 From: "STARK, BARBARA H" To: "lmap@ietf.org" Thread-Topic: framework/terminology bootstrap vs initialization Thread-Index: Ac6N2+nNqftSJOEERSWCeNJCbg8C/A== Date: Wed, 31 Jul 2013 10:51:33 +0000 Message-ID: <2D09D61DDFA73D4C884805CC7865E611303172DF@GAALPA1MSGUSR9L.ITServices.sbc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [130.10.236.91] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-RSA-Inspected: yes X-RSA-Classifications: public X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [144.160.20.145] X-AnalysisOut: [v=2.0 cv=YP8oP26x c=1 sm=0 a=ZRNLZ4dFUbCvG8UMqPvVAA==:17 a] X-AnalysisOut: [=BSMUUEfofzoA:10 a=Vh_06ZAyj-8A:10 a=ofMgfj31e3cA:10 a=BLc] X-AnalysisOut: [eEmwcHowA:10 a=kj9zAlcOel0A:10 a=zQP7CpKOAAAA:8 a=XIqpo32R] X-AnalysisOut: [AAAA:8 a=fSNjWjkXvzoA:10 a=icSr0a04TGpsIYZckY4A:9 a=CjuIK1] X-AnalysisOut: [q_8ugA:10] Subject: [lmap] framework/terminology bootstrap vs initialization X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 10:52:30 -0000 There's currently an inconsistency between the framework and terminology as= to whether to use the term bootstrap or initialize (bootstrap protocol is = used by an initializer?). I would prefer consistency of either a bootstrap = protocol used by a bootstrapper (or bootstrap function) or an initializatio= n protocol used by an initializer. I'm wondering if this happened as a resu= lt of someone being confused about what was meant by "bootstrap"? I rather = liked bootstrap, though, and saw no need for the switch. The terminology de= finition of bootstrap protocol seems clear me. Barbara From paitken@cisco.com Wed Jul 31 04:08:11 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2FCE511E8167 for ; Wed, 31 Jul 2013 04:08:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.438 X-Spam-Level: X-Spam-Status: No, score=-10.438 tagged_above=-999 required=5 tests=[AWL=0.160, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hodoe72O9SvG for ; Wed, 31 Jul 2013 04:08:02 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id 9788621F9ED1 for ; Wed, 31 Jul 2013 04:07:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=41459; q=dns/txt; s=iport; t=1375268871; x=1376478471; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to; bh=kgsTtPOoz8dr/lxDugx4Z4z/YAKclAd1wON7kfI/pKQ=; b=KRbh0DeBDoBQl6HHFGOSD/izYpd9/65lwLRj1HsBCiitG8JmT30eTm8/ 0MgW/UWMGoQoPJkUsaf9++3wk58+nCb8Dygq5cTLyqpQWyFe4DvGS9n7c yCao2B5kxORtOYa6MOrbmuYe8uIPHtl4damXpCCryn/JpyaaaIwd1uJO1 w=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ai8FAKvv+FGQ/khN/2dsb2JhbABbgkJENQGJPLUvgRgWdIIkAQEBAwEBAQEqQQoBBQsLEQECAQEBAQkWAQEGBwkDAgECARUfAwYIBgEMAQUCAQGIBgYMuGePWhwRBgEJhAIDl1+BKYR6iyqDFQ X-IronPort-AV: E=Sophos;i="4.89,786,1367971200"; d="scan'208,217";a="157790130" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-1.cisco.com with ESMTP; 31 Jul 2013 11:07:35 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id r6VB7WR6003767 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 11:07:33 GMT Received: from [10.61.88.11] (ams3-vpn-dhcp6156.cisco.com [10.61.88.11]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6VB7SC0007069; Wed, 31 Jul 2013 12:07:29 +0100 (BST) Message-ID: <51F8EFF0.7030408@cisco.com> Date: Wed, 31 Jul 2013 12:07:28 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: "Bugenhagen, Michael K" , "'Sharam Hakimi'" References: <6C7BB84D-231C-4511-AFFF-25BE7D665052@cdt.org>, <6E1DBC28-7504-4DA2-B03C-7016A9AE29EF@tik.ee.ethz.ch><51F7C3BB.50304@cisco.com>, <2013073101281916001716@cnnic.cn> <084CDC75FEC1E640B60338273BEACDFA02732F10@spboexc01.exfo.com> <084CDC75FEC1E640B60338273BEACDFA02732F25@spboexc01.exfo.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------010107030905000706070102" Cc: dengguangqing , "Jean-Francois.TremblayING" , "lmap@ietf.org" Subject: Re: [lmap] [Sender: lmap-bounces@ietf.org] Re: user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 11:08:11 -0000 This is a multi-part message in MIME format. --------------010107030905000706070102 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Michael, Sharam, I've no objections to this case since it's the architecture I just described like so: isp . ispCollector \ . / \ . / controller ---- MA -- {test peers} / . \ / . \ user . userCollector LMAP boundary Note that: * the MA may be present on the user's device * the mechanism for the user to request that the controller initiate the test is outwith the current LMAP scope. * this supposes that a single collector is not programmed into the MA, but is indicated per test. P. On 30/07/13 19:52, Bugenhagen, Michael K wrote: > > Good catch.... > > I'd call in a "2b" user case,,, but yes the ISP system test from > the Broadband Access NID aggregation device to the Internet, or to > the customer... > > *From:*Sharam Hakimi [mailto:sharam.hakimi@exfo.com] > *Sent:* Tuesday, July 30, 2013 1:51 PM > *To:* Bugenhagen, Michael K; dengguangqing; Jean-Francois.TremblayING > *Cc:* lmap@ietf.org > *Subject:* RE: [lmap] user-initiated testing > > Mike , > > I think there is one more case: > > 3) User uses an ISP test system to perform a test for the user itself > > Regards, > > Sharam > > *From:*Bugenhagen, Michael K > [mailto:Michael.K.Bugenhagen@centurylink.com] > *Sent:* Tuesday, July 30, 2013 2:30 PM > *To:* Sharam Hakimi; dengguangqing; Jean-Francois.TremblayING > *Cc:* lmap@ietf.org > *Subject:* RE: [lmap] user-initiated testing > > With user "initiated" we really need to contextualize if we mean > > 1)User testing their own Home network > > Vs. > > 2)User hooks into an ISP test system that executes a test for them. > > Of course the key issue is who ownes the Measurement point at the User > interface (DSL or Cable modem).. These are often multi-interface, so > the only aggregated interface / access test point is generally located > on the ISP side of the NID vs. the customers side. > > Regards, > > Mike > > *From:*lmap-bounces@ietf.org > [mailto:lmap-bounces@ietf.org] *On Behalf Of *Sharam Hakimi > *Sent:* Tuesday, July 30, 2013 1:11 PM > *To:* dengguangqing; Jean-Francois.TremblayING > *Cc:* lmap@ietf.org > *Subject:* Re: [lmap] user-initiated testing > > Agree with capability for some user initiated tests but it does not > have to involve the Controller necessarily. A Resource Authorization > function that would authorize a test being done can exist in a > Controller but it could also be a function outside the controller. > With large numbers of possible users a distributed test authorization > function would work better in my opinion. > > Sharam Hakimi, > > EXFO > > *From:*lmap-bounces@ietf.org > [mailto:lmap-bounces@ietf.org] *On Behalf Of *Guangqing Deng > *Sent:* Tuesday, July 30, 2013 1:28 PM > *To:* Jean-Francois.TremblayING > *Cc:* lmap@ietf.org > *Subject:* Re: [lmap] user-initiated testing > > Totally agree. What's more, different users may care about different > metrics. Such users who like watching online videos may care about > download speed very much; while those who prefer online games may care > network delay more. So, different users may perform measurements for > different metrics, if user-initiated measurements are supported. For > ISPs, they can learn more about what users are paying attention to > through such measurements (initiated by users) and then accordingly > adjust their networks to meet the demand of different users. So it > seems that having user-initiated measurement in-scope is beneficial to > both users and ISPs. > > ------------------------------------------------------------------------ > > Guangqing Deng > cnnic > > *From:*Jean-Francois.TremblayING > > > *Date:* 2013-07-30 22:41 > > *To:*lmap@ietf.org > > *Subject:* Re: [lmap] user-initiated testing > > > De : Paul Aitken > > > > > > > However, end-users aren't under control of the organisation, so they're > > > out of scope :-) > > Following this from the side lines (I am not in Berlin), I'd like to voice > > > some support to Alissa's point of vue. I believe there is some value in > > having user-initiated measurements in-scope, as long as the software used > > (either an app or a web-based interface) is consistent and is using a > > different collector than the ISP-owned MAs. > > As mentioned in the use-case draft, reliable statistics can't be produced > > from such a self-selecting group. But for an ISP, there is some value in > > seeing measurements from end-users and gathering statistics from their > > point of view. This is a rather rough measurement of the user's quality > > of experience, but it does give an idea of how that experience measures > > up to an ISP's advertised services. > > Videotron (my employer) actually already performs such measurements by > > running its own speedtest service and gathering data correlated to the > > end-user tier. It gives us a high-level view of users "happiness" levels > > with regards to their services. (see http://testvitesse.videotron.ca and > > http://ipv6.testvitesse.videotron.ca). Such data is desirable for an > > ISP and I believe should be in-scope. > > An ISP could, for example, publish an LMAP-compliant app for its users > > to test their speed and gather the results in a separate collector. > > /JF > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap > > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap --------------010107030905000706070102 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Michael, Sharam,

I've no objections to this case since it's the architecture I just described like so:


     isp   .                   ispCollector
        \  .                  /
         \ .                 /
           controller ---- MA -- {test peers}
         / .                 \
        /  .                  \
    user   .                   userCollector
         LMAP
       boundary


Note that:
    * the MA may be present on the user's device
    * the mechanism for the user to request that the controller initiate the test is outwith the current LMAP scope.
    * this supposes that a single collector is not programmed into the MA, but is indicated per test.

P.


On 30/07/13 19:52, Bugenhagen, Michael K wrote:

Good catch…. 

   I’d call in a “2b” user case,,, but yes the  ISP system test from the  Broadband Access NID aggregation device to the Internet, or to the customer…

 

 

 

From: Sharam Hakimi [mailto:sharam.hakimi@exfo.com]
Sent: Tuesday, July 30, 2013 1:51 PM
To: Bugenhagen, Michael K; dengguangqing; Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: RE: [lmap] user-initiated testing

 

Mike ,

I think there is one more case:

3)    User uses an ISP test system to perform a test for the user itself

Regards,

Sharam

 

From: Bugenhagen, Michael K [mailto:Michael.K.Bugenhagen@centurylink.com]
Sent: Tuesday, July 30, 2013 2:30 PM
To: Sharam Hakimi; dengguangqing; Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: RE: [lmap] user-initiated testing

 

With user “initiated” we really need to contextualize if we mean

1)      User testing their own Home network

Vs.

2)      User hooks into an ISP test system that executes a test for them.

Of course the key issue is who ownes the Measurement point at the User interface (DSL or Cable modem)..  These are often multi-interface, so the only aggregated interface / access test point is generally located on the ISP side of the NID vs. the customers side.

Regards,

Mike

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Sharam Hakimi
Sent: Tuesday, July 30, 2013 1:11 PM
To: dengguangqing; Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing

 

Agree with capability for some user initiated tests but it does not have to involve the Controller necessarily. A Resource Authorization function that would authorize a test being done can exist in a Controller but it could also be a function outside the controller. With large numbers of possible users a distributed test authorization function would work better in my opinion.

Sharam Hakimi, 

EXFO

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Guangqing Deng
Sent: Tuesday, July 30, 2013 1:28 PM
To: Jean-Francois.TremblayING
Cc: lmap@ietf.org
Subject: Re: [lmap] user-initiated testing

 

Totally agree. What's more, different users may care about different metrics. Such users who like watching online videos may care about download speed very much; while those who prefer online games may care network delay more. So, different users may perform measurements for different metrics, if user-initiated measurements are supported. For ISPs, they can learn more about what users are paying attention to through such measurements (initiated by users) and then accordingly adjust their networks to meet the demand of different users. So it seems that having user-initiated measurement in-scope is beneficial to both users and ISPs. 

 

Guangqing Deng
cnnic 

 

Date: 2013-07-30 22:41

Subject: Re: [lmap] user-initiated testing

> De : Paul Aitken <paitken@cisco.com>

> 

> However, end-users aren't under control of the organisation, so they're 

> out of scope :-)

 

Following this from the side lines (I am not in Berlin), I'd like to voice 

 

some support to Alissa's point of vue. I believe there is some value in 

having user-initiated measurements in-scope, as long as the software used

(either an app or a web-based interface) is consistent and is using a 

different collector than the ISP-owned MAs. 

 

As mentioned in the use-case draft, reliable statistics can't be produced

from such a self-selecting group. But for an ISP, there is some value in 

seeing measurements from end-users and gathering statistics from their 

point of view. This is a rather rough measurement of the user's quality

of experience, but it does give an idea of how that experience measures

up to an ISP's advertised services. 

 

Videotron (my employer) actually already performs such measurements by 

running its own speedtest service and gathering data correlated to the 

end-user tier. It gives us a high-level view of users "happiness" levels 

with regards to their services. (see http://testvitesse.videotron.ca and

http://ipv6.testvitesse.videotron.ca). Such data is desirable for an 

ISP and I believe should be in-scope. 

 

An ISP could, for example, publish an LMAP-compliant app for its users 

to test their speed and gather the results in a separate collector. 

 

/JF

 

 

_______________________________________________

lmap mailing list



_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap

--------------010107030905000706070102-- From j.schoenwaelder@jacobs-university.de Wed Jul 31 04:23:09 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 21C9E11E8176 for ; Wed, 31 Jul 2013 04:23:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F3XzVZcbExWA for ; Wed, 31 Jul 2013 04:23:03 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 6F05B11E8172 for ; Wed, 31 Jul 2013 04:23:02 -0700 (PDT) Received: from localhost (demetrius2.jacobs-university.de [212.201.44.47]) by hermes.jacobs-university.de (Postfix) with ESMTP id CCEE420BFE; Wed, 31 Jul 2013 13:23:01 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius2.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id mbaFLzAJZDK2; Wed, 31 Jul 2013 13:23:01 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 5139820BF8; Wed, 31 Jul 2013 13:23:01 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 9B50327A01F7; Wed, 31 Jul 2013 13:22:57 +0200 (CEST) Date: Wed, 31 Jul 2013 13:22:57 +0200 From: Juergen Schoenwaelder To: Paul Aitken Message-ID: <20130731112257.GA22368@elstar.local> Mail-Followup-To: Paul Aitken , Jean-Francois.TremblayING@videotron.com, sharam.hakimi@exfo.com, Michael.K.Bugenhagen@centurylink.com, "lmap@ietf.org" References: <51F8E728.8070403@cisco.com> <20130731103834.GA22261@elstar.local> <51F8EBED.7040907@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51F8EBED.7040907@cisco.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: sharam.hakimi@exfo.com, Jean-Francois.TremblayING@videotron.com, Michael.K.Bugenhagen@centurylink.com, "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 11:23:09 -0000 On Wed, Jul 31, 2013 at 11:50:21AM +0100, Paul Aitken wrote: > > If these are distinct MAs, then I completely agree; there is no > issue here at all. This is how I look at it. > However, if these are simply different logical sub-divisions of a > single MA, then the division is not so straightforward. > > eg, if the ISP is running a CPU-loading test which the user is > unaware of, the user-side results may be very poor (eg, user > bandwidth seems very low). > > Whereas a single controller would know that a 100% CPU test is > scheduled, and therefore not schedule other tests at the same time. > > Should we move conflict resolution into each MA? ie, should the MA > refuse the user initiated test because it conflicts with the ISP > test? What should it report? Ideally it should return as much > information as possible as to why the test could not be run, so that > the controller can potentially schedule an appropriately modified > test. > > However, if it returns too much information, it could be possible > for the user to determine a lot of information about the ISP's > testing. In a different scenario, this could be two different ISPs > so privacy is an issue... All good reasons to avoid this. (And note, the assumption of an MA is controlled by a single Controller at any point in time does not preclude implementations where the underlying runtime system can enforce proper separation and provide multiple MAs on a single box.) /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From paitken@cisco.com Wed Jul 31 04:49:48 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B4B5421F9FF1 for ; Wed, 31 Jul 2013 04:49:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.446 X-Spam-Level: X-Spam-Status: No, score=-10.446 tagged_above=-999 required=5 tests=[AWL=0.153, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NJou6JGYmpEv for ; Wed, 31 Jul 2013 04:49:43 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id E4D7E21E8130 for ; Wed, 31 Jul 2013 04:49:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=781; q=dns/txt; s=iport; t=1375271383; x=1376480983; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=CWCcS3U44aQxr4dOIbSWP6BqVbykL3yZGa3ykxRuoPs=; b=ixLY1wLE4nhdHcwwO2pHZ7mzkD/KV/yFRQp2yxlVGi+uu6cfOFDG7siy 4Q0hhmjbQmwCDzG3T8GcJyc902t4IXP/T3twRXO7k/bJB+XTAaBla8viw Bh3ykviMvwY40HzXnl3WC2noR0Q0kVh7b53YRSnMBLRhyL2m1lBgoiyxC M=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AicFAAj5+FGQ/khN/2dsb2JhbABbgwY2vmyBGBZ0giQBAQEDAThBBQsLDhMlDwJGBg0BBwEBiAYGuHqQBweECwOXX4YjiyqDFQ X-IronPort-AV: E=Sophos;i="4.89,786,1367971200"; d="scan'208";a="157791971" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-1.cisco.com with ESMTP; 31 Jul 2013 11:49:38 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id r6VBnZtw015470 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 11:49:35 GMT Received: from [10.61.88.11] (ams3-vpn-dhcp6156.cisco.com [10.61.88.11]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6VBnXDk011176; Wed, 31 Jul 2013 12:49:34 +0100 (BST) Message-ID: <51F8F9CD.50900@cisco.com> Date: Wed, 31 Jul 2013 12:49:33 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: Juergen Schoenwaelder References: <51F8E728.8070403@cisco.com> <20130731103834.GA22261@elstar.local> <51F8EBED.7040907@cisco.com> <20130731112257.GA22368@elstar.local> In-Reply-To: <20130731112257.GA22368@elstar.local> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: sharam.hakimi@exfo.com, Jean-Francois.TremblayING@videotron.com, Michael.K.Bugenhagen@centurylink.com, "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 11:49:48 -0000 Juergen, > All good reasons to avoid this. (And note, the assumption of an MA is > controlled by a single Controller at any point in time does not > preclude implementations where the underlying runtime system can > enforce proper separation and provide multiple MAs on a single box.) I've no objection to a single physical MA which is able to act as multiple virtual MAs, provided that they truly act separately and distinctly. However there's almost certainly some artificial limitation (eg, maximum CPU, maximum bandwidth) imposed by the virtual system controller - so what's being measured is not the physical limitation, but the artificial limitation. - which was said in yesterday's WG meeting: ensure you know what you're really measuring. P. From j.schoenwaelder@jacobs-university.de Wed Jul 31 05:01:07 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B8CD321F8E2A for ; Wed, 31 Jul 2013 05:01:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 73YUdG7xowC8 for ; Wed, 31 Jul 2013 05:01:02 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id EFFEF21F81FF for ; Wed, 31 Jul 2013 05:00:53 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id 93AD520BDE; Wed, 31 Jul 2013 14:00:24 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id fyVGE7shDNQt; Wed, 31 Jul 2013 14:00:24 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id A2C6720523; Wed, 31 Jul 2013 14:00:23 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id ED16F27A039F; Wed, 31 Jul 2013 14:00:19 +0200 (CEST) Date: Wed, 31 Jul 2013 14:00:19 +0200 From: Juergen Schoenwaelder To: Paul Aitken Message-ID: <20130731120019.GA22448@elstar.local> Mail-Followup-To: Paul Aitken , Jean-Francois.TremblayING@videotron.com, sharam.hakimi@exfo.com, Michael.K.Bugenhagen@centurylink.com, "lmap@ietf.org" References: <51F8E728.8070403@cisco.com> <20130731103834.GA22261@elstar.local> <51F8EBED.7040907@cisco.com> <20130731112257.GA22368@elstar.local> <51F8F9CD.50900@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51F8F9CD.50900@cisco.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: sharam.hakimi@exfo.com, Jean-Francois.TremblayING@videotron.com, Michael.K.Bugenhagen@centurylink.com, "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 12:01:07 -0000 On Wed, Jul 31, 2013 at 12:49:33PM +0100, Paul Aitken wrote: > Juergen, > > >All good reasons to avoid this. (And note, the assumption of an MA is > >controlled by a single Controller at any point in time does not > >preclude implementations where the underlying runtime system can > >enforce proper separation and provide multiple MAs on a single box.) > > I've no objection to a single physical MA which is able to act as > multiple virtual MAs, provided that they truly act separately and > distinctly. > > However there's almost certainly some artificial limitation (eg, > maximum CPU, maximum bandwidth) imposed by the virtual system > controller - so what's being measured is not the physical > limitation, but the artificial limitation. Some virtualization systems allow you for example to assign CPU cores to specific virtual domains. Of course, no system is going to be perfect - even if you deploy separate hardware boxes, they still do have limitations. > - which was said in yesterday's WG meeting: ensure you know what > you're really measuring. Yes. This is why I am a big fan of MAs that are not shared because at the MA software level, it is really difficult to control resource usage. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From david.sinicrope@gmail.com Wed Jul 31 05:12:05 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 23E4411E818D for ; Wed, 31 Jul 2013 05:12:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.203 X-Spam-Level: X-Spam-Status: No, score=-1.203 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SLtZXBQGB4tQ for ; Wed, 31 Jul 2013 05:12:04 -0700 (PDT) Received: from mail-pb0-x229.google.com (mail-pb0-x229.google.com [IPv6:2607:f8b0:400e:c01::229]) by ietfa.amsl.com (Postfix) with ESMTP id A19A921F9DBE for ; Wed, 31 Jul 2013 05:11:59 -0700 (PDT) Received: by mail-pb0-f41.google.com with SMTP id rp2so203301pbb.0 for ; Wed, 31 Jul 2013 05:11:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-mailer:mime-version:x-universally-unique-identifier:content-type :x-apple-mail-signature:message-id:content-transfer-encoding:cc :x-apple-windows-friendly:x-apple-mail-remote-attachments :x-apple-base-url:x-uniform-type-identifier:subject:from:date:to; bh=GpZ6Svbro3Tqi86PEXW8SUS+ysgCfKpEzyuh04PxBSQ=; b=UV4Om4MTOhbgZadqD5aV19LWb3lSFb+2cNUCUGbB1AFjwPFo1I4hj0IR5BTNEOcSc2 mMHBF8hyR+4FCDC4A/2SGbE6G9bxbtKYNAJTpOc6hcUMoAxW2ZxARPq5d/hK/NUBJhW9 jXOd7xkpsUZi0xWJgeJu+bX6pmss0jaOuD25f4koJRXUfqjK38bn0KTjclcwKBEYArPQ GzeGXpb+HImBHiXkVDa9c2O7FjPICErey36qNNw5ViO6I4gmAYbi4Q7P0hIryOtVJGYK aw7s7W9Fkp5oo7N2pgTSpBnK1QTkt5+cVQgjfZ/wwywKqT5kMc+85kLDxcpG+D8lqqML IFFg== X-Received: by 10.66.155.67 with SMTP id vu3mr5280360pab.42.1375272718344; Wed, 31 Jul 2013 05:11:58 -0700 (PDT) Received: from ?IPv6:2001:df8::64:e07c:386a:a6c4:a48? ([2001:df8:0:64:e07c:386a:a6c4:a48]) by mx.google.com with ESMTPSA id kc8sm1836033pbc.18.2013.07.31.05.11.56 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 31 Jul 2013 05:11:57 -0700 (PDT) X-Mailer: iPad Mail (10B329) Mime-Version: 1.0 (1.0) X-Universally-Unique-Identifier: d3534975-c10e-42c9-9a26-c2b4aa6c5f82 Content-Type: multipart/alternative; boundary=Apple-Mail-7F47765A-5548-4A0B-BF3D-1FC0872F5DCE X-Apple-Mail-Signature: Message-Id: <09AE5A45-FCCC-4CEB-A7DC-B9BE37321BA9@gmail.com> Content-Transfer-Encoding: 7bit X-Apple-Windows-Friendly: 1 X-Apple-Mail-Remote-Attachments: YES X-Apple-Base-Url: x-msg://1/ X-Uniform-Type-Identifier: com.apple.mail-draft From: David Sinicrope Date: Wed, 31 Jul 2013 14:11:54 +0200 To: "lmap@ietf.org" Cc: Alissa Cooper , Eliot Lear , David Sinicrope Subject: [lmap] Liaison reply to Broadband Forum liaisons X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 12:12:05 -0000 --Apple-Mail-7F47765A-5548-4A0B-BF3D-1FC0872F5DCE Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Hi All, As I mentioned at the mike yesterday there seems to be potential for overlap= between some of the LMAP work (e.g., framework, terminology, etc.) and the "= Broadband Access Service Attributes and Performance Metrics" (WT-304) work g= oing on in the Broadband Forum (BBF). =20 There are also several unanswered liaisons from Broadband Forum to the IETF d= ating back to last August before LMAP was formed. (See list below) Some of t= hese have detailed content from the BBF draft document and questions that th= e LMAP WG should respond to. To support coordination and cooperation between the BBF and IETF LMAP, to en= sure we don't duplicate work, and to request the latest version of the BBF w= ork, it would be advantageous to respond to the BBF liaisons and questions n= ow that the LMAP WG has been formed and there is a charter, scope of work an= d initial work on documents. Thanks, Dave Sinicrope Ericsson IETF Liaison Manager to the Broadband Forum Mar 2013 - https://datatracker.ietf.org/liaison/1243/ Dec 2012 - https://datatracker.ietf.org/liaison/1221/ Sep 2012 - https://datatracker.ietf.org/liaison/1185/ Aug 2012 - https://datatracker.ietf.org/liaison/1179/ --Apple-Mail-7F47765A-5548-4A0B-BF3D-1FC0872F5DCE Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi All,
As I men= tioned at the mike yesterday there seems to be potential for overlap between= some of the LMAP work (e.g., framework, terminology, etc.) and the "= Broadband Access Service Attribute= s and Performance Metrics" (WT-304) work going on in the Broadband Forum (BB= F).   
=
There are al= so several unanswered liaisons from Broadband Forum to the IETF dating back t= o last August before LMAP was formed.  (See list below) Some of these h= ave detailed content from the BBF draft document and questions that the LMAP= WG should respond to.

To sup= port coordination and cooperation between the BBF and IETF LMAP, to ensure w= e don't duplicate work, and to request the latest version of the BBF work, i= t would be advantageous to respond to the BBF liaisons and questions now tha= t the LMAP WG has been formed and there is a charter, scope of work and init= ial work on documents.

Thanks,
Dav= e Sinicrope
Ericsson
IETF Liaison Manager to the Broadba= nd Forum

Dec 2012 - https://datatracker.ietf.org/liaison/1221/
Aug 2012 - <= a href=3D"https://datatracker.ietf.org/liaison/1179/">https://datatracker.ie= tf.org/liaison/1179/


<= /div>

=
= --Apple-Mail-7F47765A-5548-4A0B-BF3D-1FC0872F5DCE-- From dromasca@avaya.com Wed Jul 31 05:35:06 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CB4AD11E8188 for ; Wed, 31 Jul 2013 05:34:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -104.394 X-Spam-Level: X-Spam-Status: No, score=-104.394 tagged_above=-999 required=5 tests=[AWL=1.204, BAYES_00=-2.599, GB_I_LETTER=-2, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G0NYS727mHuP for ; Wed, 31 Jul 2013 05:34:52 -0700 (PDT) Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id 6EC9F11E8194 for ; Wed, 31 Jul 2013 05:32:10 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ai8FAKMC+VHGmAcV/2dsb2JhbABbgkIjITVQgxC7DReBARZ0giQBAQEBAxIRCkwQAgEIDQQEAQELHQMCAgIwFAkIAgQBDQUIDA6HbgELnG2KPJE6EwSONBuBBy0EBgGCZTNzA54liweDFIFoBQQXIg X-IronPort-AV: E=Sophos;i="4.89,787,1367985600"; d="scan'208,217";a="18407196" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by de307622-de-outbound.net.avaya.com with ESMTP; 31 Jul 2013 08:32:06 -0400 Received: from unknown (HELO AZ-FFEXHC04.global.avaya.com) ([135.64.58.14]) by co300216-co-erhwest-out.avaya.com with ESMTP; 31 Jul 2013 08:29:09 -0400 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC04.global.avaya.com ([135.64.58.14]) with mapi id 14.03.0146.000; Wed, 31 Jul 2013 08:32:04 -0400 From: "Romascanu, Dan (Dan)" To: David Sinicrope , "lmap@ietf.org" Thread-Topic: [lmap] Liaison reply to Broadband Forum liaisons Thread-Index: AQHOjecuuVZITzIPiEesWWsJf8uePZl+tUaA Date: Wed, 31 Jul 2013 12:32:03 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA1289056B@AZ-FFEXMB04.global.avaya.com> References: <09AE5A45-FCCC-4CEB-A7DC-B9BE37321BA9@gmail.com> In-Reply-To: <09AE5A45-FCCC-4CEB-A7DC-B9BE37321BA9@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: multipart/alternative; boundary="_000_9904FB1B0159DA42B0B887B7FA8119CA1289056BAZFFEXMB04globa_" MIME-Version: 1.0 Cc: Alissa Cooper , Eliot Lear , David Sinicrope Subject: Re: [lmap] Liaison reply to Broadband Forum liaisons X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 12:35:07 -0000 --_000_9904FB1B0159DA42B0B887B7FA8119CA1289056BAZFFEXMB04globa_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SGksDQoNClRoYW5rcyBEYXZpZCBmb3IgYnJpbmdpbmcgdGhpcyB1cC4NCg0KRmlyc3Qgb2JzZXJ2 YXRpb24uIERhdmlkIHdyaXRlczoNCg0KDQrDmCAgVGhlcmUgYXJlIGFsc28gc2V2ZXJhbCB1bmFu c3dlcmVkIGxpYWlzb25zIGZyb20gQnJvYWRiYW5kIEZvcnVtIHRvIHRoZSBJRVRGIGRhdGluZyBi YWNrIHRvIGxhc3QgQXVndXN0IGJlZm9yZSBMTUFQIHdhcyBmb3JtZWQuICAoU2VlIGxpc3QgYmVs b3cpIFNvbWUgb2YgdGhlc2UgaGF2ZSBkZXRhaWxlZCBjb250ZW50IGZyb20gdGhlIEJCRiBkcmFm dCBkb2N1bWVudCBhbmQgcXVlc3Rpb25zIHRoYXQgdGhlIExNQVAgV0cgc2hvdWxkIHJlc3BvbmQg dG8uDQoNCg0KQWN0dWFsbHkgdGhlIExNQVAgV0cgd2FzIGZvcm1lZCBvbmx5IGEgZmV3IHdlZWtz IGFnbywgYW5kIG5vdCBsYXN0IEF1Z3VzdC4gQmVmb3JlIHRoZSBmb3JtYXRpb24gb2YgdGhlIFdH IHdlIHdlcmUganVzdCBhIGNvbW11bml0eSBnYXRoZXJlZCBhcm91bmQgYSBtYWlsIGxpc3QsIGFu ZCBub3QgaW4gdGhlIHBvc2l0aW9uIHRvIHJlY2VpdmUgYW5kIHJlc3BvbmQgdG8gbGlhaXNvbnMu IFRoaXMgY2hhbmdlZCBub3cuDQoNCkkgd291bGQgaW52aXRlIG90aGVyIHBhcnRpY2lwYW50cyBp biB0aGUgV0cgdG8gcmVhZCB0aGUgbGlhaXNvbiBwb2ludGVkIHRvIGJ5IERhdmlkIGFuZCBzdWdn ZXN0IGhvdyB3ZSBzaG91bGQgcmVzcG9uZC4NCg0KVGhlcmUgYXJlIHR3byBtb3JlIG9yZ2FuaXph dGlvbnMgd2hvIGhhdmUgZXhwcmVzc2VkIGludGVyZXN0IGluIGNvbW11bmljYXRpbmcgd2l0aCB0 aGUgTE1BUCBXRzoNCg0KDQotICAgICAgICAgIFRoZSBJRUVFIDgwMi4xNi4zIHdobyBzZW5kIHVz IGEgbGlhaXNvbiBsZXR0ZXIgYXZhaWxhYmxlIGF0IGh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5v cmcvbGlhaXNvbi8xMjQ0Ly4gQ29tbWVudHMgYXJlIHdlbGNvbWUgYWJvdXQgaG93IHRvIHJlc3Bv bmQgdG8gdGhpcyBvbmUgdG9vLg0KDQotICAgICAgICAgIFRoZSBOR0NPUiAoTmV4dCBHZW5lcmF0 aW9uIENvbnZlcmdlZCBPcGVyYXRpb24gUmVxdWlyZW1lbnRzKSBwcm9qZWN0LiBUaGUgY2hhaXJz IHdpbGwgZGlzY3VzcyB3aXRoIHRoZSByZXByZXNlbnRhdGl2ZXMgZnJvbSBOR0NPUiBpbiB0aGUg Y29taW5nIHdlZWtzIGFuZCB0cnkgdG8gdW5kZXJzdGFuZCB0aGUgbGV2ZWwgb2YgaW5mb3JtYXRp b24gdGhleSB3YW50IHRvIGV4Y2hhbmdlIHdpdGggTE1BUC4NCg0KUmVnYXJkcywNCg0KRGFuDQoN Cg0KRnJvbTogbG1hcC1ib3VuY2VzQGlldGYub3JnIFttYWlsdG86bG1hcC1ib3VuY2VzQGlldGYu b3JnXSBPbiBCZWhhbGYgT2YgRGF2aWQgU2luaWNyb3BlDQpTZW50OiBXZWRuZXNkYXksIEp1bHkg MzEsIDIwMTMgMzoxMiBQTQ0KVG86IGxtYXBAaWV0Zi5vcmcNCkNjOiBBbGlzc2EgQ29vcGVyOyBF bGlvdCBMZWFyOyBEYXZpZCBTaW5pY3JvcGUNClN1YmplY3Q6IFtsbWFwXSBMaWFpc29uIHJlcGx5 IHRvIEJyb2FkYmFuZCBGb3J1bSBsaWFpc29ucw0KDQpIaSBBbGwsDQpBcyBJIG1lbnRpb25lZCBh dCB0aGUgbWlrZSB5ZXN0ZXJkYXkgdGhlcmUgc2VlbXMgdG8gYmUgcG90ZW50aWFsIGZvciBvdmVy bGFwIGJldHdlZW4gc29tZSBvZiB0aGUgTE1BUCB3b3JrIChlLmcuLCBmcmFtZXdvcmssIHRlcm1p bm9sb2d5LCBldGMuKSBhbmQgdGhlICJCcm9hZGJhbmQgQWNjZXNzIFNlcnZpY2UgQXR0cmlidXRl cyBhbmQgUGVyZm9ybWFuY2UgTWV0cmljcyIgKFdULTMwNCkgd29yayBnb2luZyBvbiBpbiB0aGUg QnJvYWRiYW5kIEZvcnVtIChCQkYpLg0KDQoNClRoZXJlIGFyZSBhbHNvIHNldmVyYWwgdW5hbnN3 ZXJlZCBsaWFpc29ucyBmcm9tIEJyb2FkYmFuZCBGb3J1bSB0byB0aGUgSUVURiBkYXRpbmcgYmFj ayB0byBsYXN0IEF1Z3VzdCBiZWZvcmUgTE1BUCB3YXMgZm9ybWVkLiAgKFNlZSBsaXN0IGJlbG93 KSBTb21lIG9mIHRoZXNlIGhhdmUgZGV0YWlsZWQgY29udGVudCBmcm9tIHRoZSBCQkYgZHJhZnQg ZG9jdW1lbnQgYW5kIHF1ZXN0aW9ucyB0aGF0IHRoZSBMTUFQIFdHIHNob3VsZCByZXNwb25kIHRv Lg0KDQoNClRvIHN1cHBvcnQgY29vcmRpbmF0aW9uIGFuZCBjb29wZXJhdGlvbiBiZXR3ZWVuIHRo ZSBCQkYgYW5kIElFVEYgTE1BUCwgdG8gZW5zdXJlIHdlIGRvbid0IGR1cGxpY2F0ZSB3b3JrLCBh bmQgdG8gcmVxdWVzdCB0aGUgbGF0ZXN0IHZlcnNpb24gb2YgdGhlIEJCRiB3b3JrLCBpdCB3b3Vs ZCBiZSBhZHZhbnRhZ2VvdXMgdG8gcmVzcG9uZCB0byB0aGUgQkJGIGxpYWlzb25zIGFuZCBxdWVz dGlvbnMgbm93IHRoYXQgdGhlIExNQVAgV0cgaGFzIGJlZW4gZm9ybWVkIGFuZCB0aGVyZSBpcyBh IGNoYXJ0ZXIsIHNjb3BlIG9mIHdvcmsgYW5kIGluaXRpYWwgd29yayBvbiBkb2N1bWVudHMuDQoN ClRoYW5rcywNCkRhdmUgU2luaWNyb3BlDQpFcmljc3Nvbg0KSUVURiBMaWFpc29uIE1hbmFnZXIg dG8gdGhlIEJyb2FkYmFuZCBGb3J1bQ0KDQoNCk1hciAyMDEzIC0gaHR0cHM6Ly9kYXRhdHJhY2tl ci5pZXRmLm9yZy9saWFpc29uLzEyNDMvDQpEZWMgMjAxMiAtIGh0dHBzOi8vZGF0YXRyYWNrZXIu aWV0Zi5vcmcvbGlhaXNvbi8xMjIxLw0KU2VwIDIwMTIgLSBodHRwczovL2RhdGF0cmFja2VyLmll dGYub3JnL2xpYWlzb24vMTE4NS8NCkF1ZyAyMDEyIC0gaHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRm Lm9yZy9saWFpc29uLzExNzkvDQoNCg0KDQoNCg== --_000_9904FB1B0159DA42B0B887B7FA8119CA1289056BAZFFEXMB04globa_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTIgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 V2luZ2RpbmdzOw0KCXBhbm9zZS0xOjUgMCAwIDAgMCAwIDAgMCAwIDA7fQ0KQGZvbnQtZmFjZQ0K CXtmb250LWZhbWlseTpXaW5nZGluZ3M7DQoJcGFub3NlLTE6NSAwIDAgMCAwIDAgMCAwIDAgMDt9 DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIg MiAyIDQgMyAyIDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpUYWhvbWE7DQoJcGFub3Nl LTE6MiAxMSA2IDQgMyA1IDQgNCAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNv Tm9ybWFsLCBsaS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBpbjsNCgltYXJn aW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGlt ZXMgTmV3IFJvbWFuIiwic2VyaWYiO30NCmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsNCgl7bXNv LXN0eWxlLXByaW9yaXR5Ojk5Ow0KCWNvbG9yOmJsdWU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVy bGluZTt9DQphOnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJsaW5rRm9sbG93ZWQNCgl7bXNvLXN0eWxl LXByaW9yaXR5Ojk5Ow0KCWNvbG9yOnB1cnBsZTsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5l O30NCnAuTXNvTGlzdFBhcmFncmFwaCwgbGkuTXNvTGlzdFBhcmFncmFwaCwgZGl2Lk1zb0xpc3RQ YXJhZ3JhcGgNCgl7bXNvLXN0eWxlLXByaW9yaXR5OjM0Ow0KCW1hcmdpbi10b3A6MGluOw0KCW1h cmdpbi1yaWdodDowaW47DQoJbWFyZ2luLWJvdHRvbTowaW47DQoJbWFyZ2luLWxlZnQ6LjVpbjsN CgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWls eToiVGltZXMgTmV3IFJvbWFuIiwic2VyaWYiO30NCnNwYW4uRW1haWxTdHlsZTE3DQoJe21zby1z dHlsZS10eXBlOnBlcnNvbmFsLXJlcGx5Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1z ZXJpZiI7DQoJY29sb3I6IzFGNDk3RDt9DQouTXNvQ2hwRGVmYXVsdA0KCXttc28tc3R5bGUtdHlw ZTpleHBvcnQtb25seTsNCglmb250LXNpemU6MTAuMHB0O30NCkBwYWdlIFdvcmRTZWN0aW9uMQ0K CXtzaXplOjguNWluIDExLjBpbjsNCgltYXJnaW46MS4waW4gMS4yNWluIDEuMGluIDEuMjVpbjt9 DQpkaXYuV29yZFNlY3Rpb24xDQoJe3BhZ2U6V29yZFNlY3Rpb24xO30NCi8qIExpc3QgRGVmaW5p dGlvbnMgKi8NCkBsaXN0IGwwDQoJe21zby1saXN0LWlkOjE1MTc0NTI5NjE7DQoJbXNvLWxpc3Qt dHlwZTpoeWJyaWQ7DQoJbXNvLWxpc3QtdGVtcGxhdGUtaWRzOjE3OTk4MDkwMjggMjY4NDQ4NTQy IDY3Njk4NjkxIDY3Njk4NjkzIDY3Njk4Njg5IDY3Njk4NjkxIDY3Njk4NjkzIDY3Njk4Njg5IDY3 Njk4NjkxIDY3Njk4NjkzO30NCkBsaXN0IGwwOmxldmVsMQ0KCXttc28tbGV2ZWwtc3RhcnQtYXQ6 MDsNCgltc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ6LTsN Cgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxl ZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluOw0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1z ZXJpZiI7DQoJbXNvLWZhcmVhc3QtZm9udC1mYW1pbHk6Q2FsaWJyaTsNCgltc28tYmlkaS1mb250 LWZhbWlseTpBcmlhbDt9DQpAbGlzdCBsMQ0KCXttc28tbGlzdC1pZDoxNzY2ODc0ODE5Ow0KCW1z by1saXN0LXR5cGU6aHlicmlkOw0KCW1zby1saXN0LXRlbXBsYXRlLWlkczo0ODg5MDkwNiAxNzc2 MjA5MTMyIDY3Njk4NjkxIDY3Njk4NjkzIDY3Njk4Njg5IDY3Njk4NjkxIDY3Njk4NjkzIDY3Njk4 Njg5IDY3Njk4NjkxIDY3Njk4NjkzO30NCkBsaXN0IGwxOmxldmVsMQ0KCXttc28tbGV2ZWwtc3Rh cnQtYXQ6MDsNCgltc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRl eHQ674OYOw0KCW1zby1sZXZlbC10YWItc3RvcDpub25lOw0KCW1zby1sZXZlbC1udW1iZXItcG9z aXRpb246bGVmdDsNCgl0ZXh0LWluZGVudDotLjI1aW47DQoJZm9udC1mYW1pbHk6V2luZ2Rpbmdz Ow0KCW1zby1mYXJlYXN0LWZvbnQtZmFtaWx5OkNhbGlicmk7DQoJbXNvLWJpZGktZm9udC1mYW1p bHk6QXJpYWw7fQ0Kb2wNCgl7bWFyZ2luLWJvdHRvbTowaW47fQ0KdWwNCgl7bWFyZ2luLWJvdHRv bTowaW47fQ0KLS0+PC9zdHlsZT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlZGVm YXVsdHMgdjpleHQ9ImVkaXQiIHNwaWRtYXg9IjEwMjYiIC8+DQo8L3htbD48IVtlbmRpZl0tLT48 IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlbGF5b3V0IHY6ZXh0PSJlZGl0Ij4NCjxv OmlkbWFwIHY6ZXh0PSJlZGl0IiBkYXRhPSIxIiAvPg0KPC9vOnNoYXBlbGF5b3V0PjwveG1sPjwh W2VuZGlmXS0tPg0KPC9oZWFkPg0KPGJvZHkgbGFuZz0iRU4tVVMiIGxpbms9ImJsdWUiIHZsaW5r PSJwdXJwbGUiPg0KPGRpdiBjbGFzcz0iV29yZFNlY3Rpb24xIj4NCjxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFs JnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+SGksPG86cD48L286 cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6 ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZx dW90Oztjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFz cz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTom cXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPlRo YW5rcyBEYXZpZCBmb3IgYnJpbmdpbmcgdGhpcyB1cC4NCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4N CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQt ZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFG NDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+ PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVv dDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj5GaXJzdCBvYnNlcnZhdGlv bi4gRGF2aWQgd3JpdGVzOg0KPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJp YWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNw OzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTGlzdFBhcmFncmFwaCIgc3R5bGU9InRl eHQtaW5kZW50Oi0uMjVpbjttc28tbGlzdDpsMSBsZXZlbDEgbGZvMSI+PCFbaWYgIXN1cHBvcnRM aXN0c10+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6V2luZ2Rpbmdz Ij48c3BhbiBzdHlsZT0ibXNvLWxpc3Q6SWdub3JlIj7DmDxzcGFuIHN0eWxlPSJmb250OjcuMHB0 ICZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyI+Jm5ic3A7DQo8L3NwYW4+PC9zcGFuPjwvc3Bh bj48IVtlbmRpZl0+PHNwYW4gZGlyPSJMVFIiPjwvc3Bhbj48c3BhbiBzdHlsZT0iZm9udC1zaXpl OjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1 b3Q7Ij5UaGVyZSBhcmUgYWxzbyBzZXZlcmFsIHVuYW5zd2VyZWQgbGlhaXNvbnMgZnJvbSBCcm9h ZGJhbmQgRm9ydW0gdG8gdGhlIElFVEYgZGF0aW5nIGJhY2sgdG8gbGFzdCBBdWd1c3QgYmVmb3Jl IExNQVAgd2FzIGZvcm1lZC4gJm5ic3A7KFNlZSBsaXN0IGJlbG93KQ0KIFNvbWUgb2YgdGhlc2Ug aGF2ZSBkZXRhaWxlZCBjb250ZW50IGZyb20gdGhlIEJCRiBkcmFmdCBkb2N1bWVudCBhbmQgcXVl c3Rpb25zIHRoYXQgdGhlIExNQVAgV0cgc2hvdWxkIHJlc3BvbmQgdG8uPG86cD48L286cD48L3Nw YW4+PC9wPg0KPHAgY2xhc3M9Ik1zb0xpc3RQYXJhZ3JhcGgiPjxzcGFuIHN0eWxlPSJmb250LXNp emU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYm cXVvdDs7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xh c3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6 JnF1b3Q7QXJpYWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj5B Y3R1YWxseSB0aGUgTE1BUCBXRyB3YXMgZm9ybWVkIG9ubHkgYSBmZXcgd2Vla3MgYWdvLCBhbmQg bm90IGxhc3QgQXVndXN0LiBCZWZvcmUgdGhlIGZvcm1hdGlvbiBvZiB0aGUgV0cgd2Ugd2VyZSBq dXN0IGEgY29tbXVuaXR5IGdhdGhlcmVkIGFyb3VuZCBhIG1haWwgbGlzdCwNCiBhbmQgbm90IGlu IHRoZSBwb3NpdGlvbiB0byByZWNlaXZlIGFuZCByZXNwb25kIHRvIGxpYWlzb25zLiBUaGlzIGNo YW5nZWQgbm93LiA8bzpwPg0KPC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1 b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286 cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6 ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZx dW90Oztjb2xvcjojMUY0OTdEIj5JIHdvdWxkIGludml0ZSBvdGhlciBwYXJ0aWNpcGFudHMgaW4g dGhlIFdHIHRvIHJlYWQgdGhlIGxpYWlzb24gcG9pbnRlZCB0byBieSBEYXZpZCBhbmQgc3VnZ2Vz dCBob3cgd2Ugc2hvdWxkIHJlc3BvbmQuDQo8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFz cz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTom cXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxv OnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0 eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90 O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+VGhlcmUgYXJlIHR3byBtb3JlIG9yZ2Fu aXphdGlvbnMgd2hvIGhhdmUgZXhwcmVzc2VkIGludGVyZXN0IGluIGNvbW11bmljYXRpbmcgd2l0 aCB0aGUgTE1BUCBXRzoNCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFs JnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8 L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb0xpc3RQYXJhZ3JhcGgiIHN0eWxlPSJ0ZXh0 LWluZGVudDotLjI1aW47bXNvLWxpc3Q6bDAgbGV2ZWwxIGxmbzIiPjwhW2lmICFzdXBwb3J0TGlz dHNdPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGli cmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48c3BhbiBzdHls ZT0ibXNvLWxpc3Q6SWdub3JlIj4tPHNwYW4gc3R5bGU9ImZvbnQ6Ny4wcHQgJnF1b3Q7VGltZXMg TmV3IFJvbWFuJnF1b3Q7Ij4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsNCjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwhW2VuZGlmXT48c3BhbiBkaXI9 IkxUUiI+PC9zcGFuPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZx dW90O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+VGhl IElFRUUgODAyLjE2LjMgd2hvIHNlbmQgdXMgYSBsaWFpc29uIGxldHRlciBhdmFpbGFibGUgYXQN CjxhIGhyZWY9Imh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvbGlhaXNvbi8xMjQ0LyI+aHR0 cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9saWFpc29uLzEyNDQvPC9hPi4gQ29tbWVudHMgYXJl IHdlbGNvbWUgYWJvdXQgaG93IHRvIHJlc3BvbmQgdG8gdGhpcyBvbmUgdG9vLg0KPG86cD48L286 cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb0xpc3RQYXJhZ3JhcGgiIHN0eWxlPSJ0ZXh0LWlu ZGVudDotLjI1aW47bXNvLWxpc3Q6bDAgbGV2ZWwxIGxmbzIiPjwhW2lmICFzdXBwb3J0TGlzdHNd PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkm cXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48c3BhbiBzdHlsZT0i bXNvLWxpc3Q6SWdub3JlIj4tPHNwYW4gc3R5bGU9ImZvbnQ6Ny4wcHQgJnF1b3Q7VGltZXMgTmV3 IFJvbWFuJnF1b3Q7Ij4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsNCjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwhW2VuZGlmXT48c3BhbiBkaXI9IkxU UiI+PC9zcGFuPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90 O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+VGhlDQo8 L3NwYW4+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJp YWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjpibHVlIj5OR0NPUiAoTmV4dCBH ZW5lcmF0aW9uIENvbnZlcmdlZCBPcGVyYXRpb24gUmVxdWlyZW1lbnRzKSBwcm9qZWN0LiBUaGUg Y2hhaXJzIHdpbGwgZGlzY3VzcyB3aXRoIHRoZSByZXByZXNlbnRhdGl2ZXMgZnJvbSBOR0NPUiBp biB0aGUgY29taW5nIHdlZWtzIGFuZCB0cnkgdG8gdW5kZXJzdGFuZCB0aGUgbGV2ZWwNCiBvZiBp bmZvcm1hdGlvbiB0aGV5IHdhbnQgdG8gZXhjaGFuZ2Ugd2l0aCBMTUFQLiA8L3NwYW4+PHNwYW4g c3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssJnF1 b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48bzpwPjwvbzpwPjwvc3Bhbj48L3A+ DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250 LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMx RjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1 b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+UmVnYXJkcyw8bzpwPjwv bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1z aXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlm JnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5 OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+ RGFuPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5 bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVv dDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFu PjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0 O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztj b2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8ZGl2IHN0eWxlPSJi b3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCBibHVlIDEuNXB0O3BhZGRpbmc6MGluIDBpbiAw aW4gNC4wcHQiPg0KPGRpdj4NCjxkaXYgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci10b3A6c29s aWQgI0I1QzRERiAxLjBwdDtwYWRkaW5nOjMuMHB0IDBpbiAwaW4gMGluIj4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPjxiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZx dW90O1RhaG9tYSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7Ij5Gcm9tOjwvc3Bhbj48L2I+ PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGFob21hJnF1 b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDsiPiBsbWFwLWJvdW5jZXNAaWV0Zi5vcmcgW21haWx0 bzpsbWFwLWJvdW5jZXNAaWV0Zi5vcmddDQo8Yj5PbiBCZWhhbGYgT2YgPC9iPkRhdmlkIFNpbmlj cm9wZTxicj4NCjxiPlNlbnQ6PC9iPiBXZWRuZXNkYXksIEp1bHkgMzEsIDIwMTMgMzoxMiBQTTxi cj4NCjxiPlRvOjwvYj4gbG1hcEBpZXRmLm9yZzxicj4NCjxiPkNjOjwvYj4gQWxpc3NhIENvb3Bl cjsgRWxpb3QgTGVhcjsgRGF2aWQgU2luaWNyb3BlPGJyPg0KPGI+U3ViamVjdDo8L2I+IFtsbWFw XSBMaWFpc29uIHJlcGx5IHRvIEJyb2FkYmFuZCBGb3J1bSBsaWFpc29uczxvOnA+PC9vOnA+PC9z cGFuPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNw OzwvbzpwPjwvcD4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9y bWFsIj5IaSBBbGwsPGJyPg0KQXMgSSBtZW50aW9uZWQgYXQgdGhlIG1pa2UgeWVzdGVyZGF5IHRo ZXJlIHNlZW1zIHRvIGJlIHBvdGVudGlhbCBmb3Igb3ZlcmxhcCBiZXR3ZWVuIHNvbWUgb2YgdGhl IExNQVAgd29yayAoZS5nLiwgZnJhbWV3b3JrLCB0ZXJtaW5vbG9neSwgZXRjLikgYW5kIHRoZSAm cXVvdDtCcm9hZGJhbmQgQWNjZXNzIFNlcnZpY2UgQXR0cmlidXRlcyBhbmQgUGVyZm9ybWFuY2Ug TWV0cmljcyZxdW90OyAoV1QtMzA0KSB3b3JrIGdvaW5nIG9uIGluIHRoZSBCcm9hZGJhbmQgRm9y dW0NCiAoQkJGKS4gJm5ic3A7Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8 cCBjbGFzcz0iTXNvTm9ybWFsIj48YnI+DQo8YnI+DQo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0K PGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPlRoZXJlIGFyZSBhbHNvIHNldmVyYWwgdW5hbnN3 ZXJlZCBsaWFpc29ucyBmcm9tIEJyb2FkYmFuZCBGb3J1bSB0byB0aGUgSUVURiBkYXRpbmcgYmFj ayB0byBsYXN0IEF1Z3VzdCBiZWZvcmUgTE1BUCB3YXMgZm9ybWVkLiAmbmJzcDsoU2VlIGxpc3Qg YmVsb3cpIFNvbWUgb2YgdGhlc2UgaGF2ZSBkZXRhaWxlZCBjb250ZW50IGZyb20gdGhlIEJCRiBk cmFmdCBkb2N1bWVudCBhbmQgcXVlc3Rpb25zIHRoYXQgdGhlIExNQVANCiBXRyBzaG91bGQgcmVz cG9uZCB0by48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxicj4NCjxicj4NCjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9 Ik1zb05vcm1hbCI+VG8gc3VwcG9ydCBjb29yZGluYXRpb24gYW5kIGNvb3BlcmF0aW9uIGJldHdl ZW4gdGhlIEJCRiBhbmQgSUVURiBMTUFQLCB0byBlbnN1cmUgd2UgZG9uJ3QgZHVwbGljYXRlIHdv cmssIGFuZCB0byByZXF1ZXN0IHRoZSBsYXRlc3QgdmVyc2lvbiBvZiB0aGUgQkJGIHdvcmssIGl0 IHdvdWxkIGJlIGFkdmFudGFnZW91cyB0byByZXNwb25kIHRvIHRoZSBCQkYgbGlhaXNvbnMgYW5k IHF1ZXN0aW9ucyBub3cgdGhhdA0KIHRoZSBMTUFQIFdHIGhhcyBiZWVuIGZvcm1lZCBhbmQgdGhl cmUgaXMgYSBjaGFydGVyLCBzY29wZSBvZiB3b3JrIGFuZCBpbml0aWFsIHdvcmsgb24gZG9jdW1l bnRzLjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+ PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs Ij5UaGFua3MsPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9y bWFsIj5EYXZlIFNpbmljcm9wZTxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xh c3M9Ik1zb05vcm1hbCI+RXJpY3Nzb248bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPklFVEYgTGlhaXNvbiBNYW5hZ2VyIHRvIHRoZSBCcm9hZGJhbmQg Rm9ydW08bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi Pjxicj4NCjxicj4NCjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1z b05vcm1hbCI+TWFyIDIwMTMgLSZuYnNwOzxhIGhyZWY9Imh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0 Zi5vcmcvbGlhaXNvbi8xMjQzLyI+aHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9saWFpc29u LzEyNDMvPC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+RGVjIDIwMTIgLSZuYnNwOzxhIGhyZWY9Imh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5v cmcvbGlhaXNvbi8xMjIxLyI+aHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9saWFpc29uLzEy MjEvPC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1h bCI+U2VwIDIwMTIgLSZuYnNwOzxhIGhyZWY9Imh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcv bGlhaXNvbi8xMTg1LyI+aHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9saWFpc29uLzExODUv PC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+ QXVnIDIwMTIgLSZuYnNwOzxhIGhyZWY9Imh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvbGlh aXNvbi8xMTc5LyI+aHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9saWFpc29uLzExNzkvPC9h PjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86 cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+PGJyPg0KPGJyPg0KPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFz cz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rp dj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9odG1sPg0K --_000_9904FB1B0159DA42B0B887B7FA8119CA1289056BAZFFEXMB04globa_-- From david.sinicrope@gmail.com Wed Jul 31 05:46:28 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EC05821F9D53 for ; Wed, 31 Jul 2013 05:46:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.203 X-Spam-Level: X-Spam-Status: No, score=-2.203 tagged_above=-999 required=5 tests=[AWL=1.000, BAYES_00=-2.599, GB_I_LETTER=-2, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nxXz2jmSNOqn for ; Wed, 31 Jul 2013 05:46:25 -0700 (PDT) Received: from mail-pb0-x234.google.com (mail-pb0-x234.google.com [IPv6:2607:f8b0:400e:c01::234]) by ietfa.amsl.com (Postfix) with ESMTP id 342B221F8488 for ; Wed, 31 Jul 2013 05:45:54 -0700 (PDT) Received: by mail-pb0-f52.google.com with SMTP id wz12so723243pbc.39 for ; Wed, 31 Jul 2013 05:45:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:in-reply-to:mime-version:content-type:message-id :content-transfer-encoding:cc:x-mailer:from:subject:date:to; bh=xVYlJ2IkLizH7DVwZvXyULx7gjh0pSbYBQkrdebWXSQ=; b=L2seXGFPEN1t1rxAZiDJCXx4TDfgzQIN+KjbUmpoVTx8quZqYwDxz4KOyBGjVlk70Q PGa8HZWN7tanKOMxqn96KxAiv1a47twsYoAu9G7FVBeNjTBPbarkYnzkuiV29yANDe+K mdVLFbWlEVIulZQeQBLTxWBRbj4EgMlcg/WFrl2ieseO9vygU78aKqhdu6fNDymzhntL 78UTgOpozjziDnXyuspgqERk9U9kQhMflARC/pK754ih8beYLCSfh2/Bdq4A65Y4xn+3 3ohEm4dxT6vjhbB6OyWZDnVFOvSbPdsaxIRu2qn955MLOVBz/ZkvLV7Eh7kToz0EDwNc BB3Q== X-Received: by 10.68.93.65 with SMTP id cs1mr79056840pbb.11.1375274753874; Wed, 31 Jul 2013 05:45:53 -0700 (PDT) Received: from ?IPv6:2001:df8::64:e07c:386a:a6c4:a48? ([2001:df8:0:64:e07c:386a:a6c4:a48]) by mx.google.com with ESMTPSA id t9sm1953146pba.46.2013.07.31.05.45.52 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 31 Jul 2013 05:45:53 -0700 (PDT) References: <09AE5A45-FCCC-4CEB-A7DC-B9BE37321BA9@gmail.com> <9904FB1B0159DA42B0B887B7FA8119CA1289056B@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1289056B@AZ-FFEXMB04.global.avaya.com> Mime-Version: 1.0 (1.0) Content-Type: multipart/alternative; boundary=Apple-Mail-4F1158B0-9996-4983-8265-D4127D81AB2D Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: iPad Mail (10B329) From: David Sinicrope Date: Wed, 31 Jul 2013 14:45:49 +0200 To: "Romascanu, Dan (Dan)" Cc: David Sinicrope , Alissa Cooper , Eliot Lear , "lmap@ietf.org" Subject: Re: [lmap] Liaison reply to Broadband Forum liaisons X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 12:46:28 -0000 --Apple-Mail-4F1158B0-9996-4983-8265-D4127D81AB2D Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Thanks Dan. Exactly right. Stating my point another way: Now that LMAP is f= ormed, and we have a way to respond formally, we should address the backlog o= f BBF liaisons addressed to the IETF. (They were sent to the IETF and IESG b= ecause there was no LMAP WG to address them to.) Thanks again, Dave On Jul 31, 2013, at 2:32 PM, "Romascanu, Dan (Dan)" wro= te: > Hi, > =20 > Thanks David for bringing this up. > =20 > First observation. David writes: > =20 > =C3=98 There are also several unanswered liaisons from Broadband Forum to= the IETF dating back to last August before LMAP was formed. (See list belo= w) Some of these have detailed content from the BBF draft document and quest= ions that the LMAP WG should respond to. > =20 > Actually the LMAP WG was formed only a few weeks ago, and not last August.= Before the formation of the WG we were just a community gathered around a m= ail list, and not in the position to receive and respond to liaisons. This c= hanged now. > =20 > I would invite other participants in the WG to read the liaison pointed to= by David and suggest how we should respond. > =20 > There are two more organizations who have expressed interest in communicat= ing with the LMAP WG: > =20 > - The IEEE 802.16.3 who send us a liaison letter available at htt= ps://datatracker.ietf.org/liaison/1244/. Comments are welcome about how to r= espond to this one too. > - The NGCOR (Next Generation Converged Operation Requirements) pr= oject. The chairs will discuss with the representatives from NGCOR in the co= ming weeks and try to understand the level of information they want to excha= nge with LMAP. > =20 > Regards, > =20 > Dan > =20 > =20 > From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of Da= vid Sinicrope > Sent: Wednesday, July 31, 2013 3:12 PM > To: lmap@ietf.org > Cc: Alissa Cooper; Eliot Lear; David Sinicrope > Subject: [lmap] Liaison reply to Broadband Forum liaisons > =20 > Hi All, > As I mentioned at the mike yesterday there seems to be potential for overl= ap between some of the LMAP work (e.g., framework, terminology, etc.) and th= e "Broadband Access Service Attributes and Performance Metrics" (WT-304) wor= k going on in the Broadband Forum (BBF). =20 >=20 >=20 > There are also several unanswered liaisons from Broadband Forum to the IET= F dating back to last August before LMAP was formed. (See list below) Some o= f these have detailed content from the BBF draft document and questions that= the LMAP WG should respond to. >=20 >=20 > To support coordination and cooperation between the BBF and IETF LMAP, to e= nsure we don't duplicate work, and to request the latest version of the BBF w= ork, it would be advantageous to respond to the BBF liaisons and questions n= ow that the LMAP WG has been formed and there is a charter, scope of work an= d initial work on documents. > =20 > Thanks, > Dave Sinicrope > Ericsson > IETF Liaison Manager to the Broadband Forum >=20 >=20 > Mar 2013 - https://datatracker.ietf.org/liaison/1243/ > Dec 2012 - https://datatracker.ietf.org/liaison/1221/ > Sep 2012 - https://datatracker.ietf.org/liaison/1185/ > Aug 2012 - https://datatracker.ietf.org/liaison/1179/ > =20 >=20 >=20 > =20 --Apple-Mail-4F1158B0-9996-4983-8265-D4127D81AB2D Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Thanks Dan. &= nbsp;Exactly right.  Stating my point another way: Now that LMAP is for= med, and we have a way to respond formally, we should address the backlog of= BBF liaisons addressed to the IETF.  (They were sent to the IETF and I= ESG because there was no LMAP WG to address them to.)
Thanks again= ,
Dave



On Jul 31, 2013, at 2:32 PM, "Rom= ascanu, Dan (Dan)" <dromasca@avaya.= com> wrote:

Hi,

 

Thanks David for bringing thi= s up.

 

First observation. David writ= es:

 

=C3=98  There a= re also several unanswered liaisons from Broadband Forum to the IETF dating b= ack to last August before LMAP was formed.  (See list below) Some of these have detailed content from the BBF draft document and questio= ns that the LMAP WG should respond to.

 

Actually the LMAP WG was form= ed only a few weeks ago, and not last August. Before the formation of the WG= we were just a community gathered around a mail list, and not in the position to receive and respond to liaisons. This changed no= w.

 

I would invite other particip= ants in the WG to read the liaison pointed to by David and suggest how we sh= ould respond.

 

There are two more organizati= ons who have expressed interest in communicating with the LMAP WG:

 

- =          The IEEE 802.16.3 who send us a liaison letter available at https://datatracker.i= etf.org/liaison/1244/. Comments are welcome about how to respond to this= one too.

- =          The NGCOR (Next Generation Converged Operation Requi= rements) project. The chairs will discuss with the representatives from NGCO= R in the coming weeks and try to understand the level of information they want to exchange with LMAP.

 

Regards,

 

Dan

 

 

From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of David Sinicrope
Sent: Wednesday, July 31, 2013 3:12 PM
To: lmap@ietf.org
Cc: Alissa Cooper; Eliot Lear; David Sinicrope
Subject: [lmap] Liaison reply to Broadband Forum liaisons<= /span>

 

Hi All,
As I mentioned at the mike yesterday there seems to be potential for overlap= between some of the LMAP work (e.g., framework, terminology, etc.) and the "= Broadband Access Service Attributes and Performance Metrics" (WT-304) work g= oing on in the Broadband Forum (BBF).   



There are also several unanswered liaisons from Broad= band Forum to the IETF dating back to last August before LMAP was formed. &n= bsp;(See list below) Some of these have detailed content from the BBF draft d= ocument and questions that the LMAP WG should respond to.



To support coordination and cooperation between the B= BF and IETF LMAP, to ensure we don't duplicate work, and to request the late= st version of the BBF work, it would be advantageous to respond to the BBF l= iaisons and questions now that the LMAP WG has been formed and there is a charter, scope of work and initi= al work on documents.

 

Thanks,

Dave Sinicrope

Ericsson

IETF Liaison Manager to the Broadband Forum



 



 

= --Apple-Mail-4F1158B0-9996-4983-8265-D4127D81AB2D-- From philip.eardley@bt.com Wed Jul 31 06:01:27 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 36AED11E80E4 for ; Wed, 31 Jul 2013 06:01:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.485 X-Spam-Level: X-Spam-Status: No, score=-103.485 tagged_above=-999 required=5 tests=[AWL=0.114, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 31Qx7iGowka7 for ; Wed, 31 Jul 2013 06:01:23 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp62.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id B697721F9F6F for ; Wed, 31 Jul 2013 05:59:21 -0700 (PDT) Received: from EVMHT69-UKRD.domain1.systemhost.net (10.36.3.129) by RDW083A006ED62.smtp-e2.hygiene.service (10.187.98.11) with Microsoft SMTP Server (TLS) id 8.3.298.1; Wed, 31 Jul 2013 13:59:08 +0100 Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.2.219]) by EVMHT69-UKRD.domain1.systemhost.net ([10.36.3.129]) with mapi; Wed, 31 Jul 2013 13:59:08 +0100 From: To: , Date: Wed, 31 Jul 2013 13:57:37 +0100 Thread-Topic: draft-eardley-lmap-framework: Extensibility Thread-Index: Ac6N2sXu4qyjuxcAQV6Ll8U30MAFUAAEsIbt Message-ID: <9510D26531EF184D9017DF24659BB87F35CC70B80D@EMV65-UKRD.domain1.systemhost.net> References: <2D09D61DDFA73D4C884805CC7865E611303172BC@GAALPA1MSGUSR9L.ITServices.sbc.com> In-Reply-To: <2D09D61DDFA73D4C884805CC7865E611303172BC@GAALPA1MSGUSR9L.ITServices.sbc.com> Accept-Language: en-US, en-GB Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] draft-eardley-lmap-framework: Extensibility X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 13:01:27 -0000 Barbara If I remember, when I wrote this 'extensibility' bullet in the Intro I had = in mind the last of your options. Will think about this more ________________________________________ From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of STARK, BAR= BARA H [bs7652@att.com] Sent: 31 July 2013 11:43 To: lmap@ietf.org Subject: [lmap] draft-eardley-lmap-framework: Extensibility I read through the current draft and had a comment on the inclusion of the = "Extensible" bullet in the Introduction. I can't find anything in the rest of the framework or the charter + deliver= ables that really has anything to do with extensibility; but if it is to be= called out as a desirable feature in the Introduction, I would expect some= discussion of how to achieve it or what's being done to work towards achie= ving it. Or don't call it out so strongly. Or call it out and state that wh= ile it is desirable, the current proposed framework does nothing to address= it. Or state that extensibility of Measurement Methods is being provided by the= ippm-defined registry (which will be a "living" registry?)?; and the lmap = Information Model and protocols/data models will ensure that this extensibi= lity of Measurement Methods is supported so MAs. I think that there's also = the intention for MAs to be able to express to a controller which measureme= nts they support / what they're capable of -- which helps in the area of ex= tensibility. This would need to be in the information model. I prefer the last of these options, if that is what we're planning. BTW, extensibility areas that aren't currently in scope of lmap are being a= ble to upgrade Measurement Methods in a MA, or upgrade the MA software vers= ion. These would be functions of bootstrap/initialization, which aren't in = scope. _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap= From Jean-Francois.TremblayING@videotron.com Wed Jul 31 07:10:48 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F86811E8109 for ; Wed, 31 Jul 2013 07:10:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V4RldZYCZVlg for ; Wed, 31 Jul 2013 07:10:42 -0700 (PDT) Received: from mx01.videotron.com (mx01.videotron.com [24.201.243.152]) by ietfa.amsl.com (Postfix) with ESMTP id 8CA8A21F9E02 for ; Wed, 31 Jul 2013 07:09:26 -0700 (PDT) In-Reply-To: <51F8E728.8070403@cisco.com> References: <51F8E728.8070403@cisco.com> To: "lmap@ietf.org" MIME-Version: 1.0 X-KeepSent: CE2ECB92:C17B8B93-85257BB9:004AE050; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 8.5.3FP1 March 08, 2012 Message-ID: From: Jean-Francois.TremblayING@videotron.com Date: Wed, 31 Jul 2013 10:09:20 -0400 X-MIMETrack: Serialize by Router on DOMMSG01/SRV/GVL(Release 8.5.3FP3|November 15, 2012) at 07/31/2013 10:09:18, Serialize complete at 07/31/2013 10:09:18 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable Received-SPF: none Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 14:10:48 -0000 Paul Aitken a =E9crit sur 31/07/2013 06:30:00 AM : > Jean-Francois, Sharam, Mike, Juergen, All, > The user-initiated test scenario which you describe looks like this: >=20 > ispController ispCollector > \ / > \ / > MA--{test peers) > / \ > / \ > userController userCollector This isn't what I had in mind. The example I used was more of a ispMA on a user device, talking to a second ispCollector2.=20 ispController ispCollector \ / \ / MA--{test peers) \ \ ispCollector2 I understand that a second collector might be out of scope for=20 initial work here, but I'm trying to clear some of the confusion=20 on wording and open up the possibilities. It's possible, for=20 example, to have an ISP owned MA, ISP-initiated on an end-user=20 device.=20 user-device !=3D user-initiated !=3D user-owned MA For example, what if, as an ISP, I built a Windows 8 LMAP app=20 that the user can download and install. By running the app, the user agrees to have this app run ISP-triggered tests once a day=20 or upon demand (when he calls support for example). In this case,=20 we have:=20 Domain =3D ISP Device ownership =3D user MA ownership =3D ISP (I own the code of app) MA control =3D ISP=20 MA initiation =3D ISP Another example. Same type of app, this time on Android, on a=20 wifi mobile device: Domain =3D ISP Device ownership =3D user MA ownership =3D ISP MA control =3D ISP MA initiation =3D user In the above case, I agree with Barbara that a need for user=20 authentication might exist, but it could also be done automatically through source IP address.=20 This type of measurement can actually be useful in the regulatory=20 case as well. As an ISP, I could run MAs both on user devices and on ISP devices (integrated broadband routers for example). By=20 correlating results between both groups, I could show regulatory=20 instances that most speed issues do not arise in the last mile but actually in the last feet of the connection, in the user network (with wifi for example).=20 I'm not advocating that we bloat the scope of LMAP to huge=20 proportions here. My point is simpler: we are putting artificial and unnecessary boundaries on what LMAP can do if we assume=20 an automatic link between device ownership, domain, MA ownership=20 and measurement initiation.=20 /JF From lear@cisco.com Wed Jul 31 06:15:41 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BAA8A21E8056 for ; Wed, 31 Jul 2013 06:15:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -109.095 X-Spam-Level: X-Spam-Status: No, score=-109.095 tagged_above=-999 required=5 tests=[AWL=0.903, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_23=0.6, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id saDE31jZtuKb for ; Wed, 31 Jul 2013 06:15:37 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id E16F321F9B14 for ; Wed, 31 Jul 2013 06:15:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6300; q=dns/txt; s=iport; t=1375276525; x=1376486125; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to; bh=wVr1pbjbOOpTYhvPGuEGalJhZMxPXIvAKzTT1MwGXnc=; b=c7FaDs/Ae1fYZKeeDkqYdICAk32BM9uoSrJn/YZ+T5FCShI1rzTr1FiA m7bLsAFiJkeg1xckPJ2BaGDqX6MAUJZwUDb54PjN7qwV8IsbWBKn/gXNa 3X93JB9q2aIDOAjT3sIc5CNyUztbahfsFadumpWsthVfgAuYwA9PUnp+s A=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AikFAH4N+VGQ/khR/2dsb2JhbABbgwY1g2CFXbUwgRgWdIIkAQEBBCNVARALGAkWCwICCQMCAQIBKxoGDQEHAQGIDAynDZFOBI40gVMHgmWBJgOXX5FNgxY6 X-IronPort-AV: E=Sophos;i="4.89,729,1367971200"; d="scan'208,217";a="16185179" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-3.cisco.com with ESMTP; 31 Jul 2013 13:15:16 +0000 Received: from mctiny.local ([10.61.196.104]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id r6VDFDDO000966 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 13:15:14 GMT Message-ID: <51F90DE1.8080503@cisco.com> Date: Wed, 31 Jul 2013 15:15:13 +0200 From: Eliot Lear User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: David Sinicrope References: <09AE5A45-FCCC-4CEB-A7DC-B9BE37321BA9@gmail.com> In-Reply-To: <09AE5A45-FCCC-4CEB-A7DC-B9BE37321BA9@gmail.com> X-Enigmail-Version: 1.5.2 Content-Type: multipart/alternative; boundary="------------080108020904030806010306" X-Mailman-Approved-At: Wed, 31 Jul 2013 07:12:55 -0700 Cc: Ross Callon , David Sinicrope , Alissa Cooper , "lmap@ietf.org" Subject: Re: [lmap] Liaison reply to Broadband Forum liaisons X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 13:15:41 -0000 This is a multi-part message in MIME format. --------------080108020904030806010306 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit CC'ing Ross Callon. Eliot On 7/31/13 2:11 PM, David Sinicrope wrote: > Hi All, > As I mentioned at the mike yesterday there seems to be potential for > overlap between some of the LMAP work (e.g., framework, terminology, > etc.) and the "Broadband Access Service Attributes and Performance > Metrics" (WT-304) work going on in the Broadband Forum (BBF). > > There are also several unanswered liaisons from Broadband Forum to the > IETF dating back to last August before LMAP was formed. (See list > below) Some of these have detailed content from the BBF draft document > and questions that the LMAP WG should respond to. > > To support coordination and cooperation between the BBF and IETF LMAP, > to ensure we don't duplicate work, and to request the latest version > of the BBF work, it would be advantageous to respond to the BBF > liaisons and questions now that the LMAP WG has been formed and there > is a charter, scope of work and initial work on documents. > > Thanks, > Dave Sinicrope > Ericsson > IETF Liaison Manager to the Broadband Forum > > Mar 2013 - https://datatracker.ietf.org/liaison/1243/ > Dec 2012 - https://datatracker.ietf.org/liaison/1221/ > Sep 2012 - https://datatracker.ietf.org/liaison/1185/ > Aug 2012 - https://datatracker.ietf.org/liaison/1179/ > > > --------------080108020904030806010306 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit CC'ing Ross Callon.

Eliot
On 7/31/13 2:11 PM, David Sinicrope wrote:
Hi All,
As I mentioned at the mike yesterday there seems to be potential for overlap between some of the LMAP work (e.g., framework, terminology, etc.) and the "Broadband Access Service Attributes and Performance Metrics" (WT-304) work going on in the Broadband Forum (BBF).   

There are also several unanswered liaisons from Broadband Forum to the IETF dating back to last August before LMAP was formed.  (See list below) Some of these have detailed content from the BBF draft document and questions that the LMAP WG should respond to.

To support coordination and cooperation between the BBF and IETF LMAP, to ensure we don't duplicate work, and to request the latest version of the BBF work, it would be advantageous to respond to the BBF liaisons and questions now that the LMAP WG has been formed and there is a charter, scope of work and initial work on documents.

Thanks,
Dave Sinicrope
Ericsson
IETF Liaison Manager to the Broadband Forum





--------------080108020904030806010306-- From dave.mcdysan@verizon.com Wed Jul 31 07:40:05 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B871F11E80F4 for ; Wed, 31 Jul 2013 07:40:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.599 X-Spam-Level: X-Spam-Status: No, score=-4.599 tagged_above=-999 required=5 tests=[AWL=0.999, BAYES_00=-2.599, GB_I_LETTER=-2, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8j0QuL3igqXK for ; Wed, 31 Jul 2013 07:39:58 -0700 (PDT) Received: from omzsmtpe03.verizonbusiness.com (omzsmtpe03.verizonbusiness.com [199.249.25.208]) by ietfa.amsl.com (Postfix) with ESMTP id 3EB3721F9A3C for ; Wed, 31 Jul 2013 07:39:57 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: false Received: from unknown (HELO fldsmtpi03.verizon.com) ([166.68.71.145]) by omzsmtpe03.verizonbusiness.com with ESMTP; 31 Jul 2013 14:39:43 +0000 From: "Mcdysan, David E" X-IronPort-AV: E=Sophos;i="4.89,787,1367971200"; d="scan'208,217";a="518917136" Received: from fhdp1lumxc7hb05.verizon.com (HELO FHDP1LUMXC7HB05.us.one.verizon.com) ([166.68.59.192]) by fldsmtpi03.verizon.com with ESMTP; 31 Jul 2013 14:39:43 +0000 Received: from fhdp1lumxc7v11.us.one.verizon.com ([166.68.59.148]) by FHDP1LUMXC7HB05.us.one.verizon.com ([166.68.59.192]) with mapi; Wed, 31 Jul 2013 10:39:42 -0400 To: David Sinicrope , "Romascanu, Dan (Dan)" Date: Wed, 31 Jul 2013 10:39:36 -0400 Thread-Topic: [lmap] Liaison reply to Broadband Forum liaisons Thread-Index: Ac6N+8s32U81D/BpTkGlH2Zmg4kFsw== Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.2.5.121010 acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_CE1E9959714B0davemcdysanoneverizoncom_" MIME-Version: 1.0 Cc: Alissa Cooper , "lmap@ietf.org" , Eliot Lear , David Sinicrope Subject: Re: [lmap] Liaison reply to Broadband Forum liaisons X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 14:40:05 -0000 --_000_CE1E9959714B0davemcdysanoneverizoncom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable David, Dan, Thank you for bringing up work in BBF and IEEE that may be related to that = being considered in lamp. For similar functionality, the industry should strive to standardize on a s= ingle solution and avoid standardization of multiple solutions that provide= similar functionality. Thanks, Dave From: David Sinicrope > Date: Wednesday, July 31, 2013 8:45 AM To: "Romascanu, Dan (Dan)" > Cc: David Sinicrope >, Alissa Cooper >, Elio= t Lear >, "lmap@ietf.org" > Subject: Re: [lmap] Liaison reply to Broadband Forum liaisons Thanks Dan. Exactly right. Stating my point another way: Now that LMAP is= formed, and we have a way to respond formally, we should address the backl= og of BBF liaisons addressed to the IETF. (They were sent to the IETF and = IESG because there was no LMAP WG to address them to.) Thanks again, Dave On Jul 31, 2013, at 2:32 PM, "Romascanu, Dan (Dan)" > wrote: Hi, Thanks David for bringing this up. First observation. David writes: =D8 There are also several unanswered liaisons from Broadband Forum to the= IETF dating back to last August before LMAP was formed. (See list below) = Some of these have detailed content from the BBF draft document and questio= ns that the LMAP WG should respond to. Actually the LMAP WG was formed only a few weeks ago, and not last August. = Before the formation of the WG we were just a community gathered around a m= ail list, and not in the position to receive and respond to liaisons. This = changed now. I would invite other participants in the WG to read the liaison pointed to = by David and suggest how we should respond. There are two more organizations who have expressed interest in communicati= ng with the LMAP WG: - The IEEE 802.16.3 who send us a liaison letter available at http= s://datatracker.ietf.org/liaison/1244/. Comments are welcome about how to r= espond to this one too. - The NGCOR (Next Generation Converged Operation Requirements) pro= ject. The chairs will discuss with the representatives from NGCOR in the co= ming weeks and try to understand the level of information they want to exch= ange with LMAP. Regards, Dan From: lmap-bounces@ietf.org [mailto:lmap-boun= ces@ietf.org] On Behalf Of David Sinicrope Sent: Wednesday, July 31, 2013 3:12 PM To: lmap@ietf.org Cc: Alissa Cooper; Eliot Lear; David Sinicrope Subject: [lmap] Liaison reply to Broadband Forum liaisons Hi All, As I mentioned at the mike yesterday there seems to be potential for overla= p between some of the LMAP work (e.g., framework, terminology, etc.) and th= e "Broadband Access Service Attributes and Performance Metrics" (WT-304) wo= rk going on in the Broadband Forum (BBF). There are also several unanswered liaisons from Broadband Forum to the IETF= dating back to last August before LMAP was formed. (See list below) Some = of these have detailed content from the BBF draft document and questions th= at the LMAP WG should respond to. To support coordination and cooperation between the BBF and IETF LMAP, to e= nsure we don't duplicate work, and to request the latest version of the BBF= work, it would be advantageous to respond to the BBF liaisons and question= s now that the LMAP WG has been formed and there is a charter, scope of wor= k and initial work on documents. Thanks, Dave Sinicrope Ericsson IETF Liaison Manager to the Broadband Forum Mar 2013 - https://datatracker.ietf.org/liaison/1243/ Dec 2012 - https://datatracker.ietf.org/liaison/1221/ Sep 2012 - https://datatracker.ietf.org/liaison/1185/ Aug 2012 - https://datatracker.ietf.org/liaison/1179/ --_000_CE1E9959714B0davemcdysanoneverizoncom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
David, Dan,
Thank you for bringing up work in BBF and IEEE that may be rel= ated to that being considered in lamp.

For similar= functionality, the industry should strive to standardize on a single solut= ion and avoid standardization of multiple solutions that provide similar fu= nctionality.

Thanks,

Dave=

From: Da= vid Sinicrope <david.sinicr= ope@gmail.com>
Date: Wed= nesday, July 31, 2013 8:45 AM
To: "Romascanu, Dan (Dan)" <dromasc= a@avaya.com>
Cc: David S= inicrope <david.sinicrop= e@ericsson.com>, Alissa Cooper <acooper@cdt.org>, Eliot Lear <lear@cisco.com>, "lmap@ietf.org" <lmap@ietf.org>
Subject: Re: [lmap] Liaison reply to Broadba= nd Forum liaisons

Tha= nks Dan.  Exactly right.  Stating my point another way: Now that = LMAP is formed, and we have a way to respond formally, we should address th= e backlog of BBF liaisons addressed to the IETF.  (They were sent to t= he IETF and IESG because there was no LMAP WG to address them to.)
Thanks again,
Dave



On Jul 31, 2013, a= t 2:32 PM, "Romascanu, Dan (Dan)" <dromasca@avaya.com> wrote:

<= div>=

Hi,

<= span style=3D"font-size: 10pt; font-family: Arial, sans-serif; color: rgb(3= 1, 73, 125); "> 

Thanks David for bringing this up.

 

First observation. Dav= id writes:

 

=D8  There are also several u= nanswered liaisons from Broadband Forum to the IETF dating back to last Aug= ust before LMAP was formed.  (See list below) Some of these have detailed content from the BBF draft document and questi= ons that the LMAP WG should respond to.

 

Actually the LMAP WG was formed only a few weeks ag= o, and not last August. Before the formation of the WG we were just a commu= nity gathered around a mail list, and not in the position to receive and respond to liaisons. This changed n= ow.

 = ;

I would invite oth= er participants in the WG to read the liaison pointed to by David and sugge= st how we should respond.

 

There are two more org= anizations who have expressed interest in communicating with the LMAP WG:

 

<= span style=3D"mso-list:Ignore">-       &n= bsp;  The IEEE 802.16.3 who send us a liaison letter available at https://datatracker.= ietf.org/liaison/1244/. Comments are welcome about how to respond to th= is one too.

-      =     The NGCOR (Next Generation Converged Operation Requirements) project= . The chairs will discuss with the representatives from NGCOR in the coming= weeks and try to understand the level of information they want to exchange with LMAP.

 =

Regards,

 

Dan

 

 

From: lmap-bounces= @ietf.org [mailto:lmap-bounces= @ietf.org] On Behalf Of David Sinicrope
Sent: Wednesday, July 31, 201= 3 3:12 PM
To: lmap@ietf.org<= br>Cc: Alissa Cooper; Eliot Lear; David Sinicrope
Subject:= [lmap] Liaison reply to Broadband Forum liaisons

 

Hi All,
As I mentioned at the mike yesterday there seems to be potential for overla= p between some of the LMAP work (e.g., framework, terminology, etc.) and th= e "Broadband Access Service Attributes and Performance Metrics" (WT-304) wo= rk going on in the Broadband Forum (BBF).   


There are also several u= nanswered liaisons from Broadband Forum to the IETF dating back to last Aug= ust before LMAP was formed.  (See list below) Some of these have detai= led content from the BBF draft document and questions that the LMAP WG should respond to.


=

To support coordinatio= n and cooperation between the BBF and IETF LMAP, to ensure we don't duplica= te work, and to request the latest version of the BBF work, it would be adv= antageous to respond to the BBF liaisons and questions now that the LMAP WG has been formed and there is a charter, scope of work and init= ial work on documents.

 

Thanks,

Dave Sinicrope

Ericsson

IETF Liaison Manager to the Broadband Forum



--_000_CE1E9959714B0davemcdysanoneverizoncom_-- From Michael.K.Bugenhagen@centurylink.com Wed Jul 31 07:49:10 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 97E5811E8167 for ; Wed, 31 Jul 2013 07:49:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[AWL=1.000, BAYES_00=-2.599, GB_I_LETTER=-2, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hOykfBsv2llZ for ; Wed, 31 Jul 2013 07:49:04 -0700 (PDT) Received: from sudnp799.qwest.com (sudnp799.qwest.com [155.70.32.99]) by ietfa.amsl.com (Postfix) with ESMTP id 3B9B821F9E21 for ; Wed, 31 Jul 2013 07:48:59 -0700 (PDT) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by sudnp799.qwest.com (8.14.4/8.14.4) with ESMTP id r6VEmZwx024129 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 08:48:35 -0600 (MDT) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id A64A11E00A4; Wed, 31 Jul 2013 09:48:29 -0500 (CDT) Received: from suomp61i.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id 6B46F1E00AA; Wed, 31 Jul 2013 09:48:29 -0500 (CDT) Received: from suomp61i.qintra.com (localhost [127.0.0.1]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id r6VEmSUf016649; Wed, 31 Jul 2013 09:48:28 -0500 (CDT) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.qintra.com [151.117.206.28]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id r6VEmSiJ016635 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 31 Jul 2013 09:48:28 -0500 (CDT) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.02.0318.001; Wed, 31 Jul 2013 09:48:27 -0500 From: "Bugenhagen, Michael K" To: "'David Sinicrope'" , "Romascanu, Dan (Dan)" Thread-Topic: [lmap] Liaison reply to Broadband Forum liaisons Thread-Index: AQHOjecuKAi+bYLd00KfaQKg2Q21xZl/C+6AgAAD2YD//8VngA== Date: Wed, 31 Jul 2013 14:48:27 +0000 Message-ID: References: <09AE5A45-FCCC-4CEB-A7DC-B9BE37321BA9@gmail.com> <9904FB1B0159DA42B0B887B7FA8119CA1289056B@AZ-FFEXMB04.global.avaya.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.8] Content-Type: multipart/alternative; boundary="_000_A68F3CAC468B2E48BB775ACE2DD99B5E04785155podcwmbxex505ct_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "Cook, Charles" , Alissa Cooper , "lmap@ietf.org" , Eliot Lear , David Sinicrope Subject: Re: [lmap] Liaison reply to Broadband Forum liaisons X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 14:49:10 -0000 --_000_A68F3CAC468B2E48BB775ACE2DD99B5E04785155podcwmbxex505ct_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 RGF2aWQsIERhbiwNCg0KSSByZWFsbHkgd2lzaCBJIGNvdWxkIGhhdmUgbWFkZSB0aGUgbWVldGlu ZyDigJMgSSBwbGFuIHRvIG1ha2UgdGhlIFZhbmNvdXZlciBtZWV0aW5nLi4NCg0KICAgICBUaGFu a3MgZm9yIHJlc3VyZmFjaW5nIHRob3NlLCBtb3N0IG9mIHRoZSBleGlzdGluZyBsaWFpc29ucyBm cm9tIHRoZSBCQkYgaGF2ZSBiZWVuIGluZm9ybWF0aW9uYWwsIGJ1dCB0aGVyZSBpcyBhbiBBc2sg Zm9yIHRoZSBJRVRGIGluIG9uZSBvZiB0aGVtLg0KDQpUaGUgQkJGIHdpbGwgdmVyeSBsaWtlbHkg c2VuZCB0aGVpciBuZXh0IGRyYWZ0IHdoZW4gdGhleSBtZWV0IGluIFNlcHRlbWJlciwgc28gd2Ug c2hvdWxkIGhhdmUgYSBjbGVhbiBjb3B5IGluIFZhbmNvdXZlciAtIHdpdGggc29tZSBzaWduaWZp Y2FudCBmcmFtZXdvcmsgcGFydHMgcmVzb2x2ZWQgLyBhZG9wdGVkIHRleHQgaW4gdGhlbeKApg0K TXkgdGhvdWdodHMgaGVyZSAtICBJZiB0aGUgQkJGIGFscmVhZHkgaGFzIGFkb3B0ZWQgdGV4dCAs IHdlIGFzIGxtYXAgcHJvYmFibHkgIHNob3VsZCBjb25zaWRlciB0aGVtICB3aGVuIHByb2dyZXNz aW5nIHRoZSBsbWFwIHdvcmsgb2YgdGhlIHNhbWUgYXJlYSDigJMgZG9u4oCZdCBrbm93IGlmIGV2 ZXJ5b25lIHdvdWxkIGFncmVlIGJ1dCBpdCBtYWtlcyBzZW5zZSB0byBtZS4NCg0KVGhlbiAgaWYg bG1hcCBoYXMgYSBiZXR0ZXIgc29sdXRpb24gd2Ugc2hvdWxkIHRlbGwgdGhlIEJCRiwgZWl0aGVy IHdheSB0byBmaW5kIHRoZSBiZXN0IHNvbHV0aW9uIHdlIGNhbiBvYnNlcnZlIGJvdGguDQoNCkhl cmUgYXJlIHR3byBhcmVhcyB0aGF0IEkga25vdyBvZiwgd2hlcmUgdGhlIEJCRiBoYXMgc29tZSBh ZG9wdGVkIHRleHQgYWxyZWFkeQ0KDQoNCjEpICAgICAgIElTUCBmcmFtZXdvcmsgZm9yIHByb3Zp ZGluZyBTZXJ2aWNlIGF0dHJpYnV0ZXMgZm9yIHRlc3RpbmcgdmFsaWRhdGlvbiAmIGN1c3RvbWVy IGtub3dsZWRnZSBvZiB0aGUgc2VydmljZQ0KICAgICBUaGUgQkJGIGhhcyBpZGVudGlmaWVkIChz dGFuZGFyZGl6ZWQpIHNvbWUgYnV0IG5vdCBhbGwgIC0gICDigJxTdGFuZGFyZCBTZXJ2aWNlIEF0 dHJpYnV0ZeKAnSBkZWZpbml0aW9ucw0KDQpvICAgVGhlIGZyYW1ld29yayBjb21wb25lbnQgLSAg VGhlIEJCRiBkaWQgcmVzb2x2ZSB0aGF0IGFuIGludGVyZmFjZSBpcyByZXF1aXJlZCBvbiB0aGUg aG9tZSBMQU4gdG8gYWNjZXNzIHRoZSBzZXJ2aWNlIGF0dHJpYnV0ZXMgZm9yIE1B4oCZcyBhbmQg dGhlIGN1c3RvbWVyDQoNCsKnICAgKiogVGhlcmUgaXMgYSBCQkYgbGlhaXNvbiBhc2tpbmcgdGhl IElFVEYgd2hhdCBwcm90b2NvbCB0byB1c2UgZm9yIHRoaXMgKG5pY2UgY29vcmRpbmF0aW9uIHRo ZXJlKeKApi4NCg0KDQoNCjIpICAgICAgSWRlbnRpZmljYXRpb24gb2YgdGhlIFJlZmVyZW5jZSB0 ZXN0IHBvaW50cyBpbiB0aGUgQ2FsbC8gdHJhZmZpYyAgZmxvdyAobm90IGxpYWlzZWQgeWV0IC0g YnV0IG9uZSBhcmVhIHdlIHNob3VsZCB0byB0cnkgdG8gYWxpZ24g4oCTIElNSE8uLg0KDQphLiAg ICAgICBUaGUgQkJGIGlkZW50aWZpZWQgYW5kIG5hbWVkIHRoZSBtYWpvciB0ZXN0IHJlZmVyZW5j ZSBwb2ludHMNClRoZXkgYXJlIG5vdyB3b3JraW5nIHRvIGlkZW50aWZ5IHRoZSBzcGVjaWZpYyBh dHRyaWJ1dGVzIG9mIHdoZXJlIHRoZSB0ZXN0IGNvbm5lY3Rpb24gaXMgYXQgZWFjaCBwb2ludCAo d2hlcmUgaW4gYSBEU0wgLyBDYWJsZSBtb2RlbSwg4oCmLikNCg0KTXkgYXNrIGlmIGFueSBpcyBv ZiB0aGUgZ3JvdXAgLSAgd2hlbiB3ZSBtYWtlIGEgbG1hcCBjb250cmlidXRpb24gaW4gdGhlIGFy ZWEgd2Ugc2hvdWxkIGNvbnRyYXN0IGFuZCBjb21wYXJpbmcgZWFybGllciB2cy4gbGF0ZXIgaXMg cHJvYmFibHkgYSBnb29kIHRoaW5nIHZzLiBhIGJhZCBvbmUuLg0KUGVyc29uYWxseSBJIGxpa2Ug dGhlIEJCRiBhc2tpbmcgdGhlIElFVEYgd2hpY2ggaW50ZXJmYWNlIHRvIHVzZSBmb3IgdGhlIEhv bWUgbmV0d29yayB0byBzaGFyZSBzZXJ2aWNlIGF0dHJpYnV0ZXPigKYNCg0KDQpUaGFua3MsDQpN aWtlDQoNCg0KDQpGcm9tOiBsbWFwLWJvdW5jZXNAaWV0Zi5vcmcgW21haWx0bzpsbWFwLWJvdW5j ZXNAaWV0Zi5vcmddIE9uIEJlaGFsZiBPZiBEYXZpZCBTaW5pY3JvcGUNClNlbnQ6IFdlZG5lc2Rh eSwgSnVseSAzMSwgMjAxMyA3OjQ2IEFNDQpUbzogUm9tYXNjYW51LCBEYW4gKERhbikNCkNjOiBE YXZpZCBTaW5pY3JvcGU7IEFsaXNzYSBDb29wZXI7IEVsaW90IExlYXI7IGxtYXBAaWV0Zi5vcmcN ClN1YmplY3Q6IFJlOiBbbG1hcF0gTGlhaXNvbiByZXBseSB0byBCcm9hZGJhbmQgRm9ydW0gbGlh aXNvbnMNCg0KVGhhbmtzIERhbi4gIEV4YWN0bHkgcmlnaHQuICBTdGF0aW5nIG15IHBvaW50IGFu b3RoZXIgd2F5OiBOb3cgdGhhdCBMTUFQIGlzIGZvcm1lZCwgYW5kIHdlIGhhdmUgYSB3YXkgdG8g cmVzcG9uZCBmb3JtYWxseSwgd2Ugc2hvdWxkIGFkZHJlc3MgdGhlIGJhY2tsb2cgb2YgQkJGIGxp YWlzb25zIGFkZHJlc3NlZCB0byB0aGUgSUVURi4gIChUaGV5IHdlcmUgc2VudCB0byB0aGUgSUVU RiBhbmQgSUVTRyBiZWNhdXNlIHRoZXJlIHdhcyBubyBMTUFQIFdHIHRvIGFkZHJlc3MgdGhlbSB0 by4pDQpUaGFua3MgYWdhaW4sDQpEYXZlDQoNCg0KT24gSnVsIDMxLCAyMDEzLCBhdCAyOjMyIFBN LCAiUm9tYXNjYW51LCBEYW4gKERhbikiIDxkcm9tYXNjYUBhdmF5YS5jb208bWFpbHRvOmRyb21h c2NhQGF2YXlhLmNvbT4+IHdyb3RlOg0KSGksDQoNClRoYW5rcyBEYXZpZCBmb3IgYnJpbmdpbmcg dGhpcyB1cC4NCg0KRmlyc3Qgb2JzZXJ2YXRpb24uIERhdmlkIHdyaXRlczoNCg0KDQrDmCAgVGhl cmUgYXJlIGFsc28gc2V2ZXJhbCB1bmFuc3dlcmVkIGxpYWlzb25zIGZyb20gQnJvYWRiYW5kIEZv cnVtIHRvIHRoZSBJRVRGIGRhdGluZyBiYWNrIHRvIGxhc3QgQXVndXN0IGJlZm9yZSBMTUFQIHdh cyBmb3JtZWQuICAoU2VlIGxpc3QgYmVsb3cpIFNvbWUgb2YgdGhlc2UgaGF2ZSBkZXRhaWxlZCBj b250ZW50IGZyb20gdGhlIEJCRiBkcmFmdCBkb2N1bWVudCBhbmQgcXVlc3Rpb25zIHRoYXQgdGhl IExNQVAgV0cgc2hvdWxkIHJlc3BvbmQgdG8uDQoNCg0KQWN0dWFsbHkgdGhlIExNQVAgV0cgd2Fz IGZvcm1lZCBvbmx5IGEgZmV3IHdlZWtzIGFnbywgYW5kIG5vdCBsYXN0IEF1Z3VzdC4gQmVmb3Jl IHRoZSBmb3JtYXRpb24gb2YgdGhlIFdHIHdlIHdlcmUganVzdCBhIGNvbW11bml0eSBnYXRoZXJl ZCBhcm91bmQgYSBtYWlsIGxpc3QsIGFuZCBub3QgaW4gdGhlIHBvc2l0aW9uIHRvIHJlY2VpdmUg YW5kIHJlc3BvbmQgdG8gbGlhaXNvbnMuIFRoaXMgY2hhbmdlZCBub3cuDQoNCkkgd291bGQgaW52 aXRlIG90aGVyIHBhcnRpY2lwYW50cyBpbiB0aGUgV0cgdG8gcmVhZCB0aGUgbGlhaXNvbiBwb2lu dGVkIHRvIGJ5IERhdmlkIGFuZCBzdWdnZXN0IGhvdyB3ZSBzaG91bGQgcmVzcG9uZC4NCg0KVGhl cmUgYXJlIHR3byBtb3JlIG9yZ2FuaXphdGlvbnMgd2hvIGhhdmUgZXhwcmVzc2VkIGludGVyZXN0 IGluIGNvbW11bmljYXRpbmcgd2l0aCB0aGUgTE1BUCBXRzoNCg0KDQotICAgICAgICAgIFRoZSBJ RUVFIDgwMi4xNi4zIHdobyBzZW5kIHVzIGEgbGlhaXNvbiBsZXR0ZXIgYXZhaWxhYmxlIGF0IGh0 dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvbGlhaXNvbi8xMjQ0Ly4gQ29tbWVudHMgYXJlIHdl bGNvbWUgYWJvdXQgaG93IHRvIHJlc3BvbmQgdG8gdGhpcyBvbmUgdG9vLg0KDQotICAgICAgICAg IFRoZSBOR0NPUiAoTmV4dCBHZW5lcmF0aW9uIENvbnZlcmdlZCBPcGVyYXRpb24gUmVxdWlyZW1l bnRzKSBwcm9qZWN0LiBUaGUgY2hhaXJzIHdpbGwgZGlzY3VzcyB3aXRoIHRoZSByZXByZXNlbnRh dGl2ZXMgZnJvbSBOR0NPUiBpbiB0aGUgY29taW5nIHdlZWtzIGFuZCB0cnkgdG8gdW5kZXJzdGFu ZCB0aGUgbGV2ZWwgb2YgaW5mb3JtYXRpb24gdGhleSB3YW50IHRvIGV4Y2hhbmdlIHdpdGggTE1B UC4NCg0KUmVnYXJkcywNCg0KRGFuDQoNCg0KRnJvbTogbG1hcC1ib3VuY2VzQGlldGYub3JnPG1h aWx0bzpsbWFwLWJvdW5jZXNAaWV0Zi5vcmc+IFttYWlsdG86bG1hcC1ib3VuY2VzQGlldGYub3Jn XSBPbiBCZWhhbGYgT2YgRGF2aWQgU2luaWNyb3BlDQpTZW50OiBXZWRuZXNkYXksIEp1bHkgMzEs IDIwMTMgMzoxMiBQTQ0KVG86IGxtYXBAaWV0Zi5vcmc8bWFpbHRvOmxtYXBAaWV0Zi5vcmc+DQpD YzogQWxpc3NhIENvb3BlcjsgRWxpb3QgTGVhcjsgRGF2aWQgU2luaWNyb3BlDQpTdWJqZWN0OiBb bG1hcF0gTGlhaXNvbiByZXBseSB0byBCcm9hZGJhbmQgRm9ydW0gbGlhaXNvbnMNCg0KSGkgQWxs LA0KQXMgSSBtZW50aW9uZWQgYXQgdGhlIG1pa2UgeWVzdGVyZGF5IHRoZXJlIHNlZW1zIHRvIGJl IHBvdGVudGlhbCBmb3Igb3ZlcmxhcCBiZXR3ZWVuIHNvbWUgb2YgdGhlIExNQVAgd29yayAoZS5n LiwgZnJhbWV3b3JrLCB0ZXJtaW5vbG9neSwgZXRjLikgYW5kIHRoZSAiQnJvYWRiYW5kIEFjY2Vz cyBTZXJ2aWNlIEF0dHJpYnV0ZXMgYW5kIFBlcmZvcm1hbmNlIE1ldHJpY3MiIChXVC0zMDQpIHdv cmsgZ29pbmcgb24gaW4gdGhlIEJyb2FkYmFuZCBGb3J1bSAoQkJGKS4NCg0KDQoNClRoZXJlIGFy ZSBhbHNvIHNldmVyYWwgdW5hbnN3ZXJlZCBsaWFpc29ucyBmcm9tIEJyb2FkYmFuZCBGb3J1bSB0 byB0aGUgSUVURiBkYXRpbmcgYmFjayB0byBsYXN0IEF1Z3VzdCBiZWZvcmUgTE1BUCB3YXMgZm9y bWVkLiAgKFNlZSBsaXN0IGJlbG93KSBTb21lIG9mIHRoZXNlIGhhdmUgZGV0YWlsZWQgY29udGVu dCBmcm9tIHRoZSBCQkYgZHJhZnQgZG9jdW1lbnQgYW5kIHF1ZXN0aW9ucyB0aGF0IHRoZSBMTUFQ IFdHIHNob3VsZCByZXNwb25kIHRvLg0KDQoNCg0KVG8gc3VwcG9ydCBjb29yZGluYXRpb24gYW5k IGNvb3BlcmF0aW9uIGJldHdlZW4gdGhlIEJCRiBhbmQgSUVURiBMTUFQLCB0byBlbnN1cmUgd2Ug ZG9uJ3QgZHVwbGljYXRlIHdvcmssIGFuZCB0byByZXF1ZXN0IHRoZSBsYXRlc3QgdmVyc2lvbiBv ZiB0aGUgQkJGIHdvcmssIGl0IHdvdWxkIGJlIGFkdmFudGFnZW91cyB0byByZXNwb25kIHRvIHRo ZSBCQkYgbGlhaXNvbnMgYW5kIHF1ZXN0aW9ucyBub3cgdGhhdCB0aGUgTE1BUCBXRyBoYXMgYmVl biBmb3JtZWQgYW5kIHRoZXJlIGlzIGEgY2hhcnRlciwgc2NvcGUgb2Ygd29yayBhbmQgaW5pdGlh bCB3b3JrIG9uIGRvY3VtZW50cy4NCg0KVGhhbmtzLA0KRGF2ZSBTaW5pY3JvcGUNCkVyaWNzc29u DQpJRVRGIExpYWlzb24gTWFuYWdlciB0byB0aGUgQnJvYWRiYW5kIEZvcnVtDQoNCg0KDQpNYXIg MjAxMyAtIGh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvbGlhaXNvbi8xMjQzLw0KRGVjIDIw MTIgLSBodHRwczovL2RhdGF0cmFja2VyLmlldGYub3JnL2xpYWlzb24vMTIyMS8NClNlcCAyMDEy IC0gaHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9saWFpc29uLzExODUvDQpBdWcgMjAxMiAt IGh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvbGlhaXNvbi8xMTc5Lw0KDQoNCg0KDQoNCg== --_000_A68F3CAC468B2E48BB775ACE2DD99B5E04785155podcwmbxex505ct_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTIgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 V2luZ2RpbmdzOw0KCXBhbm9zZS0xOjUgMCAwIDAgMCAwIDAgMCAwIDA7fQ0KQGZvbnQtZmFjZQ0K CXtmb250LWZhbWlseTpXaW5nZGluZ3M7DQoJcGFub3NlLTE6NSAwIDAgMCAwIDAgMCAwIDAgMDt9 DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIg MiAyIDQgMyAyIDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpUYWhvbWE7DQoJcGFub3Nl LTE6MiAxMSA2IDQgMyA1IDQgNCAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNv Tm9ybWFsLCBsaS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBpbjsNCgltYXJn aW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGlt ZXMgTmV3IFJvbWFuIiwic2VyaWYiO30NCmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsNCgl7bXNv LXN0eWxlLXByaW9yaXR5Ojk5Ow0KCWNvbG9yOmJsdWU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVy bGluZTt9DQphOnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJsaW5rRm9sbG93ZWQNCgl7bXNvLXN0eWxl LXByaW9yaXR5Ojk5Ow0KCWNvbG9yOnB1cnBsZTsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5l O30NCnAuTXNvQWNldGF0ZSwgbGkuTXNvQWNldGF0ZSwgZGl2Lk1zb0FjZXRhdGUNCgl7bXNvLXN0 eWxlLXByaW9yaXR5Ojk5Ow0KCW1zby1zdHlsZS1saW5rOiJCYWxsb29uIFRleHQgQ2hhciI7DQoJ bWFyZ2luOjBpbjsNCgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjguMHB0Ow0K CWZvbnQtZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlmIjt9DQpwLk1zb0xpc3RQYXJhZ3JhcGgs IGxpLk1zb0xpc3RQYXJhZ3JhcGgsIGRpdi5Nc29MaXN0UGFyYWdyYXBoDQoJe21zby1zdHlsZS1w cmlvcml0eTozNDsNCgltYXJnaW4tdG9wOjBpbjsNCgltYXJnaW4tcmlnaHQ6MGluOw0KCW1hcmdp bi1ib3R0b206MGluOw0KCW1hcmdpbi1sZWZ0Oi41aW47DQoJbWFyZ2luLWJvdHRvbTouMDAwMXB0 Ow0KCWZvbnQtc2l6ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsInNl cmlmIjt9DQpzcGFuLkVtYWlsU3R5bGUxOA0KCXttc28tc3R5bGUtdHlwZTpwZXJzb25hbDsNCglm b250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiOw0KCWNvbG9yOiMxRjQ5N0Q7fQ0Kc3Bh bi5CYWxsb29uVGV4dENoYXINCgl7bXNvLXN0eWxlLW5hbWU6IkJhbGxvb24gVGV4dCBDaGFyIjsN Cgltc28tc3R5bGUtcHJpb3JpdHk6OTk7DQoJbXNvLXN0eWxlLWxpbms6IkJhbGxvb24gVGV4dCI7 DQoJZm9udC1mYW1pbHk6IlRhaG9tYSIsInNhbnMtc2VyaWYiO30NCnNwYW4uRW1haWxTdHlsZTIx DQoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsLXJlcGx5Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJp Iiwic2Fucy1zZXJpZiI7DQoJY29sb3I6IzFGNDk3RDt9DQouTXNvQ2hwRGVmYXVsdA0KCXttc28t c3R5bGUtdHlwZTpleHBvcnQtb25seTsNCglmb250LXNpemU6MTAuMHB0O30NCkBwYWdlIFdvcmRT ZWN0aW9uMQ0KCXtzaXplOjguNWluIDExLjBpbjsNCgltYXJnaW46MS4waW4gMS4yNWluIDEuMGlu IDEuMjVpbjt9DQpkaXYuV29yZFNlY3Rpb24xDQoJe3BhZ2U6V29yZFNlY3Rpb24xO30NCi8qIExp c3QgRGVmaW5pdGlvbnMgKi8NCkBsaXN0IGwwDQoJe21zby1saXN0LWlkOjk1Nzc1NjQzOTsNCglt c28tbGlzdC10eXBlOmh5YnJpZDsNCgltc28tbGlzdC10ZW1wbGF0ZS1pZHM6MTYzNDIxODQ4NCAt NTMxMzM1ODIwIDY3Njk4NjkxIDY3Njk4NjkzIDY3Njk4Njg5IDY3Njk4NjkxIDY3Njk4NjkzIDY3 Njk4Njg5IDY3Njk4NjkxIDY3Njk4NjkzO30NCkBsaXN0IGwwOmxldmVsMQ0KCXttc28tbGV2ZWwt bnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ6LTsNCgltc28tbGV2ZWwtdGFi LXN0b3A6bm9uZTsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJbWFyZ2luLWxl ZnQ6Ljc1aW47DQoJdGV4dC1pbmRlbnQ6LS4yNWluOw0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIiwi c2Fucy1zZXJpZiI7DQoJbXNvLWZhcmVhc3QtZm9udC1mYW1pbHk6Q2FsaWJyaTsNCgltc28tYmlk aS1mb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIjt9DQpAbGlzdCBsMDpsZXZlbDINCgl7bXNv LWxldmVsLW51bWJlci1mb3JtYXQ6YnVsbGV0Ow0KCW1zby1sZXZlbC10ZXh0Om87DQoJbXNvLWxl dmVsLXRhYi1zdG9wOm5vbmU7DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjpsZWZ0Ow0KCW1h cmdpbi1sZWZ0OjEuMjVpbjsNCgl0ZXh0LWluZGVudDotLjI1aW47DQoJZm9udC1mYW1pbHk6IkNv dXJpZXIgTmV3Ijt9DQpAbGlzdCBsMDpsZXZlbDMNCgl7bXNvLWxldmVsLW51bWJlci1mb3JtYXQ6 YnVsbGV0Ow0KCW1zby1sZXZlbC10ZXh0Ou+CpzsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsN Cgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJbWFyZ2luLWxlZnQ6MS43NWluOw0K CXRleHQtaW5kZW50Oi0uMjVpbjsNCglmb250LWZhbWlseTpXaW5nZGluZ3M7fQ0KQGxpc3QgbDEN Cgl7bXNvLWxpc3QtaWQ6MTE1NzMwODU4NzsNCgltc28tbGlzdC10eXBlOmh5YnJpZDsNCgltc28t bGlzdC10ZW1wbGF0ZS1pZHM6Njc1OTM4MzU0IDkzMjYzOTI4MiA2NzY5ODY5MSA2NzY5ODY5MyA2 NzY5ODY4OSA2NzY5ODY5MSA2NzY5ODY5MyA2NzY5ODY4OSA2NzY5ODY5MSA2NzY5ODY5Mzt9DQpA bGlzdCBsMTpsZXZlbDENCgl7bXNvLWxldmVsLW51bWJlci1mb3JtYXQ6YnVsbGV0Ow0KCW1zby1s ZXZlbC10ZXh0Oi07DQoJbXNvLWxldmVsLXRhYi1zdG9wOm5vbmU7DQoJbXNvLWxldmVsLW51bWJl ci1wb3NpdGlvbjpsZWZ0Ow0KCW1hcmdpbi1sZWZ0OjEuMjVpbjsNCgl0ZXh0LWluZGVudDotLjI1 aW47DQoJZm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjsNCgltc28tZmFyZWFzdC1m b250LWZhbWlseTpDYWxpYnJpOw0KCW1zby1iaWRpLWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9t YW4iO30NCkBsaXN0IGwxOmxldmVsMg0KCXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7 DQoJbXNvLWxldmVsLXRleHQ6bzsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2 ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJbWFyZ2luLWxlZnQ6MS43NWluOw0KCXRleHQtaW5k ZW50Oi0uMjVpbjsNCglmb250LWZhbWlseToiQ291cmllciBOZXciO30NCkBsaXN0IGwyDQoJe21z by1saXN0LWlkOjEyNDU0NTQ5NDk7DQoJbXNvLWxpc3QtdHlwZTpoeWJyaWQ7DQoJbXNvLWxpc3Qt dGVtcGxhdGUtaWRzOjQ1MTU5NDUxMCAxNTU1MDU4NzE4IDY3Njk4NjkxIDY3Njk4NjkzIDY3Njk4 Njg5IDY3Njk4NjkxIDY3Njk4NjkzIDY3Njk4Njg5IDY3Njk4NjkxIDY3Njk4NjkzO30NCkBsaXN0 IGwyOmxldmVsMQ0KCXttc28tbGV2ZWwtc3RhcnQtYXQ6MjsNCgltc28tbGV2ZWwtbnVtYmVyLWZv cm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ6LTsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9u ZTsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWlu Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7DQoJbXNvLWZhcmVhc3QtZm9u dC1mYW1pbHk6Q2FsaWJyaTsNCgltc28tYmlkaS1mb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFu Ijt9DQpAbGlzdCBsMw0KCXttc28tbGlzdC1pZDoxNTE3NDUyOTYxOw0KCW1zby1saXN0LXR5cGU6 aHlicmlkOw0KCW1zby1saXN0LXRlbXBsYXRlLWlkczoxNzk5ODA5MDI4IDI2ODQ0ODU0MiA2NzY5 ODY5MSA2NzY5ODY5MyA2NzY5ODY4OSA2NzY5ODY5MSA2NzY5ODY5MyA2NzY5ODY4OSA2NzY5ODY5 MSA2NzY5ODY5Mzt9DQpAbGlzdCBsMzpsZXZlbDENCgl7bXNvLWxldmVsLXN0YXJ0LWF0OjA7DQoJ bXNvLWxldmVsLW51bWJlci1mb3JtYXQ6YnVsbGV0Ow0KCW1zby1sZXZlbC10ZXh0Oi07DQoJbXNv LWxldmVsLXRhYi1zdG9wOm5vbmU7DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjpsZWZ0Ow0K CXRleHQtaW5kZW50Oi0uMjVpbjsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYi Ow0KCW1zby1mYXJlYXN0LWZvbnQtZmFtaWx5OkNhbGlicmk7DQoJbXNvLWJpZGktZm9udC1mYW1p bHk6QXJpYWw7fQ0KQGxpc3QgbDM6bGV2ZWwyDQoJe21zby1sZXZlbC10YWItc3RvcDoxLjBpbjsN Cgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluO30N CkBsaXN0IGwzOmxldmVsMw0KCXttc28tbGV2ZWwtdGFiLXN0b3A6MS41aW47DQoJbXNvLWxldmVs LW51bWJlci1wb3NpdGlvbjpsZWZ0Ow0KCXRleHQtaW5kZW50Oi0uMjVpbjt9DQpAbGlzdCBsMzps ZXZlbDQNCgl7bXNvLWxldmVsLXRhYi1zdG9wOjIuMGluOw0KCW1zby1sZXZlbC1udW1iZXItcG9z aXRpb246bGVmdDsNCgl0ZXh0LWluZGVudDotLjI1aW47fQ0KQGxpc3QgbDM6bGV2ZWw1DQoJe21z by1sZXZlbC10YWItc3RvcDoyLjVpbjsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7 DQoJdGV4dC1pbmRlbnQ6LS4yNWluO30NCkBsaXN0IGwzOmxldmVsNg0KCXttc28tbGV2ZWwtdGFi LXN0b3A6My4waW47DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjpsZWZ0Ow0KCXRleHQtaW5k ZW50Oi0uMjVpbjt9DQpAbGlzdCBsMzpsZXZlbDcNCgl7bXNvLWxldmVsLXRhYi1zdG9wOjMuNWlu Ow0KCW1zby1sZXZlbC1udW1iZXItcG9zaXRpb246bGVmdDsNCgl0ZXh0LWluZGVudDotLjI1aW47 fQ0KQGxpc3QgbDM6bGV2ZWw4DQoJe21zby1sZXZlbC10YWItc3RvcDo0LjBpbjsNCgltc28tbGV2 ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluO30NCkBsaXN0IGwz OmxldmVsOQ0KCXttc28tbGV2ZWwtdGFiLXN0b3A6NC41aW47DQoJbXNvLWxldmVsLW51bWJlci1w b3NpdGlvbjpsZWZ0Ow0KCXRleHQtaW5kZW50Oi0uMjVpbjt9DQpAbGlzdCBsNA0KCXttc28tbGlz dC1pZDoxNTM0MjY0MzA1Ow0KCW1zby1saXN0LXR5cGU6aHlicmlkOw0KCW1zby1saXN0LXRlbXBs YXRlLWlkczo2MTMwNDI4NTQgNjc2OTg3MDUgNjc2OTg3MTMgNjc2OTg3MTUgNjc2OTg3MDMgNjc2 OTg3MTMgNjc2OTg3MTUgNjc2OTg3MDMgNjc2OTg3MTMgNjc2OTg3MTU7fQ0KQGxpc3QgbDQ6bGV2 ZWwxDQoJe21zby1sZXZlbC10ZXh0OiIlMVwpIjsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsN Cgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluO30N CkBsaXN0IGw0OmxldmVsMg0KCXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDphbHBoYS1sb3dlcjsN Cgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxl ZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluO30NCkBsaXN0IGw1DQoJe21zby1saXN0LWlkOjE2OTY2 OTAyMzk7DQoJbXNvLWxpc3QtdHlwZTpoeWJyaWQ7DQoJbXNvLWxpc3QtdGVtcGxhdGUtaWRzOi03 MTIxMDAyOTAgNjc2OTg3MDUgNjc2OTg3MTMgNjc2OTg3MTUgNjc2OTg3MDMgNjc2OTg3MTMgNjc2 OTg3MTUgNjc2OTg3MDMgNjc2OTg3MTMgNjc2OTg3MTU7fQ0KQGxpc3QgbDU6bGV2ZWwxDQoJe21z by1sZXZlbC10ZXh0OiIlMVwpIjsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2 ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluO30NCkBsaXN0IGw1 OmxldmVsMg0KCXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDphbHBoYS1sb3dlcjsNCgltc28tbGV2 ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4 dC1pbmRlbnQ6LS4yNWluO30NCkBsaXN0IGw1OmxldmVsMw0KCXttc28tbGV2ZWwtbnVtYmVyLWZv cm1hdDpyb21hbi1sb3dlcjsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2ZWwt bnVtYmVyLXBvc2l0aW9uOnJpZ2h0Ow0KCXRleHQtaW5kZW50Oi05LjBwdDt9DQpAbGlzdCBsNg0K CXttc28tbGlzdC1pZDoxNzY2ODc0ODE5Ow0KCW1zby1saXN0LXR5cGU6aHlicmlkOw0KCW1zby1s aXN0LXRlbXBsYXRlLWlkczo0ODg5MDkwNiAxNzc2MjA5MTMyIDY3Njk4NjkxIDY3Njk4NjkzIDY3 Njk4Njg5IDY3Njk4NjkxIDY3Njk4NjkzIDY3Njk4Njg5IDY3Njk4NjkxIDY3Njk4NjkzO30NCkBs aXN0IGw2OmxldmVsMQ0KCXttc28tbGV2ZWwtc3RhcnQtYXQ6MDsNCgltc28tbGV2ZWwtbnVtYmVy LWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ674OYOw0KCW1zby1sZXZlbC10YWItc3Rv cDpub25lOw0KCW1zby1sZXZlbC1udW1iZXItcG9zaXRpb246bGVmdDsNCgl0ZXh0LWluZGVudDot LjI1aW47DQoJZm9udC1mYW1pbHk6V2luZ2RpbmdzOw0KCW1zby1mYXJlYXN0LWZvbnQtZmFtaWx5 OkNhbGlicmk7DQoJbXNvLWJpZGktZm9udC1mYW1pbHk6QXJpYWw7fQ0KQGxpc3QgbDY6bGV2ZWwy DQoJe21zby1sZXZlbC10YWItc3RvcDoxLjBpbjsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9u OmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluO30NCkBsaXN0IGw2OmxldmVsMw0KCXttc28tbGV2 ZWwtdGFiLXN0b3A6MS41aW47DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjpsZWZ0Ow0KCXRl eHQtaW5kZW50Oi0uMjVpbjt9DQpAbGlzdCBsNjpsZXZlbDQNCgl7bXNvLWxldmVsLXRhYi1zdG9w OjIuMGluOw0KCW1zby1sZXZlbC1udW1iZXItcG9zaXRpb246bGVmdDsNCgl0ZXh0LWluZGVudDot LjI1aW47fQ0KQGxpc3QgbDY6bGV2ZWw1DQoJe21zby1sZXZlbC10YWItc3RvcDoyLjVpbjsNCglt c28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluO30NCkBs aXN0IGw2OmxldmVsNg0KCXttc28tbGV2ZWwtdGFiLXN0b3A6My4waW47DQoJbXNvLWxldmVsLW51 bWJlci1wb3NpdGlvbjpsZWZ0Ow0KCXRleHQtaW5kZW50Oi0uMjVpbjt9DQpAbGlzdCBsNjpsZXZl bDcNCgl7bXNvLWxldmVsLXRhYi1zdG9wOjMuNWluOw0KCW1zby1sZXZlbC1udW1iZXItcG9zaXRp b246bGVmdDsNCgl0ZXh0LWluZGVudDotLjI1aW47fQ0KQGxpc3QgbDY6bGV2ZWw4DQoJe21zby1s ZXZlbC10YWItc3RvcDo0LjBpbjsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJ dGV4dC1pbmRlbnQ6LS4yNWluO30NCkBsaXN0IGw2OmxldmVsOQ0KCXttc28tbGV2ZWwtdGFiLXN0 b3A6NC41aW47DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjpsZWZ0Ow0KCXRleHQtaW5kZW50 Oi0uMjVpbjt9DQpvbA0KCXttYXJnaW4tYm90dG9tOjBpbjt9DQp1bA0KCXttYXJnaW4tYm90dG9t OjBpbjt9DQotLT48L3N0eWxlPjwhLS1baWYgZ3RlIG1zbyA5XT48eG1sPg0KPG86c2hhcGVkZWZh dWx0cyB2OmV4dD0iZWRpdCIgc3BpZG1heD0iMTAyNiIgLz4NCjwveG1sPjwhW2VuZGlmXS0tPjwh LS1baWYgZ3RlIG1zbyA5XT48eG1sPg0KPG86c2hhcGVsYXlvdXQgdjpleHQ9ImVkaXQiPg0KPG86 aWRtYXAgdjpleHQ9ImVkaXQiIGRhdGE9IjEiIC8+DQo8L286c2hhcGVsYXlvdXQ+PC94bWw+PCFb ZW5kaWZdLS0+DQo8L2hlYWQ+DQo8Ym9keSBsYW5nPSJFTi1VUyIgbGluaz0iYmx1ZSIgdmxpbms9 InB1cnBsZSI+DQo8ZGl2IGNsYXNzPSJXb3JkU2VjdGlvbjEiPg0KPHAgY2xhc3M9Ik1zb05vcm1h bCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJy aSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPkRhdmlkLCBEYW4s PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9 ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtz YW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwv cD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2Zv bnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xv cjojMUY0OTdEIj5JIHJlYWxseSB3aXNoIEkgY291bGQgaGF2ZSBtYWRlIHRoZSBtZWV0aW5nIOKA kyBJIHBsYW4gdG8gbWFrZSB0aGUgVmFuY291dmVyIG1lZXRpbmcuLg0KPG86cD48L286cD48L3Nw YW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4w cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7 O2NvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90 O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj4mbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsgVGhhbmtzIGZvciByZXN1cmZhY2luZyB0aG9zZSwgbW9zdCBv ZiB0aGUgZXhpc3RpbmcgbGlhaXNvbnMgZnJvbSB0aGUgQkJGIGhhdmUgYmVlbiBpbmZvcm1hdGlv bmFsLCBidXQgdGhlcmUgaXMgYW4gQXNrIGZvciB0aGUgSUVURiBpbiBvbmUgb2YgdGhlbS48bzpw PjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9u dC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMt c2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0K PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1m YW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMx RjQ5N0QiPlRoZSBCQkYgd2lsbCB2ZXJ5IGxpa2VseSBzZW5kIHRoZWlyIG5leHQgZHJhZnQgd2hl biB0aGV5IG1lZXQgaW4gU2VwdGVtYmVyLCBzbyB3ZSBzaG91bGQgaGF2ZSBhIGNsZWFuIGNvcHkg aW4gVmFuY291dmVyIC0gd2l0aCBzb21lIHNpZ25pZmljYW50IGZyYW1ld29yayBwYXJ0cw0KIHJl c29sdmVkIC8gYWRvcHRlZCB0ZXh0IGluIHRoZW3igKYgPG86cD48L286cD48L3NwYW4+PC9wPg0K PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1m YW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMx RjQ5N0QiPk15IHRob3VnaHRzIGhlcmUgLSZuYnNwOyBJZiB0aGUgQkJGIGFscmVhZHkgaGFzDQo8 Yj48dT5hZG9wdGVkIHRleHQ8L3U+PC9iPiAsIHdlIGFzIGxtYXAgcHJvYmFibHkgJm5ic3A7c2hv dWxkIGNvbnNpZGVyIHRoZW0gJm5ic3A7d2hlbiBwcm9ncmVzc2luZyB0aGUgbG1hcCB3b3JrIG9m IHRoZSBzYW1lIGFyZWEg4oCTIGRvbuKAmXQga25vdyBpZiBldmVyeW9uZSB3b3VsZCBhZ3JlZSBi dXQgaXQgbWFrZXMgc2Vuc2UgdG8gbWUuPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9 Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1 b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxv OnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0 eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1 b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj5UaGVuICZuYnNwO2lmIGxtYXAgaGFz IGEgYmV0dGVyIHNvbHV0aW9uIHdlIHNob3VsZCB0ZWxsIHRoZSBCQkYsIGVpdGhlciB3YXkgdG8g ZmluZCB0aGUgYmVzdCBzb2x1dGlvbiB3ZSBjYW4gb2JzZXJ2ZSBib3RoLg0KPG86cD48L286cD48 L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox MS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1 b3Q7O2NvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNz PSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZx dW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj5I ZXJlIGFyZSB0d28gYXJlYXMgdGhhdCBJIGtub3cgb2YsIHdoZXJlIHRoZSBCQkYgaGFzIHNvbWUg YWRvcHRlZCB0ZXh0IGFscmVhZHkNCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90 O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48bzpw PiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTGlzdFBhcmFncmFwaCIgc3R5 bGU9InRleHQtaW5kZW50Oi0uMjVpbjttc28tbGlzdDpsNSBsZXZlbDEgbGZvOCI+PCFbaWYgIXN1 cHBvcnRMaXN0c10+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1 b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxz cGFuIHN0eWxlPSJtc28tbGlzdDpJZ25vcmUiPjEpPHNwYW4gc3R5bGU9ImZvbnQ6Ny4wcHQgJnF1 b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7Ij4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsN Cjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwhW2VuZGlmXT48dT48c3BhbiBzdHlsZT0iZm9udC1zaXpl OjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYm cXVvdDs7Y29sb3I6IzFGNDk3RCI+Jm5ic3A7SVNQIGZyYW1ld29yayBmb3IgcHJvdmlkaW5nIFNl cnZpY2UgYXR0cmlidXRlcyBmb3IgdGVzdGluZyB2YWxpZGF0aW9uICZhbXA7IGN1c3RvbWVyIGtu b3dsZWRnZSBvZiB0aGUgc2VydmljZTxvOnA+PC9vOnA+PC9zcGFuPjwvdT48L3A+DQo8cCBjbGFz cz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTom cXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+ Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IFRoZSBCQkYgaGFzIGlkZW50aWZpZWQgKHN0YW5kYXJk aXplZCkgc29tZSBidXQgbm90IGFsbCAmbmJzcDstICZuYnNwOyZuYnNwO+KAnFN0YW5kYXJkIFNl cnZpY2UgQXR0cmlidXRl4oCdIGRlZmluaXRpb25zICZuYnNwOzxvOnA+PC9vOnA+PC9zcGFuPjwv cD4NCjxwIGNsYXNzPSJNc29MaXN0UGFyYWdyYXBoIiBzdHlsZT0ibWFyZ2luLWxlZnQ6MS4yNWlu O3RleHQtaW5kZW50Oi0uMjVpbjttc28tbGlzdDpsMCBsZXZlbDIgbGZvNyI+DQo8IVtpZiAhc3Vw cG9ydExpc3RzXT48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVv dDtDb3VyaWVyIE5ldyZxdW90Oztjb2xvcjojMUY0OTdEIj48c3BhbiBzdHlsZT0ibXNvLWxpc3Q6 SWdub3JlIj5vPHNwYW4gc3R5bGU9ImZvbnQ6Ny4wcHQgJnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1 b3Q7Ij4mbmJzcDsmbmJzcDsNCjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwhW2VuZGlmXT48c3BhbiBz dHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZx dW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+VGhlIGZyYW1ld29yayBjb21wb25l bnQgLSZuYnNwOyBUaGUgQkJGIGRpZCByZXNvbHZlIHRoYXQgYW4gaW50ZXJmYWNlIGlzIHJlcXVp cmVkIG9uIHRoZSBob21lIExBTiB0byBhY2Nlc3MgdGhlIHNlcnZpY2UgYXR0cmlidXRlcyBmb3Ig TUHigJlzIGFuZCB0aGUgY3VzdG9tZXI8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0i TXNvTGlzdFBhcmFncmFwaCIgc3R5bGU9Im1hcmdpbi1sZWZ0OjEuNzVpbjt0ZXh0LWluZGVudDot LjI1aW47bXNvLWxpc3Q6bDAgbGV2ZWwzIGxmbzciPg0KPCFbaWYgIXN1cHBvcnRMaXN0c10+PHNw YW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6V2luZ2RpbmdzO2NvbG9yOiMx RjQ5N0QiPjxzcGFuIHN0eWxlPSJtc28tbGlzdDpJZ25vcmUiPsKnPHNwYW4gc3R5bGU9ImZvbnQ6 Ny4wcHQgJnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7Ij4mbmJzcDsNCjwvc3Bhbj48L3NwYW4+ PC9zcGFuPjwhW2VuZGlmXT48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWls eTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3 RCI+Jm5ic3A7PHU+KiogVGhlcmUgaXMgYSBCQkYgbGlhaXNvbiBhc2tpbmcgdGhlIElFVEYgd2hh dCBwcm90b2NvbCB0byB1c2UgZm9yIHRoaXMgKG5pY2UgY29vcmRpbmF0aW9uIHRoZXJlKeKApi48 L3U+PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb0xpc3RQYXJhZ3JhcGgiIHN0 eWxlPSJtYXJnaW4tbGVmdDoxLjI1aW4iPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2Zv bnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xv cjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTGlz dFBhcmFncmFwaCIgc3R5bGU9InRleHQtaW5kZW50Oi0uMjVpbjttc28tbGlzdDpsNSBsZXZlbDEg bGZvOCI+PCFbaWYgIXN1cHBvcnRMaXN0c10+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7 Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2Nv bG9yOiMxRjQ5N0QiPjxzcGFuIHN0eWxlPSJtc28tbGlzdDpJZ25vcmUiPjIpPHNwYW4gc3R5bGU9 ImZvbnQ6Ny4wcHQgJnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7Ij4mbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsNCjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwhW2VuZGlmXT48c3BhbiBzdHls ZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90 O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+SWRlbnRpZmljYXRpb24gb2YgdGhlIFJl ZmVyZW5jZSB0ZXN0IHBvaW50cyBpbiB0aGUgQ2FsbC8gdHJhZmZpYyAmbmJzcDtmbG93IChub3Qg bGlhaXNlZCB5ZXQgLSBidXQgb25lIGFyZWEgd2Ugc2hvdWxkIHRvIHRyeSB0byBhbGlnbiDigJMg SU1ITy4uPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb0xpc3RQYXJhZ3JhcGgi IHN0eWxlPSJtYXJnaW4tbGVmdDoxLjBpbjt0ZXh0LWluZGVudDotLjI1aW47bXNvLWxpc3Q6bDUg bGV2ZWwyIGxmbzgiPg0KPCFbaWYgIXN1cHBvcnRMaXN0c10+PHNwYW4gc3R5bGU9ImZvbnQtc2l6 ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlm JnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxzcGFuIHN0eWxlPSJtc28tbGlzdDpJZ25vcmUiPmEuPHNw YW4gc3R5bGU9ImZvbnQ6Ny4wcHQgJnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7Ij4mbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsNCjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwhW2Vu ZGlmXT48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxp YnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+VGhlIEJCRiBp ZGVudGlmaWVkIGFuZCBuYW1lZCB0aGUgbWFqb3IgdGVzdCByZWZlcmVuY2UgcG9pbnRzPG86cD48 L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1hcmdpbi1sZWZ0 OjEuMGluO3RleHQtaW5kZW50Oi41aW4iPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2Zv bnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xv cjojMUY0OTdEIj5UaGV5IGFyZSBub3cgd29ya2luZyB0byBpZGVudGlmeSB0aGUgc3BlY2lmaWMg YXR0cmlidXRlcyBvZiB3aGVyZSB0aGUgdGVzdCBjb25uZWN0aW9uIGlzIGF0IGVhY2ggcG9pbnQg KHdoZXJlIGluDQogYSBEU0wgLyBDYWJsZSBtb2RlbSwg4oCmLik8bzpwPjwvbzpwPjwvc3Bhbj48 L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtm b250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29s b3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2Fs aWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPk15IGFzayBp ZiBhbnkgaXMgb2YgdGhlIGdyb3VwIC0mbmJzcDsgd2hlbiB3ZSBtYWtlIGEgbG1hcCBjb250cmli dXRpb24gaW4gdGhlIGFyZWEgd2Ugc2hvdWxkIGNvbnRyYXN0IGFuZCBjb21wYXJpbmcgZWFybGll ciB2cy4gbGF0ZXIgaXMgcHJvYmFibHkgYSBnb29kIHRoaW5nIHZzLg0KIGEgYmFkIG9uZS4uIDxv OnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJm b250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fu cy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj5QZXJzb25hbGx5IEkgbGlrZSB0aGUgQkJGIGFz a2luZyB0aGUgSUVURiB3aGljaCBpbnRlcmZhY2UgdG8gdXNlIGZvciB0aGUgSG9tZSBuZXR3b3Jr IHRvIHNoYXJlIHNlcnZpY2UgYXR0cmlidXRlc+KApjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFt aWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0 OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48 c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1 b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286 cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6 ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlm JnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPlRoYW5rcyw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBj bGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWls eTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3 RCI+TWlrZTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFu IHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDss JnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwv c3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEx LjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVv dDs7Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9 Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1 b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPjxv OnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxkaXY+DQo8ZGl2IHN0eWxlPSJib3JkZXI6bm9u ZTtib3JkZXItdG9wOnNvbGlkICNCNUM0REYgMS4wcHQ7cGFkZGluZzozLjBwdCAwaW4gMGluIDBp biI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBw dDtmb250LWZhbWlseTomcXVvdDtUYWhvbWEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90OyI+ RnJvbTo8L3NwYW4+PC9iPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5 OiZxdW90O1RhaG9tYSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7Ij4gbG1hcC1ib3VuY2Vz QGlldGYub3JnIFttYWlsdG86bG1hcC1ib3VuY2VzQGlldGYub3JnXQ0KPGI+T24gQmVoYWxmIE9m IDwvYj5EYXZpZCBTaW5pY3JvcGU8YnI+DQo8Yj5TZW50OjwvYj4gV2VkbmVzZGF5LCBKdWx5IDMx LCAyMDEzIDc6NDYgQU08YnI+DQo8Yj5Ubzo8L2I+IFJvbWFzY2FudSwgRGFuIChEYW4pPGJyPg0K PGI+Q2M6PC9iPiBEYXZpZCBTaW5pY3JvcGU7IEFsaXNzYSBDb29wZXI7IEVsaW90IExlYXI7IGxt YXBAaWV0Zi5vcmc8YnI+DQo8Yj5TdWJqZWN0OjwvYj4gUmU6IFtsbWFwXSBMaWFpc29uIHJlcGx5 IHRvIEJyb2FkYmFuZCBGb3J1bSBsaWFpc29uczxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2 Pg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxk aXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+VGhhbmtzIERhbi4gJm5ic3A7RXhhY3Rs eSByaWdodC4gJm5ic3A7U3RhdGluZyBteSBwb2ludCBhbm90aGVyIHdheTogTm93IHRoYXQgTE1B UCBpcyBmb3JtZWQsIGFuZCB3ZSBoYXZlIGEgd2F5IHRvIHJlc3BvbmQgZm9ybWFsbHksIHdlIHNo b3VsZCBhZGRyZXNzIHRoZSBiYWNrbG9nIG9mIEJCRiBsaWFpc29ucyBhZGRyZXNzZWQgdG8gdGhl IElFVEYuICZuYnNwOyhUaGV5IHdlcmUgc2VudCB0byB0aGUgSUVURiBhbmQgSUVTRyBiZWNhdXNl DQogdGhlcmUgd2FzIG5vIExNQVAgV0cgdG8gYWRkcmVzcyB0aGVtIHRvLik8bzpwPjwvbzpwPjwv cD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPlRoYW5rcyBhZ2Fpbiw8bzpw PjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJt YXJnaW4tYm90dG9tOjEyLjBwdCI+RGF2ZTxicj4NCjxicj4NCjxvOnA+PC9vOnA+PC9wPg0KPC9k aXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1hcmdpbi1ib3R0b206MTIu MHB0Ij48YnI+DQpPbiBKdWwgMzEsIDIwMTMsIGF0IDI6MzIgUE0sICZxdW90O1JvbWFzY2FudSwg RGFuIChEYW4pJnF1b3Q7ICZsdDs8YSBocmVmPSJtYWlsdG86ZHJvbWFzY2FAYXZheWEuY29tIj5k cm9tYXNjYUBhdmF5YS5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0K PGJsb2NrcXVvdGUgc3R5bGU9Im1hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLWJvdHRvbTo1LjBwdCI+ DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4w cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztj b2xvcjojMUY0OTdEIj5IaSw8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9y bWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlh bCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPiZuYnNwOzwvc3Bh bj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250 LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMtc2Vy aWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+VGhhbmtzIERhdmlkIGZvciBicmluZ2luZyB0aGlzIHVw Lg0KPC9zcGFuPjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5 bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssJnF1b3Q7 c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286cD48 L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtm b250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9y OiMxRjQ5N0QiPkZpcnN0IG9ic2VydmF0aW9uLiBEYXZpZCB3cml0ZXM6DQo8L3NwYW4+PG86cD48 L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEw LjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7 O2NvbG9yOiMxRjQ5N0QiPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJN c29MaXN0UGFyYWdyYXBoIiBzdHlsZT0idGV4dC1pbmRlbnQ6LS4yNWluO21zby1saXN0Omw2IGxl dmVsMSBsZm8yIj48IVtpZiAhc3VwcG9ydExpc3RzXT48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6 V2luZ2RpbmdzIj48c3BhbiBzdHlsZT0ibXNvLWxpc3Q6SWdub3JlIj7DmDxzcGFuIHN0eWxlPSJm b250OjcuMHB0ICZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyI+Jm5ic3A7DQo8L3NwYW4+PC9z cGFuPjwvc3Bhbj48IVtlbmRpZl0+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1m YW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90OyI+VGhlcmUgYXJl IGFsc28gc2V2ZXJhbCB1bmFuc3dlcmVkIGxpYWlzb25zIGZyb20gQnJvYWRiYW5kIEZvcnVtIHRv IHRoZSBJRVRGIGRhdGluZyBiYWNrIHRvIGxhc3QgQXVndXN0IGJlZm9yZSBMTUFQIHdhcyBmb3Jt ZWQuICZuYnNwOyhTZWUgbGlzdCBiZWxvdykgU29tZSBvZiB0aGVzZSBoYXZlDQogZGV0YWlsZWQg Y29udGVudCBmcm9tIHRoZSBCQkYgZHJhZnQgZG9jdW1lbnQgYW5kIHF1ZXN0aW9ucyB0aGF0IHRo ZSBMTUFQIFdHIHNob3VsZCByZXNwb25kIHRvLjwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNs YXNzPSJNc29MaXN0UGFyYWdyYXBoIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250 LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMx RjQ5N0QiPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1 b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFGNDk3RCI+QWN0dWFsbHkgdGhlIExN QVAgV0cgd2FzIGZvcm1lZCBvbmx5IGEgZmV3IHdlZWtzIGFnbywgYW5kIG5vdCBsYXN0IEF1Z3Vz dC4gQmVmb3JlIHRoZSBmb3JtYXRpb24gb2YgdGhlIFdHIHdlIHdlcmUganVzdCBhIGNvbW11bml0 eSBnYXRoZXJlZCBhcm91bmQgYSBtYWlsIGxpc3QsDQogYW5kIG5vdCBpbiB0aGUgcG9zaXRpb24g dG8gcmVjZWl2ZSBhbmQgcmVzcG9uZCB0byBsaWFpc29ucy4gVGhpcyBjaGFuZ2VkIG5vdy4gPC9z cGFuPg0KPG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0i Zm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5z LXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD4N CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQt ZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6IzFG NDk3RCI+SSB3b3VsZCBpbnZpdGUgb3RoZXIgcGFydGljaXBhbnRzIGluIHRoZSBXRyB0byByZWFk IHRoZSBsaWFpc29uIHBvaW50ZWQgdG8gYnkgRGF2aWQgYW5kIHN1Z2dlc3QgaG93IHdlIHNob3Vs ZCByZXNwb25kLg0KPC9zcGFuPjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+ PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVv dDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86 cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXpl OjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1 b3Q7O2NvbG9yOiMxRjQ5N0QiPlRoZXJlIGFyZSB0d28gbW9yZSBvcmdhbml6YXRpb25zIHdobyBo YXZlIGV4cHJlc3NlZCBpbnRlcmVzdCBpbiBjb21tdW5pY2F0aW5nIHdpdGggdGhlIExNQVAgV0c6 DQo8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHls ZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtz YW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwv cD4NCjxwIGNsYXNzPSJNc29MaXN0UGFyYWdyYXBoIiBzdHlsZT0idGV4dC1pbmRlbnQ6LS4yNWlu O21zby1saXN0OmwzIGxldmVsMSBsZm80Ij48IVtpZiAhc3VwcG9ydExpc3RzXT48c3BhbiBzdHls ZT0iZm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7 Ij48c3BhbiBzdHlsZT0ibXNvLWxpc3Q6SWdub3JlIj4tPHNwYW4gc3R5bGU9ImZvbnQ6Ny4wcHQg JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7Ij4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsNCjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwhW2VuZGlm XT48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZx dW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPlRoZSBJRUVFIDgwMi4x Ni4zIHdobyBzZW5kIHVzIGEgbGlhaXNvbiBsZXR0ZXIgYXZhaWxhYmxlIGF0DQo8YSBocmVmPSJo dHRwczovL2RhdGF0cmFja2VyLmlldGYub3JnL2xpYWlzb24vMTI0NC8iPmh0dHBzOi8vZGF0YXRy YWNrZXIuaWV0Zi5vcmcvbGlhaXNvbi8xMjQ0LzwvYT4uIENvbW1lbnRzIGFyZSB3ZWxjb21lIGFi b3V0IGhvdyB0byByZXNwb25kIHRvIHRoaXMgb25lIHRvby4NCjwvc3Bhbj48bzpwPjwvbzpwPjwv cD4NCjxwIGNsYXNzPSJNc29MaXN0UGFyYWdyYXBoIiBzdHlsZT0idGV4dC1pbmRlbnQ6LS4yNWlu O21zby1saXN0OmwzIGxldmVsMSBsZm80Ij48IVtpZiAhc3VwcG9ydExpc3RzXT48c3BhbiBzdHls ZT0iZm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7 Ij48c3BhbiBzdHlsZT0ibXNvLWxpc3Q6SWdub3JlIj4tPHNwYW4gc3R5bGU9ImZvbnQ6Ny4wcHQg JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7Ij4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsNCjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwhW2VuZGlm XT48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZx dW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPlRoZQ0KPC9zcGFuPjxz cGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7 LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6Ymx1ZSI+TkdDT1IgKE5leHQgR2VuZXJhdGlv biBDb252ZXJnZWQgT3BlcmF0aW9uIFJlcXVpcmVtZW50cykgcHJvamVjdC4gVGhlIGNoYWlycyB3 aWxsIGRpc2N1c3Mgd2l0aCB0aGUgcmVwcmVzZW50YXRpdmVzIGZyb20gTkdDT1IgaW4gdGhlIGNv bWluZyB3ZWVrcyBhbmQgdHJ5IHRvIHVuZGVyc3RhbmQgdGhlIGxldmVsDQogb2YgaW5mb3JtYXRp b24gdGhleSB3YW50IHRvIGV4Y2hhbmdlIHdpdGggTE1BUC4gPC9zcGFuPjxvOnA+PC9vOnA+PC9w Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9u dC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjoj MUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFs Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZx dW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0QiPlJlZ2FyZHMsPC9zcGFu PjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQt c2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJp ZiZxdW90Oztjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBj bGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWls eTomcXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOiMxRjQ5N0Qi PkRhbjwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0 eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssJnF1 b3Q7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286 cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBw dDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7 Y29sb3I6IzFGNDk3RCI+Jm5ic3A7PC9zcGFuPjxvOnA+PC9vOnA+PC9wPg0KPGRpdiBzdHlsZT0i Ym9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgYmx1ZSAxLjVwdDtwYWRkaW5nOjBpbiAwaW4g MGluIDQuMHB0Ij4NCjxkaXY+DQo8ZGl2IHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItdG9wOnNv bGlkICNCNUM0REYgMS4wcHQ7cGFkZGluZzozLjBwdCAwaW4gMGluIDBpbiI+DQo8cCBjbGFzcz0i TXNvTm9ybWFsIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTom cXVvdDtUYWhvbWEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90OyI+RnJvbTo8L3NwYW4+PC9i PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1RhaG9tYSZx dW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7Ij4NCjxhIGhyZWY9Im1haWx0bzpsbWFwLWJvdW5j ZXNAaWV0Zi5vcmciPmxtYXAtYm91bmNlc0BpZXRmLm9yZzwvYT4gWzxhIGhyZWY9Im1haWx0bzps bWFwLWJvdW5jZXNAaWV0Zi5vcmciPm1haWx0bzpsbWFwLWJvdW5jZXNAaWV0Zi5vcmc8L2E+XQ0K PGI+T24gQmVoYWxmIE9mIDwvYj5EYXZpZCBTaW5pY3JvcGU8YnI+DQo8Yj5TZW50OjwvYj4gV2Vk bmVzZGF5LCBKdWx5IDMxLCAyMDEzIDM6MTIgUE08YnI+DQo8Yj5Ubzo8L2I+IDxhIGhyZWY9Im1h aWx0bzpsbWFwQGlldGYub3JnIj5sbWFwQGlldGYub3JnPC9hPjxicj4NCjxiPkNjOjwvYj4gQWxp c3NhIENvb3BlcjsgRWxpb3QgTGVhcjsgRGF2aWQgU2luaWNyb3BlPGJyPg0KPGI+U3ViamVjdDo8 L2I+IFtsbWFwXSBMaWFpc29uIHJlcGx5IHRvIEJyb2FkYmFuZCBGb3J1bSBsaWFpc29uczwvc3Bh bj48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj4m bmJzcDs8bzpwPjwvbzpwPjwvcD4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFz cz0iTXNvTm9ybWFsIj5IaSBBbGwsPGJyPg0KQXMgSSBtZW50aW9uZWQgYXQgdGhlIG1pa2UgeWVz dGVyZGF5IHRoZXJlIHNlZW1zIHRvIGJlIHBvdGVudGlhbCBmb3Igb3ZlcmxhcCBiZXR3ZWVuIHNv bWUgb2YgdGhlIExNQVAgd29yayAoZS5nLiwgZnJhbWV3b3JrLCB0ZXJtaW5vbG9neSwgZXRjLikg YW5kIHRoZSAmcXVvdDtCcm9hZGJhbmQgQWNjZXNzIFNlcnZpY2UgQXR0cmlidXRlcyBhbmQgUGVy Zm9ybWFuY2UgTWV0cmljcyZxdW90OyAoV1QtMzA0KSB3b3JrIGdvaW5nIG9uIGluIHRoZSBCcm9h ZGJhbmQgRm9ydW0NCiAoQkJGKS4gJm5ic3A7Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4N CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48YnI+DQo8YnI+DQo8YnI+DQo8bzpwPjwvbzpw PjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPlRoZXJlIGFyZSBhbHNv IHNldmVyYWwgdW5hbnN3ZXJlZCBsaWFpc29ucyBmcm9tIEJyb2FkYmFuZCBGb3J1bSB0byB0aGUg SUVURiBkYXRpbmcgYmFjayB0byBsYXN0IEF1Z3VzdCBiZWZvcmUgTE1BUCB3YXMgZm9ybWVkLiAm bmJzcDsoU2VlIGxpc3QgYmVsb3cpIFNvbWUgb2YgdGhlc2UgaGF2ZSBkZXRhaWxlZCBjb250ZW50 IGZyb20gdGhlIEJCRiBkcmFmdCBkb2N1bWVudCBhbmQgcXVlc3Rpb25zIHRoYXQgdGhlIExNQVAN CiBXRyBzaG91bGQgcmVzcG9uZCB0by48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPjxicj4NCjxicj4NCjxicj4NCjxvOnA+PC9vOnA+PC9wPg0KPC9k aXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+VG8gc3VwcG9ydCBjb29yZGluYXRpb24g YW5kIGNvb3BlcmF0aW9uIGJldHdlZW4gdGhlIEJCRiBhbmQgSUVURiBMTUFQLCB0byBlbnN1cmUg d2UgZG9uJ3QgZHVwbGljYXRlIHdvcmssIGFuZCB0byByZXF1ZXN0IHRoZSBsYXRlc3QgdmVyc2lv biBvZiB0aGUgQkJGIHdvcmssIGl0IHdvdWxkIGJlIGFkdmFudGFnZW91cyB0byByZXNwb25kIHRv IHRoZSBCQkYgbGlhaXNvbnMgYW5kIHF1ZXN0aW9ucyBub3cgdGhhdA0KIHRoZSBMTUFQIFdHIGhh cyBiZWVuIGZvcm1lZCBhbmQgdGhlcmUgaXMgYSBjaGFydGVyLCBzY29wZSBvZiB3b3JrIGFuZCBp bml0aWFsIHdvcmsgb24gZG9jdW1lbnRzLjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0K PHAgY2xhc3M9Ik1zb05vcm1hbCI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+ DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5UaGFua3MsPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxk aXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5EYXZlIFNpbmljcm9wZTxvOnA+PC9vOnA+PC9wPg0K PC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+RXJpY3Nzb248bzpwPjwvbzpwPjwv cD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPklFVEYgTGlhaXNvbiBNYW5h Z2VyIHRvIHRoZSBCcm9hZGJhbmQgRm9ydW08bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4N CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxicj4NCjxicj4NCjxicj4NCjxvOnA+PC9vOnA+PC9wPg0K PC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+TWFyIDIwMTMgLSZuYnNwOzxhIGhy ZWY9Imh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvbGlhaXNvbi8xMjQzLyI+aHR0cHM6Ly9k YXRhdHJhY2tlci5pZXRmLm9yZy9saWFpc29uLzEyNDMvPC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9k aXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+RGVjIDIwMTIgLSZuYnNwOzxhIGhyZWY9 Imh0dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvbGlhaXNvbi8xMjIxLyI+aHR0cHM6Ly9kYXRh dHJhY2tlci5pZXRmLm9yZy9saWFpc29uLzEyMjEvPC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+ DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+U2VwIDIwMTIgLSZuYnNwOzxhIGhyZWY9Imh0 dHBzOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvbGlhaXNvbi8xMTg1LyI+aHR0cHM6Ly9kYXRhdHJh Y2tlci5pZXRmLm9yZy9saWFpc29uLzExODUvPC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8 ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+QXVnIDIwMTIgLSZuYnNwOzxhIGhyZWY9Imh0dHBz Oi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvbGlhaXNvbi8xMTc5LyI+aHR0cHM6Ly9kYXRhdHJhY2tl ci5pZXRmLm9yZy9saWFpc29uLzExNzkvPC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2 Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxk aXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PGJyPg0KPGJyPg0KPGJyPg0KPG86cD48 L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj4mbmJzcDs8bzpw PjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rp dj4NCjwvZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRt bD4NCg== --_000_A68F3CAC468B2E48BB775ACE2DD99B5E04785155podcwmbxex505ct_-- From trevor.burbridge@bt.com Wed Jul 31 08:22:29 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9D9F121F9E27 for ; Wed, 31 Jul 2013 08:22:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.199 X-Spam-Level: X-Spam-Status: No, score=-3.199 tagged_above=-999 required=5 tests=[AWL=0.400, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I19Lij-g0mJ9 for ; Wed, 31 Jul 2013 08:22:13 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp62.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id A9D5121F9E35 for ; Wed, 31 Jul 2013 08:22:12 -0700 (PDT) Received: from EVMHT63-UKRD.domain1.systemhost.net (10.36.3.100) by RDW083A006ED62.smtp-e2.hygiene.service (10.187.98.11) with Microsoft SMTP Server (TLS) id 8.3.298.1; Wed, 31 Jul 2013 16:22:10 +0100 Received: from EMV64-UKRD.domain1.systemhost.net ([169.254.1.34]) by EVMHT63-UKRD.domain1.systemhost.net ([10.36.3.100]) with mapi; Wed, 31 Jul 2013 16:22:10 +0100 From: To: , Date: Wed, 31 Jul 2013 16:22:09 +0100 Thread-Topic: [lmap] user-initiated testing Thread-Index: Ac6N98QYiouGxQz4TOGsNSg+E6sidAACauvQ Message-ID: References: <51F8E728.8070403@cisco.com> In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 15:22:29 -0000 >-----Original Message----- >From: lmap-bounces@ietf.org [mailto:lmap-bounces@ietf.org] On Behalf Of >Jean-Francois.TremblayING@videotron.com >Sent: 31 July 2013 15:09 >To: lmap@ietf.org >Subject: Re: [lmap] user-initiated testing > >Paul Aitken a =E9crit sur 31/07/2013 06:30:00 AM : > >> Jean-Francois, Sharam, Mike, Juergen, All, The user-initiated test >> scenario which you describe looks like this: >> >> ispController ispCollector >> \ / >> \ / >> MA--{test peers) >> / \ >> / \ >> userController userCollector > >This isn't what I had in mind. The example I used was more of a ispMA on a >user device, talking to a second ispCollector2. > > ispController ispCollector > \ / > \ / > MA--{test peers) > \ > \ > ispCollector2 > >I understand that a second collector might be out of scope for initial wor= k >here, but I'm trying to clear some of the confusion on wording and open up >the possibilities. It's possible, for example, to have an ISP owned MA, IS= P- >initiated on an end-user device. There's no problem with multiple Collectors. The Information Model supports= this through multiple Report Channels. Part of the motivation for this was= to allow on-demand tests to be directed to Collectors which could be the u= ser or call agent consoles. Trevor. From Michael.K.Bugenhagen@centurylink.com Wed Jul 31 09:09:20 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E3B7A21F9F85 for ; Wed, 31 Jul 2013 09:09:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.932 X-Spam-Level: X-Spam-Status: No, score=-2.932 tagged_above=-999 required=5 tests=[AWL=-0.333, BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uObsjnhQjzLJ for ; Wed, 31 Jul 2013 09:09:13 -0700 (PDT) Received: from sudnp799.qwest.com (sudnp799.qwest.com [155.70.32.99]) by ietfa.amsl.com (Postfix) with ESMTP id 527B821F9FED for ; Wed, 31 Jul 2013 09:09:11 -0700 (PDT) Received: from lxdenvmpc030.qintra.com (emailout.qintra.com [10.1.51.30]) by sudnp799.qwest.com (8.14.4/8.14.4) with ESMTP id r6VG98dE017966 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 10:09:08 -0600 (MDT) Received: from lxdenvmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 2A3F01E0076; Wed, 31 Jul 2013 10:09:00 -0600 (MDT) Received: from sudnp797.qintra.com (unknown [151.119.91.93]) by lxdenvmpc030.qintra.com (Postfix) with ESMTP id 0A6381E0068; Wed, 31 Jul 2013 10:09:00 -0600 (MDT) Received: from sudnp797.qintra.com (localhost [127.0.0.1]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r6VG8xlF020235; Wed, 31 Jul 2013 10:08:59 -0600 (MDT) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.qintra.com [151.117.206.28]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r6VG8w4k020195 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 31 Jul 2013 10:08:59 -0600 (MDT) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.02.0318.001; Wed, 31 Jul 2013 11:08:58 -0500 From: "Bugenhagen, Michael K" To: "'Juergen Schoenwaelder'" , "Paul Aitken" Thread-Topic: [lmap] user-initiated testing Thread-Index: AQHOjdjo+A7wEvhTU0SLWsc0HC7415l+7FYAgAADS4CAAAkbgIAAB2+AgAADAoD///CsMA== Date: Wed, 31 Jul 2013 16:08:58 +0000 Message-ID: References: <51F8E728.8070403@cisco.com> <20130731103834.GA22261@elstar.local> <51F8EBED.7040907@cisco.com> <20130731112257.GA22368@elstar.local> <51F8F9CD.50900@cisco.com> <20130731120019.GA22448@elstar.local> In-Reply-To: <20130731120019.GA22448@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.8] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "sharam.hakimi@exfo.com" , "Jean-Francois.TremblayING@videotron.com" , "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 16:09:20 -0000 Single vs. multiple controller question - I don't think anything in the charter keeps us from describing the creation= of a "client or user" interface for others to execute and or schedule test= s via a single controller. Customer Controller Question --- Do we really assume the customer would have their own controller ? Personally I had assumed we may define a MA, on a ISP NID that has two inte= rfaces, one facing the customer that doesn't require control (a responder),= and another that in an initiator that can send tests in either direction. = This spit MA view enable a good amount of flexibility, however both still= require some type of local resources available check to make sure they can= perform the task at hand. -----Original Message----- From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs-university.de]=20 Sent: Wednesday, July 31, 2013 7:00 AM To: Paul Aitken Cc: Jean-Francois.TremblayING@videotron.com; sharam.hakimi@exfo.com; Bugenh= agen, Michael K; lmap@ietf.org Subject: Re: [lmap] user-initiated testing On Wed, Jul 31, 2013 at 12:49:33PM +0100, Paul Aitken wrote: > Juergen, >=20 > >All good reasons to avoid this. (And note, the assumption of an MA is=20 > >controlled by a single Controller at any point in time does not=20 > >preclude implementations where the underlying runtime system can=20 > >enforce proper separation and provide multiple MAs on a single box.) >=20 > I've no objection to a single physical MA which is able to act as=20 > multiple virtual MAs, provided that they truly act separately and=20 > distinctly. >=20 > However there's almost certainly some artificial limitation (eg,=20 > maximum CPU, maximum bandwidth) imposed by the virtual system=20 > controller - so what's being measured is not the physical limitation,=20 > but the artificial limitation. Some virtualization systems allow you for example to assign CPU cores to sp= ecific virtual domains. Of course, no system is going to be perfect - even = if you deploy separate hardware boxes, they still do have limitations. > - which was said in yesterday's WG meeting: ensure you know what=20 > you're really measuring. Yes. This is why I am a big fan of MAs that are not shared because at the M= A software level, it is really difficult to control resource usage. /js --=20 Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Wed Jul 31 09:36:29 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5F39221F9814 for ; Wed, 31 Jul 2013 09:36:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zq7Gwrjlgg7D for ; Wed, 31 Jul 2013 09:36:13 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id F192321E804E for ; Wed, 31 Jul 2013 09:36:06 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id 13FEF20BFC; Wed, 31 Jul 2013 18:36:06 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id AwRXNAc9WNDT; Wed, 31 Jul 2013 18:36:05 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 663F120BF4; Wed, 31 Jul 2013 18:36:05 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id A08B827A0C5E; Wed, 31 Jul 2013 18:36:01 +0200 (CEST) Date: Wed, 31 Jul 2013 18:36:01 +0200 From: Juergen Schoenwaelder To: "Bugenhagen, Michael K" Message-ID: <20130731163601.GA23170@elstar.local> Mail-Followup-To: "Bugenhagen, Michael K" , Paul Aitken , "sharam.hakimi@exfo.com" , "Jean-Francois.TremblayING@videotron.com" , "lmap@ietf.org" References: <51F8E728.8070403@cisco.com> <20130731103834.GA22261@elstar.local> <51F8EBED.7040907@cisco.com> <20130731112257.GA22368@elstar.local> <51F8F9CD.50900@cisco.com> <20130731120019.GA22448@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "sharam.hakimi@exfo.com" , "Jean-Francois.TremblayING@videotron.com" , Paul Aitken , "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 16:36:29 -0000 On Wed, Jul 31, 2013 at 04:08:58PM +0000, Bugenhagen, Michael K wrote: > > Single vs. multiple controller question - > > I don't think anything in the charter keeps us from describing the creation of a "client or user" interface for others to execute and or schedule tests via a single controller. > Not sure what "describing the creation of" means. If you want to have the possibility of such an interface mentioned in say the framework, fine. If you want to work on details of such an interface, I would say that is not covered by the current charter. I think Phil had a much larger 'picture' on his slides since there are a number of components surrounding the components the LMAP working group is currently chartered to work on. > Customer Controller Question --- > > Do we really assume the customer would have their own controller ? > > Personally I had assumed we may define a MA, on a ISP NID that has two interfaces, one facing the customer that doesn't require control (a responder), and another that in an initiator that can send tests in either direction. This spit MA view enable a good amount of flexibility, however both still require some type of local resources available check to make sure they can perform the task at hand. > What is a 'responder'? Are you saying you assume that Measurement Agent and a Measurement Peer are always packaged together? I think the functional separation should not require this. (And there are some large measurement systems today where Measurement Agent and Measurement Peer are clearly not packaged together.) /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From Michael.K.Bugenhagen@centurylink.com Wed Jul 31 09:40:56 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B1E1D21F9EF0 for ; Wed, 31 Jul 2013 09:40:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.849 X-Spam-Level: X-Spam-Status: No, score=-2.849 tagged_above=-999 required=5 tests=[AWL=-0.250, BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ECV1SixTSwZ7 for ; Wed, 31 Jul 2013 09:40:50 -0700 (PDT) Received: from sudnp799.qwest.com (sudnp799.qwest.com [155.70.32.99]) by ietfa.amsl.com (Postfix) with ESMTP id 5201821F9EB2 for ; Wed, 31 Jul 2013 09:40:42 -0700 (PDT) Received: from lxdenvmpc030.qintra.com (emailout.qintra.com [10.1.51.30]) by sudnp799.qwest.com (8.14.4/8.14.4) with ESMTP id r6VGeciK012989 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 10:40:38 -0600 (MDT) Received: from lxdenvmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 2BC7E1E0067; Wed, 31 Jul 2013 10:40:33 -0600 (MDT) Received: from sudnp796.qintra.com (unknown [151.119.91.93]) by lxdenvmpc030.qintra.com (Postfix) with ESMTP id 0E2E81E004E; Wed, 31 Jul 2013 10:40:33 -0600 (MDT) Received: from sudnp796.qintra.com (localhost [127.0.0.1]) by sudnp796.qintra.com (8.14.4/8.14.4) with ESMTP id r6VGeW2j016319; Wed, 31 Jul 2013 10:40:32 -0600 (MDT) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.qintra.com [151.117.206.28]) by sudnp796.qintra.com (8.14.4/8.14.4) with ESMTP id r6VGeWrG016308 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 31 Jul 2013 10:40:32 -0600 (MDT) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.02.0318.001; Wed, 31 Jul 2013 11:40:31 -0500 From: "Bugenhagen, Michael K" To: "'Juergen Schoenwaelder'" Thread-Topic: [lmap] user-initiated testing Thread-Index: AQHOjdjo+A7wEvhTU0SLWsc0HC7415l+7FYAgAADS4CAAAkbgIAAB2+AgAADAoD///CsMIAAXFyA//+sgaA= Date: Wed, 31 Jul 2013 16:40:31 +0000 Message-ID: References: <51F8E728.8070403@cisco.com> <20130731103834.GA22261@elstar.local> <51F8EBED.7040907@cisco.com> <20130731112257.GA22368@elstar.local> <51F8F9CD.50900@cisco.com> <20130731120019.GA22448@elstar.local> <20130731163601.GA23170@elstar.local> In-Reply-To: <20130731163601.GA23170@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.8] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "sharam.hakimi@exfo.com" , "Jean-Francois.TremblayING@videotron.com" , Paul Aitken , "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 16:40:56 -0000 Question from JS - What is a 'responder'? Are you saying you assume that Measurement Agent and= a Measurement Peer are always packaged together? I think the functional se= paration should not require this. (And there are some large measurement sys= tems today where Measurement Agent and Measurement Peer are clearly not pac= kaged together.) /js Answer - I'm assuming that a MA, can include a test initator, and or, a test resp= onder function. These are distinct existing functions in use today. My Deeper Answer - we should have a Abstract MA concept here - My MA assumption is that a MA should in fact be capable of supporting an= y new test function without being re-invented. =20 Therefore - a Test MA function is really an abstracted subset of functions = that support any and all types of testing frameworks, and the MA abstractio= n model should allow any existing test to be ported to it. Regards, Mike =20 -----Original Message----- From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs-university.de]=20 Sent: Wednesday, July 31, 2013 11:36 AM To: Bugenhagen, Michael K Cc: Paul Aitken; sharam.hakimi@exfo.com; Jean-Francois.TremblayING@videotro= n.com; lmap@ietf.org Subject: Re: [lmap] user-initiated testing On Wed, Jul 31, 2013 at 04:08:58PM +0000, Bugenhagen, Michael K wrote: >=20 > Single vs. multiple controller question - >=20 > I don't think anything in the charter keeps us from describing the creati= on of a "client or user" interface for others to execute and or schedule te= sts via a single controller. >=20 Not sure what "describing the creation of" means. If you want to have the p= ossibility of such an interface mentioned in say the framework, fine. If yo= u want to work on details of such an interface, I would say that is not cov= ered by the current charter. I think Phil had a much larger 'picture' on hi= s slides since there are a number of components surrounding the components = the LMAP working group is currently chartered to work on. =20 > Customer Controller Question --- >=20 > Do we really assume the customer would have their own controller ? >=20 > Personally I had assumed we may define a MA, on a ISP NID that has two in= terfaces, one facing the customer that doesn't require control (a responder= ), and another that in an initiator that can send tests in either direction= . This spit MA view enable a good amount of flexibility, however both sti= ll require some type of local resources available check to make sure they c= an perform the task at hand. >=20 What is a 'responder'? Are you saying you assume that Measurement Agent and= a Measurement Peer are always packaged together? I think the functional se= paration should not require this. (And there are some large measurement sys= tems today where Measurement Agent and Measurement Peer are clearly not pac= kaged together.) /js --=20 Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From j.schoenwaelder@jacobs-university.de Wed Jul 31 12:55:59 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 023D921F9C55 for ; Wed, 31 Jul 2013 12:55:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.249 X-Spam-Level: X-Spam-Status: No, score=-103.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mMh9Zf2fphCe for ; Wed, 31 Jul 2013 12:55:53 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by ietfa.amsl.com (Postfix) with ESMTP id 8951A21F9BCE for ; Wed, 31 Jul 2013 12:55:52 -0700 (PDT) Received: from localhost (demetrius1.jacobs-university.de [212.201.44.46]) by hermes.jacobs-university.de (Postfix) with ESMTP id 7260E20BF0; Wed, 31 Jul 2013 21:55:51 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius1.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id inkfcHxT7dfI; Wed, 31 Jul 2013 21:55:51 +0200 (CEST) Received: from elstar.local (elstar.jacobs.jacobs-university.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 93B5220BEF; Wed, 31 Jul 2013 21:55:50 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 7EF6827A1147; Wed, 31 Jul 2013 21:55:47 +0200 (CEST) Date: Wed, 31 Jul 2013 21:55:47 +0200 From: Juergen Schoenwaelder To: "Bugenhagen, Michael K" Message-ID: <20130731195547.GB23513@elstar.local> Mail-Followup-To: "Bugenhagen, Michael K" , Paul Aitken , "sharam.hakimi@exfo.com" , "Jean-Francois.TremblayING@videotron.com" , "lmap@ietf.org" References: <51F8E728.8070403@cisco.com> <20130731103834.GA22261@elstar.local> <51F8EBED.7040907@cisco.com> <20130731112257.GA22368@elstar.local> <51F8F9CD.50900@cisco.com> <20130731120019.GA22448@elstar.local> <20130731163601.GA23170@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "sharam.hakimi@exfo.com" , "Jean-Francois.TremblayING@videotron.com" , Paul Aitken , "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Juergen Schoenwaelder List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 19:55:59 -0000 On Wed, Jul 31, 2013 at 04:40:31PM +0000, Bugenhagen, Michael K wrote: > > I'm assuming that a MA, can include a test initator, and or, a test responder function. > These are distinct existing functions in use today. I think we are facing terminology clashes. Can we agree to use one terminology? I assume what you call a 'test initiator' is what [1] calles a Measurement Agent and what you call a 'test responder' is what [1] calls a Measurement Peer. These are functions. How you package them is an implementation decision. > My Deeper Answer - we should have a Abstract MA concept here - > > My MA assumption is that a MA should in fact be capable of supporting any new test function without being re-invented. > Therefore - a Test MA function is really an abstracted subset of functions that support any and all types of testing frameworks, and the MA abstraction model should allow any existing test to be ported to it. > I am trying to translate from what I assume is your terminology but I get somewhere lost in the second sentence. /js [1] http://tools.ietf.org/html/draft-eardley-lmap-terminology-02 -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 From paitken@cisco.com Wed Jul 31 13:05:35 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CB79221F9FDA for ; Wed, 31 Jul 2013 13:05:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.472 X-Spam-Level: X-Spam-Status: No, score=-10.472 tagged_above=-999 required=5 tests=[AWL=0.127, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S-W9nh-0kaJT for ; Wed, 31 Jul 2013 13:05:30 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 0D67A21F9F51 for ; Wed, 31 Jul 2013 13:05:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2423; q=dns/txt; s=iport; t=1375301130; x=1376510730; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=vR7fiq4rJzEAVd49x+dweu9NoUSAe15UImsr6CTEr/g=; b=Xgi4DO5lKgZeOAjwtR9qkjQF0uRr/3lHqHAEV5qnQwDraaJvVy9BGhWc mprVq2YtkRf6oksJ9AvI8ZZTl6uAc+EpJttzmBgNAcGdbv/UttT1DblH7 2jAwXQ+pI7ArSH2bdcaDM9quOR+BhardjsuhW58t+N+q38GRe3XQwcqX1 k=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AicFAApt+VGQ/khR/2dsb2JhbABbgwY2vnmBGRZ0giQBAQEDATIBBUABBQsLEg8WDwkDAgECATcBDRMBBQIBAYgGBrktjj+BSAcWg3UDl1+GI4sqgxWBcA X-IronPort-AV: E=Sophos;i="4.89,789,1367971200"; d="scan'208";a="85322174" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-2.cisco.com with ESMTP; 31 Jul 2013 20:05:28 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id r6VK5PlS024790 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 20:05:26 GMT Received: from [10.61.169.133] ([10.61.169.133]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id r6VK5Ow5017518; Wed, 31 Jul 2013 21:05:24 +0100 (BST) Message-ID: <51F96E04.4010503@cisco.com> Date: Wed, 31 Jul 2013 21:05:24 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: Jean-Francois.TremblayING@videotron.com References: <51F8E728.8070403@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 20:05:35 -0000 Jean-Francois, > Paul Aitken a écrit sur 31/07/2013 06:30:00 AM : > >> Jean-Francois, Sharam, Mike, Juergen, All, >> The user-initiated test scenario which you describe looks like this: >> >> ispController ispCollector >> \ / >> \ / >> MA--{test peers) >> / \ >> / \ >> userController userCollector > This isn't what I had in mind. The example I used was more of a ispMA > on a user device, talking to a second ispCollector2. > > ispController ispCollector > \ / > \ / > MA--{test peers) > \ > \ > ispCollector2 This is the same topography that I showed in my later email: isp . ispCollector \ . / \ . / controller ---- MA -- {test peers} / . \ / . \ user . userCollector LMAP boundary We all seem to agree that this is a valid and likely use case. > I understand that a second collector might be out of scope for > initial work here, but I'm trying to clear some of the confusion > on wording and open up the possibilities. It's possible, for > example, to have an ISP owned MA, ISP-initiated on an end-user > device. > > user-device != user-initiated != user-owned MA > > For example, what if, as an ISP, I built a Windows 8 LMAP app > that the user can download and install. By running the app, the > user agrees to have this app run ISP-triggered tests once a day > or upon demand (when he calls support for example). In this case, > we have: > Domain = ISP > Device ownership = user > MA ownership = ISP (I own the code of app) > MA control = ISP > MA initiation = ISP > > Another example. Same type of app, this time on Android, on a > wifi mobile device: > Domain = ISP > Device ownership = user > MA ownership = ISP > MA control = ISP > MA initiation = user In the architecture we're discussing, the MA is instructed from a controller. Whether the end-user has access to the controller and what sort of access they have, is beyond the scope of LMAP. P. From Michael.K.Bugenhagen@centurylink.com Wed Jul 31 14:13:36 2013 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C301521E80F9 for ; Wed, 31 Jul 2013 14:13:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.799 X-Spam-Level: X-Spam-Status: No, score=-2.799 tagged_above=-999 required=5 tests=[AWL=-0.200, BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YYRz8VWr1Fb9 for ; Wed, 31 Jul 2013 14:13:20 -0700 (PDT) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id D0EBD11E80FF for ; Wed, 31 Jul 2013 14:13:18 -0700 (PDT) Received: from lxomavmpc030.qintra.com (emailout.qintra.com [151.117.207.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id r6VLDGgo000846 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 31 Jul 2013 16:13:16 -0500 (CDT) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 319A91E006B; Wed, 31 Jul 2013 16:13:11 -0500 (CDT) Received: from sudnp797.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id DE6A71E0075; Wed, 31 Jul 2013 16:13:10 -0500 (CDT) Received: from sudnp797.qintra.com (localhost [127.0.0.1]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r6VLDAY3027226; Wed, 31 Jul 2013 15:13:10 -0600 (MDT) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.qintra.com [151.117.206.28]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id r6VLD7Ec027100 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 31 Jul 2013 15:13:10 -0600 (MDT) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.02.0318.001; Wed, 31 Jul 2013 16:13:03 -0500 From: "Bugenhagen, Michael K" To: "'Juergen Schoenwaelder'" Thread-Topic: [lmap] user-initiated testing Thread-Index: AQHOjdjo+A7wEvhTU0SLWsc0HC7415l+7FYAgAADS4CAAAkbgIAAB2+AgAADAoD///CsMIAAXFyA//+sgaCAAItPgP//v6aA Date: Wed, 31 Jul 2013 21:13:02 +0000 Message-ID: References: <51F8E728.8070403@cisco.com> <20130731103834.GA22261@elstar.local> <51F8EBED.7040907@cisco.com> <20130731112257.GA22368@elstar.local> <51F8F9CD.50900@cisco.com> <20130731120019.GA22448@elstar.local> <20130731163601.GA23170@elstar.local> <20130731195547.GB23513@elstar.local> In-Reply-To: <20130731195547.GB23513@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.8] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "sharam.hakimi@exfo.com" , "Jean-Francois.TremblayING@videotron.com" , Paul Aitken , "lmap@ietf.org" Subject: Re: [lmap] user-initiated testing X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 21:13:36 -0000 Nope I was decomposing a Measurement agent... somewhat like this -------------------- | Measurement Agent | | Control function | -------------------- =20 / \ =09 / \ -------------------- ----------------- | Test initiator | | test responder | =20 | function | | function | =20 -------------------- -----------------=20 Where a Measurement agent has two separate sub-functions under it, both bid= ing for resources of running a test. The measurement agent control function talks to the Measurement controller. The test initiator / responder functions roll up under the Measurement agen= t, but can work independently of each other. -----Original Message----- From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs-university.de]=20 Sent: Wednesday, July 31, 2013 2:56 PM To: Bugenhagen, Michael K Cc: Paul Aitken; sharam.hakimi@exfo.com; Jean-Francois.TremblayING@videotro= n.com; lmap@ietf.org Subject: Re: [lmap] user-initiated testing On Wed, Jul 31, 2013 at 04:40:31PM +0000, Bugenhagen, Michael K wrote: >=20 > I'm assuming that a MA, can include a test initator, and or, a test re= sponder function. > These are distinct existing functions in use today. I think we are facing terminology clashes. Can we agree to use one terminol= ogy? I assume what you call a 'test initiator' is what [1] calles a Measure= ment Agent and what you call a 'test responder' is what [1] calls a Measure= ment Peer. These are functions. How you package them is an implementation d= ecision. > My Deeper Answer - we should have a Abstract MA concept here - >=20 > My MA assumption is that a MA should in fact be capable of supporting = any new test function without being re-invented. =20 > Therefore - a Test MA function is really an abstracted subset of function= s that support any and all types of testing frameworks, and the MA abstract= ion model should allow any existing test to be ported to it. >=20 I am trying to translate from what I assume is your terminology but I get s= omewhere lost in the second sentence. /js [1] http://tools.ietf.org/html/draft-eardley-lmap-terminology-02 --=20 Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103