From Michael.K.Bugenhagen@centurylink.com Mon Jan 20 06:22:01 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 42FF01A0186 for ; Mon, 20 Jan 2014 06:22:01 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b_BWaoPzPRhj for ; Mon, 20 Jan 2014 06:21:59 -0800 (PST) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id 949061A0180 for ; Mon, 20 Jan 2014 06:21:59 -0800 (PST) Received: from lxdenvmpc030.qintra.com (lxdenvmpc030.qintra.com [10.1.51.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id s0KELmii003668 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 20 Jan 2014 08:21:49 -0600 (CST) Received: from lxdenvmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 7249E1E0049; Mon, 20 Jan 2014 07:21:43 -0700 (MST) Received: from suomp61i.qintra.com (unknown [151.119.91.93]) by lxdenvmpc030.qintra.com (Postfix) with ESMTP id 3CBA31E0061; Mon, 20 Jan 2014 07:21:43 -0700 (MST) Received: from suomp61i.qintra.com (localhost [127.0.0.1]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id s0KELgcN008420; Mon, 20 Jan 2014 08:21:42 -0600 (CST) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.ctl.intranet [151.117.206.28]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id s0KELgFJ008410 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 20 Jan 2014 08:21:42 -0600 (CST) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.03.0158.001; Mon, 20 Jan 2014 08:21:42 -0600 From: "Bugenhagen, Michael K" To: "lmap@ietf.org" , "'philip.eardley@bt.com'" , "'trevor.burbridge@bt.com'" , "Cook, Charles" , "'bs7652@att.com'" Thread-Topic: LMAP detection of "BB line service state" Thread-Index: AQHPFervcrGkjOuz1k2FT8ZycGCyng== Date: Mon, 20 Jan 2014 14:21:41 +0000 Message-ID: References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> In-Reply-To: <20131112091731.GA2503@elstar.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.8] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Subject: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 14:22:01 -0000 One thing that is key to the metric "NOT" producing false positive results = (negative types), is the ability of the MA to know the "service state" of t= he broadband line. I haven't heard this discussion so I'll contribute - There are multiple types of ISP service "states" that dynamically change de= pending upon the terms of service. Some of the more common ones - 1) Exceeding a cap results in a Bandwidth profile decrease - Many wireless carriers / SAT. providers have these in place (read their = websites). Some are "gentle" enforcements and may not be implemented right at the thr= eshold crossing, others are hard coded. Complications to this - Some ISP's have a buyout (pay to increase your cap and then you get the = full speed back, others don't but it rests on your billing cycle, or possib= ly at the end of the month. =20 Either way 1) Fair "service tests" are conducted against the terms of service - Provided the "terms of service" are made available / publically know ..= . they shouldn't be part of performance testing. =09 So the problem gets to - How does a MA know what the bandwidth state the li= ne is currently based on the "terms of service". =20 Presumption - If we want this to scale, and have low cost then we can't ass= ume manual anything for this "learning" of state. Recommendation - the BBF liaison had a proposal to share that "state" infor= mation with the MA, which seems to be the easiest answer. Systemically build a function into the MA that "asks" what the line stat= e information is. To do this we'd have to=20 1) add that step to the MA doing a test 2) Choose what protocols to use to have the MA ask via the home network wh= at the service state is to conduct the test. (current speed, service state, .... ) Cheers, Mike From philip.eardley@bt.com Tue Jan 21 01:44:24 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BA4DA1A02B8 for ; Tue, 21 Jan 2014 01:44:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.401 X-Spam-Level: X-Spam-Status: No, score=-1.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cRV3N2zSKVT4 for ; Tue, 21 Jan 2014 01:44:22 -0800 (PST) Received: from smtpe1.intersmtp.com (smtpe1.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id F03371A00A2 for ; Tue, 21 Jan 2014 01:44:21 -0800 (PST) Received: from EVMHT68-UKRD.domain1.systemhost.net (10.36.3.105) by RDW083A006ED62.bt.com (10.187.98.11) with Microsoft SMTP Server (TLS) id 14.3.169.1; Tue, 21 Jan 2014 09:44:23 +0000 Received: from EMV67-UKRD.domain1.systemhost.net ([169.254.1.39]) by EVMHT68-UKRD.domain1.systemhost.net ([10.36.3.105]) with mapi; Tue, 21 Jan 2014 09:44:20 +0000 From: To: , , , , Date: Tue, 21 Jan 2014 09:44:19 +0000 Thread-Topic: LMAP detection of "BB line service state" Thread-Index: AQHPFervcrGkjOuz1k2FT8ZycGCynpqO6HHQ Message-ID: References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 09:44:24 -0000 Michael, I think this may depend on who's operating the measurement system. For an operator-controlled measurement system, then they should know what c= ontract the BB user is on and whether they're currently being capped. In th= e light of this they can work out what tests they want done. The controller= instructs the MA. I'm not sure how up-to-date the Controller can be kept i= nformed about dynamic capping (this info is going via OAM, which isn't very= dynamic). =20 For a regulator-controlled measurement system, there is likely to be some o= ff-line way that the ISP tells the regulator what contract the BB user is o= n - so that they can run sensible tests. After results collection, perhaps = the ISP could tell the regulator that user y had exceeded their cap on Jan = 21st (so treat those results with caution). I think defining a real-time in= terface about caps is unlikely. Partly technical (dynamic info, many differ= ent capping approaches) and partly commercial. For an end-user controlled measurement system, I think it would be quite us= eful for the end user (& their MA) to be able to confirm what contract they= are on. Again I think defining a real-time interface about caps is difficu= lt. I think it's something some ISPs might find it useful to tell a subscri= ber (eg via an email or webpage) but doubtful about standards or about real= time (more likely a warning before it starts and, after a time lag, inform= ing them it's happening or happened) So, rather than dynamically informing the MA about capping, I'd take a diff= erent approach. If, for a particular test, it really matters to understand = about capping, then there should be a preliminary part of the test where it= sends some probe traffic in order to assess this.=20 Best wishes phil > -----Original Message----- > From: Bugenhagen, Michael K > [mailto:Michael.K.Bugenhagen@centurylink.com] > Sent: 20 January 2014 14:22 > To: lmap@ietf.org; Eardley,PL,Philip,TUB8 R; Burbridge,T,Trevor,TUB8 R; > Cook, Charles; 'bs7652@att.com' > Subject: LMAP detection of "BB line service state" >=20 > One thing that is key to the metric "NOT" producing false positive > results (negative types), is the ability of the MA to know the "service > state" of the broadband line. >=20 > I haven't heard this discussion so I'll contribute - >=20 > There are multiple types of ISP service "states" that dynamically > change depending upon the terms of service. >=20 > Some of the more common ones - >=20 > 1) Exceeding a cap results in a Bandwidth profile decrease > - Many wireless carriers / SAT. providers have these in place > (read their websites). > Some are "gentle" enforcements and may not be implemented right at > the threshold crossing, others are hard coded. >=20 > Complications to this > - Some ISP's have a buyout (pay to increase your cap and then you > get the full speed back, others don't but it rests on your billing > cycle, or possibly at the end of the month. >=20 > Either way >=20 > 1) Fair "service tests" are conducted against the terms of service > - Provided the "terms of service" are made available / publically > know ... they shouldn't be part of performance testing. >=20 >=20 > So the problem gets to - How does a MA know what the bandwidth state > the line is currently based on the "terms of service". >=20 > Presumption - If we want this to scale, and have low cost then we can't > assume manual anything for this "learning" of state. >=20 > Recommendation - the BBF liaison had a proposal to share that "state" > information with the MA, which seems to be the easiest answer. > Systemically build a function into the MA that "asks" > what the line state information is. >=20 > To do this we'd have to > 1) add that step to the MA doing a test > 2) Choose what protocols to use to have the MA ask via the home > network what the service state is to conduct the test. > (current speed, service state, .... ) >=20 > Cheers, > Mike From ken.ko@adtran.com Tue Jan 21 05:11:08 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8AF5E1A00BC for ; Tue, 21 Jan 2014 05:11:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3 X-Spam-Level: X-Spam-Status: No, score=-3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w8dpaKFB1p_2 for ; Tue, 21 Jan 2014 05:11:03 -0800 (PST) Received: from p02c12o148.mxlogic.net (p02c12o148.mxlogic.net [208.65.145.81]) by ietfa.amsl.com (Postfix) with ESMTP id D722C1A00D6 for ; Tue, 21 Jan 2014 05:11:02 -0800 (PST) Received: from unknown [76.164.174.83] (EHLO ex-hc2.corp.adtran.com) by p02c12o148.mxlogic.net(mxl_mta-7.2.2-0) with ESMTP id 7e17ed25.2b29c5007940.75339.00-503.161082.p02c12o148.mxlogic.net (envelope-from ); Tue, 21 Jan 2014 06:11:03 -0700 (MST) X-MXL-Hash: 52de71e77f75f7a0-5cece09e28e33d1d314ae86bba70882d97e776dc Received: from unknown [76.164.174.83] (EHLO ex-hc2.corp.adtran.com) by p02c12o148.mxlogic.net(mxl_mta-7.2.2-0) over TLS secured channel with ESMTP id 3a17ed25.0.74732.00-355.159690.p02c12o148.mxlogic.net (envelope-from ); Tue, 21 Jan 2014 06:09:58 -0700 (MST) X-MXL-Hash: 52de71a61c1f498b-2455ccd43e1ed518eb7aa3d9f63d9a079d250986 Received: from ex-mb1.corp.adtran.com ([fe80::51a3:972d:5f16:9952]) by ex-hc2.corp.adtran.com ([fe80::a019:449b:3f62:28e5%10]) with mapi id 14.01.0438.000; Tue, 21 Jan 2014 07:09:54 -0600 From: KEN KO To: "philip.eardley@bt.com" , "Michael.K.Bugenhagen@centurylink.com" , "lmap@ietf.org" , "trevor.burbridge@bt.com" , "Charles.Cook2@centurylink.com" , "bs7652@att.com" Thread-Topic: [lmap] LMAP detection of "BB line service state" Thread-Index: AQHPFervcrGkjOuz1k2FT8ZycGCynpqO6HHQgAA9hDA= Date: Tue, 21 Jan 2014 13:09:54 +0000 Message-ID: References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.5.197] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-AnalysisOut: [v=2.0 cv=KJPt+i5o c=1 sm=1 a=5zDNsY1we+1mvVcp/5+1jQ==:17 a] X-AnalysisOut: [=7z528Rzh2MIA:10 a=qZHQZMT3apYA:10 a=FBzmcnhlJawA:10 a=BLc] X-AnalysisOut: [eEmwcHowA:10 a=kj9zAlcOel0A:10 a=xqWC_Br6kY4A:10 a=eJNrpio] X-AnalysisOut: [GAAAA:8 a=jVcuj6UKxdgA:10 a=48vgC7mUAAAA:8 a=e9qsufxtAAAA:] X-AnalysisOut: [8 a=J_N6KFswAAAA:8 a=zQP7CpKOAAAA:8 a=r9XCnccb2MN2rW289ngA] X-AnalysisOut: [:9 a=CjuIK1q_8ugA:10 a=lZB815dzVvQA:10 a=W1qU_X6G3J8A:10 a] X-AnalysisOut: [=Pwbduc0PQ3sA:10 a=Hz7IrDYlS0cA:10] X-Spam: [F=0.2000000000; CM=0.500; MH=0.500(2014012106); S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [76.164.174.83] Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 13:11:08 -0000 Mobile operators currently provide traffic volume vs. data cap reporting to= subscribers in "near-real time" - that is, current to within about the las= t 24 hours. So it seems like a similar capability should be feasible with r= egard to service state for lmap. I think it's a valid question whether that= data can be made real time, and whether anything less is actionable by the= MA. -----Original Message----- From: lmap [mailto:lmap-bounces@ietf.org] On Behalf Of philip.eardley@bt.co= m Sent: Tuesday, January 21, 2014 4:44 AM To: Michael.K.Bugenhagen@centurylink.com; lmap@ietf.org; trevor.burbridge@b= t.com; Charles.Cook2@centurylink.com; bs7652@att.com Subject: Re: [lmap] LMAP detection of "BB line service state" Michael, I think this may depend on who's operating the measurement system. For an operator-controlled measurement system, then they should know what c= ontract the BB user is on and whether they're currently being capped. In th= e light of this they can work out what tests they want done. The controller= instructs the MA. I'm not sure how up-to-date the Controller can be kept i= nformed about dynamic capping (this info is going via OAM, which isn't very= dynamic). =20 For a regulator-controlled measurement system, there is likely to be some o= ff-line way that the ISP tells the regulator what contract the BB user is o= n - so that they can run sensible tests. After results collection, perhaps = the ISP could tell the regulator that user y had exceeded their cap on Jan = 21st (so treat those results with caution). I think defining a real-time in= terface about caps is unlikely. Partly technical (dynamic info, many differ= ent capping approaches) and partly commercial. For an end-user controlled measurement system, I think it would be quite us= eful for the end user (& their MA) to be able to confirm what contract they= are on. Again I think defining a real-time interface about caps is difficu= lt. I think it's something some ISPs might find it useful to tell a subscri= ber (eg via an email or webpage) but doubtful about standards or about real= time (more likely a warning before it starts and, after a time lag, inform= ing them it's happening or happened) So, rather than dynamically informing the MA about capping, I'd take a diff= erent approach. If, for a particular test, it really matters to understand = about capping, then there should be a preliminary part of the test where it= sends some probe traffic in order to assess this.=20 Best wishes phil > -----Original Message----- > From: Bugenhagen, Michael K > [mailto:Michael.K.Bugenhagen@centurylink.com] > Sent: 20 January 2014 14:22 > To: lmap@ietf.org; Eardley,PL,Philip,TUB8 R; Burbridge,T,Trevor,TUB8=20 > R; Cook, Charles; 'bs7652@att.com' > Subject: LMAP detection of "BB line service state" >=20 > One thing that is key to the metric "NOT" producing false positive=20 > results (negative types), is the ability of the MA to know the=20 > "service state" of the broadband line. >=20 > I haven't heard this discussion so I'll contribute - >=20 > There are multiple types of ISP service "states" that dynamically=20 > change depending upon the terms of service. >=20 > Some of the more common ones - >=20 > 1) Exceeding a cap results in a Bandwidth profile decrease > - Many wireless carriers / SAT. providers have these in place (read=20 > their websites). > Some are "gentle" enforcements and may not be implemented right at=20 > the threshold crossing, others are hard coded. >=20 > Complications to this > - Some ISP's have a buyout (pay to increase your cap and then you get=20 > the full speed back, others don't but it rests on your billing cycle,=20 > or possibly at the end of the month. >=20 > Either way >=20 > 1) Fair "service tests" are conducted against the terms of service > - Provided the "terms of service" are made available / publically=20 > know ... they shouldn't be part of performance testing. >=20 >=20 > So the problem gets to - How does a MA know what the bandwidth state=20 > the line is currently based on the "terms of service". >=20 > Presumption - If we want this to scale, and have low cost then we=20 > can't assume manual anything for this "learning" of state. >=20 > Recommendation - the BBF liaison had a proposal to share that "state" > information with the MA, which seems to be the easiest answer. > Systemically build a function into the MA that "asks" > what the line state information is. >=20 > To do this we'd have to > 1) add that step to the MA doing a test > 2) Choose what protocols to use to have the MA ask via the home=20 > network what the service state is to conduct the test. > (current speed, service state, .... ) >=20 > Cheers, > Mike _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap From charles.cook2@centurylink.com Tue Jan 21 05:36:43 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1834F1A00F0 for ; Tue, 21 Jan 2014 05:36:43 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MGx6VXNhFx_d for ; Tue, 21 Jan 2014 05:36:41 -0800 (PST) Received: from sudnp799.qwest.com (sudnp799.qwest.com [155.70.32.99]) by ietfa.amsl.com (Postfix) with ESMTP id 89A6D1A00D3 for ; Tue, 21 Jan 2014 05:36:41 -0800 (PST) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by sudnp799.qwest.com (8.14.4/8.14.4) with ESMTP id s0LDaZ8H000481 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jan 2014 06:36:36 -0700 (MST) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 9025D1E006F; Tue, 21 Jan 2014 07:36:30 -0600 (CST) Received: from sudnp797.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id 5674B1E0059; Tue, 21 Jan 2014 07:36:30 -0600 (CST) Received: from sudnp797.qintra.com (localhost [127.0.0.1]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id s0LDaTtn008972; Tue, 21 Jan 2014 06:36:29 -0700 (MST) Received: from [10.188.247.82] (x1069818.dhcp.intranet [10.188.247.82]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id s0LDaSrd008951; Tue, 21 Jan 2014 06:36:28 -0700 (MST) Message-ID: <52DE77DA.7070109@centurylink.com> Date: Tue, 21 Jan 2014 06:36:26 -0700 From: Charles Cook User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121005 Thunderbird/16.0 MIME-Version: 1.0 To: philip.eardley@bt.com References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> In-Reply-To: X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-CFilter-Loop: Reflected Cc: bs7652@att.com, trevor.burbridge@bt.com, Michael.K.Bugenhagen@centurylink.com, lmap@ietf.org Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: charles.cook2@centurylink.com List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 13:36:43 -0000 Service Caps can be an issue. Since no one has a crystal ball regarding whether a subscriber will operate close to the service cap, and since it would not be appropriate for a subscriber to be charged for a measurement test that they did not request, one possibility is that when the MA registers with the Measurement Controller, information regarding the existence of a Service Cap is provided. If there is a service cap, then the Measurement Controller simply does not schedule any bandwidth/throughput tests with that MA. If the service provider that owns the MA runs the test, then I agree that the service provider can meter their own test and adjust billing to make the subscriber whole. If a third party must run a bandwidth/throughput test on an MA with a Service Cap, then somehow there needs to be a mechanism to meter the test and charge the third party for their usage of the subscriber's allocation. Obviously this would be much more complex. Another possibility is that the third-party could negotiate with the service provider increase the service cap beyond what the customer has subscribed to so that any tests that are run would not eat into the amount of bandwidth that the customer contracted for with the service provider. However it is done, we need to ensure that the subscriber is whole. Charles On 1/21/2014 2:44 AM, philip.eardley@bt.com wrote: > Michael, > > I think this may depend on who's operating the measurement system. > > For an operator-controlled measurement system, then they should know what contract the BB user is on and whether they're currently being capped. In the light of this they can work out what tests they want done. The controller instructs the MA. I'm not sure how up-to-date the Controller can be kept informed about dynamic capping (this info is going via OAM, which isn't very dynamic). > > For a regulator-controlled measurement system, there is likely to be some off-line way that the ISP tells the regulator what contract the BB user is on - so that they can run sensible tests. After results collection, perhaps the ISP could tell the regulator that user y had exceeded their cap on Jan 21st (so treat those results with caution). I think defining a real-time interface about caps is unlikely. Partly technical (dynamic info, many different capping approaches) and partly commercial. > > For an end-user controlled measurement system, I think it would be quite useful for the end user (& their MA) to be able to confirm what contract they are on. Again I think defining a real-time interface about caps is difficult. I think it's something some ISPs might find it useful to tell a subscriber (eg via an email or webpage) but doubtful about standards or about real time (more likely a warning before it starts and, after a time lag, informing them it's happening or happened) > > So, rather than dynamically informing the MA about capping, I'd take a different approach. If, for a particular test, it really matters to understand about capping, then there should be a preliminary part of the test where it sends some probe traffic in order to assess this. > > Best wishes > phil > >> -----Original Message----- >> From: Bugenhagen, Michael K >> [mailto:Michael.K.Bugenhagen@centurylink.com] >> Sent: 20 January 2014 14:22 >> To: lmap@ietf.org; Eardley,PL,Philip,TUB8 R; Burbridge,T,Trevor,TUB8 R; >> Cook, Charles; 'bs7652@att.com' >> Subject: LMAP detection of "BB line service state" >> >> One thing that is key to the metric "NOT" producing false positive >> results (negative types), is the ability of the MA to know the "service >> state" of the broadband line. >> >> I haven't heard this discussion so I'll contribute - >> >> There are multiple types of ISP service "states" that dynamically >> change depending upon the terms of service. >> >> Some of the more common ones - >> >> 1) Exceeding a cap results in a Bandwidth profile decrease >> - Many wireless carriers / SAT. providers have these in place >> (read their websites). >> Some are "gentle" enforcements and may not be implemented right at >> the threshold crossing, others are hard coded. >> >> Complications to this >> - Some ISP's have a buyout (pay to increase your cap and then you >> get the full speed back, others don't but it rests on your billing >> cycle, or possibly at the end of the month. >> >> Either way >> >> 1) Fair "service tests" are conducted against the terms of service >> - Provided the "terms of service" are made available / publically >> know ... they shouldn't be part of performance testing. >> >> >> So the problem gets to - How does a MA know what the bandwidth state >> the line is currently based on the "terms of service". >> >> Presumption - If we want this to scale, and have low cost then we can't >> assume manual anything for this "learning" of state. >> >> Recommendation - the BBF liaison had a proposal to share that "state" >> information with the MA, which seems to be the easiest answer. >> Systemically build a function into the MA that "asks" >> what the line state information is. >> >> To do this we'd have to >> 1) add that step to the MA doing a test >> 2) Choose what protocols to use to have the MA ask via the home >> network what the service state is to conduct the test. >> (current speed, service state, .... ) >> >> Cheers, >> Mike -- Charles Cook Principal Architect Network 5325 Zuni Street; Suite 224 Denver, CO 80221 Tel: 303.992.8952 Fax: 925.281.0662 charles.cook2@centurylink.com From Michael.K.Bugenhagen@centurylink.com Tue Jan 21 05:51:01 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6BF2B1A0127 for ; Tue, 21 Jan 2014 05:51:01 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 45Qx3v5NKLYX for ; Tue, 21 Jan 2014 05:51:00 -0800 (PST) Received: from sudnp799.qwest.com (sudnp799.qwest.com [155.70.32.99]) by ietfa.amsl.com (Postfix) with ESMTP id 463B01A0121 for ; Tue, 21 Jan 2014 05:51:00 -0800 (PST) Received: from lxdenvmpc030.qintra.com (emailout.qintra.com [10.1.51.30]) by sudnp799.qwest.com (8.14.4/8.14.4) with ESMTP id s0LDorET007154 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jan 2014 06:50:53 -0700 (MST) Received: from lxdenvmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 5950F1E005E; Tue, 21 Jan 2014 06:50:48 -0700 (MST) Received: from suomp60i.qintra.com (unknown [151.119.91.93]) by lxdenvmpc030.qintra.com (Postfix) with ESMTP id 2708D1E0035; Tue, 21 Jan 2014 06:50:48 -0700 (MST) Received: from suomp60i.qintra.com (localhost [127.0.0.1]) by suomp60i.qintra.com (8.14.4/8.14.4) with ESMTP id s0LDolAJ005850; Tue, 21 Jan 2014 07:50:47 -0600 (CST) Received: from vodcwhubex501.ctl.intranet (vodcwhubex501.ctl.intranet [151.117.206.27]) by suomp60i.qintra.com (8.14.4/8.14.4) with ESMTP id s0LDolNl005843 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 21 Jan 2014 07:50:47 -0600 (CST) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex501.ctl.intranet ([151.117.206.27]) with mapi id 14.03.0158.001; Tue, 21 Jan 2014 07:50:46 -0600 From: "Bugenhagen, Michael K" To: "Cook, Charles" , "philip.eardley@bt.com" Thread-Topic: LMAP detection of "BB line service state" Thread-Index: AQHPFq3Kc4NOS8IBa0+x7dIZfDqi/JqPMBoA Date: Tue, 21 Jan 2014 13:50:46 +0000 Message-ID: References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> <52DE77DA.7070109@centurylink.com> In-Reply-To: <52DE77DA.7070109@centurylink.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [155.70.16.191] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "trevor.burbridge@bt.com" , "bs7652@att.com" , "lmap@ietf.org" Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 13:51:01 -0000 The key point I had in mind was that the test result really needs to have t= he "service state" information in the actual result to make this truly scal= able. Simple math wise In a single ISP per speed tier I would assume (and they have many speed tie= rs) 10k lines 12 measurements a day (minimum required granularity that lets one see a pea= k hour) 30 days per month. Yields - 360k test results.... makes a very large spreadsheet...=20 So scalable for any LMAP test IMHO means having sort-able key fields in = the test result. Maybe a really, really small ISP could do a manual processes but if you are= doing this in scale per market you'd have to hire a person to do that. So I don't like added costs on my ISP line - I'll vote for trying to kept t= his cheap vs. doing to make our jobs a bit easier by not addressing it. -----Original Message----- From: Cook, Charles=20 Sent: Tuesday, January 21, 2014 7:36 AM To: philip.eardley@bt.com Cc: Bugenhagen, Michael K; lmap@ietf.org; trevor.burbridge@bt.com; bs7652@a= tt.com Subject: Re: LMAP detection of "BB line service state" Service Caps can be an issue. Since no one has a crystal ball regarding whether a subscriber will operate close to the service cap, and s= ince it would not be appropriate for a subscriber to be charged for a measu= rement test that they did not request, one possibility is that when the MA = registers with the Measurement Controller, information regarding the existe= nce of a Service Cap is provided. If there is a service cap, then the Meas= urement Controller simply does not schedule any bandwidth/throughput tests = with that MA.=20 If the service provider that owns the MA runs the test, then I agree that = the service provider can meter their own test and adjust billing to make th= e subscriber whole. If a third party must run a bandwidth/throughput test on an MA with a Servi= ce Cap, then somehow there needs to be a mechanism to meter the test and ch= arge the third party for their usage of the subscriber's allocation. Obvio= usly this would be much more complex. Another possibility is that the third-party could negotiate with the servic= e provider increase the service cap beyond what the customer has subscribed= to so that any tests that are run would not eat into the amount of bandwid= th that the customer contracted for with the service provider. However it is done, we need to ensure that the subscriber is whole. Charles On 1/21/2014 2:44 AM, philip.eardley@bt.com wrote: > Michael, > > I think this may depend on who's operating the measurement system. > > For an operator-controlled measurement system, then they should know what= contract the BB user is on and whether they're currently being capped. In = the light of this they can work out what tests they want done. The controll= er instructs the MA. I'm not sure how up-to-date the Controller can be kept= informed about dynamic capping (this info is going via OAM, which isn't ve= ry dynamic). > =20 > For a regulator-controlled measurement system, there is likely to be some= off-line way that the ISP tells the regulator what contract the BB user is= on - so that they can run sensible tests. After results collection, perhap= s the ISP could tell the regulator that user y had exceeded their cap on Ja= n 21st (so treat those results with caution). I think defining a real-time = interface about caps is unlikely. Partly technical (dynamic info, many diff= erent capping approaches) and partly commercial. > > For an end-user controlled measurement system, I think it would be=20 > quite useful for the end user (& their MA) to be able to confirm what=20 > contract they are on. Again I think defining a real-time interface=20 > about caps is difficult. I think it's something some ISPs might find=20 > it useful to tell a subscriber (eg via an email or webpage) but=20 > doubtful about standards or about real time (more likely a warning=20 > before it starts and, after a time lag, informing them it's happening=20 > or happened) > > So, rather than dynamically informing the MA about capping, I'd take a di= fferent approach. If, for a particular test, it really matters to understan= d about capping, then there should be a preliminary part of the test where = it sends some probe traffic in order to assess this.=20 > > Best wishes > phil > >> -----Original Message----- >> From: Bugenhagen, Michael K >> [mailto:Michael.K.Bugenhagen@centurylink.com] >> Sent: 20 January 2014 14:22 >> To: lmap@ietf.org; Eardley,PL,Philip,TUB8 R; Burbridge,T,Trevor,TUB8=20 >> R; Cook, Charles; 'bs7652@att.com' >> Subject: LMAP detection of "BB line service state" >> >> One thing that is key to the metric "NOT" producing false positive=20 >> results (negative types), is the ability of the MA to know the=20 >> "service state" of the broadband line. >> >> I haven't heard this discussion so I'll contribute - >> >> There are multiple types of ISP service "states" that dynamically=20 >> change depending upon the terms of service. >> >> Some of the more common ones - >> >> 1) Exceeding a cap results in a Bandwidth profile decrease >> - Many wireless carriers / SAT. providers have these in place (read=20 >> their websites). >> Some are "gentle" enforcements and may not be implemented right at=20 >> the threshold crossing, others are hard coded. >> >> Complications to this >> - Some ISP's have a buyout (pay to increase your cap and then you=20 >> get the full speed back, others don't but it rests on your billing=20 >> cycle, or possibly at the end of the month. >> >> Either way >> >> 1) Fair "service tests" are conducted against the terms of service >> - Provided the "terms of service" are made available / publically=20 >> know ... they shouldn't be part of performance testing. >> >> >> So the problem gets to - How does a MA know what the bandwidth state=20 >> the line is currently based on the "terms of service". >> >> Presumption - If we want this to scale, and have low cost then we=20 >> can't assume manual anything for this "learning" of state. >> >> Recommendation - the BBF liaison had a proposal to share that "state" >> information with the MA, which seems to be the easiest answer. >> Systemically build a function into the MA that "asks" >> what the line state information is. >> >> To do this we'd have to >> 1) add that step to the MA doing a test >> 2) Choose what protocols to use to have the MA ask via the home=20 >> network what the service state is to conduct the test. >> (current speed, service state, .... ) >> >> Cheers, >> Mike --=20 Charles Cook Principal Architect Network 5325 Zuni Street; Suite 224 Denver, CO 80221 Tel: 303.992.8952 Fax: 925.281.0662 charles.cook2@centurylink.com From Michael.K.Bugenhagen@centurylink.com Tue Jan 21 06:02:06 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6600C1A00E8 for ; Tue, 21 Jan 2014 06:02:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cjYbtZOi_Roe for ; Tue, 21 Jan 2014 06:02:04 -0800 (PST) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id 6E2BB1A00DB for ; Tue, 21 Jan 2014 06:02:04 -0800 (PST) Received: from lxdenvmpc030.qintra.com (lxdenvmpc030.qintra.com [10.1.51.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id s0LE1x41014755 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jan 2014 08:02:00 -0600 (CST) Received: from lxdenvmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 9390A1E0061; Tue, 21 Jan 2014 07:01:54 -0700 (MST) Received: from suomp60i.qintra.com (unknown [151.119.91.93]) by lxdenvmpc030.qintra.com (Postfix) with ESMTP id 5CD171E007D; Tue, 21 Jan 2014 07:01:54 -0700 (MST) Received: from suomp60i.qintra.com (localhost [127.0.0.1]) by suomp60i.qintra.com (8.14.4/8.14.4) with ESMTP id s0LE1rYk001302; Tue, 21 Jan 2014 08:01:53 -0600 (CST) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.ctl.intranet [151.117.206.28]) by suomp60i.qintra.com (8.14.4/8.14.4) with ESMTP id s0LE1rjV001258 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 21 Jan 2014 08:01:53 -0600 (CST) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.03.0158.001; Tue, 21 Jan 2014 08:01:52 -0600 From: "Bugenhagen, Michael K" To: "'philip.eardley@bt.com'" , "lmap@ietf.org" , "trevor.burbridge@bt.com" , "Cook, Charles" , "bs7652@att.com" Thread-Topic: LMAP detection of "BB line service state" Thread-Index: AQHPFo1emNEQBXarl0utcDXK34x84pqPNArQ Date: Tue, 21 Jan 2014 14:01:52 +0000 Message-ID: References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [155.70.16.191] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 14:02:06 -0000 Phil, I think the BBF guys felt the MA should do a Pull on the ISP system vs. = the other way around. I would agree with that approach. I'm also finding out that there are many more types of service "terms" out = there than I thought - beyond caps I remember Matt Mathis saying that Malwa= re could impact line performance (part of the line in use all the time), an= d things like failure to pay (yes I have a teenager) ends up placing you in= a walled garden or changes your bandwidth profile. =20 So there are other "line states" out there that we probably don't know ab= out in an ISP terms of service. So - On the take the high road side, a customer having more information abo= ut their line / service is always good.... so making this poll-able from t= he customer end for a MA that may or may not be and ISP MA sounds like a wi= n-win-win... Best Regards, Mike -----Original Message----- From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 Sent: Tuesday, January 21, 2014 3:44 AM To: Bugenhagen, Michael K; lmap@ietf.org; trevor.burbridge@bt.com; Cook, Ch= arles; bs7652@att.com Subject: RE: LMAP detection of "BB line service state"=20 Michael, I think this may depend on who's operating the measurement system. For an operator-controlled measurement system, then they should know what c= ontract the BB user is on and whether they're currently being capped. In th= e light of this they can work out what tests they want done. The controller= instructs the MA. I'm not sure how up-to-date the Controller can be kept i= nformed about dynamic capping (this info is going via OAM, which isn't very= dynamic). =20 For a regulator-controlled measurement system, there is likely to be some o= ff-line way that the ISP tells the regulator what contract the BB user is o= n - so that they can run sensible tests. After results collection, perhaps = the ISP could tell the regulator that user y had exceeded their cap on Jan = 21st (so treat those results with caution). I think defining a real-time in= terface about caps is unlikely. Partly technical (dynamic info, many differ= ent capping approaches) and partly commercial. For an end-user controlled measurement system, I think it would be quite us= eful for the end user (& their MA) to be able to confirm what contract they= are on. Again I think defining a real-time interface about caps is difficu= lt. I think it's something some ISPs might find it useful to tell a subscri= ber (eg via an email or webpage) but doubtful about standards or about real= time (more likely a warning before it starts and, after a time lag, inform= ing them it's happening or happened) So, rather than dynamically informing the MA about capping, I'd take a diff= erent approach. If, for a particular test, it really matters to understand = about capping, then there should be a preliminary part of the test where it= sends some probe traffic in order to assess this.=20 Best wishes phil > -----Original Message----- > From: Bugenhagen, Michael K > [mailto:Michael.K.Bugenhagen@centurylink.com] > Sent: 20 January 2014 14:22 > To: lmap@ietf.org; Eardley,PL,Philip,TUB8 R; Burbridge,T,Trevor,TUB8=20 > R; Cook, Charles; 'bs7652@att.com' > Subject: LMAP detection of "BB line service state" >=20 > One thing that is key to the metric "NOT" producing false positive=20 > results (negative types), is the ability of the MA to know the=20 > "service state" of the broadband line. >=20 > I haven't heard this discussion so I'll contribute - >=20 > There are multiple types of ISP service "states" that dynamically=20 > change depending upon the terms of service. >=20 > Some of the more common ones - >=20 > 1) Exceeding a cap results in a Bandwidth profile decrease > - Many wireless carriers / SAT. providers have these in place (read=20 > their websites). > Some are "gentle" enforcements and may not be implemented right at=20 > the threshold crossing, others are hard coded. >=20 > Complications to this > - Some ISP's have a buyout (pay to increase your cap and then you get=20 > the full speed back, others don't but it rests on your billing cycle,=20 > or possibly at the end of the month. >=20 > Either way >=20 > 1) Fair "service tests" are conducted against the terms of service > - Provided the "terms of service" are made available / publically=20 > know ... they shouldn't be part of performance testing. >=20 >=20 > So the problem gets to - How does a MA know what the bandwidth state=20 > the line is currently based on the "terms of service". >=20 > Presumption - If we want this to scale, and have low cost then we=20 > can't assume manual anything for this "learning" of state. >=20 > Recommendation - the BBF liaison had a proposal to share that "state" > information with the MA, which seems to be the easiest answer. > Systemically build a function into the MA that "asks" > what the line state information is. >=20 > To do this we'd have to > 1) add that step to the MA doing a test > 2) Choose what protocols to use to have the MA ask via the home=20 > network what the service state is to conduct the test. > (current speed, service state, .... ) >=20 > Cheers, > Mike From ken.ko@adtran.com Tue Jan 21 06:03:23 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 889AF1A0116 for ; Tue, 21 Jan 2014 06:03:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3 X-Spam-Level: X-Spam-Status: No, score=-3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D5bwisvLhHWi for ; Tue, 21 Jan 2014 06:03:21 -0800 (PST) Received: from p02c11o149.mxlogic.net (p02c11o149.mxlogic.net [208.65.144.82]) by ietfa.amsl.com (Postfix) with ESMTP id 21F531A00DB for ; Tue, 21 Jan 2014 06:03:21 -0800 (PST) Received: from unknown [76.164.174.81] (EHLO p02c11o149.mxlogic.net) by p02c11o149.mxlogic.net(mxl_mta-7.2.2-0) with ESMTP id 92e7ed25.2afd37204940.7839.00-575.17807.p02c11o149.mxlogic.net (envelope-from ); Tue, 21 Jan 2014 07:03:21 -0700 (MST) X-MXL-Hash: 52de7e294782d176-eb793fb045a7c8afa3cc4224246903028f128256 Received: from unknown [76.164.174.81] by p02c11o149.mxlogic.net(mxl_mta-7.2.2-0) with SMTP id b0e7ed25.0.5534.00-334.17027.p02c11o149.mxlogic.net (envelope-from ); Tue, 21 Jan 2014 07:02:53 -0700 (MST) X-MXL-Hash: 52de7e0d426f7a80-6c6c619668fc2ae15236e696a5d4b2c1c7fff5ce Received: from ex-mb1.corp.adtran.com ([fe80::51a3:972d:5f16:9952]) by ex-hc1.corp.adtran.com ([fe80::a43f:7ea6:7688:37b%13]) with mapi id 14.01.0438.000; Tue, 21 Jan 2014 08:01:28 -0600 From: KEN KO To: "Bugenhagen, Michael K" , "Cook, Charles" , "philip.eardley@bt.com" Thread-Topic: LMAP detection of "BB line service state" Thread-Index: AQHPFq/UTHqK/xoOiUOadd2ucHkDSpqPM83Q Date: Tue, 21 Jan 2014 14:01:27 +0000 Message-ID: References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> <52DE77DA.7070109@centurylink.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.5.197] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-AnalysisOut: [v=2.0 cv=YLEoP26x c=1 sm=1 a=0XgpNN2/4a34ymu16SVwsQ==:17 a] X-AnalysisOut: [=ywCGDIam2kAA:10 a=qZHQZMT3apYA:10 a=FBzmcnhlJawA:10 a=BLc] X-AnalysisOut: [eEmwcHowA:10 a=kj9zAlcOel0A:10 a=xqWC_Br6kY4A:10 a=eJNrpio] X-AnalysisOut: [GAAAA:8 a=MnYPV9Kup-sA:10 a=48vgC7mUAAAA:8 a=e9qsufxtAAAA:] X-AnalysisOut: [8 a=zQP7CpKOAAAA:8 a=J_N6KFswAAAA:8 a=pNcg6rUHDc-NljChcV0A] X-AnalysisOut: [:9 a=CjuIK1q_8ugA:10 a=fK2py77DiAcA:10 a=pjBFNBMRyLAA:10 a] X-AnalysisOut: [=lZB815dzVvQA:10 a=W1qU_X6G3J8A:10 a=Hz7IrDYlS0cA:10 a=Pwb] X-AnalysisOut: [duc0PQ3sA:10 a=ZSyFC8DWXvJwHtMM:21 a=unF7eO84AbMpC31l:21] X-Spam: [F=0.2000000000; CM=0.500; MH=0.500(2014012107); S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [76.164.174.81] Cc: "trevor.burbridge@bt.com" , "bs7652@att.com" , "lmap@ietf.org" Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 14:03:23 -0000 I think the key point was brought up by Charles - any solution that reports= after the fact that a subscriber's cap was exceeded due to traffic added b= y a test represents a failure condition. The subscriber cannot be disadvant= aged by testing conducted by either the operator or a third party.=20 As Charles also pointed out, there are multiple ways to prevent this. Some = are contractual, not technical (e.g., operator and third party agree to inc= rease data cap by the planned volume of test traffic + margin). -----Original Message----- From: lmap [mailto:lmap-bounces@ietf.org] On Behalf Of Bugenhagen, Michael = K Sent: Tuesday, January 21, 2014 8:51 AM To: Cook, Charles; philip.eardley@bt.com Cc: trevor.burbridge@bt.com; bs7652@att.com; lmap@ietf.org Subject: Re: [lmap] LMAP detection of "BB line service state" The key point I had in mind was that the test result really needs to have t= he "service state" information in the actual result to make this truly scal= able. Simple math wise In a single ISP per speed tier I would assume (and they have many speed tie= rs) 10k lines 12 measurements a day (minimum required granularity that lets one see a pea= k hour) 30 days per month. Yields - 360k test results.... makes a very large spreadsheet...=20 So scalable for any LMAP test IMHO means having sort-able key fields in = the test result. Maybe a really, really small ISP could do a manual processes but if you are= doing this in scale per market you'd have to hire a person to do that. So I don't like added costs on my ISP line - I'll vote for trying to kept t= his cheap vs. doing to make our jobs a bit easier by not addressing it. -----Original Message----- From: Cook, Charles Sent: Tuesday, January 21, 2014 7:36 AM To: philip.eardley@bt.com Cc: Bugenhagen, Michael K; lmap@ietf.org; trevor.burbridge@bt.com; bs7652@a= tt.com Subject: Re: LMAP detection of "BB line service state" Service Caps can be an issue. Since no one has a crystal ball regarding whether a subscriber will operate close to the service cap, and s= ince it would not be appropriate for a subscriber to be charged for a measu= rement test that they did not request, one possibility is that when the MA = registers with the Measurement Controller, information regarding the existe= nce of a Service Cap is provided. If there is a service cap, then the Meas= urement Controller simply does not schedule any bandwidth/throughput tests = with that MA.=20 If the service provider that owns the MA runs the test, then I agree that = the service provider can meter their own test and adjust billing to make th= e subscriber whole. If a third party must run a bandwidth/throughput test on an MA with a Servi= ce Cap, then somehow there needs to be a mechanism to meter the test and ch= arge the third party for their usage of the subscriber's allocation. Obvio= usly this would be much more complex. Another possibility is that the third-party could negotiate with the servic= e provider increase the service cap beyond what the customer has subscribed= to so that any tests that are run would not eat into the amount of bandwid= th that the customer contracted for with the service provider. However it is done, we need to ensure that the subscriber is whole. Charles On 1/21/2014 2:44 AM, philip.eardley@bt.com wrote: > Michael, > > I think this may depend on who's operating the measurement system. > > For an operator-controlled measurement system, then they should know what= contract the BB user is on and whether they're currently being capped. In = the light of this they can work out what tests they want done. The controll= er instructs the MA. I'm not sure how up-to-date the Controller can be kept= informed about dynamic capping (this info is going via OAM, which isn't ve= ry dynamic). > =20 > For a regulator-controlled measurement system, there is likely to be some= off-line way that the ISP tells the regulator what contract the BB user is= on - so that they can run sensible tests. After results collection, perhap= s the ISP could tell the regulator that user y had exceeded their cap on Ja= n 21st (so treat those results with caution). I think defining a real-time = interface about caps is unlikely. Partly technical (dynamic info, many diff= erent capping approaches) and partly commercial. > > For an end-user controlled measurement system, I think it would be=20 > quite useful for the end user (& their MA) to be able to confirm what=20 > contract they are on. Again I think defining a real-time interface=20 > about caps is difficult. I think it's something some ISPs might find=20 > it useful to tell a subscriber (eg via an email or webpage) but=20 > doubtful about standards or about real time (more likely a warning=20 > before it starts and, after a time lag, informing them it's happening=20 > or happened) > > So, rather than dynamically informing the MA about capping, I'd take a di= fferent approach. If, for a particular test, it really matters to understan= d about capping, then there should be a preliminary part of the test where = it sends some probe traffic in order to assess this.=20 > > Best wishes > phil > >> -----Original Message----- >> From: Bugenhagen, Michael K >> [mailto:Michael.K.Bugenhagen@centurylink.com] >> Sent: 20 January 2014 14:22 >> To: lmap@ietf.org; Eardley,PL,Philip,TUB8 R; Burbridge,T,Trevor,TUB8=20 >> R; Cook, Charles; 'bs7652@att.com' >> Subject: LMAP detection of "BB line service state" >> >> One thing that is key to the metric "NOT" producing false positive=20 >> results (negative types), is the ability of the MA to know the=20 >> "service state" of the broadband line. >> >> I haven't heard this discussion so I'll contribute - >> >> There are multiple types of ISP service "states" that dynamically=20 >> change depending upon the terms of service. >> >> Some of the more common ones - >> >> 1) Exceeding a cap results in a Bandwidth profile decrease >> - Many wireless carriers / SAT. providers have these in place (read=20 >> their websites). >> Some are "gentle" enforcements and may not be implemented right at=20 >> the threshold crossing, others are hard coded. >> >> Complications to this >> - Some ISP's have a buyout (pay to increase your cap and then you=20 >> get the full speed back, others don't but it rests on your billing=20 >> cycle, or possibly at the end of the month. >> >> Either way >> >> 1) Fair "service tests" are conducted against the terms of service >> - Provided the "terms of service" are made available / publically=20 >> know ... they shouldn't be part of performance testing. >> >> >> So the problem gets to - How does a MA know what the bandwidth state=20 >> the line is currently based on the "terms of service". >> >> Presumption - If we want this to scale, and have low cost then we=20 >> can't assume manual anything for this "learning" of state. >> >> Recommendation - the BBF liaison had a proposal to share that "state" >> information with the MA, which seems to be the easiest answer. >> Systemically build a function into the MA that "asks" >> what the line state information is. >> >> To do this we'd have to >> 1) add that step to the MA doing a test >> 2) Choose what protocols to use to have the MA ask via the home=20 >> network what the service state is to conduct the test. >> (current speed, service state, .... ) >> >> Cheers, >> Mike --=20 Charles Cook Principal Architect Network 5325 Zuni Street; Suite 224 Denver, CO 80221 Tel: 303.992.8952 Fax: 925.281.0662 charles.cook2@centurylink.com _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap From sharam.hakimi@exfo.com Tue Jan 21 06:31:13 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BF71C1A014D for ; Tue, 21 Jan 2014 06:31:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.235 X-Spam-Level: X-Spam-Status: No, score=-1.235 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6, RP_MATCHES_RCVD=-0.535] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x5Z5u9O2vBs7 for ; Tue, 21 Jan 2014 06:31:11 -0800 (PST) Received: from smtpinqc.exfo.com (smtpinqc.exfo.com [206.162.164.97]) by ietfa.amsl.com (Postfix) with ESMTP id 9B56B1A014E for ; Tue, 21 Jan 2014 06:31:10 -0800 (PST) Received: from spqcexc04.exfo.com ([172.16.48.171]) by smtpinqc.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 21 Jan 2014 09:31:09 -0500 Received: from spboexc01.exfo.com ([10.10.10.16]) by spqcexc04.exfo.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 21 Jan 2014 09:31:09 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Date: Tue, 21 Jan 2014 09:21:51 -0500 Message-ID: <084CDC75FEC1E640B60338273BEACDFA02ACAEB0@spboexc01.exfo.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [lmap] LMAP detection of "BB line service state" Thread-Index: AQHPFq/UTHqK/xoOiUOadd2ucHkDSpqPM83QgAAEMgA= References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> <52DE77DA.7070109@centurylink.com> From: "Sharam Hakimi" To: "KEN KO" , "Bugenhagen, Michael K" , "Cook, Charles" , X-OriginalArrivalTime: 21 Jan 2014 14:31:09.0363 (UTC) FILETIME=[6D2D9830:01CF16B5] Cc: trevor.burbridge@bt.com, bs7652@att.com, lmap@ietf.org Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 14:31:14 -0000 These are part of PRE and POST processing of the measurement task and data and MA does not need to know these details to perform measurements in my opinion. The task will be allocated based on the contract and the results would be matched against any contractual detail. =20 Thanks, Sharam -----Original Message----- From: lmap [mailto:lmap-bounces@ietf.org] On Behalf Of KEN KO Sent: Tuesday, January 21, 2014 9:01 AM To: Bugenhagen, Michael K; Cook, Charles; philip.eardley@bt.com Cc: trevor.burbridge@bt.com; bs7652@att.com; lmap@ietf.org Subject: Re: [lmap] LMAP detection of "BB line service state" I think the key point was brought up by Charles - any solution that reports after the fact that a subscriber's cap was exceeded due to traffic added by a test represents a failure condition. The subscriber cannot be disadvantaged by testing conducted by either the operator or a third party.=20 As Charles also pointed out, there are multiple ways to prevent this. Some are contractual, not technical (e.g., operator and third party agree to increase data cap by the planned volume of test traffic + margin). -----Original Message----- From: lmap [mailto:lmap-bounces@ietf.org] On Behalf Of Bugenhagen, Michael K Sent: Tuesday, January 21, 2014 8:51 AM To: Cook, Charles; philip.eardley@bt.com Cc: trevor.burbridge@bt.com; bs7652@att.com; lmap@ietf.org Subject: Re: [lmap] LMAP detection of "BB line service state" The key point I had in mind was that the test result really needs to have the "service state" information in the actual result to make this truly scalable. Simple math wise In a single ISP per speed tier I would assume (and they have many speed tiers) 10k lines 12 measurements a day (minimum required granularity that lets one see a peak hour) 30 days per month. Yields - 360k test results.... makes a very large spreadsheet...=20 So scalable for any LMAP test IMHO means having sort-able key fields in the test result. Maybe a really, really small ISP could do a manual processes but if you are doing this in scale per market you'd have to hire a person to do that. So I don't like added costs on my ISP line - I'll vote for trying to kept this cheap vs. doing to make our jobs a bit easier by not addressing it. -----Original Message----- From: Cook, Charles Sent: Tuesday, January 21, 2014 7:36 AM To: philip.eardley@bt.com Cc: Bugenhagen, Michael K; lmap@ietf.org; trevor.burbridge@bt.com; bs7652@att.com Subject: Re: LMAP detection of "BB line service state" Service Caps can be an issue. Since no one has a crystal ball regarding whether a subscriber will operate close to the service cap, and since it would not be appropriate for a subscriber to be charged for a measurement test that they did not request, one possibility is that when the MA registers with the Measurement Controller, information regarding the existence of a Service Cap is provided. If there is a service cap, then the Measurement Controller simply does not schedule any bandwidth/throughput tests with that MA.=20 If the service provider that owns the MA runs the test, then I agree that the service provider can meter their own test and adjust billing to make the subscriber whole. If a third party must run a bandwidth/throughput test on an MA with a Service Cap, then somehow there needs to be a mechanism to meter the test and charge the third party for their usage of the subscriber's allocation. Obviously this would be much more complex. Another possibility is that the third-party could negotiate with the service provider increase the service cap beyond what the customer has subscribed to so that any tests that are run would not eat into the amount of bandwidth that the customer contracted for with the service provider. However it is done, we need to ensure that the subscriber is whole. Charles On 1/21/2014 2:44 AM, philip.eardley@bt.com wrote: > Michael, > > I think this may depend on who's operating the measurement system. > > For an operator-controlled measurement system, then they should know what contract the BB user is on and whether they're currently being capped. In the light of this they can work out what tests they want done. The controller instructs the MA. I'm not sure how up-to-date the Controller can be kept informed about dynamic capping (this info is going via OAM, which isn't very dynamic). > =20 > For a regulator-controlled measurement system, there is likely to be some off-line way that the ISP tells the regulator what contract the BB user is on - so that they can run sensible tests. After results collection, perhaps the ISP could tell the regulator that user y had exceeded their cap on Jan 21st (so treat those results with caution). I think defining a real-time interface about caps is unlikely. Partly technical (dynamic info, many different capping approaches) and partly commercial. > > For an end-user controlled measurement system, I think it would be=20 > quite useful for the end user (& their MA) to be able to confirm what=20 > contract they are on. Again I think defining a real-time interface=20 > about caps is difficult. I think it's something some ISPs might find=20 > it useful to tell a subscriber (eg via an email or webpage) but=20 > doubtful about standards or about real time (more likely a warning=20 > before it starts and, after a time lag, informing them it's happening=20 > or happened) > > So, rather than dynamically informing the MA about capping, I'd take a different approach. If, for a particular test, it really matters to understand about capping, then there should be a preliminary part of the test where it sends some probe traffic in order to assess this.=20 > > Best wishes > phil > >> -----Original Message----- >> From: Bugenhagen, Michael K >> [mailto:Michael.K.Bugenhagen@centurylink.com] >> Sent: 20 January 2014 14:22 >> To: lmap@ietf.org; Eardley,PL,Philip,TUB8 R; Burbridge,T,Trevor,TUB8=20 >> R; Cook, Charles; 'bs7652@att.com' >> Subject: LMAP detection of "BB line service state" >> >> One thing that is key to the metric "NOT" producing false positive=20 >> results (negative types), is the ability of the MA to know the=20 >> "service state" of the broadband line. >> >> I haven't heard this discussion so I'll contribute - >> >> There are multiple types of ISP service "states" that dynamically=20 >> change depending upon the terms of service. >> >> Some of the more common ones - >> >> 1) Exceeding a cap results in a Bandwidth profile decrease >> - Many wireless carriers / SAT. providers have these in place (read=20 >> their websites). >> Some are "gentle" enforcements and may not be implemented right at=20 >> the threshold crossing, others are hard coded. >> >> Complications to this >> - Some ISP's have a buyout (pay to increase your cap and then you=20 >> get the full speed back, others don't but it rests on your billing=20 >> cycle, or possibly at the end of the month. >> >> Either way >> >> 1) Fair "service tests" are conducted against the terms of service >> - Provided the "terms of service" are made available / publically=20 >> know ... they shouldn't be part of performance testing. >> >> >> So the problem gets to - How does a MA know what the bandwidth state=20 >> the line is currently based on the "terms of service". >> >> Presumption - If we want this to scale, and have low cost then we=20 >> can't assume manual anything for this "learning" of state. >> >> Recommendation - the BBF liaison had a proposal to share that "state" >> information with the MA, which seems to be the easiest answer. >> Systemically build a function into the MA that "asks" >> what the line state information is. >> >> To do this we'd have to >> 1) add that step to the MA doing a test >> 2) Choose what protocols to use to have the MA ask via the home=20 >> network what the service state is to conduct the test. >> (current speed, service state, .... ) >> >> Cheers, >> Mike --=20 Charles Cook Principal Architect Network 5325 Zuni Street; Suite 224 Denver, CO 80221 Tel: 303.992.8952 Fax: 925.281.0662 charles.cook2@centurylink.com _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap From Michael.K.Bugenhagen@centurylink.com Tue Jan 21 06:45:12 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D6A521A014C for ; Tue, 21 Jan 2014 06:45:12 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_72=0.6, J_CHICKENPOX_91=0.6] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Rp5hpj-GJBxg for ; Tue, 21 Jan 2014 06:45:10 -0800 (PST) Received: from sudnp799.qwest.com (sudnp799.qwest.com [155.70.32.99]) by ietfa.amsl.com (Postfix) with ESMTP id AB03F1A014D for ; Tue, 21 Jan 2014 06:45:09 -0800 (PST) Received: from lxomavmpc030.qintra.com (emailout.qintra.com [151.117.207.30]) by sudnp799.qwest.com (8.14.4/8.14.4) with ESMTP id s0LEiwqG027707 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jan 2014 07:44:58 -0700 (MST) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id EE5191E00A8; Tue, 21 Jan 2014 08:44:52 -0600 (CST) Received: from suomp61i.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id C4F961E0069; Tue, 21 Jan 2014 08:44:52 -0600 (CST) Received: from suomp61i.qintra.com (localhost [127.0.0.1]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id s0LEiqc7010149; Tue, 21 Jan 2014 08:44:52 -0600 (CST) Received: from vodcwhubex501.ctl.intranet (vodcwhubex501.ctl.intranet [151.117.206.27]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id s0LEiqEA010136 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 21 Jan 2014 08:44:52 -0600 (CST) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex501.ctl.intranet ([151.117.206.27]) with mapi id 14.03.0158.001; Tue, 21 Jan 2014 08:44:51 -0600 From: "Bugenhagen, Michael K" To: "'Sharam Hakimi'" , "'KEN KO'" , "Cook, Charles" , "'philip.eardley@bt.com'" Thread-Topic: [lmap] LMAP detection of "BB line service state" Thread-Index: AQHPFq3Kc4NOS8IBa0+x7dIZfDqi/JqPM83QgAAEMgCAAAi8sA== Date: Tue, 21 Jan 2014 14:44:50 +0000 Message-ID: References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> <52DE77DA.7070109@centurylink.com> <084CDC75FEC1E640B60338273BEACDFA02ACAEB0@spboexc01.exfo.com> In-Reply-To: <084CDC75FEC1E640B60338273BEACDFA02ACAEB0@spboexc01.exfo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.8] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "'trevor.burbridge@bt.com'" , "'bs7652@att.com'" , "'lmap@ietf.org'" Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 14:45:13 -0000 Keep in mind beside testing - It's been suggested that the API or website to find out what you service= state is should be made global as a standard so people don't have to call = their ISP to find out what is going on.... Do you like your ISP's music on hold feature.... ? :) -----Original Message----- From: Sharam Hakimi [mailto:sharam.hakimi@exfo.com]=20 Sent: Tuesday, January 21, 2014 8:22 AM To: KEN KO; Bugenhagen, Michael K; Cook, Charles; philip.eardley@bt.com Cc: trevor.burbridge@bt.com; bs7652@att.com; lmap@ietf.org Subject: RE: [lmap] LMAP detection of "BB line service state" These are part of PRE and POST processing of the measurement task and data = and MA does not need to know these details to perform measurements in my op= inion. The task will be allocated based on the contract and the results wou= ld be matched against any contractual detail. =20 Thanks, Sharam -----Original Message----- From: lmap [mailto:lmap-bounces@ietf.org] On Behalf Of KEN KO Sent: Tuesday, January 21, 2014 9:01 AM To: Bugenhagen, Michael K; Cook, Charles; philip.eardley@bt.com Cc: trevor.burbridge@bt.com; bs7652@att.com; lmap@ietf.org Subject: Re: [lmap] LMAP detection of "BB line service state" I think the key point was brought up by Charles - any solution that reports= after the fact that a subscriber's cap was exceeded due to traffic added b= y a test represents a failure condition. The subscriber cannot be disadvant= aged by testing conducted by either the operator or a third party.=20 As Charles also pointed out, there are multiple ways to prevent this. Some are contractual, not technical (e.g., operator and third party agree t= o increase data cap by the planned volume of test traffic + margin). -----Original Message----- From: lmap [mailto:lmap-bounces@ietf.org] On Behalf Of Bugenhagen, Michael = K Sent: Tuesday, January 21, 2014 8:51 AM To: Cook, Charles; philip.eardley@bt.com Cc: trevor.burbridge@bt.com; bs7652@att.com; lmap@ietf.org Subject: Re: [lmap] LMAP detection of "BB line service state" The key point I had in mind was that the test result really needs to have t= he "service state" information in the actual result to make this truly scal= able. Simple math wise In a single ISP per speed tier I would assume (and they have many speed tiers) 10k lines 12 measurements a day (minimum required granularity that lets one see a pea= k hour) 30 days per month. Yields - 360k test results.... makes a very large spreadsheet...=20 So scalable for any LMAP test IMHO means having sort-able key fields in = the test result. Maybe a really, really small ISP could do a manual processes but if you are= doing this in scale per market you'd have to hire a person to do that. So I don't like added costs on my ISP line - I'll vote for trying to kept t= his cheap vs. doing to make our jobs a bit easier by not addressing it. -----Original Message----- From: Cook, Charles Sent: Tuesday, January 21, 2014 7:36 AM To: philip.eardley@bt.com Cc: Bugenhagen, Michael K; lmap@ietf.org; trevor.burbridge@bt.com; bs7652@a= tt.com Subject: Re: LMAP detection of "BB line service state" Service Caps can be an issue. Since no one has a crystal ball regarding whether a subscriber will operate close to the service cap, and s= ince it would not be appropriate for a subscriber to be charged for a measu= rement test that they did not request, one possibility is that when the MA = registers with the Measurement Controller, information regarding the existe= nce of a Service Cap is provided. If there is a service cap, then the Meas= urement Controller simply does not schedule any bandwidth/throughput tests = with that MA.=20 If the service provider that owns the MA runs the test, then I agree that = the service provider can meter their own test and adjust billing to make th= e subscriber whole. If a third party must run a bandwidth/throughput test on an MA with a Servi= ce Cap, then somehow there needs to be a mechanism to meter the test and ch= arge the third party for their usage of the subscriber's allocation. Obvio= usly this would be much more complex. Another possibility is that the third-party could negotiate with the servic= e provider increase the service cap beyond what the customer has subscribed= to so that any tests that are run would not eat into the amount of bandwid= th that the customer contracted for with the service provider. However it is done, we need to ensure that the subscriber is whole. Charles On 1/21/2014 2:44 AM, philip.eardley@bt.com wrote: > Michael, > > I think this may depend on who's operating the measurement system. > > For an operator-controlled measurement system, then they should know what contract the BB user is on and whether they're currently being capped.= In the light of this they can work out what tests they want done. The cont= roller instructs the MA. I'm not sure how up-to-date the Controller can be = kept informed about dynamic capping (this info is going via OAM, which isn'= t very dynamic). > =20 > For a regulator-controlled measurement system, there is likely to be some off-line way that the ISP tells the regulator what contract the BB use= r is on - so that they can run sensible tests. After results collection, pe= rhaps the ISP could tell the regulator that user y had exceeded their cap o= n Jan 21st (so treat those results with caution). I think defining a real-t= ime interface about caps is unlikely. Partly technical (dynamic info, many = different capping approaches) and partly commercial. > > For an end-user controlled measurement system, I think it would be=20 > quite useful for the end user (& their MA) to be able to confirm what=20 > contract they are on. Again I think defining a real-time interface=20 > about caps is difficult. I think it's something some ISPs might find=20 > it useful to tell a subscriber (eg via an email or webpage) but=20 > doubtful about standards or about real time (more likely a warning=20 > before it starts and, after a time lag, informing them it's happening=20 > or happened) > > So, rather than dynamically informing the MA about capping, I'd take a different approach. If, for a particular test, it really matters to underst= and about capping, then there should be a preliminary part of the test wher= e it sends some probe traffic in order to assess this.=20 > > Best wishes > phil > >> -----Original Message----- >> From: Bugenhagen, Michael K >> [mailto:Michael.K.Bugenhagen@centurylink.com] >> Sent: 20 January 2014 14:22 >> To: lmap@ietf.org; Eardley,PL,Philip,TUB8 R; Burbridge,T,Trevor,TUB8=20 >> R; Cook, Charles; 'bs7652@att.com' >> Subject: LMAP detection of "BB line service state" >> >> One thing that is key to the metric "NOT" producing false positive=20 >> results (negative types), is the ability of the MA to know the=20 >> "service state" of the broadband line. >> >> I haven't heard this discussion so I'll contribute - >> >> There are multiple types of ISP service "states" that dynamically=20 >> change depending upon the terms of service. >> >> Some of the more common ones - >> >> 1) Exceeding a cap results in a Bandwidth profile decrease >> - Many wireless carriers / SAT. providers have these in place (read=20 >> their websites). >> Some are "gentle" enforcements and may not be implemented right at=20 >> the threshold crossing, others are hard coded. >> >> Complications to this >> - Some ISP's have a buyout (pay to increase your cap and then you=20 >> get the full speed back, others don't but it rests on your billing=20 >> cycle, or possibly at the end of the month. >> >> Either way >> >> 1) Fair "service tests" are conducted against the terms of service >> - Provided the "terms of service" are made available / publically=20 >> know ... they shouldn't be part of performance testing. >> >> >> So the problem gets to - How does a MA know what the bandwidth state=20 >> the line is currently based on the "terms of service". >> >> Presumption - If we want this to scale, and have low cost then we=20 >> can't assume manual anything for this "learning" of state. >> >> Recommendation - the BBF liaison had a proposal to share that "state" >> information with the MA, which seems to be the easiest answer. >> Systemically build a function into the MA that "asks" >> what the line state information is. >> >> To do this we'd have to >> 1) add that step to the MA doing a test >> 2) Choose what protocols to use to have the MA ask via the home=20 >> network what the service state is to conduct the test. >> (current speed, service state, .... ) >> >> Cheers, >> Mike --=20 Charles Cook Principal Architect Network 5325 Zuni Street; Suite 224 Denver, CO 80221 Tel: 303.992.8952 Fax: 925.281.0662 charles.cook2@centurylink.com _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap _______________________________________________ lmap mailing list lmap@ietf.org https://www.ietf.org/mailman/listinfo/lmap From internet-drafts@ietf.org Tue Jan 21 06:56:51 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 376CD1A0167; Tue, 21 Jan 2014 06:56:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4C5-5QWX-wG8; Tue, 21 Jan 2014 06:56:49 -0800 (PST) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id E1A391A00E4; Tue, 21 Jan 2014 06:56:49 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.90.p2 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20140121145649.5390.53341.idtracker@ietfa.amsl.com> Date: Tue, 21 Jan 2014 06:56:49 -0800 Cc: lmap@ietf.org Subject: [lmap] I-D Action: draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 14:56:51 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the Large-Scale Measurement of Broadband Perf= ormance Working Group of the IETF. Title : A framework for large-scale measurement platforms= (LMAP) Authors : Philip Eardley Al Morton Marcelo Bagnulo Trevor Burbridge Paul Aitken Aamer Akhter Filename : draft-ietf-lmap-framework-03.txt Pages : 45 Date : 2014-01-21 Abstract: Measuring broadband service on a large scale requires a description of the logical architecture and standardisation of the key protocols that coordinate interactions between the components. The document presents an overall framework for large-scale measurements. It also defines terminology for LMAP (large-scale measurement platforms). The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-lmap-framework/ There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-lmap-framework-03 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-lmap-framework-03 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From philip.eardley@bt.com Tue Jan 21 07:00:24 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 073381A0167 for ; Tue, 21 Jan 2014 07:00:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i8svOrNEAcKx for ; Tue, 21 Jan 2014 07:00:19 -0800 (PST) Received: from smtpe1.intersmtp.com (smtpe1.intersmtp.com [62.239.224.234]) by ietfa.amsl.com (Postfix) with ESMTP id 903221A014A for ; Tue, 21 Jan 2014 07:00:19 -0800 (PST) Received: from EVMHT68-UKRD.domain1.systemhost.net (10.36.3.105) by RDW083A005ED61.bt.com (10.187.98.10) with Microsoft SMTP Server (TLS) id 14.3.169.1; Tue, 21 Jan 2014 15:00:22 +0000 Received: from EMV67-UKRD.domain1.systemhost.net ([169.254.1.39]) by EVMHT68-UKRD.domain1.systemhost.net ([10.36.3.105]) with mapi; Tue, 21 Jan 2014 15:00:18 +0000 From: To: Date: Tue, 21 Jan 2014 15:00:16 +0000 Thread-Topic: New Version Notification for draft-ietf-lmap-framework-03.txt Thread-Index: Ac8WuQcQ5fE68h3gS3O4ZHX8Q/xVrgAABpJA Message-ID: References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> In-Reply-To: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 15:00:24 -0000 V2Ugc3VibWl0dGVkIGEgbmV3IHZlcnNpb24gb2YgdGhlIGZyYW1ld29yayBkb2N1bWVudC4NCg0K aHR0cDovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0Zi1sbWFwLWZyYW1ld29yayANCg0K SGVyZSdzIGEgc3VtbWFyeSBvZiB0aGUgY2hhbmdlczoNCiAgIG8gIGFsaWdubWVudCB3aXRoIHRo ZSBJbmZvcm1hdGlvbiBNb2RlbA0KICAgICAgW0ktRC5idXJicmlkZ2UtbG1hcC1pbmZvcm1hdGlv bi1tb2RlbF0gYXMgdGhpcyBpcyBhZ3JlZWQgYXMgYSBXRw0KICAgICAgZG9jdW1lbnQNCg0KICAg byAgT25lLW9mZiBhbmQgcGVyaW9kaWMgTWVhc3VyZW1lbnQgU2NoZWR1bGVzIGFyZSBrZXB0IHNl cGFyYXRlLCBzbw0KICAgICAgdGhhdCB0aGV5IGNhbiBiZSB1cGRhdGVkIGluZGVwZW5kZW50bHkN Cg0KICAgbyAgTWVhc3VyZW1lbnQgU3VwcHJlc3Npb24gaW4gYSBzZXBhcmF0ZSBzdWItc2VjdGlv bi4gIENhbiBub3cNCiAgICAgIG9wdGlvbmFsbHkgaW5jbHVkZSBwYXJ0aWN1bGFyIE1lYXN1cmVt ZW50IFRhc2tzICYvb3IgU2NoZWR1bGVzIHRvDQogICAgICBzdXBwcmVzcywgYW5kIHN0YXJ0L3N0 b3AgdGltZQ0KDQogICBvICBmb3IgY2xhcml0eSwgY29uY2VwdCBvZiBDaGFubmVsIHNwbGl0IGlu dG8gQ29udHJvbCwgUmVwb3J0IGFuZCBNQS0NCiAgICAgIHRvLUNvbnRyb2xsZXIgQ2hhbm5lbHMN Cg0KICAgbyAgbnVtZXJvdXMgZWRpdG9yaWFsIGNoYW5nZXMsIG1haW5seSBhcmlzaW5nIGZyb20g YSB2ZXJ5IGRldGFpbGVkDQogICAgICByZXZpZXcgYnkgQ2hhcmxlcyBDb29rDQoNCmJlc3Qgd2lz aGVzLA0KcGhpbCwgYWwsIG1hcmNlbG8sIHRyZXZvciwgcGF1bCwgYWFtZXINCg0K From bs7652@att.com Tue Jan 21 07:00:25 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A32D01A014A for ; Tue, 21 Jan 2014 07:00:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.735 X-Spam-Level: X-Spam-Status: No, score=-4.735 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.535] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b7x99VyVH6DN for ; Tue, 21 Jan 2014 07:00:20 -0800 (PST) Received: from nbfkord-smmo06.seg.att.com (nbfkord-smmo06.seg.att.com [209.65.160.94]) by ietfa.amsl.com (Postfix) with ESMTP id 7E0881A014C for ; Tue, 21 Jan 2014 07:00:20 -0800 (PST) Received: from unknown [144.160.229.23] (EHLO alpi154.enaf.aldc.att.com) by nbfkord-smmo06.seg.att.com(mxl_mta-7.2.1-0) with ESMTP id 48b8ed25.2ba13422c940.4440018.00-2492.12266690.nbfkord-smmo06.seg.att.com (envelope-from ); Tue, 21 Jan 2014 15:00:20 +0000 (UTC) X-MXL-Hash: 52de8b846503faae-ea77ed7118a26ac86cc2a9f2dbe93c1322cadff5 Received: from unknown [144.160.229.23] (EHLO alpi154.enaf.aldc.att.com) by nbfkord-smmo06.seg.att.com(mxl_mta-7.2.1-0) over TLS secured channel with ESMTP id 55b8ed25.0.4439528.00-2114.12265337.nbfkord-smmo06.seg.att.com (envelope-from ); Tue, 21 Jan 2014 14:59:34 +0000 (UTC) X-MXL-Hash: 52de8b564fe85d1f-2eceae013a867a7b323e385edcdff376739e93e8 Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi154.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id s0LExWI5028636; Tue, 21 Jan 2014 09:59:33 -0500 Received: from alpi131.aldc.att.com (alpi131.aldc.att.com [130.8.218.69]) by alpi154.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id s0LExQhX028573 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jan 2014 09:59:27 -0500 Received: from GAALPA1MSGHUB9F.ITServices.sbc.com (GAALPA1MSGHUB9F.itservices.sbc.com [130.8.36.92]) by alpi131.aldc.att.com (RSA Interceptor); Tue, 21 Jan 2014 14:59:06 GMT Received: from GAALPA1MSGUSR9L.ITServices.sbc.com ([130.8.36.69]) by GAALPA1MSGHUB9F.ITServices.sbc.com ([130.8.36.92]) with mapi id 14.03.0174.001; Tue, 21 Jan 2014 09:59:06 -0500 From: "STARK, BARBARA H" To: KEN KO , "Bugenhagen, Michael K" , "Cook, Charles" , "philip.eardley@bt.com" Thread-Topic: LMAP detection of "BB line service state" Thread-Index: AQHPFq3UAq7jEDlmvUyq2+ds0U8dXpqPhgcAgAAC/ID//64VQA== Date: Tue, 21 Jan 2014 14:59:05 +0000 Message-ID: <2D09D61DDFA73D4C884805CC7865E611303D3D4C@GAALPA1MSGUSR9L.ITServices.sbc.com> References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> <52DE77DA.7070109@centurylink.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.61.166.199] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-RSA-Inspected: yes X-RSA-Classifications: public X-AnalysisOut: [v=2.0 cv=MLLXbrll c=1 sm=1 a=VXHOiMMwGAwA+y4G3/O+aw==:17 a] X-AnalysisOut: [=ywCGDIam2kAA:10 a=ofMgfj31e3cA:10 a=-45HWa1Ubl0A:10 a=BLc] X-AnalysisOut: [eEmwcHowA:10 a=kj9zAlcOel0A:10 a=zQP7CpKOAAAA:8 a=XIqpo32R] X-AnalysisOut: [AAAA:8 a=MnYPV9Kup-sA:10 a=ccMQh9G98-qaW-izp-wA:9 a=CjuIK1] X-AnalysisOut: [q_8ugA:10] X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [144.160.229.23] Cc: "trevor.burbridge@bt.com" , "lmap@ietf.org" Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 15:00:25 -0000 We keep coming back to this, over and over again. My opinion on dealing wit= h this hasn't changed since the last time we discussed it. There are several components to this: 1. Information model to include in a report all aspects of the environment = a test was conducted under. This includes every little aspect of the access= service (its states, its configuration, etc.) that we can dream of. I've b= een trying to do this dreaming of access service aspects in BBF and not IET= F, because (IMO) the competency to understand these aspects of access servi= ce exist much more in BBF than in IETF. That's why in my last contribution = to BBF, I greatly expanded this list in the current BBF draft. My hope is t= hat we can liaise that draft to IETF soon and suggest that lmap include the= se aspects in their data model, as info that can be included when reporting= measurement results. But I don't think that IETF is the right place to arg= ue about what all the right aspects are. Like I said, I think the competenc= y to characterize access services exists more in BBF. 2. Architectural framework for how to get the info to wherever it needs to = go. We've discussed (and I thought agreed) that there are various ways this= can be architected. As Phil described, some ISPs might choose to do send t= he data from one of their backend systems to another. Or from their backend= systems to regulator or 3rd party systems. I thought we had agreed that th= is was a valid approach that is allowed but not mandated. I also thought we= agreed that lmap (and BBF) will provide no guidance on how to do this (it'= s out-of-scope for lmap). We also discussed that some ISPs might choose to = send this info to their managed device, through the management interface. A= gain, this would be allowed but not mandated. The management interface is o= ut-of-scope for lmap. I'm quite sure we agreed on that. But in BBF we have = a management protocol called TR-069 whose data model we know how to expand = to include the ability to supply this info to an RG. OMA-DM could do the sa= me thing for devices managed through the OMA-DM protocol. But all of this i= s outside the scope of IETF, and outside IETF's sphere of influence. So dis= cussing this sort of stuff in IETF (where the management interface for lmap= has been stated to be out-of-scope) is a waste of time. The discussion to = expand the data models for these protocols needs to happen in BBF and OMA-D= M. 3. Getting the info to devices not managed by the ISP and where there will = be no backend system interfaces. I'm still trying to get around to that con= tribution to homenet for using DNS-SD and mDNS to get info from an ISP-mana= ged device to another device in the home network. But homenet isn't lmap. O= f course, this would only help where there exists an ISP-managed device in = the home network that can be supplied the info by some mechanism like TR-06= 9 or OMA-DM. It does nothing for the case where no ISP-managed device exist= s in the home network. There are ways to do this, but I thought we'd agreed= that we weren't going to try to tackle "standardizing" or creating a super= -simple solution for that use case right now, because we have enough on our= plates. Which leads me right back to where I was before, regarding my personal to-d= o list: 1. Shore up service attributes in BBF 2. Get the BBF work liaised to IETF and try to get the service attributes i= ncluded as (optional) "report" elements in the lmap information model. [The= y are optional because some ISPs might do this association in the backend. = They need to be in the report part of the information model because some IS= Ps might want to send the info to the MA (using the out-of-scope management= protocol) for attachment to the reported data.] 3. Do the homenet contribution for DNS-SD / mDNS. 4. Get the service attributes in the TR-069 data model both as info to be i= ncluded with reported data, and as info that can be provided to the managed= device to accomplish this association. [OMA-DM is not on my personal to-do= list.] So Mike and others: What exactly are you wanting from lmap right now? Do yo= u want to move the discussion regarding defining service attributes to IETF= and out of BBF? Do you want to try to put the management protocol and inte= rface back in scope in lmap? Do you want lmap to work on this DNS-SD/mDNS t= hing that I think belongs in homenet and that really needs to have the cont= ribution written describing it (and I'm really trying to get to it for Lond= on)? Do you want to forbid ISPs from being allowed to send this info around= using their backend systems? Do you want to try to create a standardized s= olution for the case where there is no ISP-managed device in the home netwo= rk? I'm really not understanding the reason for bringing this up in lmap r= ight now. Barbara From bs7652@att.com Tue Jan 21 07:19:51 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D34B51A0150 for ; Tue, 21 Jan 2014 07:19:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.735 X-Spam-Level: X-Spam-Status: No, score=-4.735 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.535] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SzBPqnvrrA5Z for ; Tue, 21 Jan 2014 07:19:50 -0800 (PST) Received: from nbfkord-smmo05.seg.att.com (nbfkord-smmo05.seg.att.com [209.65.160.92]) by ietfa.amsl.com (Postfix) with ESMTP id 5D5731A0121 for ; Tue, 21 Jan 2014 07:19:50 -0800 (PST) Received: from unknown [144.160.229.23] (EHLO alpi154.enaf.aldc.att.com) by nbfkord-smmo05.seg.att.com(mxl_mta-7.2.1-0) with ESMTP id 6109ed25.2b5ab225c940.4444968.00-2428.12305455.nbfkord-smmo05.seg.att.com (envelope-from ); Tue, 21 Jan 2014 15:19:50 +0000 (UTC) X-MXL-Hash: 52de901620324e05-3362a7d3404fdd895e76c4b025bcfda04e055b9a Received: from unknown [144.160.229.23] (EHLO alpi154.enaf.aldc.att.com) by nbfkord-smmo05.seg.att.com(mxl_mta-7.2.1-0) over TLS secured channel with ESMTP id 3ef8ed25.0.4444399.00-2276.12303847.nbfkord-smmo05.seg.att.com (envelope-from ); Tue, 21 Jan 2014 15:19:00 +0000 (UTC) X-MXL-Hash: 52de8fe431e2582a-fa3f8e38a9341f73319fb4ef802987604904eed3 Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi154.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id s0LFIvIc024980; Tue, 21 Jan 2014 10:18:59 -0500 Received: from alpi132.aldc.att.com (alpi132.aldc.att.com [130.8.217.2]) by alpi154.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id s0LFIr2o024874 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jan 2014 10:18:54 -0500 Received: from GAALPA1MSGHUB9D.ITServices.sbc.com (GAALPA1MSGHUB9D.itservices.sbc.com [130.8.36.90]) by alpi132.aldc.att.com (RSA Interceptor); Tue, 21 Jan 2014 15:18:39 GMT Received: from GAALPA1MSGUSR9L.ITServices.sbc.com ([130.8.36.69]) by GAALPA1MSGHUB9D.ITServices.sbc.com ([130.8.36.90]) with mapi id 14.03.0174.001; Tue, 21 Jan 2014 10:18:39 -0500 From: "STARK, BARBARA H" To: "Bugenhagen, Michael K" , "'Sharam Hakimi'" , "'KEN KO'" , "Cook, Charles" , "'philip.eardley@bt.com'" Thread-Topic: [lmap] LMAP detection of "BB line service state" Thread-Index: AQHPFq3UAq7jEDlmvUyq2+ds0U8dXpqPM83QgAAEMgCAAF0jAP//sDpg Date: Tue, 21 Jan 2014 15:18:37 +0000 Message-ID: <2D09D61DDFA73D4C884805CC7865E611303D3D82@GAALPA1MSGUSR9L.ITServices.sbc.com> References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> <52DE77DA.7070109@centurylink.com> <084CDC75FEC1E640B60338273BEACDFA02ACAEB0@spboexc01.exfo.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.61.166.199] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-RSA-Inspected: yes X-RSA-Classifications: public X-AnalysisOut: [v=2.0 cv=Y/9PRGiN c=1 sm=1 a=VXHOiMMwGAwA+y4G3/O+aw==:17 a] X-AnalysisOut: [=7z528Rzh2MIA:10 a=ofMgfj31e3cA:10 a=-45HWa1Ubl0A:10 a=BLc] X-AnalysisOut: [eEmwcHowA:10 a=kj9zAlcOel0A:10 a=zQP7CpKOAAAA:8 a=XIqpo32R] X-AnalysisOut: [AAAA:8 a=jVcuj6UKxdgA:10 a=8DlFnhS3aAD3NAjqOqEA:9 a=CjuIK1] X-AnalysisOut: [q_8ugA:10] X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010122901)] X-MAIL-FROM: X-SOURCE-IP: [144.160.229.23] Cc: "'trevor.burbridge@bt.com'" , "'lmap@ietf.org'" Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 15:19:52 -0000 > Keep in mind beside testing - >=20 > It's been suggested that the API or website to find out what you servi= ce > state is should be made global as a standard so people don't have to call= their > ISP to find out what is going on.... >=20 >=20 > Do you like your ISP's music on hold feature.... ? >=20 > :) I just sent a much longer message, but I'll send an excerpt to respond to t= his.=20 As I mentioned in the other email, I don't think lmap (IETF) is the right p= lace to discuss defining service attributes. I agree that a website is possible for providing this info. I don't think l= map is the right place to discuss this. I think it's out-of-scope for lmap.= I think that once the service attributes are defined (and I want to do thi= s in BBF where competency exists for understanding service attributes), and= the xml for providing this info via TR-059 has been specified, and the xml= for that homenet contribution that I keep meaning to get to exists, that p= osting this xml on a website is so simple that I don't need lmap to tell me= how. Once xml of service attributes is published somewhere, I know exactly= how to create a website that people can log in to or have their MA device = log into to read the xml. I don't need lmap to do anything with this. I nee= d lmap to stick to what they've already chartered themselves to do. Mike, I have no clue what specifically it is that you are wanting lmap to d= o at this time. You're aware of the work we're doing in BBF. And I recently= put a lot of effort into expanding the service attribute section in WT-304= . Maybe you could provide comments against that, in BBF. I've said I want t= o do a homenet contribution to have a DNS-SD / mDNS way to get service attr= ibutes from an ISP-managed device in the home network to some other device = in the home network. But sending emails to lmap isn't going to make my cont= ribution to homenet happen any quicker. Barbara From Michael.K.Bugenhagen@centurylink.com Tue Jan 21 07:58:00 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0AB541A03DB for ; Tue, 21 Jan 2014 07:58:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3wSncXd0R8Ip for ; Tue, 21 Jan 2014 07:57:58 -0800 (PST) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id E13031A0346 for ; Tue, 21 Jan 2014 07:57:56 -0800 (PST) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id s0LFvhMa002214 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jan 2014 09:57:43 -0600 (CST) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 6EAF31E007E; Tue, 21 Jan 2014 09:57:38 -0600 (CST) Received: from sudnp797.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id 3CB2F1E0078; Tue, 21 Jan 2014 09:57:38 -0600 (CST) Received: from sudnp797.qintra.com (localhost [127.0.0.1]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id s0LFvb0D014178; Tue, 21 Jan 2014 08:57:37 -0700 (MST) Received: from vodcwhubex502.ctl.intranet (vodcwhubex502.ctl.intranet [151.117.206.28]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id s0LFvbbw014166 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 21 Jan 2014 08:57:37 -0700 (MST) Received: from PODCWMBXEX505.ctl.intranet ([fe80::f87e:fe44:ad72:b610]) by vodcwhubex502.ctl.intranet ([2002:9775:ce1c::9775:ce1c]) with mapi id 14.03.0158.001; Tue, 21 Jan 2014 09:57:37 -0600 From: "Bugenhagen, Michael K" To: "'STARK, BARBARA H'" , "'KEN KO'" , "Cook, Charles" , "'philip.eardley@bt.com'" Thread-Topic: LMAP detection of "BB line service state" Thread-Index: AQHPFq3Kc4NOS8IBa0+x7dIZfDqi/JqPMBoAgABprYCAABAagP//qk8Q Date: Tue, 21 Jan 2014 15:57:36 +0000 Message-ID: References: <20131111182839.GA21454@idrathernotsay.com> <20131111204218.GB770@elstar.local> <20131112091731.GA2503@elstar.local> <52DE77DA.7070109@centurylink.com> <2D09D61DDFA73D4C884805CC7865E611303D3D4C@GAALPA1MSGUSR9L.ITServices.sbc.com> In-Reply-To: <2D09D61DDFA73D4C884805CC7865E611303D3D4C@GAALPA1MSGUSR9L.ITServices.sbc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [151.117.206.8] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "'trevor.burbridge@bt.com'" , "'lmap@ietf.org'" Subject: Re: [lmap] LMAP detection of "BB line service state" X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 15:58:00 -0000 >So Mike and others: What exactly are you wanting from lmap right now? Do y= ou want to move the discussion regarding defining service attributes to IET= F and out of BBF? Do you want to try to put the management protocol and int= erface >back in scope in lmap? I would like the interface approach. Why - it's future proof, and by this I mean "cloud proof," or more correctl= y Cloud Framework aligned. (aka it aligns with NFV frameworks) What most people don't realize about Cloud is that it's a framework - NOT= an architecture. Cloud requires a Common information model about infrastructure - like a API= at a location - So if we identify some interface when Cloud comes along it can be identifi= ed as part of the Infrastructure Layer at that location (the interface / AP= I that is). But then again I don't think there is a right now given this discussion has= been going on for almost a year :) =20 =20 -----Original Message----- From: STARK, BARBARA H [mailto:bs7652@att.com]=20 Sent: Tuesday, January 21, 2014 8:59 AM To: KEN KO; Bugenhagen, Michael K; Cook, Charles; philip.eardley@bt.com Cc: trevor.burbridge@bt.com; lmap@ietf.org Subject: RE: LMAP detection of "BB line service state" We keep coming back to this, over and over again. My opinion on dealing wit= h this hasn't changed since the last time we discussed it. There are several components to this: 1. Information model to include in a report all aspects of the environment = a test was conducted under. This includes every little aspect of the access= service (its states, its configuration, etc.) that we can dream of. I've b= een trying to do this dreaming of access service aspects in BBF and not IET= F, because (IMO) the competency to understand these aspects of access servi= ce exist much more in BBF than in IETF. That's why in my last contribution = to BBF, I greatly expanded this list in the current BBF draft. My hope is t= hat we can liaise that draft to IETF soon and suggest that lmap include the= se aspects in their data model, as info that can be included when reporting= measurement results. But I don't think that IETF is the right place to arg= ue about what all the right aspects are. Like I said, I think the competenc= y to characterize access services exists more in BBF. 2. Architectural framework for how to get the info to wherever it needs to = go. We've discussed (and I thought agreed) that there are various ways this= can be architected. As Phil described, some ISPs might choose to do send t= he data from one of their backend systems to another. Or from their backend= systems to regulator or 3rd party systems. I thought we had agreed that th= is was a valid approach that is allowed but not mandated. I also thought we= agreed that lmap (and BBF) will provide no guidance on how to do this (it'= s out-of-scope for lmap). We also discussed that some ISPs might choose to = send this info to their managed device, through the management interface. A= gain, this would be allowed but not mandated. The management interface is o= ut-of-scope for lmap. I'm quite sure we agreed on that. But in BBF we have = a management protocol called TR-069 whose data model we know how to expand = to include the ability to supply this info to an RG. OMA-DM could do the sa= me thing for devices managed through the OMA-DM protocol. But all of this i= s outside the scope of IETF, and outside IETF's sphere of influence. So dis= cussing this sort of stuff in IETF (where the management interface for lmap= has been stated to be out-of-scope) is a waste of time. The discussion to = expand the data models for these protocols needs to happen in BBF and OMA-D= M. 3. Getting the info to devices not managed by the ISP and where there will = be no backend system interfaces. I'm still trying to get around to that con= tribution to homenet for using DNS-SD and mDNS to get info from an ISP-mana= ged device to another device in the home network. But homenet isn't lmap. O= f course, this would only help where there exists an ISP-managed device in = the home network that can be supplied the info by some mechanism like TR-06= 9 or OMA-DM. It does nothing for the case where no ISP-managed device exist= s in the home network. There are ways to do this, but I thought we'd agreed= that we weren't going to try to tackle "standardizing" or creating a super= -simple solution for that use case right now, because we have enough on our= plates. Which leads me right back to where I was before, regarding my personal to-d= o list: 1. Shore up service attributes in BBF 2. Get the BBF work liaised to IETF and try to get the service attributes i= ncluded as (optional) "report" elements in the lmap information model. [The= y are optional because some ISPs might do this association in the backend. = They need to be in the report part of the information model because some IS= Ps might want to send the info to the MA (using the out-of-scope management= protocol) for attachment to the reported data.] 3. Do the homenet contribu= tion for DNS-SD / mDNS. 4. Get the service attributes in the TR-069 data model both as info to be i= ncluded with reported data, and as info that can be provided to the managed= device to accomplish this association. [OMA-DM is not on my personal to-do= list.] So Mike and others: What exactly are you wanting from lmap right now? Do yo= u want to move the discussion regarding defining service attributes to IETF= and out of BBF? Do you want to try to put the management protocol and inte= rface back in scope in lmap? Do you want lmap to work on this DNS-SD/mDNS t= hing that I think belongs in homenet and that really needs to have the cont= ribution written describing it (and I'm really trying to get to it for Lond= on)? Do you want to forbid ISPs from being allowed to send this info around= using their backend systems? Do you want to try to create a standardized s= olution for the case where there is no ISP-managed device in the home netwo= rk? I'm really not understanding the reason for bringing this up in lmap r= ight now. Barbara From dromasca@avaya.com Wed Jan 22 02:32:56 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9FD7D1A0379 for ; Wed, 22 Jan 2014 02:32:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.435 X-Spam-Level: X-Spam-Status: No, score=-2.435 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.535] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HMLymAmATKk5 for ; Wed, 22 Jan 2014 02:32:55 -0800 (PST) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id ED60D1A02C0 for ; Wed, 22 Jan 2014 02:32:54 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ah0FAP6c31LGmAcV/2dsb2JhbABbgmohOFa7W4ERFnSCJQEBAQEDAQEBD1wJDgQCAQYCDQQEAQELHQcnCxQIAQgBAQQBEggah2MBDIkgkTCmexeOSzgGgx6BFASZVIU9iymDLYIq X-IronPort-AV: E=Sophos;i="4.95,698,1384318800"; d="scan'208";a="46137677" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 22 Jan 2014 05:32:53 -0500 Received: from unknown (HELO AZ-FFEXHC03.global.avaya.com) ([135.64.58.13]) by co300216-co-erhwest-out.avaya.com with ESMTP/TLS/AES128-SHA; 22 Jan 2014 05:22:47 -0500 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC03.global.avaya.com ([135.64.58.13]) with mapi id 14.03.0158.001; Wed, 22 Jan 2014 05:32:52 -0500 From: "Romascanu, Dan (Dan)" To: "philip.eardley@bt.com" , "lmap@ietf.org" Thread-Topic: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt Thread-Index: Ac8WuQcQ5fE68h3gS3O4ZHX8Q/xVrgAABpJAACjxEZA= Date: Wed, 22 Jan 2014 10:32:52 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA243D468F@AZ-FFEXMB04.global.avaya.com> References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: text/plain; charset="windows-1255" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 10:32:56 -0000 Phil and co-authors, Thanks for the submission and for the good work until now.=20 Question - is this version ready for WGLC? If not, what's left to be done?= =20 Everybody's opinions are welcome.=20 Regards, Dan > -----Original Message----- > From: lmap [mailto:lmap-bounces@ietf.org] On Behalf Of > philip.eardley@bt.com > Sent: =E9=E5=ED=A0=E2 21 =E9=F0=E5=E0=F8 2014 17:00 > To: lmap@ietf.org > Subject: [lmap] FW: New Version Notification for draft-ietf-lmap-framewor= k- > 03.txt >=20 > We submitted a new version of the framework document. >=20 > http://tools.ietf.org/html/draft-ietf-lmap-framework >=20 > Here's a summary of the changes: > o alignment with the Information Model > [I-D.burbridge-lmap-information-model] as this is agreed as a WG > document >=20 > o One-off and periodic Measurement Schedules are kept separate, so > that they can be updated independently >=20 > o Measurement Suppression in a separate sub-section. Can now > optionally include particular Measurement Tasks &/or Schedules to > suppress, and start/stop time >=20 > o for clarity, concept of Channel split into Control, Report and MA- > to-Controller Channels >=20 > o numerous editorial changes, mainly arising from a very detailed > review by Charles Cook >=20 > best wishes, > phil, al, marcelo, trevor, paul, aamer >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From marcelo@it.uc3m.es Wed Jan 22 02:57:45 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C44CE1A02C2 for ; Wed, 22 Jan 2014 02:57:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -104.07 X-Spam-Level: X-Spam-Status: No, score=-104.07 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.535, SPF_SOFTFAIL=0.665, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t4-mAPlc3iCv for ; Wed, 22 Jan 2014 02:57:42 -0800 (PST) Received: from smtp01.uc3m.es (smtp01.uc3m.es [163.117.176.131]) by ietfa.amsl.com (Postfix) with ESMTP id A9CEA1A00A4 for ; Wed, 22 Jan 2014 02:57:42 -0800 (PST) Received: from smtp01.uc3m.es (localhost [127.0.0.1]) by localhost.uc3m.es (Postfix) with ESMTP id 3F9D1CD5492 for ; Wed, 22 Jan 2014 11:57:41 +0100 (CET) X-uc3m-safe: yes Received: from dummyhost11.it.uc3m.es (unknown [163.117.139.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: marcelo@smtp01.uc3m.es) by smtp01.uc3m.es (Postfix) with ESMTPSA id 32049CC51A0 for ; Wed, 22 Jan 2014 11:57:41 +0100 (CET) Message-ID: <52DFA42A.3070709@it.uc3m.es> Date: Wed, 22 Jan 2014 11:57:46 +0100 From: marcelo bagnulo braun User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: lmap@ietf.org References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> <9904FB1B0159DA42B0B887B7FA8119CA243D468F@AZ-FFEXMB04.global.avaya.com> In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA243D468F@AZ-FFEXMB04.global.avaya.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelistedACL 138 matched, not delayed by milter-greylist-4.2.7 (smtp01.uc3m.es); Wed, 22 Jan 2014 11:57:41 +0100 (CET) X-TM-AS-Product-Ver: IMSS-7.1.0.1224-7.5.0.1017-20450.006 Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 10:57:46 -0000 IMO, this is ready for WGLC Regards, marcelo El 22/01/14 11:32, Romascanu, Dan (Dan) escribi³: > Phil and co-authors, > > Thanks for the submission and for the good work until now. > > Question - is this version ready for WGLC? If not, what's left to be done? > > Everybody's opinions are welcome. > > Regards, > > Dan > > >> -----Original Message----- >> From: lmap [mailto:lmap-bounces@ietf.org] On Behalf Of >> philip.eardley@bt.com >> Sent: יום ג 21 ינואר 2014 17:00 >> To: lmap@ietf.org >> Subject: [lmap] FW: New Version Notification for draft-ietf-lmap-framework- >> 03.txt >> >> We submitted a new version of the framework document. >> >> http://tools.ietf.org/html/draft-ietf-lmap-framework >> >> Here's a summary of the changes: >> o alignment with the Information Model >> [I-D.burbridge-lmap-information-model] as this is agreed as a WG >> document >> >> o One-off and periodic Measurement Schedules are kept separate, so >> that they can be updated independently >> >> o Measurement Suppression in a separate sub-section. Can now >> optionally include particular Measurement Tasks &/or Schedules to >> suppress, and start/stop time >> >> o for clarity, concept of Channel split into Control, Report and MA- >> to-Controller Channels >> >> o numerous editorial changes, mainly arising from a very detailed >> review by Charles Cook >> >> best wishes, >> phil, al, marcelo, trevor, paul, aamer >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From philip.eardley@bt.com Wed Jan 22 03:50:08 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E2FCB1A043E for ; Wed, 22 Jan 2014 03:50:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NhJDYsSCmiQd for ; Wed, 22 Jan 2014 03:50:06 -0800 (PST) Received: from smtpe1.intersmtp.com (smtpe1.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 653FB1A0439 for ; Wed, 22 Jan 2014 03:50:06 -0800 (PST) Received: from EVMHT68-UKRD.domain1.systemhost.net (10.36.3.105) by RDW083A007ED63.bt.com (10.187.98.12) with Microsoft SMTP Server (TLS) id 14.3.169.1; Wed, 22 Jan 2014 11:50:28 +0000 Received: from EMV67-UKRD.domain1.systemhost.net ([169.254.1.39]) by EVMHT68-UKRD.domain1.systemhost.net ([10.36.3.105]) with mapi; Wed, 22 Jan 2014 11:50:04 +0000 From: To: Date: Wed, 22 Jan 2014 11:50:03 +0000 Thread-Topic: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt Thread-Index: Ac8XYM6kXWAy4Q0uQ92GMEZJ7lFH8gAB0I1A Message-ID: References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> <9904FB1B0159DA42B0B887B7FA8119CA243D468F@AZ-FFEXMB04.global.avaya.com> <52DFA42A.3070709@it.uc3m.es> In-Reply-To: <52DFA42A.3070709@it.uc3m.es> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 11:50:09 -0000 SSBhZ3JlZQ0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IGxtYXAgW21h aWx0bzpsbWFwLWJvdW5jZXNAaWV0Zi5vcmddIE9uIEJlaGFsZiBPZiBtYXJjZWxvIGJhZ251bG8N Cj4gYnJhdW4NCj4gU2VudDogMjIgSmFudWFyeSAyMDE0IDEwOjU4DQo+IFRvOiBsbWFwQGlldGYu b3JnDQo+IFN1YmplY3Q6IFJlOiBbbG1hcF0gRlc6IE5ldyBWZXJzaW9uIE5vdGlmaWNhdGlvbiBm b3IgZHJhZnQtaWV0Zi1sbWFwLQ0KPiBmcmFtZXdvcmstMDMudHh0DQo+IA0KPiBJTU8sIHRoaXMg aXMgcmVhZHkgZm9yIFdHTEMNCj4gUmVnYXJkcywgbWFyY2Vsbw0KPiANCj4gDQo+IEVsIDIyLzAx LzE0IDExOjMyLCBSb21hc2NhbnUsIERhbiAoRGFuKSBlc2NyaWJpw7M6DQo+ID4gUGhpbCBhbmQg Y28tYXV0aG9ycywNCj4gPg0KPiA+IFRoYW5rcyBmb3IgdGhlIHN1Ym1pc3Npb24gYW5kIGZvciB0 aGUgZ29vZCB3b3JrIHVudGlsIG5vdy4NCj4gPg0KPiA+IFF1ZXN0aW9uIC0gaXMgdGhpcyB2ZXJz aW9uIHJlYWR5IGZvciBXR0xDPyBJZiBub3QsIHdoYXQncyBsZWZ0IHRvIGJlDQo+IGRvbmU/DQo+ ID4NCj4gPiBFdmVyeWJvZHkncyBvcGluaW9ucyBhcmUgd2VsY29tZS4NCj4gPg0KPiA+IFJlZ2Fy ZHMsDQo+ID4NCj4gPiBEYW4NCj4gPg0KPiA+DQo+ID4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0t LS0tDQo+ID4+IEZyb206IGxtYXAgW21haWx0bzpsbWFwLWJvdW5jZXNAaWV0Zi5vcmddIE9uIEJl aGFsZiBPZg0KPiA+PiBwaGlsaXAuZWFyZGxleUBidC5jb20NCj4gPj4gU2VudDog15nXldedINeS IDIxINeZ16DXldeQ16ggMjAxNCAxNzowMA0KPiA+PiBUbzogbG1hcEBpZXRmLm9yZw0KPiA+PiBT dWJqZWN0OiBbbG1hcF0gRlc6IE5ldyBWZXJzaW9uIE5vdGlmaWNhdGlvbiBmb3INCj4gPj4gZHJh ZnQtaWV0Zi1sbWFwLWZyYW1ld29yay0gMDMudHh0DQo+ID4+DQo+ID4+IFdlIHN1Ym1pdHRlZCBh IG5ldyB2ZXJzaW9uIG9mIHRoZSBmcmFtZXdvcmsgZG9jdW1lbnQuDQo+ID4+DQo+ID4+IGh0dHA6 Ly90b29scy5pZXRmLm9yZy9odG1sL2RyYWZ0LWlldGYtbG1hcC1mcmFtZXdvcmsNCj4gPj4NCj4g Pj4gSGVyZSdzIGEgc3VtbWFyeSBvZiB0aGUgY2hhbmdlczoNCj4gPj4gICAgIG8gIGFsaWdubWVu dCB3aXRoIHRoZSBJbmZvcm1hdGlvbiBNb2RlbA0KPiA+PiAgICAgICAgW0ktRC5idXJicmlkZ2Ut bG1hcC1pbmZvcm1hdGlvbi1tb2RlbF0gYXMgdGhpcyBpcyBhZ3JlZWQgYXMgYQ0KPiBXRw0KPiA+ PiAgICAgICAgZG9jdW1lbnQNCj4gPj4NCj4gPj4gICAgIG8gIE9uZS1vZmYgYW5kIHBlcmlvZGlj IE1lYXN1cmVtZW50IFNjaGVkdWxlcyBhcmUga2VwdCBzZXBhcmF0ZSwNCj4gc28NCj4gPj4gICAg ICAgIHRoYXQgdGhleSBjYW4gYmUgdXBkYXRlZCBpbmRlcGVuZGVudGx5DQo+ID4+DQo+ID4+ICAg ICBvICBNZWFzdXJlbWVudCBTdXBwcmVzc2lvbiBpbiBhIHNlcGFyYXRlIHN1Yi1zZWN0aW9uLiAg Q2FuIG5vdw0KPiA+PiAgICAgICAgb3B0aW9uYWxseSBpbmNsdWRlIHBhcnRpY3VsYXIgTWVhc3Vy ZW1lbnQgVGFza3MgJi9vcg0KPiBTY2hlZHVsZXMgdG8NCj4gPj4gICAgICAgIHN1cHByZXNzLCBh bmQgc3RhcnQvc3RvcCB0aW1lDQo+ID4+DQo+ID4+ICAgICBvICBmb3IgY2xhcml0eSwgY29uY2Vw dCBvZiBDaGFubmVsIHNwbGl0IGludG8gQ29udHJvbCwgUmVwb3J0DQo+IGFuZCBNQS0NCj4gPj4g ICAgICAgIHRvLUNvbnRyb2xsZXIgQ2hhbm5lbHMNCj4gPj4NCj4gPj4gICAgIG8gIG51bWVyb3Vz IGVkaXRvcmlhbCBjaGFuZ2VzLCBtYWlubHkgYXJpc2luZyBmcm9tIGEgdmVyeQ0KPiBkZXRhaWxl ZA0KPiA+PiAgICAgICAgcmV2aWV3IGJ5IENoYXJsZXMgQ29vaw0KPiA+Pg0KPiA+PiBiZXN0IHdp c2hlcywNCj4gPj4gcGhpbCwgYWwsIG1hcmNlbG8sIHRyZXZvciwgcGF1bCwgYWFtZXINCj4gPj4N Cj4gPj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4g Pj4gbG1hcCBtYWlsaW5nIGxpc3QNCj4gPj4gbG1hcEBpZXRmLm9yZw0KPiA+PiBodHRwczovL3d3 dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xtYXANCj4gPiBfX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KPiA+IGxtYXAgbWFpbGluZyBsaXN0DQo+ID4g bG1hcEBpZXRmLm9yZw0KPiA+IGh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8v bG1hcA0KPiANCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X18NCj4gbG1hcCBtYWlsaW5nIGxpc3QNCj4gbG1hcEBpZXRmLm9yZw0KPiBodHRwczovL3d3dy5p ZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xtYXANCg== From trevor.burbridge@bt.com Wed Jan 22 03:54:32 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0C1271A02C2 for ; Wed, 22 Jan 2014 03:54:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9sAhbTy7V5K8 for ; Wed, 22 Jan 2014 03:54:30 -0800 (PST) Received: from smtpe1.intersmtp.com (smtpe1.intersmtp.com [62.239.224.234]) by ietfa.amsl.com (Postfix) with ESMTP id CCDDE1A0443 for ; Wed, 22 Jan 2014 03:54:29 -0800 (PST) Received: from EVMHT64-UKRD.domain1.systemhost.net (10.36.3.101) by RDW083A005ED61.bt.com (10.187.98.10) with Microsoft SMTP Server (TLS) id 14.3.169.1; Wed, 22 Jan 2014 11:54:34 +0000 Received: from EMV64-UKRD.domain1.systemhost.net ([169.254.1.159]) by EVMHT64-UKRD.domain1.systemhost.net ([10.36.3.101]) with mapi; Wed, 22 Jan 2014 11:54:28 +0000 From: To: Date: Wed, 22 Jan 2014 11:54:27 +0000 Thread-Topic: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt Thread-Index: Ac8XYM6kXWAy4Q0uQ92GMEZJ7lFH8gAB0I1AAAAivMA= Message-ID: References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> <9904FB1B0159DA42B0B887B7FA8119CA243D468F@AZ-FFEXMB04.global.avaya.com> <52DFA42A.3070709@it.uc3m.es> In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 11:54:32 -0000 WWVzIGZpbmUgZm9yIG1lLg0KDQpUcmV2b3IgQnVyYnJpZGdlDQoNCj4tLS0tLU9yaWdpbmFsIE1l c3NhZ2UtLS0tLQ0KPkZyb206IGxtYXAgW21haWx0bzpsbWFwLWJvdW5jZXNAaWV0Zi5vcmddIE9u IEJlaGFsZiBPZiBwaGlsaXAuZWFyZGxleUBidC5jb20NCj5TZW50OiAyMiBKYW51YXJ5IDIwMTQg MTE6NTANCj5UbzogbG1hcEBpZXRmLm9yZw0KPlN1YmplY3Q6IFJlOiBbbG1hcF0gRlc6IE5ldyBW ZXJzaW9uIE5vdGlmaWNhdGlvbiBmb3IgZHJhZnQtaWV0Zi1sbWFwLWZyYW1ld29yay0NCj4wMy50 eHQNCj4NCj5JIGFncmVlDQo+DQo+PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPj4gRnJv bTogbG1hcCBbbWFpbHRvOmxtYXAtYm91bmNlc0BpZXRmLm9yZ10gT24gQmVoYWxmIE9mIG1hcmNl bG8gYmFnbnVsbw0KPj4gYnJhdW4NCj4+IFNlbnQ6IDIyIEphbnVhcnkgMjAxNCAxMDo1OA0KPj4g VG86IGxtYXBAaWV0Zi5vcmcNCj4+IFN1YmplY3Q6IFJlOiBbbG1hcF0gRlc6IE5ldyBWZXJzaW9u IE5vdGlmaWNhdGlvbiBmb3IgZHJhZnQtaWV0Zi1sbWFwLQ0KPj4gZnJhbWV3b3JrLTAzLnR4dA0K Pj4NCj4+IElNTywgdGhpcyBpcyByZWFkeSBmb3IgV0dMQw0KPj4gUmVnYXJkcywgbWFyY2Vsbw0K Pj4NCj4+DQo+PiBFbCAyMi8wMS8xNCAxMTozMiwgUm9tYXNjYW51LCBEYW4gKERhbikgZXNjcmli acOzOg0KPj4gPiBQaGlsIGFuZCBjby1hdXRob3JzLA0KPj4gPg0KPj4gPiBUaGFua3MgZm9yIHRo ZSBzdWJtaXNzaW9uIGFuZCBmb3IgdGhlIGdvb2Qgd29yayB1bnRpbCBub3cuDQo+PiA+DQo+PiA+ IFF1ZXN0aW9uIC0gaXMgdGhpcyB2ZXJzaW9uIHJlYWR5IGZvciBXR0xDPyBJZiBub3QsIHdoYXQn cyBsZWZ0IHRvIGJlDQo+PiBkb25lPw0KPj4gPg0KPj4gPiBFdmVyeWJvZHkncyBvcGluaW9ucyBh cmUgd2VsY29tZS4NCj4+ID4NCj4+ID4gUmVnYXJkcywNCj4+ID4NCj4+ID4gRGFuDQo+PiA+DQo+ PiA+DQo+PiA+PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPj4gPj4gRnJvbTogbG1hcCBb bWFpbHRvOmxtYXAtYm91bmNlc0BpZXRmLm9yZ10gT24gQmVoYWxmIE9mDQo+PiA+PiBwaGlsaXAu ZWFyZGxleUBidC5jb20NCj4+ID4+IFNlbnQ6INeZ15XXnSDXkiAyMSDXmdeg15XXkNeoIDIwMTQg MTc6MDANCj4+ID4+IFRvOiBsbWFwQGlldGYub3JnDQo+PiA+PiBTdWJqZWN0OiBbbG1hcF0gRlc6 IE5ldyBWZXJzaW9uIE5vdGlmaWNhdGlvbiBmb3INCj4+ID4+IGRyYWZ0LWlldGYtbG1hcC1mcmFt ZXdvcmstIDAzLnR4dA0KPj4gPj4NCj4+ID4+IFdlIHN1Ym1pdHRlZCBhIG5ldyB2ZXJzaW9uIG9m IHRoZSBmcmFtZXdvcmsgZG9jdW1lbnQuDQo+PiA+Pg0KPj4gPj4gaHR0cDovL3Rvb2xzLmlldGYu b3JnL2h0bWwvZHJhZnQtaWV0Zi1sbWFwLWZyYW1ld29yaw0KPj4gPj4NCj4+ID4+IEhlcmUncyBh IHN1bW1hcnkgb2YgdGhlIGNoYW5nZXM6DQo+PiA+PiAgICAgbyAgYWxpZ25tZW50IHdpdGggdGhl IEluZm9ybWF0aW9uIE1vZGVsDQo+PiA+PiAgICAgICAgW0ktRC5idXJicmlkZ2UtbG1hcC1pbmZv cm1hdGlvbi1tb2RlbF0gYXMgdGhpcyBpcyBhZ3JlZWQgYXMNCj4+ID4+IGENCj4+IFdHDQo+PiA+ PiAgICAgICAgZG9jdW1lbnQNCj4+ID4+DQo+PiA+PiAgICAgbyAgT25lLW9mZiBhbmQgcGVyaW9k aWMgTWVhc3VyZW1lbnQgU2NoZWR1bGVzIGFyZSBrZXB0DQo+PiA+PiBzZXBhcmF0ZSwNCj4+IHNv DQo+PiA+PiAgICAgICAgdGhhdCB0aGV5IGNhbiBiZSB1cGRhdGVkIGluZGVwZW5kZW50bHkNCj4+ ID4+DQo+PiA+PiAgICAgbyAgTWVhc3VyZW1lbnQgU3VwcHJlc3Npb24gaW4gYSBzZXBhcmF0ZSBz dWItc2VjdGlvbi4gIENhbiBub3cNCj4+ID4+ICAgICAgICBvcHRpb25hbGx5IGluY2x1ZGUgcGFy dGljdWxhciBNZWFzdXJlbWVudCBUYXNrcyAmL29yDQo+PiBTY2hlZHVsZXMgdG8NCj4+ID4+ICAg ICAgICBzdXBwcmVzcywgYW5kIHN0YXJ0L3N0b3AgdGltZQ0KPj4gPj4NCj4+ID4+ICAgICBvICBm b3IgY2xhcml0eSwgY29uY2VwdCBvZiBDaGFubmVsIHNwbGl0IGludG8gQ29udHJvbCwgUmVwb3J0 DQo+PiBhbmQgTUEtDQo+PiA+PiAgICAgICAgdG8tQ29udHJvbGxlciBDaGFubmVscw0KPj4gPj4N Cj4+ID4+ICAgICBvICBudW1lcm91cyBlZGl0b3JpYWwgY2hhbmdlcywgbWFpbmx5IGFyaXNpbmcg ZnJvbSBhIHZlcnkNCj4+IGRldGFpbGVkDQo+PiA+PiAgICAgICAgcmV2aWV3IGJ5IENoYXJsZXMg Q29vaw0KPj4gPj4NCj4+ID4+IGJlc3Qgd2lzaGVzLA0KPj4gPj4gcGhpbCwgYWwsIG1hcmNlbG8s IHRyZXZvciwgcGF1bCwgYWFtZXINCj4+ID4+DQo+PiA+PiBfX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fXw0KPj4gPj4gbG1hcCBtYWlsaW5nIGxpc3QNCj4+ID4+ IGxtYXBAaWV0Zi5vcmcNCj4+ID4+IGh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGlu Zm8vbG1hcA0KPj4gPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXw0KPj4gPiBsbWFwIG1haWxpbmcgbGlzdA0KPj4gPiBsbWFwQGlldGYub3JnDQo+PiA+IGh0 dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vbG1hcA0KPj4NCj4+IF9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQo+PiBsbWFwIG1haWxpbmcg bGlzdA0KPj4gbG1hcEBpZXRmLm9yZw0KPj4gaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9s aXN0aW5mby9sbWFwDQo+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18NCj5sbWFwIG1haWxpbmcgbGlzdA0KPmxtYXBAaWV0Zi5vcmcNCj5odHRwczovL3d3dy5p ZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xtYXANCg== From jason.weil@twcable.com Wed Jan 22 11:37:24 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 535FA1A0200 for ; Wed, 22 Jan 2014 11:37:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.3 X-Spam-Level: X-Spam-Status: No, score=-0.3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, RP_MATCHES_RCVD=-0.535, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EuNLBNapf6EF for ; Wed, 22 Jan 2014 11:37:22 -0800 (PST) Received: from cdcipgw01.twcable.com (cdcipgw01.twcable.com [165.237.91.110]) by ietfa.amsl.com (Postfix) with ESMTP id 6C2AE1A01B7 for ; Wed, 22 Jan 2014 11:37:22 -0800 (PST) X-SENDER-IP: 10.136.163.15 X-SENDER-REPUTATION: None X-IronPort-AV: E=Sophos;i="4.95,701,1384318800"; d="scan'208";a="66387699" Received: from unknown (HELO PRVPEXHUB06.corp.twcable.com) ([10.136.163.15]) by cdcipgw01.twcable.com with ESMTP/TLS/RC4-MD5; 22 Jan 2014 14:37:11 -0500 Received: from PRVPEXVS06.corp.twcable.com ([10.136.163.32]) by PRVPEXHUB06.corp.twcable.com ([10.136.163.15]) with mapi; Wed, 22 Jan 2014 14:37:20 -0500 From: "Weil, Jason" To: "philip.eardley@bt.com" , "lmap@ietf.org" Date: Wed, 22 Jan 2014 14:37:19 -0500 Thread-Topic: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt Thread-Index: Ac8XqV2hd1POu0kmSduA09/z9uIdag== Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.8.130913 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 19:37:24 -0000 Phil et al, Overall I think the document is coming together nicely. I have 3 comments after reading the latest version with chair hat removed: Section 5.1 - It is not clear what role and scope the Initial Controller plays. Is this a new device type/function - does it need to be described somewhere other than a cursory glance that is presented here - or does this represent a device that is part of the bootstrap operation that is out of scope? It is not very clear here. Maybe rename to Bootstrap Controller instead? Section 5.2.1 & 5.3 These sections seem to contradict each other slightly. Suppression seems to imply that the Controller is capable of actively managing the MA. This includes the challenges stated in Section 6.2.2. I am wondering if we should state optional or if required contingent on transport protocol operation? Section 5.3 then goes on to say that the MA acts autonomously once it receives the Measurement and Report Schedules from the Controller. This sentence seems to contradict 5.2.1 operation. Section 6.2 - "A single site (home, branch office etc.) that is participating in a measurement could make use of one or multiple Measurement Agents in a single measurement." This concept needs more details if it is going to be included I believe. How does a measurement task employ multiple MAs for a single test? Which MA is responsible for reporting the results? Is there some kind of master/slave function implied here or is Group-ID tie these devices together? Would all the MAs participating in this test group need to be associated with the same controller and does that controller need to be aware of the group? Does the Collector need to be aware of the test group as well? Thanks, Jason On 1/21/14 10:00 AM, "philip.eardley@bt.com" wrote: >We submitted a new version of the framework document. > >http://tools.ietf.org/html/draft-ietf-lmap-framework > >Here's a summary of the changes: > o alignment with the Information Model > [I-D.burbridge-lmap-information-model] as this is agreed as a WG > document > > o One-off and periodic Measurement Schedules are kept separate, so > that they can be updated independently > > o Measurement Suppression in a separate sub-section. Can now > optionally include particular Measurement Tasks &/or Schedules to > suppress, and start/stop time > > o for clarity, concept of Channel split into Control, Report and MA- > to-Controller Channels > > o numerous editorial changes, mainly arising from a very detailed > review by Charles Cook > >best wishes, >phil, al, marcelo, trevor, paul, aamer > >_______________________________________________ >lmap mailing list >lmap@ietf.org >https://www.ietf.org/mailman/listinfo/lmap This E-mail and any of its attachments may contain Time Warner Cable propri= etary information, which is privileged, confidential, or subject to copyrig= ht belonging to Time Warner Cable. This E-mail is intended solely for the u= se of the individual or entity to which it is addressed. If you are not the= intended recipient of this E-mail, you are hereby notified that any dissem= ination, distribution, copying, or action taken in relation to the contents= of and attachments to this E-mail is strictly prohibited and may be unlawf= ul. If you have received this E-mail in error, please notify the sender imm= ediately and permanently delete the original and any copy of this E-mail an= d any printout. From dromasca@avaya.com Thu Jan 23 05:30:53 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A5DE51A00CB for ; Thu, 23 Jan 2014 05:30:53 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.434 X-Spam-Level: X-Spam-Status: No, score=-2.434 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.535] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ggzUA9QJE8kf for ; Thu, 23 Jan 2014 05:30:52 -0800 (PST) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id E5D8E1A00C6 for ; Thu, 23 Jan 2014 05:30:51 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AsUQAKsY4VLGmAcV/2dsb2JhbABbgkgjIThWplsDBZUPgQ4WdIInAQEDEgsQXgEMCRVWJgEEGwEZh2MBDJojhFCmc44vAQEegmcPZoEUBJ8SiymDLYFxOQ X-IronPort-AV: E=Sophos;i="4.95,706,1384318800"; d="scan'208,217";a="46345714" Received: from unknown (HELO co300216-co-erhwest-exch.avaya.com) ([198.152.7.21]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 23 Jan 2014 08:30:50 -0500 Received: from unknown (HELO AZ-FFEXHC01.global.avaya.com) ([135.64.58.11]) by co300216-co-erhwest-out.avaya.com with ESMTP/TLS/AES128-SHA; 23 Jan 2014 08:20:40 -0500 Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC01.global.avaya.com ([135.64.58.11]) with mapi id 14.03.0158.001; Thu, 23 Jan 2014 14:30:49 +0100 From: "Romascanu, Dan (Dan)" To: "lmap@ietf.org" Thread-Topic: WGLC for http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt Thread-Index: Ac8YP1MxATKR0Iw1RyKcPlwz98oRLA== Date: Thu, 23 Jan 2014 13:30:48 +0000 Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA243D862C@AZ-FFEXMB04.global.avaya.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.64.58.45] Content-Type: multipart/alternative; boundary="_000_9904FB1B0159DA42B0B887B7FA8119CA243D862CAZFFEXMB04globa_" MIME-Version: 1.0 Subject: [lmap] WGLC for http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 13:30:53 -0000 --_000_9904FB1B0159DA42B0B887B7FA8119CA243D862CAZFFEXMB04globa_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable This is a Working Group Last Call for the Internet-Draft 'A framework for l= arge-scale measurement platforms (LMAP)' which can be found at http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt. Please send your = comments about this I-D to the WG mail list until 2/6/13. If you have no c= omments but you believe that the document is ready to be submitted to the I= ESG for consideration as Informational RFC a short mail stating this is als= o welcome. Thanks and Regards, Dan --_000_9904FB1B0159DA42B0B887B7FA8119CA243D862CAZFFEXMB04globa_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable 
This is a Working Group Last Call for the Internet-Draft ‘A fram=
ework for large-scale measurement platforms (LMAP)’ which can be foun=
d at

 http://www.ietf.org/id/draft-ietf-lmap-framework-03.t= xt. Please send your comments about this I-D to the WG mail list until = 2/6/13.  If you have no comments but you believe that the document is ready to be submitted to the IESG for conside= ration as Informational RFC a short mail stating this is also welcome.

 

Thanks and Regards,

 

Dan

--_000_9904FB1B0159DA42B0B887B7FA8119CA243D862CAZFFEXMB04globa_-- From philip.eardley@bt.com Fri Jan 24 03:46:54 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 673B01A02B8 for ; Fri, 24 Jan 2014 03:46:54 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.001 X-Spam-Level: X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_72=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b3We6rkfp08P for ; Fri, 24 Jan 2014 03:46:51 -0800 (PST) Received: from smtpe1.intersmtp.com (smtpe1.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id 326DA1A02BA for ; Fri, 24 Jan 2014 03:46:50 -0800 (PST) Received: from EVMHT69-UKRD.domain1.systemhost.net (10.36.3.129) by RDW083A008ED64.bt.com (10.187.98.13) with Microsoft SMTP Server (TLS) id 14.3.169.1; Fri, 24 Jan 2014 11:46:54 +0000 Received: from EMV67-UKRD.domain1.systemhost.net ([169.254.1.39]) by EVMHT69-UKRD.domain1.systemhost.net ([10.36.3.129]) with mapi; Fri, 24 Jan 2014 11:46:09 +0000 From: To: , Date: Fri, 24 Jan 2014 11:46:08 +0000 Thread-Topic: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt Thread-Index: Ac8XqV2hd1POu0kmSduA09/z9uIdagBS2lXg Message-ID: References: In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 11:46:54 -0000 Jason, Thanks for your comments and careful reading In-line phil > -----Original Message----- > From: Weil, Jason [mailto:jason.weil@twcable.com] > Sent: 22 January 2014 19:37 > To: Eardley,PL,Philip,TUB8 R; lmap@ietf.org > Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap- > framework-03.txt >=20 > Phil et al, >=20 > Overall I think the document is coming together nicely. I have 3 > comments after reading the latest version with chair hat removed: >=20 > Section 5.1 - It is not clear what role and scope the Initial > Controller plays. Is this a new device type/function - does it need to > be described somewhere other than a cursory glance that is presented > here - or does this represent a device that is part of the bootstrap > operation that is out of scope? It is not very clear here. Maybe rename > to Bootstrap Controller instead? I think you're right that this section isn't very clear.=20 To my mind, we're simply trying to say that the bootstrap process isn't one= we're going to define but it needs to get the MA-id, maybe a group-id and = details of the various channels. And here are some options about how to get= the info on the MA. i think it would be better to scrap the picture - as i= t will be read as implying we're defining things that we aren't. One other possibility that we should mention: The bootstrapping information might be configured on the MA.=20 (Marcelo mentioned this during our editing, sorry I forgot to include it)=20 >=20 > Section 5.2.1 & 5.3 These sections seem to contradict each other > slightly. > Suppression seems to imply that the Controller is capable of actively > managing the MA. This includes the challenges stated in Section 6.2.2. > I am wondering if we should state optional or if required contingent on > transport protocol operation? Section 5.3 then goes on to say that the > MA acts autonomously once it receives the Measurement and Report > Schedules from the Controller. This sentence seems to contradict 5.2.1 > operation. You're right that the MA "acts autonomously" should have a supplement "unti= l it gets a new Instruction (including perhaps an Instruction about Measure= ment Suppression)"=20 I don't think suppression should be optional (to implement). It seems a ver= y useful capability.=20 I agree that if the MA is behind a NAT, then that presents issues for how l= map operates over the transport protocol - but I don't think they're differ= ent for suppression & the ordinary instruction.=20 >=20 > Section 6.2 - "A single site (home, branch office etc.) that is > participating in a measurement could make use of one or multiple > Measurement Agents in a single measurement." > This concept needs more details if it is going to be included I > believe. > How does a measurement task employ multiple MAs for a single test? > Which MA is responsible for reporting the results? Is there some kind > of master/slave function implied here or is Group-ID tie these devices > together? Would all the MAs participating in this test group need to be > associated with the same controller and does that controller need to be > aware of the group? Does the Collector need to be aware of the test > group as well? I think that's a mistake. It shouldn't mention the possibility of multiple = MAs.=20 If I remember a discussion quite a while ago, we agreed that if you want a = multi agent test, then it would be up to you to define two Measurement Task= s and schedule them at the same time - but that wouldn't be a concern of lm= ap. Thanks! phil >=20 >=20 > Thanks, >=20 > Jason >=20 >=20 > On 1/21/14 10:00 AM, "philip.eardley@bt.com" > wrote: >=20 > >We submitted a new version of the framework document. > > > >http://tools.ietf.org/html/draft-ietf-lmap-framework > > > >Here's a summary of the changes: > > o alignment with the Information Model > > [I-D.burbridge-lmap-information-model] as this is agreed as a WG > > document > > > > o One-off and periodic Measurement Schedules are kept separate, so > > that they can be updated independently > > > > o Measurement Suppression in a separate sub-section. Can now > > optionally include particular Measurement Tasks &/or Schedules > to > > suppress, and start/stop time > > > > o for clarity, concept of Channel split into Control, Report and > MA- > > to-Controller Channels > > > > o numerous editorial changes, mainly arising from a very detailed > > review by Charles Cook > > > >best wishes, > >phil, al, marcelo, trevor, paul, aamer > > > >_______________________________________________ > >lmap mailing list > >lmap@ietf.org > >https://www.ietf.org/mailman/listinfo/lmap >=20 >=20 > This E-mail and any of its attachments may contain Time Warner Cable > proprietary information, which is privileged, confidential, or subject > to copyright belonging to Time Warner Cable. This E-mail is intended > solely for the use of the individual or entity to which it is > addressed. If you are not the intended recipient of this E-mail, you > are hereby notified that any dissemination, distribution, copying, or > action taken in relation to the contents of and attachments to this E- > mail is strictly prohibited and may be unlawful. If you have received > this E-mail in error, please notify the sender immediately and > permanently delete the original and any copy of this E-mail and any > printout. From internet-drafts@ietf.org Fri Jan 24 10:02:40 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A71221A00BB; Fri, 24 Jan 2014 10:02:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fG6OaP_nNa8Z; Fri, 24 Jan 2014 10:02:39 -0800 (PST) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F23E1A009F; Fri, 24 Jan 2014 10:02:39 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.90.p2 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20140124180238.24848.30797.idtracker@ietfa.amsl.com> Date: Fri, 24 Jan 2014 10:02:38 -0800 Cc: lmap@ietf.org Subject: [lmap] I-D Action: draft-ietf-lmap-use-cases-02.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 18:02:40 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the Large-Scale Measurement of Broadband Perf= ormance Working Group of the IETF. Title : Large-Scale Broadband Measurement Use Cases Authors : Marc Linsner Philip Eardley Trevor Burbridge Frode Sorensen Filename : draft-ietf-lmap-use-cases-02.txt Pages : 16 Date : 2014-01-24 Abstract: Measuring broadband performance on a large scale is important for network diagnostics by providers and users, as well as for public policy. To conduct such measurements, user networks gather data instructed by a measurement controller, and then upload the measurement results to a designated measurement server. Understanding the various scenarios and users of measuring broadband performance is essential to development of the framework, information model and protocol. The details of the measurement metrics themselves are beyond the scope of this document. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-lmap-use-cases/ There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-lmap-use-cases-02 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-lmap-use-cases-02 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From mlinsner@cisco.com Fri Jan 24 10:06:23 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A9DA31A00C0 for ; Fri, 24 Jan 2014 10:06:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -15.036 X-Spam-Level: X-Spam-Status: No, score=-15.036 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.535, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kGcPusbXvk4b for ; Fri, 24 Jan 2014 10:06:22 -0800 (PST) Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com [173.37.86.74]) by ietfa.amsl.com (Postfix) with ESMTP id 2A7A81A0047 for ; Fri, 24 Jan 2014 10:06:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2226; q=dns/txt; s=iport; t=1390586781; x=1391796381; h=from:to:subject:date:message-id:in-reply-to:content-id: content-transfer-encoding:mime-version; bh=rgNosI+/vECyaRHQ71VC/bRwhrXAo9PtWzaEyWR6Oo4=; b=fZ1r7Kky3khyR598Rmoif9i6f6YEJB7zhTGw7waNphoSbaICXrUcUP+m BBAE2GegD2of5tEPyJJ6sPSwBgZ8MTJIAMZz5eymAtyucll5OxB0iGKyd Qm68L7+WnXYkNjbXQJ25RAQ4gqkgzKUDlHcf+NSk1ZPzB6wpWlBdfK9t7 Q=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ag8FAFur4lKtJV2Y/2dsb2JhbABagww4UAa8NYENFnSCJQEBAQQBAQE3NB0BCDY3CxsBBgMCBBMJh3wIBZxGq3IXjxOEOASYJ4EykGyDLYIq X-IronPort-AV: E=Sophos;i="4.95,714,1384300800"; d="scan'208";a="299551953" Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by rcdn-iport-3.cisco.com with ESMTP; 24 Jan 2014 18:05:58 +0000 Received: from xhc-rcd-x14.cisco.com (xhc-rcd-x14.cisco.com [173.37.183.88]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id s0OI5vBL028878 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Fri, 24 Jan 2014 18:05:57 GMT Received: from xmb-rcd-x08.cisco.com ([169.254.8.39]) by xhc-rcd-x14.cisco.com ([173.37.183.88]) with mapi id 14.03.0123.003; Fri, 24 Jan 2014 12:05:57 -0600 From: "Marc Linsner (mlinsner)" To: "lmap@ietf.org" Thread-Topic: [lmap] I-D Action: draft-ietf-lmap-use-cases-02.txt Thread-Index: AQHPGS6KaVfl7jP2p0uptMxdWtNXb5qUPECA Date: Fri, 24 Jan 2014 18:05:56 +0000 Message-ID: In-Reply-To: <20140124180238.24848.30797.idtracker@ietfa.amsl.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.116.195.118] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [lmap] FW: I-D Action: draft-ietf-lmap-use-cases-02.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 18:06:23 -0000 We updated the flow of the draft, pulled conclusions for each use case into a single section. More alignment to the framework draft. And handled comments to the authors both from the list and privately. Please review and sent comments to the list. Thanks, Marc, Philip, Trevor, Frode > >A New Internet-Draft is available from the on-line Internet-Drafts >directories. > This draft is a work item of the Large-Scale Measurement of Broadband >Performance Working Group of the IETF. > > Title : Large-Scale Broadband Measurement Use Cases > Authors : Marc Linsner > Philip Eardley > Trevor Burbridge > Frode Sorensen > Filename : draft-ietf-lmap-use-cases-02.txt > Pages : 16 > Date : 2014-01-24 > >Abstract: > Measuring broadband performance on a large scale is important for > network diagnostics by providers and users, as well as for public > policy. To conduct such measurements, user networks gather data > instructed by a measurement controller, and then upload the > measurement results to a designated measurement server. Understanding > the various scenarios and users of measuring broadband performance is > essential to development of the framework, information model and > protocol. The details of the measurement metrics themselves are > beyond the scope of this document. > > > >The IETF datatracker status page for this draft is: >https://datatracker.ietf.org/doc/draft-ietf-lmap-use-cases/ > >There's also a htmlized version available at: >http://tools.ietf.org/html/draft-ietf-lmap-use-cases-02 > >A diff from the previous version is available at: >http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-lmap-use-cases-02 > > >Please note that it may take a couple of minutes from the time of >submission >until the htmlized version and diff are available at tools.ietf.org. > >Internet-Drafts are also available by anonymous FTP at: >ftp://ftp.ietf.org/internet-drafts/ > >_______________________________________________ >lmap mailing list >lmap@ietf.org >https://www.ietf.org/mailman/listinfo/lmap From charles.cook2@centurylink.com Mon Jan 27 12:33:08 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D5E3B1A029E for ; Mon, 27 Jan 2014 12:33:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j24wbJ-dfKfh for ; Mon, 27 Jan 2014 12:33:05 -0800 (PST) Received: from sudnp799.qwest.com (sudnp799.qwest.com [155.70.32.99]) by ietfa.amsl.com (Postfix) with ESMTP id 2F75E1A00F6 for ; Mon, 27 Jan 2014 12:33:05 -0800 (PST) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by sudnp799.qwest.com (8.14.4/8.14.4) with ESMTP id s0RKWwqZ009573 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 27 Jan 2014 13:32:58 -0700 (MST) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 02AE81E005D; Mon, 27 Jan 2014 14:32:53 -0600 (CST) Received: from sudnp796.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id D26F71E0073; Mon, 27 Jan 2014 14:32:52 -0600 (CST) Received: from sudnp796.qintra.com (localhost [127.0.0.1]) by sudnp796.qintra.com (8.14.4/8.14.4) with ESMTP id s0RKWqDW024651; Mon, 27 Jan 2014 13:32:52 -0700 (MST) Received: from [10.188.109.177] (x1069818.dhcp.intranet [10.188.109.177]) by sudnp796.qintra.com (8.14.4/8.14.4) with ESMTP id s0RKWp0D024631; Mon, 27 Jan 2014 13:32:51 -0700 (MST) Message-ID: <52E6C274.9090401@centurylink.com> Date: Mon, 27 Jan 2014 13:32:52 -0700 From: Charles Cook User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121005 Thunderbird/16.0 MIME-Version: 1.0 To: philip.eardley@bt.com References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> In-Reply-To: X-Enigmail-Version: 1.4.6 Content-Type: multipart/alternative; boundary="------------010606090107040302030400" X-CFilter-Loop: Reflected Cc: lmap@ietf.org Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: charles.cook2@centurylink.com List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jan 2014 20:33:09 -0000 This is a multi-part message in MIME format. --------------010606090107040302030400 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Phil, I am in the process of reviewing the draft. I like how many things have been improved. Thanks for the good work. That said, I need some clarity of certain terms and how they relate to each other, because I am getting confused as I read through the draft. The terms I am concerned about are: * Registry * Metric * Measurement Task * Measurement Method It appears that Measurement Task and Measurement Method are sometimes used synonymously, and sometimes they appear to be very different. Sometimes a Measurement Method appears to be a Metric with its associated parameters not populated. Sometimes Measurement Method seems to refer to a Measurement Task with variables that have not been populated. Generally the term "method" is a synonym with "procedure", which is generally how I interpret the term "method". Maybe the use of "method" rather than "procedure" is causing part of my confusion. Maybe part of my problem is that the Registry, its role, and what is stored in the Registry is not well described. Here is what I thought was going on: 1. A registry exists somewhere that contains a list of the standardised metrics of interest. 2. The metric may be composed of some parameters that can be varied. For example, a metric called "Throughput" may have parameters like, originating end point, terminating endpoint, traffic type (UDP/TCP), etc. 3. A Measurement Task is the result of the Controller populating the variables of the standardised metric. 4. A Measurement Method would define particular constraints that the measurements must be performed under in order for the Measurement Results to be comparable with other Measurement Results of the same Measurement Task performed at different times and/or locations. This could be verification that there are sufficient resources to perform the Measurement Task (the MA has sufficient processing power), the path being measured is a direct path versus a derived path, a result that requires multiple iterations averaged together has confirmation that the same path is used for each iteration, etc. Note that is it not enough for two Measurement Tasks to be comparable simply because their parameters are the same. Certain external conditions must also be the same in order for the results to be comparable. Here is what the draft appears to say: 1. A registry exists somewhere that contains a list of standardised Measurement Methods. 2. Measurement Methods consist of specific actions that have parameters associated with them that have not been populated. 3. A Measurement Task is the result of the Controller populating the parameters of the standardised Measurement Method. 4. No discussion of what mechanism is used to ensure that Measurement Tasks conducted at different times and places can be comparable. Is there a gap? After we clarify what the intent is, I can re-read the draft and suggest clarifying comments. Thanks, Charles On 1/21/2014 8:00 AM, philip.eardley@bt.com wrote: > We submitted a new version of the framework document. > > http://tools.ietf.org/html/draft-ietf-lmap-framework > > Here's a summary of the changes: > o alignment with the Information Model > [I-D.burbridge-lmap-information-model] as this is agreed as a WG > document > > o One-off and periodic Measurement Schedules are kept separate, so > that they can be updated independently > > o Measurement Suppression in a separate sub-section. Can now > optionally include particular Measurement Tasks &/or Schedules to > suppress, and start/stop time > > o for clarity, concept of Channel split into Control, Report and MA- > to-Controller Channels > > o numerous editorial changes, mainly arising from a very detailed > review by Charles Cook > > best wishes, > phil, al, marcelo, trevor, paul, aamer > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap -- Charles Cook Principal Architect Network 5325 Zuni Street; Suite 224 Denver, CO 80221 Tel: 303.992.8952 Fax: 925.281.0662 charles.cook2@centurylink.com --------------010606090107040302030400 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Phil,

I am in the process of reviewing the draft.  I like how many things have been improved.  Thanks for the good work.

That said, I need some clarity of certain terms and how they relate to each other, because I am getting confused as I read through the draft.

The terms I am concerned about are:
  • Registry
  • Metric
  • Measurement Task
  • Measurement Method

It appears that Measurement Task and Measurement Method are sometimes used synonymously, and sometimes they appear to be very different.  Sometimes a Measurement Method appears to be a Metric with its associated parameters not populated.  Sometimes Measurement Method seems to refer to a Measurement Task with variables that have not been populated.  Generally the term "method" is a synonym with "procedure", which is generally how I interpret the term "method".  Maybe the use of "method" rather than "procedure" is causing part of my confusion.  Maybe part of my problem is that the Registry, its role, and what is stored in the Registry is not well described. 

Here is what I thought was going on:
  1. A registry exists somewhere that contains a list of the standardised metrics of interest.
  2. The metric may be composed of some parameters that can be varied.  For example, a metric called "Throughput" may have parameters like, originating end point, terminating endpoint, traffic type (UDP/TCP), etc.
  3. A Measurement Task is the result of the Controller populating the variables of the standardised metric.
  4. A Measurement Method would define particular constraints that the measurements must be performed under in order for the Measurement Results to be comparable with other Measurement Results of the same Measurement Task performed at different times and/or locations.  This could be verification that there are sufficient resources to perform the Measurement Task (the MA has sufficient processing power), the path being measured is a direct path versus a derived path, a result that requires multiple iterations averaged together has confirmation that the same path is used for each iteration, etc.  Note that is it not enough for two Measurement Tasks to be comparable simply because their parameters are the same.  Certain external conditions must also be the same in order for the results to be comparable.
Here is what the draft appears to say:
  1. A registry exists somewhere  that contains a list of standardised Measurement Methods.
  2. Measurement Methods consist of specific actions that have parameters associated with them that have not been populated.
  3. A Measurement Task is the result of the Controller populating the parameters of the standardised Measurement Method.
  4. No discussion of what mechanism is used to ensure that Measurement Tasks conducted at different times and places can be comparable.
Is there a gap?  After we clarify what the intent is, I can re-read the draft and suggest clarifying comments.

Thanks,
Charles





On 1/21/2014 8:00 AM, philip.eardley@bt.com wrote:
We submitted a new version of the framework document.

http://tools.ietf.org/html/draft-ietf-lmap-framework 

Here's a summary of the changes:
   o  alignment with the Information Model
      [I-D.burbridge-lmap-information-model] as this is agreed as a WG
      document

   o  One-off and periodic Measurement Schedules are kept separate, so
      that they can be updated independently

   o  Measurement Suppression in a separate sub-section.  Can now
      optionally include particular Measurement Tasks &/or Schedules to
      suppress, and start/stop time

   o  for clarity, concept of Channel split into Control, Report and MA-
      to-Controller Channels

   o  numerous editorial changes, mainly arising from a very detailed
      review by Charles Cook

best wishes,
phil, al, marcelo, trevor, paul, aamer

_______________________________________________
lmap mailing list
lmap@ietf.org
https://www.ietf.org/mailman/listinfo/lmap


-- 

Charles Cook 
Principal Architect
Network
5325 Zuni Street; Suite 224
Denver, CO  80221
Tel:  303.992.8952  Fax:  925.281.0662
charles.cook2@centurylink.com
--------------010606090107040302030400-- From marcelo@it.uc3m.es Mon Jan 27 12:43:25 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D05511A0371 for ; Mon, 27 Jan 2014 12:43:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -104.736 X-Spam-Level: X-Spam-Status: No, score=-104.736 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.535, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qp1qhatQz4wW for ; Mon, 27 Jan 2014 12:43:22 -0800 (PST) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by ietfa.amsl.com (Postfix) with ESMTP id A70021A00F6 for ; Mon, 27 Jan 2014 12:43:21 -0800 (PST) Received: from smtp02.uc3m.es (localhost [127.0.0.1]) by localhost.uc3m.es (Postfix) with ESMTP id 31C24897042 for ; Mon, 27 Jan 2014 21:43:18 +0100 (CET) X-uc3m-safe: yes Received: from [163.117.203.86] (unknown [163.117.203.86]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: marcelo@smtp02.uc3m.es) by smtp02.uc3m.es (Postfix) with ESMTPSA id F2529894F5F for ; Mon, 27 Jan 2014 21:43:17 +0100 (CET) Message-ID: <52E6C4E6.3060705@it.uc3m.es> Date: Mon, 27 Jan 2014 21:43:18 +0100 From: marcelo bagnulo braun User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: lmap@ietf.org References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> <52E6C274.9090401@centurylink.com> In-Reply-To: <52E6C274.9090401@centurylink.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelistedACL 131 matched, not delayed by milter-greylist-4.2.7 (smtp02.uc3m.es); Mon, 27 Jan 2014 21:43:18 +0100 (CET) X-TM-AS-Product-Ver: IMSS-7.1.0.1224-7.5.0.1017-20466.002 X-TM-AS-Result: No--37.022-7.0-31-1 X-imss-scan-details: No--37.022-7.0-31-1 Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jan 2014 20:43:26 -0000 Hi Charles, thanks for the comments. In section 3 of the draft the following defintions are included: Measurement Method: The process for assessing the value of a Metric; the process of measuring some performance or reliability parameter; the generalisation of a Measurement Task. Measurement Task: The act that yields a single Measurement Result; the act consisting of the (single) operation of the Measurement Method at a particular time and with all its parameters set to specific values. Measurement Result: The output of a single Measurement Task (the value obtained for the parameter of interest or Metric). Metric: The quantity related to the performance and reliability of the network that we'd like to know the value of, and that is carefully specified. Registry is not defined, nor metric registry nor entries of the registry. So, my question is, do you find the above definitions confusing/inaccurate/incomplete, or do you believe that the terms are used with a different meaning in the draft or else? I mean, i wonder if we should focus our effort in improving the defintion or how the terms are used in the body of the draft. Regards, marcelo El 27/01/14 21:32, Charles Cook escribió: > Phil, > > I am in the process of reviewing the draft. I like how many things > have been improved. Thanks for the good work. > > That said, I need some clarity of certain terms and how they relate to > each other, because I am getting confused as I read through the draft. > > The terms I am concerned about are: > > * Registry > * Metric > * Measurement Task > * Measurement Method > > > It appears that Measurement Task and Measurement Method are sometimes > used synonymously, and sometimes they appear to be very different. > Sometimes a Measurement Method appears to be a Metric with its > associated parameters not populated. Sometimes Measurement Method > seems to refer to a Measurement Task with variables that have not been > populated. Generally the term "method" is a synonym with "procedure", > which is generally how I interpret the term "method". Maybe the use > of "method" rather than "procedure" is causing part of my confusion. > Maybe part of my problem is that the Registry, its role, and what is > stored in the Registry is not well described. > > Here is what I thought was going on: > > 1. A registry exists somewhere that contains a list of the > standardised metrics of interest. > 2. The metric may be composed of some parameters that can be varied. > For example, a metric called "Throughput" may have parameters > like, originating end point, terminating endpoint, traffic type > (UDP/TCP), etc. > 3. A Measurement Task is the result of the Controller populating the > variables of the standardised metric. > 4. A Measurement Method would define particular constraints that the > measurements must be performed under in order for the Measurement > Results to be comparable with other Measurement Results of the > same Measurement Task performed at different times and/or > locations. This could be verification that there are sufficient > resources to perform the Measurement Task (the MA has sufficient > processing power), the path being measured is a direct path versus > a derived path, a result that requires multiple iterations > averaged together has confirmation that the same path is used for > each iteration, etc. Note that is it not enough for two > Measurement Tasks to be comparable simply because their parameters > are the same. Certain external conditions must also be the same > in order for the results to be comparable. > > Here is what the draft appears to say: > > 1. A registry exists somewhere that contains a list of standardised > Measurement Methods. > 2. Measurement Methods consist of specific actions that have > parameters associated with them that have not been populated. > 3. A Measurement Task is the result of the Controller populating the > parameters of the standardised Measurement Method. > 4. No discussion of what mechanism is used to ensure that Measurement > Tasks conducted at different times and places can be comparable. > > Is there a gap? After we clarify what the intent is, I can re-read > the draft and suggest clarifying comments. > > Thanks, > Charles > > > > > > On 1/21/2014 8:00 AM, philip.eardley@bt.com wrote: >> We submitted a new version of the framework document. >> >> http://tools.ietf.org/html/draft-ietf-lmap-framework >> >> Here's a summary of the changes: >> o alignment with the Information Model >> [I-D.burbridge-lmap-information-model] as this is agreed as a WG >> document >> >> o One-off and periodic Measurement Schedules are kept separate, so >> that they can be updated independently >> >> o Measurement Suppression in a separate sub-section. Can now >> optionally include particular Measurement Tasks &/or Schedules to >> suppress, and start/stop time >> >> o for clarity, concept of Channel split into Control, Report and MA- >> to-Controller Channels >> >> o numerous editorial changes, mainly arising from a very detailed >> review by Charles Cook >> >> best wishes, >> phil, al, marcelo, trevor, paul, aamer >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap > > -- > > Charles Cook > Principal Architect > Network > 5325 Zuni Street; Suite 224 > Denver, CO 80221 > Tel: 303.992.8952 Fax: 925.281.0662 > charles.cook2@centurylink.com > > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From charles.cook2@centurylink.com Mon Jan 27 12:54:00 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 152861A0368 for ; Mon, 27 Jan 2014 12:54:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DbfHXVrcFufp for ; Mon, 27 Jan 2014 12:53:57 -0800 (PST) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id A8CA11A00F6 for ; Mon, 27 Jan 2014 12:53:57 -0800 (PST) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id s0RKrsS4027064 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 27 Jan 2014 14:53:54 -0600 (CST) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 297791E012D; Mon, 27 Jan 2014 14:53:49 -0600 (CST) Received: from suomp61i.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id 182651E005D; Mon, 27 Jan 2014 14:53:49 -0600 (CST) Received: from suomp61i.qintra.com (localhost [127.0.0.1]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id s0RKrm5Y024802; Mon, 27 Jan 2014 14:53:48 -0600 (CST) Received: from [10.188.109.177] (x1069818.dhcp.intranet [10.188.109.177]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id s0RKrlQb024742; Mon, 27 Jan 2014 14:53:47 -0600 (CST) Message-ID: <52E6C75C.7070504@centurylink.com> Date: Mon, 27 Jan 2014 13:53:48 -0700 From: Charles Cook User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121005 Thunderbird/16.0 MIME-Version: 1.0 To: marcelo bagnulo braun References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> <52E6C274.9090401@centurylink.com> <52E6C4E6.3060705@it.uc3m.es> In-Reply-To: <52E6C4E6.3060705@it.uc3m.es> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-CFilter-Loop: Reflected Cc: lmap@ietf.org Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: charles.cook2@centurylink.com List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jan 2014 20:54:00 -0000 Marcelo, For the most part, I am finding that the terms are used with a different meanings in the draft. I also find use of the term "generalisation" to be confusing. However, I would focus more on how the terms are used in the draft. Charles On 1/27/2014 1:43 PM, marcelo bagnulo braun wrote: > Hi Charles, > > thanks for the comments. > > In section 3 of the draft the following defintions are included: > > Measurement Method: The process for assessing the value of a Metric; > the process of measuring some performance or reliability parameter; > the generalisation of a Measurement Task. > > Measurement Task: The act that yields a single Measurement Result; > the act consisting of the (single) operation of the Measurement > Method at a particular time and with all its parameters set to > specific values. > > Measurement Result: The output of a single Measurement Task (the > value obtained for the parameter of interest or Metric). > > Metric: The quantity related to the performance and reliability of > the network that we'd like to know the value of, and that is > carefully specified. > > > Registry is not defined, nor metric registry nor entries of the registry. > > So, my question is, do you find the above definitions > confusing/inaccurate/incomplete, or do you believe that the terms are > used with a different meaning in the draft or else? > > I mean, i wonder if we should focus our effort in improving the > defintion or how the terms are used in the body of the draft. > > Regards, marcelo > > > > > > El 27/01/14 21:32, Charles Cook escribió: >> Phil, >> >> I am in the process of reviewing the draft. I like how many things >> have been improved. Thanks for the good work. >> >> That said, I need some clarity of certain terms and how they relate >> to each other, because I am getting confused as I read through the >> draft. >> >> The terms I am concerned about are: >> >> * Registry >> * Metric >> * Measurement Task >> * Measurement Method >> >> >> It appears that Measurement Task and Measurement Method are sometimes >> used synonymously, and sometimes they appear to be very different. >> Sometimes a Measurement Method appears to be a Metric with its >> associated parameters not populated. Sometimes Measurement Method >> seems to refer to a Measurement Task with variables that have not >> been populated. Generally the term "method" is a synonym with >> "procedure", which is generally how I interpret the term "method". >> Maybe the use of "method" rather than "procedure" is causing part of >> my confusion. Maybe part of my problem is that the Registry, its >> role, and what is stored in the Registry is not well described. >> >> Here is what I thought was going on: >> >> 1. A registry exists somewhere that contains a list of the >> standardised metrics of interest. >> 2. The metric may be composed of some parameters that can be varied. >> For example, a metric called "Throughput" may have parameters >> like, originating end point, terminating endpoint, traffic type >> (UDP/TCP), etc. >> 3. A Measurement Task is the result of the Controller populating the >> variables of the standardised metric. >> 4. A Measurement Method would define particular constraints that the >> measurements must be performed under in order for the Measurement >> Results to be comparable with other Measurement Results of the >> same Measurement Task performed at different times and/or >> locations. This could be verification that there are sufficient >> resources to perform the Measurement Task (the MA has sufficient >> processing power), the path being measured is a direct path versus >> a derived path, a result that requires multiple iterations >> averaged together has confirmation that the same path is used for >> each iteration, etc. Note that is it not enough for two >> Measurement Tasks to be comparable simply because their parameters >> are the same. Certain external conditions must also be the same >> in order for the results to be comparable. >> >> Here is what the draft appears to say: >> >> 1. A registry exists somewhere that contains a list of standardised >> Measurement Methods. >> 2. Measurement Methods consist of specific actions that have >> parameters associated with them that have not been populated. >> 3. A Measurement Task is the result of the Controller populating the >> parameters of the standardised Measurement Method. >> 4. No discussion of what mechanism is used to ensure that Measurement >> Tasks conducted at different times and places can be comparable. >> >> Is there a gap? After we clarify what the intent is, I can re-read >> the draft and suggest clarifying comments. >> >> Thanks, >> Charles >> >> >> >> >> >> On 1/21/2014 8:00 AM, philip.eardley@bt.com wrote: >>> We submitted a new version of the framework document. >>> >>> http://tools.ietf.org/html/draft-ietf-lmap-framework >>> Here's a summary of the changes: >>> o alignment with the Information Model >>> [I-D.burbridge-lmap-information-model] as this is agreed as a WG >>> document >>> >>> o One-off and periodic Measurement Schedules are kept separate, so >>> that they can be updated independently >>> >>> o Measurement Suppression in a separate sub-section. Can now >>> optionally include particular Measurement Tasks &/or >>> Schedules to >>> suppress, and start/stop time >>> >>> o for clarity, concept of Channel split into Control, Report >>> and MA- >>> to-Controller Channels >>> >>> o numerous editorial changes, mainly arising from a very detailed >>> review by Charles Cook >>> >>> best wishes, >>> phil, al, marcelo, trevor, paul, aamer >>> >>> _______________________________________________ >>> lmap mailing list >>> lmap@ietf.org >>> https://www.ietf.org/mailman/listinfo/lmap >> >> -- >> >> Charles Cook >> Principal Architect >> Network >> 5325 Zuni Street; Suite 224 >> Denver, CO 80221 >> Tel: 303.992.8952 Fax: 925.281.0662 >> charles.cook2@centurylink.com >> >> >> _______________________________________________ >> lmap mailing list >> lmap@ietf.org >> https://www.ietf.org/mailman/listinfo/lmap > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap -- Charles Cook Principal Architect Network 5325 Zuni Street; Suite 224 Denver, CO 80221 Tel: 303.992.8952 Fax: 925.281.0662 charles.cook2@centurylink.com From gregimirsky@gmail.com Mon Jan 27 16:46:28 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9F7F01A02DA for ; Mon, 27 Jan 2014 16:46:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NaFVVcPPccDI for ; Mon, 27 Jan 2014 16:46:26 -0800 (PST) Received: from mail-ve0-x22d.google.com (mail-ve0-x22d.google.com [IPv6:2607:f8b0:400c:c01::22d]) by ietfa.amsl.com (Postfix) with ESMTP id E40731A008F for ; Mon, 27 Jan 2014 16:46:25 -0800 (PST) Received: by mail-ve0-f173.google.com with SMTP id oz11so3946935veb.18 for ; Mon, 27 Jan 2014 16:46:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pi9DH/SgDr+Ba1cYJm6KKtOKOUAYz53b/h0cVPOi3hQ=; b=SQOGKjW0PYjrBE3AWdtxGzcJMO5/IR7j3ESisIADJoqTYHOShvF0jIVm8PG7a7NPEI J90Cq7eoQl0N7fg9G4F0knhHoIjdH8OgzeKm/99yTv20t00ee/hUGcdYs3Cp8XxzF/3Y w2ZMLABrmSMVgyporz5A94CC1HlVJEN5QRRR7pcGERdbEZG+QlHVFw75GRgmEuOqq3R9 zqW2dogJvaQz4y/Rav/sHD3LqSZxvEfSYdVCDHVac5AQjekmrEl6KIv23JVT0dxK2KPx SJFOpEYw25ZT2MxZxj3B5Fy9wy0qrQW/RMKBBb1NR8l73CRTFT7c/WZDP7lNPZYbhhbP DHgQ== MIME-Version: 1.0 X-Received: by 10.52.246.133 with SMTP id xw5mr105829vdc.32.1390869983224; Mon, 27 Jan 2014 16:46:23 -0800 (PST) Received: by 10.220.129.207 with HTTP; Mon, 27 Jan 2014 16:46:23 -0800 (PST) In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA243D862C@AZ-FFEXMB04.global.avaya.com> References: <9904FB1B0159DA42B0B887B7FA8119CA243D862C@AZ-FFEXMB04.global.avaya.com> Date: Mon, 27 Jan 2014 16:46:23 -0800 Message-ID: From: Greg Mirsky To: "Romascanu, Dan (Dan)" Content-Type: multipart/alternative; boundary=001a1133f0b8477c4e04f0fd277d Cc: "lmap@ietf.org" Subject: Re: [lmap] WGLC for http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jan 2014 00:46:28 -0000 --001a1133f0b8477c4e04f0fd277d Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Dear Authors, please find my comments to the document: - Document needs table of Acronyms - Introduction, second paragraph. I think that reference to out-of-scope use cases is vague as charter of the LMAP WG may change over time. Perhaps it can be edited to "The LMAP framework should be useful for these such as to help end users to run diagnostic checks". - text of the third paragraph in Introduction needs tightening. Use of "perhaps" makes it less clear. - I'd consider to use Measurement Method rather than Measurement Task considering that Sec.2 describes the latter as instantiation of the form= er. - I think that "Measurement Tasks may be Active (the MA or Measurement Peer generates Active Measurement Traffic), Passive (the MA observes use= r traffic), or some hybrid form of the two" refers to Measurement Methods, not Measurement Tasks. - s/but that are/but these are/ - I think that achieving Standardization of Measurement Methods across arbitrary domain of heterogeneous Measurement Agents would encompass goa= ls one and three outlined in the Introduction. - Section 2 and Figure 1. I think that limiting scope of LMAP to only single-ended two-way Measurement Methods is too restrictive. For example= , Passive Measurement Methods would likely to be one-way and Controller wo= uld be required to coordinate Measurement Task on both Measurement Agent and Measurement Peer (yes, I disagree with the assumption that passive metho= d requires participation of only MA). And it is likely that the Measuremen= t Peer would be exporting Measurement Results to a Collector. If the Figur= e 1 considered as reference model for LMAP it has to present more generic ca= se. - Section 2 "The next components we consider are the Measurement Agent (MA), Controller and Collector." Should it list Measurement Peer or acknowledge that Measurement Peer is just an instance of Measurement Age= nt. - Section 3. In definition of Active Measurement Method does not include case when both MA and Management Peer inject Active Measurement Traffic. Please consider "... where the Measurement Agent, the Measurement Peer o= r both inject Active Measurement Traffic ..." - Section 3. Please consider s/Capabilities/Measurement Capabilities/ - Section 3. I think that Control Channel exists not only between Controller and MA but between Controller and Measurement Peer. - Section 3. What differentiates Control Channel and MA-to-Controller Channel? And why Control Channel not listed in definition of a Channel along with other types of LMAP channels? Regards, Greg On Thu, Jan 23, 2014 at 5:30 AM, Romascanu, Dan (Dan) w= rote: > This is a Working Group Last Call for the Internet-Draft =91A framework = for large-scale measurement platforms (LMAP)=92 which can be found at > > http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt. Please send > your comments about this I-D to the WG mail list until 2/6/13. If you ha= ve > no comments but you believe that the document is ready to be submitted to > the IESG for consideration as Informational RFC a short mail stating this > is also welcome. > > > > Thanks and Regards, > > > > Dan > > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap > > --001a1133f0b8477c4e04f0fd277d Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Dear Authors,
please find my comme= nts to the document:
  • Document needs table of Acronyms
  • In= troduction, second paragraph. I think that reference to out-of-scope use ca= ses is vague as charter of the LMAP WG may change over time. Perhaps it= can be edited to "The LMAP framework should be useful for these such as to help end users to run diagnostic checks".
  • = text of the third paragraph in Introduction needs tightening. Use of "= perhaps" makes it less clear.
  • I'd consider to use Measurem= ent Method rather than Measurement Task considering that Sec.2 describes th= e latter as instantiation of the former.
  • I think that "Measurement Tasks may be Active (the MA or Meas= urement Peer generates Active Measurement Traffic), Passive (the MA observes user traffic), or some hybrid form of the two" refers to Measurement Methods, not = Measurement Tasks.
  • s/but that are/but these are/
  • I think th= at achieving Standardization of Measurement Methods across arbitrary domain= of heterogeneous Measurement Agents would encompass goals one and three ou= tlined in the Introduction.
  • Section 2 and Figure 1. I think that limiting scope of LMAP to only sin= gle-ended two-way Measurement Methods is too restrictive. For example, Pass= ive Measurement Methods would likely to be one-way and Controller would be = required to coordinate Measurement Task on both Measurement Agent and Measu= rement Peer (yes, I disagree with the assumption that passive method requir= es participation of only MA). And it is likely that the Measurement Peer wo= uld be exporting Measurement Results to a Collector. If the Figure 1 consid= ered as reference model for LMAP it has to present more generic case.
  • Section 2 "The next components we consider are the Measuremen= t Agent (MA), Controller and Collector." Should it list Measurement Peer or ackno= wledge that Measurement Peer is just an instance of Measurement Agent.
    • =
  • Section 3. In definition of Active Measurement Method does not include = case when both MA and Management Peer inject Active Measurement Traffic. Pl= ease consider "... where the Measurement Agent, the Measurement Peer or both inject Active Measurement Traffic ..."
  • Section 3. P= lease consider s/Capabilities/Measurement Capabilities/
  • Section 3. = I think that Control Channel exists not only between Controller and MA but = between Controller and Measurement Peer.
  • Section 3. What differentiates Control Channel and MA-to-Controller Cha= nnel? And why Control Channel not listed in definition of a Channel along w= ith other types of LMAP channels?
Regards,
Greg


On Thu, Jan 23, 2014 at 5:30 AM, Romascanu, Dan (Dan) &l= t;dromasca@avaya.co= m> wrote:
This is a Working Group Last Call for the Internet-Draft =91A framewor=
k for large-scale measurement platforms (LMAP)=92 which can be found at =

=A0http://www.ietf.org/id/draft-ietf-lmap= -framework-03.txt. Please send your comments about this I-D to the WG m= ail list until 2/6/13. =A0If you have no comments but you believe that the document is ready to be submitted to the IESG for conside= ration as Informational RFC a short mail stating this is also welcome.

=A0

Thanks and Regards,

=A0

Dan


_______________________________________________
lmap mailing list
lmap@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/lmap


--001a1133f0b8477c4e04f0fd277d-- From philip.eardley@bt.com Tue Jan 28 04:24:40 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6EED21A01E4 for ; Tue, 28 Jan 2014 04:24:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z78rMv9BXhsH for ; Tue, 28 Jan 2014 04:24:37 -0800 (PST) Received: from smtpe1.intersmtp.com (smtpe1.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id 71B141A01E8 for ; Tue, 28 Jan 2014 04:24:36 -0800 (PST) Received: from EVMHT69-UKRD.domain1.systemhost.net (10.36.3.129) by RDW083A006ED62.bt.com (10.187.98.11) with Microsoft SMTP Server (TLS) id 14.3.169.1; Tue, 28 Jan 2014 12:24:35 +0000 Received: from EMV67-UKRD.domain1.systemhost.net ([169.254.2.146]) by EVMHT69-UKRD.domain1.systemhost.net ([10.36.3.129]) with mapi; Tue, 28 Jan 2014 12:24:32 +0000 From: To: , Date: Tue, 28 Jan 2014 12:24:31 +0000 Thread-Topic: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt Thread-Index: Ac8boej4wfKB3uBkSG2WZzu4qHLocQAaylPA Message-ID: References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> <52E6C274.9090401@centurylink.com> <52E6C4E6.3060705@it.uc3m.es> <52E6C75C.7070504@centurylink.com> In-Reply-To: <52E6C75C.7070504@centurylink.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: lmap@ietf.org Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jan 2014 12:24:40 -0000 Charles, Thanks for the comments! > >> Here is what the draft appears to say: > >> > >> 1. A registry exists somewhere that contains a list of > standardised > >> Measurement Methods. > >> 2. Measurement Methods consist of specific actions that have > >> parameters associated with them that have not been populated. > >> 3. A Measurement Task is the result of the Controller populating > the > >> parameters of the standardised Measurement Method. > >> 4. No discussion of what mechanism is used to ensure that > Measurement > >> Tasks conducted at different times and places can be comparable. I think the above is basically correct. > >> It appears that Measurement Task and Measurement Method are > sometimes > >> used synonymously,=20 [phil] such uses are wrong > >> and sometimes they appear to be very different. > >> Sometimes a Measurement Method appears to be a Metric with its > >> associated parameters not populated. =20 [phil] such a use is wrong. A Measurement Method is how the value of a Metr= ic is measured. > >> Sometimes Measurement Method > >> seems to refer to a Measurement Task with variables that have not > >> been populated. Generally the term "method" is a synonym with > >> "procedure", which is generally how I interpret the term "method". > >> Maybe the use of "method" rather than "procedure" is causing part of > >> my confusion. Maybe part of my problem is that the Registry, its > >> role, and what is stored in the Registry is not well described. [phil] I can sympathise with this. We're working on an overall draft descri= bing a Registry for performance metrics. This will appear in IPPM WG soon (= hopefully).=20 This will be a registry of IETF registered metrics. An entry in the registr= y defines the Measurement Method. However (at least in my mind) the operator of the measurement system may de= fine their own registry with their own Measurement Methods.=20 > >> Here is what I thought was going on: > >> > >> 1. A registry exists somewhere that contains a list of the > >> standardised metrics of interest. [phil] the ietf-iana registry has the list, yes. > >> 2. The metric may be composed of some parameters that can be > varied. > >> For example, a metric called "Throughput" may have parameters > >> like, originating end point, terminating endpoint, traffic type > >> (UDP/TCP), etc. [phil] the Measurement Method has some variable parameters that don't affec= t the fundamental nature of the test. End points are good examples. Traffic= type is not. The Metric is the thing measured by the Measurement Method - the thing we w= ant to know the value of.=20 So if the Metric is "UDP One-way-packet delay variation 95% poisson", the M= ethod might be "sections 2.4 and 3.4 of [RFC3393] with 200 bit packet, dscp= =3D0, poisson distribution with lambda =3D1sec, output 95%-ile, " =09 > >> 3. A Measurement Task is the result of the Controller populating > the > >> variables of the standardised metric. [phil] the Measurement Task is a particular measurement made using the Meas= urement Method with its Parameters set and at a particular time. Yes, the C= ontroller tells the MA the Measurement Method, Parameters & the Schedule.=20 > >> 4. A Measurement Method would define particular constraints that > the > >> measurements must be performed under in order for the > Measurement > >> Results to be comparable with other Measurement Results of the > >> same Measurement Task performed at different times and/or > >> locations. =20 [phil] the Measurement Method should be well enough defined that Measuremen= t Tasks (ie particular measurements that use the Measurement Method) produc= e Measurement Results that it is reasonable to compare > >> This could be verification that there are sufficient > >> resources to perform the Measurement Task (the MA has sufficient > >> processing power), the path being measured is a direct path > versus > >> a derived path, a result that requires multiple iterations > >> averaged together has confirmation that the same path is used > for > >> each iteration, etc. Note that is it not enough for two > >> Measurement Tasks to be comparable simply because their > parameters > >> are the same. Certain external conditions must also be the same > >> in order for the results to be comparable. [phil] it depends on how the Measurement Method is defined. The Method could include a check that the end user isn't active, the path f= ollowed is the one expected etc, and if it fails then the Task doesn't send= Active Measurement Traffic. Then the Result is "No Active Measurement Traf= fic sent due to xy" On the other hand...=09 The Method could include a check that the end user isn't active, the path f= ollowed is the one expected etc, but (whatever the result of the check) Act= ive Measurement Traffic is sent. Then the Result is "Metric measured as 47,= but caution - measurement made under xy conditions so Result may be suspec= t" In both cases, it is reasonable to compare the Measurement Results, but in = the second case the analysis needs to exercise more caution. Of course, comparability may need some extra work. For example, a regulator= comparing two operators would need to make sure that measurements were mad= e at similar times (not busy hour vs off peak), on similar access lines (no= t ones next to exchange vs furthest point) etc.=20 Re verification that the MA has sufficient processing power. I'd put this i= n a different category to your other checks. It should be reported as "Task= not run - failed due to lack of MA processing power"; there is no Measurem= ent Result. The operator of the measurement system should know the capabili= ties of the device with the MA, and shouldn't be anywhere near stretching i= ts capabilities, unless there's some failure condition like the device is i= n the middle of crashing.=20 Thanks phil > -----Original Message----- > From: lmap [mailto:lmap-bounces@ietf.org] On Behalf Of Charles Cook > Sent: 27 January 2014 20:54 > To: marcelo bagnulo braun > Cc: lmap@ietf.org > Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap- > framework-03.txt >=20 > Marcelo, >=20 > For the most part, I am finding that the terms are used with a > different meanings in the draft. I also find use of the term > "generalisation" to be confusing. However, I would focus more on how > the terms are used in the draft. >=20 > Charles >=20 > On 1/27/2014 1:43 PM, marcelo bagnulo braun wrote: > > Hi Charles, > > > > thanks for the comments. > > > > In section 3 of the draft the following defintions are included: > > > > Measurement Method: The process for assessing the value of a > Metric; > > the process of measuring some performance or reliability > parameter; > > the generalisation of a Measurement Task. > > > > Measurement Task: The act that yields a single Measurement Result; > > the act consisting of the (single) operation of the Measurement > > Method at a particular time and with all its parameters set to > > specific values. > > > > Measurement Result: The output of a single Measurement Task (the > > value obtained for the parameter of interest or Metric). > > > > Metric: The quantity related to the performance and reliability of > > the network that we'd like to know the value of, and that is > > carefully specified. > > > > > > Registry is not defined, nor metric registry nor entries of the > registry. > > > > So, my question is, do you find the above definitions > > confusing/inaccurate/incomplete, or do you believe that the terms are > > used with a different meaning in the draft or else? > > > > I mean, i wonder if we should focus our effort in improving the > > defintion or how the terms are used in the body of the draft. > > > > Regards, marcelo > > > > > > > > > > > > El 27/01/14 21:32, Charles Cook escribi=F3: > >> Phil, > >> > >> I am in the process of reviewing the draft. I like how many things > >> have been improved. Thanks for the good work. > >> > >> That said, I need some clarity of certain terms and how they relate > >> to each other, because I am getting confused as I read through the > >> draft. > >> > >> The terms I am concerned about are: > >> > >> * Registry > >> * Metric > >> * Measurement Task > >> * Measurement Method > >> > >> > >> It appears that Measurement Task and Measurement Method are > sometimes > >> used synonymously, and sometimes they appear to be very different. > >> Sometimes a Measurement Method appears to be a Metric with its > >> associated parameters not populated. Sometimes Measurement Method > >> seems to refer to a Measurement Task with variables that have not > >> been populated. Generally the term "method" is a synonym with > >> "procedure", which is generally how I interpret the term "method". > >> Maybe the use of "method" rather than "procedure" is causing part of > >> my confusion. Maybe part of my problem is that the Registry, its > >> role, and what is stored in the Registry is not well described. > >> > >> Here is what I thought was going on: > >> > >> 1. A registry exists somewhere that contains a list of the > >> standardised metrics of interest. > >> 2. The metric may be composed of some parameters that can be > varied. > >> For example, a metric called "Throughput" may have parameters > >> like, originating end point, terminating endpoint, traffic type > >> (UDP/TCP), etc. > >> 3. A Measurement Task is the result of the Controller populating > the > >> variables of the standardised metric. > >> 4. A Measurement Method would define particular constraints that > the > >> measurements must be performed under in order for the > Measurement > >> Results to be comparable with other Measurement Results of the > >> same Measurement Task performed at different times and/or > >> locations. This could be verification that there are sufficient > >> resources to perform the Measurement Task (the MA has sufficient > >> processing power), the path being measured is a direct path > versus > >> a derived path, a result that requires multiple iterations > >> averaged together has confirmation that the same path is used > for > >> each iteration, etc. Note that is it not enough for two > >> Measurement Tasks to be comparable simply because their > parameters > >> are the same. Certain external conditions must also be the same > >> in order for the results to be comparable. > >> > >> Here is what the draft appears to say: > >> > >> 1. A registry exists somewhere that contains a list of > standardised > >> Measurement Methods. > >> 2. Measurement Methods consist of specific actions that have > >> parameters associated with them that have not been populated. > >> 3. A Measurement Task is the result of the Controller populating > the > >> parameters of the standardised Measurement Method. > >> 4. No discussion of what mechanism is used to ensure that > Measurement > >> Tasks conducted at different times and places can be comparable. > >> > >> Is there a gap? After we clarify what the intent is, I can re-read > >> the draft and suggest clarifying comments. > >> > >> Thanks, > >> Charles > >> > >> > >> > >> > >> > >> On 1/21/2014 8:00 AM, philip.eardley@bt.com wrote: > >>> We submitted a new version of the framework document. > >>> > >>> http://tools.ietf.org/html/draft-ietf-lmap-framework > >>> Here's a summary of the changes: > >>> o alignment with the Information Model > >>> [I-D.burbridge-lmap-information-model] as this is agreed as > a WG > >>> document > >>> > >>> o One-off and periodic Measurement Schedules are kept > separate, so > >>> that they can be updated independently > >>> > >>> o Measurement Suppression in a separate sub-section. Can now > >>> optionally include particular Measurement Tasks &/or > >>> Schedules to > >>> suppress, and start/stop time > >>> > >>> o for clarity, concept of Channel split into Control, Report > >>> and MA- > >>> to-Controller Channels > >>> > >>> o numerous editorial changes, mainly arising from a very > detailed > >>> review by Charles Cook > >>> > >>> best wishes, > >>> phil, al, marcelo, trevor, paul, aamer > >>> > >>> _______________________________________________ > >>> lmap mailing list > >>> lmap@ietf.org > >>> https://www.ietf.org/mailman/listinfo/lmap > >> > >> -- > >> > >> Charles Cook > >> Principal Architect > >> Network > >> 5325 Zuni Street; Suite 224 > >> Denver, CO 80221 > >> Tel: 303.992.8952 Fax: 925.281.0662 charles.cook2@centurylink.com > >> > >> > >> _______________________________________________ > >> lmap mailing list > >> lmap@ietf.org > >> https://www.ietf.org/mailman/listinfo/lmap > > > > _______________________________________________ > > lmap mailing list > > lmap@ietf.org > > https://www.ietf.org/mailman/listinfo/lmap >=20 > -- >=20 > Charles Cook > Principal Architect > Network > 5325 Zuni Street; Suite 224 > Denver, CO 80221 > Tel: 303.992.8952 Fax: 925.281.0662 > charles.cook2@centurylink.com >=20 > _______________________________________________ > lmap mailing list > lmap@ietf.org > https://www.ietf.org/mailman/listinfo/lmap From charles.cook2@centurylink.com Tue Jan 28 06:02:40 2014 Return-Path: X-Original-To: lmap@ietfa.amsl.com Delivered-To: lmap@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 18DA91A020E for ; Tue, 28 Jan 2014 06:02:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xgnAjJ0WFgHp for ; Tue, 28 Jan 2014 06:02:39 -0800 (PST) Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by ietfa.amsl.com (Postfix) with ESMTP id ECABC1A0216 for ; Tue, 28 Jan 2014 06:02:38 -0800 (PST) Received: from lxomavmpc030.qintra.com (lxomavmpc030.qintra.com [151.117.207.30]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id s0SE2I0L009242 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 28 Jan 2014 08:02:19 -0600 (CST) Received: from lxomavmpc030.qintra.com (unknown [127.0.0.1]) by IMSA (Postfix) with ESMTP id 6FB191E005D; Tue, 28 Jan 2014 08:02:13 -0600 (CST) Received: from sudnp797.qintra.com (unknown [10.6.10.61]) by lxomavmpc030.qintra.com (Postfix) with ESMTP id 4AD501E0058; Tue, 28 Jan 2014 08:02:13 -0600 (CST) Received: from sudnp797.qintra.com (localhost [127.0.0.1]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id s0SE2CRA029698; Tue, 28 Jan 2014 07:02:12 -0700 (MST) Received: from [10.188.126.111] (x1069818.dhcp.intranet [10.188.126.111]) by sudnp797.qintra.com (8.14.4/8.14.4) with ESMTP id s0SE2CBl029692; Tue, 28 Jan 2014 07:02:12 -0700 (MST) Message-ID: <52E7B863.5050209@centurylink.com> Date: Tue, 28 Jan 2014 07:02:11 -0700 From: Charles Cook User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121005 Thunderbird/16.0 MIME-Version: 1.0 To: philip.eardley@bt.com References: <20140121145650.5390.63718.idtracker@ietfa.amsl.com> <52E6C274.9090401@centurylink.com> <52E6C4E6.3060705@it.uc3m.es> <52E6C75C.7070504@centurylink.com> In-Reply-To: X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-CFilter-Loop: Reflected Cc: marcelo@it.uc3m.es, lmap@ietf.org Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt X-BeenThere: lmap@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: charles.cook2@centurylink.com List-Id: Large Scale Measurement of Access network Performance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jan 2014 14:02:40 -0000 Phil, Thanks for the clarifications. They are helpful. I plan to re-read the draft based on these clarifications. Charles On 1/28/2014 5:24 AM, philip.eardley@bt.com wrote: >>>> > >> parameters associated with them that have not been populated. >>>> > >> 3. A Measurement Task is the result of the Controller populating >> > the >>>> > >> parameters of the standardised Measurement Method. >>>> > >> 4. No discussion of what mechanism is used to ensure that >> > Measurement >>>> > >> Tasks conducted at different times and places can be comparable. > I think the above is basically correct. > >>>> > >> It appears that Measurement Task and Measurement Method are -- Charles Cook Principal Architect Network 5325 Zuni Street; Suite 224 Denver, CO 80221 Tel: 303.992.8952 Fax: 925.281.0662 charles.cook2@centurylink.com