Subject: [Maprg] Measurement Tools and Platforms

From: [email-masked]
Date: Thu, 20 Oct 2016 20:50:55 +1100

Hi all,

I wonder why measurement Lab https://www.measurementlab.net/ is not 
listed in Measurement Tools and Platforms, under the MAPRG wiki page?

Any comments?

Thanks,
-x

---

From: [email-masked]
Date: Thu, 20 Oct 2016 20:47:12 +1100

Hi WG and Chairs,

I notice the first call for contribution deadline has passed. Don't know 
if there is a second round call. If so, I'd send a request in between. 
:) Either way, please give a feedback.

I'd like request 20 minutes on Maprg Soul meeting to present the 
following topic (which has been presented at AusNoG Sep. 2016)

A link to the presentation:
http://www.ausnog.net/sites/default/files/ausnog-2016/presentations/1.6_Xiaohong_Deng_AusNOG2016.pdf

Abstract of the talk:
Broadband network performance is multi-faceted, dependent on factors 
such as access link characteristics, speed-tier, server distance, host 
buffers, ISP network dimensioning, and time-of-day. Daily or monthly 
aggregates, published by content providers such as Netflix and Youtube, 
present a highly simplified view that do not distinguish the impact of 
the above factors, making the comparisons potentially unfair and 
incorrect. Our paper revisits broadband performance using open data from 
M-Lab, utilizing nearly 5 million measurements taken over a 12-month 
period across 19 ISPs in 3 continents. We first develop a tool that 
allows a non-expert to query, process, and visualize M-Lab data by 
applying various filters and granularities via a simple web-based 
interface. Using this tool, we characterize the influence of the various 
factors, and find that there are significant biases affecting the 
averages, including disparities amongst households in their frequency of 
testing and access speed-tier, that contribute to fluctuations and 
biases in ISP performance comparisons. We then apply statistical and 
causal inference techniques to reduce the sampling bias, and find that 
the disparity amongst ISPs is lower than current averaging methods 
indicate, with hourly variations giving a truer indication of 
well-dimensioned ISP networks.

Thank you,
-x

---

From: Dave Plonka <[email-masked]>
Date: Thu, 20 Oct 2016 10:19:01 -0500

Hi Xiaohong,

Mirja and I will be meeting next week to consider the set of submissions
for MAPRG in Seoul.

I won't be there because IETF collides with IMC (in Santa Monica) that
week, but I will present a WIP on MAPRG at IMC.

We haven't determined a need for a second call, but we can consider yours
amongst them if there might be time available in the meeting.

Thanks,
Dave

On Thu, Oct 20, 2016 at 4:47 AM, <[email-masked]> wrote:

> if there is a second round call. If so, I'd send a request in between. :)
> Either way, please give a feedback.
>
> I'd like request 20 minutes on Maprg Soul meeting to present the following
> topic (which has been presented at AusNoG Sep. 2016)
>
> Broadband network performance is multi-faceted, dependent on factors such
> as access link characteristics, speed-tier, server distance, host buffers,
> ISP network dimensioning, and time-of-day. Daily or monthly aggregates,
> published by content providers such as Netflix and Youtube, present a
> highly simplified view that do not distinguish the impact of the above
> factors, making the comparisons potentially unfair and incorrect. Our paper
> revisits broadband performance using open data from M-Lab, utilizing nearly
> 5 million measurements taken over a 12-month period across 19 ISPs in 3
> continents. We first develop a tool that allows a non-expert to query,
> process, and visualize M-Lab data by applying various filters and
> granularities via a simple web-based interface. Using this tool, we
> characterize the influence of the various factors, and find that there are
> significant biases affecting the averages, including disparities amongst
> households in their frequency of testing and access speed-tier, that
> contribute to fluctuations and biases in ISP performance comparisons. We
> then apply statistical and causal inference techniques to reduce the
> sampling bias, and find that the disparity amongst ISPs is lower than
> current averaging methods indicate, with hourly variations giving a truer
> indication of well-dimensioned ISP networks.
>
> Thank you,
> -x
>

-- 

[email-masked]  http://www.cs.wisc.edu/~plonka/

---

From: Dave Plonka <[email-masked]>
Date: Thu, 20 Oct 2016 10:27:22 -0500

Assuming you meant to request it be added, I did so. :)

https://trac.tools.ietf.org/group/irtf/trac/wiki/MeasurementTools

Thanks,
Dave

--

---

From: [email-masked]
Date: Fri, 21 Oct 2016 16:22:12 +1100

Exactly. Thank you, Dave.

-x
在 2016-10-21 02:27，Dave Plonka 写道：
>> I wonder why measurement Lab https://www.measurementlab.net/ [1] is
>> not listed in Measurement Tools and Platforms, under the MAPRG wiki
>> page?
>> 
>> Any comments?
>> 
>> Thanks,
>> -x
> 
> --
> 
> Links:
> ------
> [1] https://www.measurementlab.net/

---

From: [email-masked]
Date: Mon, 24 Oct 2016 23:55:11 +1100

Hi Amreesh and Jose,

Thank you for your interest.

A paper is now under working for a Sigmetrics submission. Yet, we do 
have preliminary results published on this website:
http://fair-isp-compari.internet-measurement.com/

You may also like to try our tool:
http://vis-mlab.internet-measurement.com/
to gain some insights from visualised ISP performance and individual 
household performance and its correlation with impacting factors such as 
household Speed Tier, server client Distance and Host Buffer size, etc.

The NDT data presented with our vis tool website is from Jan 2016 to Sep 
2016 (last month). And we'll keep updating the data accordingly to NDT 
data availability.

Kind Regards,
Xiaohong
  2016-10-20 22:34，Amreesh Phokeer ：
> Hello Xiaohong,
> 
> Interesting abstract. Do you have the full paper you could share?
> 
> Thanks,
> Amreesh
> 
> On Thu, Oct 20, 2016 at 11:47 AM, <[email-masked]> wrote:
> 
>> Hi WG and Chairs,
>> 
>> I notice the first call for contribution deadline has passed. Don't
>> know if there is a second round call. If so, I'd send a request in
>> between. :) Either way, please give a feedback.
>> 
>> [1]
>> such as access link characteristics, speed-tier, server distance,
>> host buffers, ISP network dimensioning, and time-of-day. Daily or
>> monthly aggregates, published by content providers such as Netflix
>> and Youtube, present a highly simplified view that do not
>> distinguish the impact of the above factors, making the comparisons
>> potentially unfair and incorrect. Our paper revisits broadband
>> performance using open data from M-Lab, utilizing nearly 5 million
>> measurements taken over a 12-month period across 19 ISPs in 3
>> continents. We first develop a tool that allows a non-expert to
>> query, process, and visualize M-Lab data by applying various filters
>> and granularities via a simple web-based interface. Using this tool,
>> we characterize the influence of the various factors, and find that
>> there are significant biases affecting the averages, including
>> disparities amongst households in their frequency of testing and
>> access speed-tier, that contribute to fluctuations and biases in ISP
>> performance comparisons. We then apply statistical and causal
>> inference techniques to reduce the sampling bias, and find that the
>> disparity amongst ISPs is lower than current averaging methods
>> _______________________________________________
>> Maprg mailing list
>> [email-masked]
>> https://www.irtf.org/mailman/listinfo/maprg [2]
> 
> --
> Amreesh Phokeer
> 
> 
> Links:
> ------
> [1]
> [2] https://www.irtf.org/mailman/listinfo/maprg