Subject: Re: [ippm] [M-Lab-Steering] Testing TCP Throughput Capacity in Operator Networks

From: Matt Mathis <[email-masked]>
Date: Sat, 10 Oct 2009 23:48:12 -0400

Barry,

NPAD uses essentially the same algorithm as your proposed test.
While it works quite well as a diagnostic, it doesn't work so well as
a metric, because the results can't be well calibrated or modeled.

I have been thinking about this problem for a very long time (It was
my goal when I was the founding co-chair of the IPPM BOF, at IETF 35).
Following my experiences developing NPAD (which was yet another
iteration on the problem) have come to the conclusion that part of the
problem is that we have been asking the wrong question all these
years.

TCP performance is to unstable and unpredictable to be a good metric
of anything.  A better metric would be a packet loss test at high data
rate (e.g. 80% of the link capacity), that asks a pass-fail question
(is the loss rate lower than X).  With the proper parameters the test
guarantees a lower bound on ideal TCP performance, and at the same
time supports an algebra:  if the separate sections of a path each
pass the metric, then you can predict if the concatenated sections do
as well.

For an SLA the most useful metric would actually be the statistics of
the test: what fraction of the time does it pass for what fraction of
the endpoints?

I will be at Hiroshima - will you?

Thanks,
--MM--
-------------------------------------------
Matt Mathis      http://staff.psc.edu/mathis
Work:412.268.3319   Home/Cell:412.654.7529
-------------------------------------------
Evil is defined by mortals who think they know
"The Truth" and use force to apply it to others.

On Tue, Oct 6, 2009 at 6:49 PM, Stephen Stuart <[email-masked]> wrote:
> Hi, Barry. I saw your post on the IPPM list:
>
>    http://www.ietf.org/mail-archive/web/ippm/current/msg02176.html
>
> and I think it would be worth your while to have a look at the tools
> used in Measurement Lab - http://measurementlab.net/ - NPAD and NDT in
> particular. I'm cc'ing the rest of the steering committee so that they
> can see the pointer to your IPPM post and so that you can follow up
> with any questions regarding the platform, tools, test methodology,
> etc.
>
> Thanks,
> Stephen

---

From: Gilles Forget <[email-masked]>
Date: Sun, 11 Oct 2009 13:09:47 -0400

Barry,

I am an independent consultant in the telecommunication area with
several decades of experience working for a network provider and I
completely support your idea.

In the circuit switching days, network providers were able to measure
their end to end path performances with Bit Error Rate Test patterns
like 2047, QRSS and others. Instruments like the FIREBERD 6000 from TTC
were used as reference test sets and network path performances were
measured against ITU-T standards like G.821 or ANSI standards. 

Network providers infrastructures have evolved from layer one only to
layer two network paths using Frame relay and ATM protocols and then
later on layer 3 network paths using IP and MPLS protocols. 

Now, in the packet switching days, there is a need for more precision in
the way tests are conducted. Network providers can not provide their
customers with a simple pass/fail result any more. They should be able
to provide their customers with more complex feedback results. These
results should help their business customers optimizing their complete
IP network performances.

Thanks.    
-- 
Gilles Forget, CCNP
T : 450.473.5718
C : 514.895.8212
@ : [email-masked]