Subject: [ippm] Testing TCP Throughput Capacity in Operator Networks

From: "Barry Constantine" <[email-masked]>
Date: Tue, 6 Oct 2009 15:06:07 -0700

Hello,

I work in the communications test industry and we have seen a growing
need to verify the capacity of a network in terms of end-end TCP
throughput.  

Most network operators use RFC-2544 based Layer 3 tests to verify the
SLA of the network, but this has significant shortcomings since it does
not verify the ability of the network to carry a customer's TCP traffic.
Ideally, a TCP layer throughput test would detect issues with
prioritization, queuing, policing, etc.  This type of test could also
detect issues such as TCP Tail drop phenomena in the network.

I have spoken to several network operators and equipment providers and
there is a consistent desire to test throughput at the stateful TCP
layer, at line rate speeds (1G-10G), and to conduct these tests in a
standardized manner.

At a very high level, a standardized TCP layer throughput test would:

-        Run a latency test and automatically compute the ideal TCP
window size for the test network

-        Conduct a series of single and multiple connection TCP tests
and vary the window size to verify throughput per window size

-        Conduct QoS testing, tagging the TCP connections with
appropriate prioritization (DSCP, etc.) and test in the midst of
background traffic loads

This is a very simplified description of the test workflow.

I would like to solicit the interest of members within the IPPM
workgroup and would like to move forward with a draft document to
describe this test in detail.

Thank you,

Barry

Principal Member of Technical Staff

JDSU Communication Test (formerly Acterna)

Emerging Markets and Technology Research         

One Milestone Center Court                              

Germantown, MD 20876                                         

(W) 240-404-2227                                                

(C) 301-325-7069

---

From: <[email-masked]>
Date: Tue, 13 Oct 2009 11:47:39 +0200

Hello Barry,
 
I'm not sure, whether IPPM is chartered to specify transport layer performance metrics. I've checked the PMOL charter and it clearly says PMOL works above transport layer. A fair conclusion would be, that IPPM is chartered to investigate transport layer performance.
 
Three questions to the chairs and to the IPPM WG come to my mind:
 
Chairs: Is IPPM chartered to execute the work proposed by Barry?
 
IPPM WG: If not, is there interest to charter IPPM for such an activity?
 
Chairs: If not, to which WG can Barry submit his proposal?
 
My personal interest in this activity is limited, saying no support, no objection. The feedback I received from within the company I work for is, that such an activity is relevant for application or service providers, but not for carriers. 
 
Regards,
 
Ruediger
 
Deutsche Telekom Netzproduktion GmbH
Zentrum Technik Einführung
Technik Internet Backbone, TE142-19
Rüdiger Geib
Heinrich Hertz Str. 3-7
64297 Darmstadt
Tel.: 06151/6282747
Fax: 0251/7985109

Deutsche Telekom Netzproduktion GmbH
Aufsichtsrat: Dr. Steffen Roehn (Vorsitzender)
Geschäftsführung: Dr. Bruno Jacobfeuerborn (Vorsitzender), Albert Matheis, Klaus Peren
Handelsregister: Amtsgericht Bonn HRB 14190
Sitz der Gesellschaft: Bonn
USt-IdNr.: DE 814645262

  _____  

From: [email-masked] [mailto:[email-masked]] On Behalf Of Barry Constantine
Sent: Wednesday, October 07, 2009 12:06 AM
To: [email-masked]
Subject: [ippm] Testing TCP Throughput Capacity in Operator Networks

Hello,

I work in the communications test industry and we have seen a growing need to verify the capacity of a network in terms of end-end TCP throughput.  

Most network operators use RFC-2544 based Layer 3 tests to verify the SLA of the network, but this has significant shortcomings since it does not verify the ability of the network to carry a customer's TCP traffic.  Ideally, a TCP layer throughput test would detect issues with prioritization, queuing, policing, etc.  This type of test could also detect issues such as TCP Tail drop phenomena in the network.

I have spoken to several network operators and equipment providers and there is a consistent desire to test throughput at the stateful TCP layer, at line rate speeds (1G-10G), and to conduct these tests in a standardized manner.

-        Run a latency test and automatically compute the ideal TCP window size for the test network

-        Conduct a series of single and multiple connection TCP tests and vary the window size to verify throughput per window size

-        Conduct QoS testing, tagging the TCP connections with appropriate prioritization (DSCP, etc.) and test in the midst of background traffic loads

I would like to solicit the interest of members within the IPPM workgroup and would like to move forward with a draft document to describe this test in detail.

Thank you,

Barry

Germantown, MD 20876                                         

(W) 240-404-2227                                                

(C) 301-325-7069

---

From: Henk Uijterwaal <[email-masked]>
Date: Tue, 13 Oct 2009 12:17:55 +0200

Hi Ruediger,

Personal opinion, not discussed with Matt or AD's yet

> I'm not sure, whether IPPM is chartered to specify transport layer 
> performance metrics. I've checked the PMOL charter and it clearly says 
> PMOL works above transport layer. A fair conclusion would be, that IPPM 
> is chartered to investigate transport layer performance.

I looked at it from a slightly different angle.  Network capacity has
always been a topic of interest for IPPM, so this work would be in our
charter.  PMOL deals with higher layers, besides that (AFAIK) the PMOL
WG is in some form of hybernation.

So, my take on this is that we should see if there is interest to work
on this topic _and_ have a first version of a draft, we can decide later
where exactly this works fits in the IETF.

Henk

------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.xs4all.nl/~henku
P.O.Box 10096          Singel 258         Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
The Netherlands        The Netherlands    Mobile: +31.6.55861746
------------------------------------------------------------------------------

Belgium: an unsolvable problem, discussed in endless meetings, with no
          hope for a solution, where everybody still lives happily.

---

From: Al Morton <[email-masked]>
Date: Tue, 13 Oct 2009 08:27:34 -0400

At 06:17 AM 10/13/2009, Henk Uijterwaal wrote:
>...PMOL deals with higher layers, besides that (AFAIK) the PMOL

Expect to see more action in PMOL when the revised draft of the
framework and guidelines is published (new editor).
Also, there is considerable activity on the SIP metrics draft
at the IESG level.

Al
pmol co-chair

---

From: Henk Uijterwaal <[email-masked]>
Date: Tue, 13 Oct 2009 15:26:16 +0200

Al,

> framework and guidelines is published (new editor).

I wasn't aware of this activity.  That said, do you agree that this
work belongs in IPPM rather than PMOL?

Henk

--

---

From: Matthew J Zekauskas <[email-masked]>
Date: Tue, 13 Oct 2009 10:32:18 -0400

My thoughts are similar to Henk's.  IPPM has always been interested in
the area (see the bulk transport capacity work), so I'd claim the area
is in-scope, but doing work would require AD discussion.

I think Matt Mathis' last message to the list is also worth thought;
it's hard to create a good metric.  In some sense what has been proposed
is documenting current practice (we do something similar for quality
control on the R&E networks, and NPAD is used for debugging).

I cannot be in Hiroshima do to a conflicting conference, but look
forward to the results of any discussion there.

--Matt

On 10/13/09 6:17 AM, Henk Uijterwaal wrote:
>> PMOL works above transport layer. A fair conclusion would be, that
>> IPPM is chartered to investigate transport layer performance.
> 
> _______________________________________________
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm

---

From: "Barry Constantine" <[email-masked]>
Date: Tue, 13 Oct 2009 07:41:42 -0700

Henk and Ruediger,

Thank for your comments and I would like to clarify a few points
concerning the proposal for a draft related to TCP Throughput Capacity
test methodology and metrics.

When I first addressed the subject with Henk and Matt Zekauskas, I was
pointed to RFC 3148, draft-ietf-ippm-treno-btc-03, and
draft-ietf-ippm-btc-cap-00 as a starting point.  By the way, I found
these to be very interesting and useful papers.

The TCP Throughput Capacity draft would reference these papers but would
focus more at the network level metrics of TCP performance, not the
specific behavior of various TCP mechanisms and implementations (i.e.
CAC, False time-outs, Self-clocking, etc.).

In other words, given a large complex network with various QoS and
queuing techniques employed, the TCP throughput technique would allow a
network provider to quantify the ability of the entire network to carry
TCP traffic at specified SLA levels.  Varying window sizes and MSS sizes
is one component of this type of test. Another key component to this
test is the throughput achieved with varying background traffic (perhaps
UDP) and verifying proper prioritization between the TCP traffic and the
background traffic.  

It is very difficult for a network provider to extrapolate TCP layer
performance from packet loss studies and metrics.  RFC-2544 is widely
used today and makes it difficult (if not impossible) to adequately
characterize how TCP will perform.

Concerning the interest level based upon the responses, the sample size
is small, but I have received 3-4 who are interested and 3-4 with little
interest.  Referencing Henk's final remarks; does this constitute enough
interest from this working group to work on a draft?

Thanks,

Barry

Germantown, MD 20876                                         

(W) 240-404-2227                                                

(C) 301-325-7069

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Henk Uijterwaal
Sent: Tuesday, October 13, 2009 6:18 AM
To: [email-masked]
Cc: Barry Constantine; [email-masked]
Subject: Re: [ippm] Testing TCP Throughput Capacity in Operator Networks

Hi Ruediger,

> PMOL works above transport layer. A fair conclusion would be, that
IPPM 

Henk

------------------------------------------------------------------------
------
Henk Uijterwaal                           Email:
henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.xs4all.nl/~henku
P.O.Box 10096          Singel 258         Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
The Netherlands        The Netherlands    Mobile: +31.6.55861746
------------------------------------------------------------------------
------

---

From: Gilles Forget <[email-masked]>
Date: Tue, 13 Oct 2009 11:23:31 -0400

Bonjour Ruediger,

If I may, I would like to comment on the feedback you received from
within your company.

Here in North America, over the last 30 years, we have seen the
monopolistic carriers evolve to competitive Service Providers and even
Application Service Providers. 

There are still pure Carrier Transport SONET type equipments involved in
their core network, but most of the business customer services are now
delivered via an MPLS infrastructure using IP routers and multi-layer
switches in the core, distribution and access layers of the network.

There are also Professional Services groups and Network Security groups
within the organizations involved in business services delivery for some
major customers.

I am pretty sure, that most European PT&Ts have done the same. 

So I do not see why you still make differences between application or
service providers and carriers!

Thanks,
Gilles Forget, CCNP.

On Tue, 2009-10-13 at 11:47 +0200, [email-masked] wrote:
> My personal interest in this activity is limited, saying no support,
> no objection. The feedback I received from within the company I work
> for is, that such an activity is relevant for application or service
> providers, but not for carriers. 
>  
> Regards,
>  
> Ruediger
>  
> ______________________________________________________________________
> SLA of the network, but this has significant shortcomings since it
> does not verify the ability of the network to carry a customer’s TCP
> traffic.  Ideally, a TCP layer throughput test would detect issues
> with prioritization, queuing, policing, etc.  This type of test could
> also detect issues such as TCP Tail drop phenomena in the network.
> 
> 
> 
-- 
Gilles Forget, CCNP
T : 450.473.5718
C : 514.895.8212
@ : [email-masked]

---

From: Al Morton <[email-masked]>
Date: Tue, 13 Oct 2009 11:56:15 -0400

>work belongs in IPPM rather than PMOL?

I think that TCP performance has always been in IPPM's charter.

But like a few others, I have doubts that TCP-based throughput
measurements will produce useful results, starting with the
issue of what TCP version and other configuration options to use.
However, Matt Mathis' suggestion deserves attention.

I should also add that Ethernet-layer throughput testing has
been proposed in ITU-T SG12 Q17, and contributions are expected on
this topic at the November meeting (this is scheduled at the
same time as IETF, so I won't be in Hiroshima in person).

Al

---

From: "Fardid, Reza" <[email-masked]>
Date: Tue, 13 Oct 2009 12:15:17 -0700

Barry,

Other factors to consider are the effects of network path asymmetry on TCP performance, which are of particular interest to Access Service Providers. Some of the PILC (concluded WG) recommendations from RFC 3449 may be relevant to your proposal, regardless of which WG shepherds it.

Regards,
Reza Fardid
Covad Communications

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Barry Constantine
Sent: Tuesday, October 13, 2009 7:42 AM
To: Henk Uijterwaal; [email-masked]
Cc: [email-masked]
Subject: Re: [ippm] Testing TCP Throughput Capacity in Operator Networks

Henk and Ruediger,

Thanks,

Barry

Germantown, MD 20876

(W) 240-404-2227

(C) 301-325-7069

-----Original Message-----

Hi Ruediger,

IPPM

Henk

---

From: Lars Eggert <[email-masked]>
Date: Wed, 14 Oct 2009 08:13:09 +0900

Hi,

> is in-scope, but doing work would require AD discussion.

I think it wouldn't be unreasonable for IPPM to take this on, if we  
more clearly nail down what "this" is. I can also follow the argument  
that it could go into PMOL - to me, it doesn't matter very much,  
although since we're talking TCP I believe there is a stronger  
affinity with the transport area, i.e., IPPM.

> it's hard to create a good metric.

I agree that while the idea sounds simple, in reality it isn't. It  
already starts to get complicated when we try to say what we mean by  
"TCP" - pretty much all stacks implement a different set of TCP  
extensions, and even stacks that implement the same extension usually  
implement a different set of the SHOULDs and MAYs. Other stacks (e.g.,  
Linux) even implement a congestion control scheme that is completely  
non-standard (Cubic).

If the intent is to look at bulk throughput figures on otherwise idle  
paths with little to no loss, maybe those differences won't matter,  
but they will start to influence the result in more realistic scenarios.

So while I can see why TCP throughout metrics would be interesting to  
have, I believe it'll take a bit of work to define them in a useful way.

Lars

---

From: <[email-masked]>
Date: Wed, 14 Oct 2009 09:11:27 +0200

Hi Gilles,

the company I work for is rather big and we differentiate between 
branches providing products/services and those which provide transport. 

I agree that one can't completely separate both (you probably are 
aware that European regulation strongly favours a complete separation
of transport and service).

TCP performance is relevant for products. To my department that means, 
if a product manager is interested, he may push standardisation of a 
TCP throughput measurement system. Currently, I'm not aware of any 
demand (but as I said, it's a big company). Different products and 
services may have different typical TCP communication patterns (like 
different RTTs). All different products are served across the same 
backbone. I doubt that product specific backbone engineering makes 
economical sense in a competitive environment. Different products may 
however apply differentiated transport quality for their communication.

The company I work for operates an IPPM compliant SLA validation 
system for nearly 10 years. Several service/product validation systems 
are operated in parallel. I can't recall that there ever has been a 
discussion whether to disable a backbone SLA validation system because 
there is a service validation system working in parallel. They serve 
different purposes, even if they partially measure similar metrics 
or along shared network sections.

Regards,

Ruediger

-----Original Message-----
From: Gilles Forget [mailto:[email-masked]] 
Sent: Tuesday, October 13, 2009 5:24 PM
To: Geib, Rüdiger
Cc: [email-masked]
Subject: Re: [ippm] Testing TCP Throughput Capacity in Operator Networks

Bonjour Ruediger,

Thanks,
Gilles Forget, CCNP.

On Tue, 2009-10-13 at 11:47 +0200, [email-masked] wrote:
> does not verify the ability of the network to carry a customer's TCP

---

From: Henk Uijterwaal <[email-masked]>
Date: Wed, 14 Oct 2009 13:19:56 +0200

Lars Eggert wrote:
> I think it wouldn't be unreasonable for IPPM to take this on, if we more 
> clearly nail down what "this" is.

In order to move forward: I suggest that Barry writes a -00 draft (including
a definition of "this") before Hiroshima.  3-4 people showing interest should
be sufficient to review it.  In Hiroshima, we can discuss the draft further,
afterwards we can decide on the mailing list and with the AD to pick it up
as a WG item or not.

Barry, can you produce something before Hiroshima?

Henk

---

From: Matt Mathis <[email-masked]>
Date: Thu, 15 Oct 2009 14:52:07 -0400

>> clearly nail down what "this" is.

You might want to consider finishing the work started in RFCs 2330 and
3148 as a sufficient definition of "this".

Developing a Bulk Transport Metric has been sort of a holy grail of
IPPM from the very beginning.    It was my primary agenda when I
co-chaired the BOF at the Danvers IETF, April 1995).    After
publishing a tool [TReno] and making some important progress [RFC2330,
RFC3148, and Marl Alman's[CAP] the effort was abandoned essentially
because the results were not sufficiently robust to be useful to
support SLAs.  [NPAD] was my latest chapter of my efforts in this
area.

Although there are many difficulties with using TCP (notably
differences in implementations as noted in other messages) the real
killer is this:

TCP congestion control is an equilibrium process.  Correctly
functioning TCP with sufficient data ALWAYS causes congestion
somewhere in the network.  This congestion manifests itself by raising
the RTT and/or the packet loss until the data rate is consistent with
the the model[MODEL]:
date_rate=(MSS/RTT)*(0.7/squrt(loss_probability))   The problem is you
can't tell how much of the congestion was caused by your measurement,
and how much was already present in the network.   This causes sort of
a double Heisenberg problem where you can't even tell if you are
measuring bowling balls with pingpong balls or vice versa.  This is
because the "stiffness" of a TCP flow (think "first derivative of the
model") depends on the RTT, so when there are multiple flows sharing a
bottleneck, each flow yields a different amount, depending on their
relative RTTs.   If the measurement flow has a very short RTT, it will
push nearly all other traffic out of the bottleneck, if the cross
traffic has a short RTT, it will greatly depress the data rate of the
measured folw.  As a consequence a simple measurement with a single
TCP flow is useless for predicting performance of another flow through
the same bottleneck.   Note that a non-predictive measurement is
useless to the point where it can't really be called a metric.

THIS IS REALLY IMPORTANT: even with an ideal TCP implementation and
network, equilibrium TCP performance is useless as a metric.

The trick (used by NPAD) is the throttle TCP with a controlled
bottleneck, such that it does not cause congestion....

This can be done using an RFC 4898 instrumented stack under real
applications.   I will describe it in email in a bit, and a
presentation at IPPM, if people are interested.    No I do not have
enough cycles to generate an ID in time for the submission deadline.

Thanks,
--MM--
-------------------------------------------
Matt Mathis      http://staff.psc.edu/mathis
Work:412.268.3319   Home/Cell:412.654.7529
-------------------------------------------
Evil is defined by mortals who think they know
"The Truth" and use force to apply it to others.

---

From: Matt Mathis <[email-masked]>
Date: Thu, 15 Oct 2009 15:46:17 -0400

Opps I forgot to paste in the references:

[NPAD] M. Mathis, J. Heffner, P. O'Neil, P. Siemsen, "Pathdiag:
Automated TCP Diagnosis", PAM 2008.  See also:
http://www.psc.edu/networking/projects/pathdiag/

[CAP] M. Allman "Measuring End-to-End Bulk Transport Capacity" ACM
SIGCOMM Internet Measurement Workshop 2001.

[RFC3148] M. Mathis, M. Allman, "A Framework for Defining Empirical
Bulk Transfer Capacity Metrics", RFC3148, July 2001.

[RFC2330] V. Paxson, G. Almes, J. Mahdavi, M. Mathis, "Framework for
IP Performance Metrics", RFC2330, May 1998.

[MODEL] M. Mathis, J. Semke, J. Mahdavi, T. Ott, "The Macroscopic
Behavior of the TCP Congestion Avoidance Agorithm", Computer
ommunication Review, volume 27, number 3, July 1997.

[TReno] M. Mathis, "Diagnosing Internet Congestion with a Transport
Layer Performance Tool", Proceedings of INET'96, June, 1996, Montreal,
Quebec.

Thanks,
--MM--

---

From: "Barry Constantine" <[email-masked]>
Date: Thu, 15 Oct 2009 14:45:57 -0700

Hi Matt,

I understand your concerns with relation to TCP and you undoubtedly have
vast experience in detailed TCP implementations and studies.  I have
read several of your papers and they are all excellent.

Let me try to clearly lay out the problem statement and then determine
if it is worthy of a -00 draft.  I tried to keep this email concise,
sorry if it is a little long-winded.

In my work with network providers and network equipment manufacturers
(NEMs), there is a growing realization that RFC-2544 packet level
testing on today's networks is not adequate.  This community desires to
measure network throughput performance at the TCP layer, to gain a much
more meaningful measure that the network can meet the end user's
application SLA (and ultimately reach some level of TCP testing
interoperability which does not exist today).  The complexity of the
network grows and the various queuing mechanisms in the network greatly
affect TCP layer performance (i.e. improper default router settings for
queuing, etc.) and devices such as firewalls, proxies, load-balancers
can actively alter the TCP settings as a TCP session traverses the
network (such as window size, MSS, etc.).  These are all very complex
topics to the general network community, and I can't overemphasize the
desire for a standard test methodology/guideline at the TCP layer.

So the intent behind this draft TCP Throughput work would be to define a
methodology for testing TCP layer performance, and guidelines for
expected TCP throughput results that *should* be experienced in the
network under test.  Network providers and NEMs are wrestling with
end-end complexities of the above (queuing, active proxy devices, etc.);
they desire to standardize the methodology to validate end-end TCP
performance, as this is the precursor to acceptable end-user application
performance.

Before RFC-2544 testing existed, network providers and NEMs deployed a
variety of ad hoc test techniques to verify the Layer2/3 performance of
the network.  RFC-2544 was a huge step forward in the network test
world, standardizing the Layer 2/3 test methodology which greatly
improved the quality of the network (and reduced operational test
expenses).

In the case of TCP, several network providers that I work with employ
the following TCP testing methodology (high level simplified example):

1. Run tests to determine average end-end latency and end-end bottleneck
bandwidth (the minimum bandwidth may be known by design).
2. Calculate bandwidth delay product (BDP) for this situation then run a
series of window size experiments with TCP hosts on each end of the
network.  The network under test should be able to support the full TCP
capacity unless there are packet loss conditions.  This may point to
devices in the network that are altering the window size IF
retransmissions are not the issue and throughput is not achieved.
3. Run multiple connection tests (i.e. 32 connections) to over-utilize
the link.  Proper queuing techniques in the network should allow the TCP
connections to maintain their "fair share" of the available bandwidth.
Improper queuing (which is very common) would display bandwidth
volatility between the connections.  This is very valuable method as
well.
4. Another example is by either running single or multiple TCP
connections with a variety of MSS values (similar to varying the frame
size in RFC-2544).

Right now there is no established methodology for any of the above tests
in the network provider / NEM space.  So even taking a first step toward
standardizing the test methodology at the TCP layer, would be a giant
step forward.  As a matter of fact, this paper would probably be
co-authored with myself and at least one network provider and/or NEM.

So again, sorry for the long winded response, but I hope that this
provides better insight into the intent of such a draft. 

Thanks,
Barry

Germantown, MD 20876                                         

(W) 240-404-2227                                                

(C) 301-325-7069

-----Original Message-----
From: Matt Mathis [mailto:[email-masked]] 
Sent: Thursday, October 15, 2009 2:52 PM
To: Henk Uijterwaal
Cc: Lars Eggert; Barry Constantine; Matthew J Zekauskas; [email-masked]
Subject: Re: [ippm] Testing TCP Throughput Capacity in Operator Networks

>> I think it wouldn't be unreasonable for IPPM to take this on, if we
more

Thanks,
--MM--

---

From: Matt Mathis <[email-masked]>
Date: Thu, 15 Oct 2009 18:38:02 -0400

Your procedure makes perfect sense for bench testing gear in a
laboratory, where you have complete control over all of the traffic.

NPAD's algorithm is essentially the same as your steps 1&2.  (It uses
LimCwnd in the web100 prototype of 4898 to scan window sizes with one
TCP connection).  I suspect that NPAD's queue measurement algorithm
captures most of the effects detected in your step 3, although not all
of them (For example NPAD would not capture effects due to loss
synchronization between flows).

IPPM has traditionally been focused on in situ testing of Internet
infrastructure, which also includes the effects of uncontrolled cross
traffic.  Please excuse me it this changed while I wasn't looking.
BMWG was focused on Bench Testing, as the name implies.  While it is
true that many metrics might be the same both both arenas, this is not
at all true for BTC.

I completely agree that a good BTC bench test is needed.  Your
proposal is a very good first step in this direction.

However, the world also desperately needs a way to write TCP
performance SLAs, which was my whole reason for starting IPPM some 14
years ago.  (Do people remember that it was called IP Provider Metrics
at one time?)  We knew we didn't have a clue how to do BTC, so we
started with easy metrics like availability, loss and reordering.
Remember, at that time we started, the basic TCP performance model
would not be published for another couple of years.

I think it is time to attempt to do a BTC for in situ Internet
infrastructure, which is designed to support BTC SLAs.   (BTW the
algorithm I have in mind is not suitable for bench testing, so there
is no overlap.)

Thanks,
--MM--

---

From: "Barry Constantine" <[email-masked]>
Date: Thu, 15 Oct 2009 16:26:53 -0700

Hi Matt, 

Thanks for the quick reply and comments.

For business class networks, it is somewhat of a hybrid.  The networks are privately managed so they are not exactly "bench" networks, but certainly not the Internet.  The business class networks are staged, then provisioned (pre-prod), then new services are "turned-up" for business customers on managed production network.  There is a mix of business traffic on these virtual private networks, but prioritization (DSCP, VLANs, MPLS, etc) is supposed to be correct (you can imagine what that means) to ensure new customer TCP traffic flows as expected.  This prioritization + TCP testing is another entire test method step that I did not mention in the outline but would be covered in the draft.

As a matter of fact, RFC-2544 was originally written for bench testing devices and became (and still is) the defacto standard in these privately managed networks.  

These managed networks are intended to be predictable, but therein lies the problem.  Even in a staged provider network, errors in device configuration can go undetected with RFC-2544 testing and can later show up in production networks. 

So I can understand why TCP testing methodology in a non-controlled, non-managed network (internet) is inappropriate, but I hope this explanation concerning managed, private networks helps to clarify the intent.
 
Thanks again,

Barry

Germantown, MD 20876                                         

(W) 240-404-2227                                                

(C) 301-325-7069

-----Original Message-----
From: Matt Mathis [mailto:[email-masked]] 
Sent: Thursday, October 15, 2009 6:38 PM
To: Barry Constantine
Cc: Henk Uijterwaal; Lars Eggert; Matthew J Zekauskas; [email-masked]
Subject: Re: [ippm] Testing TCP Throughput Capacity in Operator Networks

Thanks,
--MM--