Subject: [ippm] questions of TCP metrics

From: "Loki Jorgenson" <[email-masked]>
Date: Tue, 13 Oct 2009 10:37:44 -0700

Please pardon my relative naïveté and probable ignorance of past discussions on this topic - but there are two points that I am not clear on:

1) "TCP performance has always been in IPPM's charter" - does this statement derive entirely from historical precedence?  

In light of the existence of PMOL (regardless of its current state of quiescence), it would seem obvious that TCP is part of the "other layers" scheme.  Specifically, it would seem that TCP performs as consequence of an application's implementation and therefore that TCP performance would be non-specific to (although clearly a function of) IP-level characteristics, just as MOS would related but non-specific in the case of VoIP.

The IPPM charter comes as close as referencing "bulk transport capacity" but it is not trivial (for me) to conclude that a specific mechanism for enabling bulk transport is necessarily within scope.  Prior to PMOL, IPPM would seem the likeliest group to have undertaken a TCP metric.  And, certainly, if another group such as PMOL were undertaking a TCP metric, it would be obvious that IPPM would be involved.

This may well be simply a matter of philosophical leaning - but so that I am clear - on what basis do you say, Al, that TCP performance is unquestionably an IPPM matter?

2) There are at least two regimes of TCP performance that seem important to consider:
    i) non-equilibrium behaviors such as during slow-start
   ii) steady-state behaviors
Apart from the vagaries of different TCP implementations, does not the work of Padhye et al and Mathis et al (such as listed below) offer the basis for at least developing useful steady-state descriptions?  

Matt Mathis himself has said he does not see it being viable developing metrics along these lines.  While I can imagine they are not 100% accurate under all circumstances, they would at least seem like useful extrapolations of IP-level characteristics.  What then are the most important shortcomings of these models?

Refs:

Modeling TCP Reno Performance: A Simple Model and its Empirical Validation
J. Padhye, V. Firoiu, D. Towsley and J. Kurose,
IEEE/ACM Transactions on Networking, April 2000.

[1] M. Mathis, J. Semske, J. Mahdavi and T. Ott, The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm, Computer Communication Review, Vol 27, Number 3, July 1997

Al Morton writes:
>Message: 6
>Date: Tue, 13 Oct 2009 11:56:15 -0400
>From: Al Morton <[email-masked]>
>Subject: Re: [ippm] Testing TCP Throughput Capacity in Operator Networks
>To: Henk Uijterwaal <[email-masked]>
>Cc: [email-masked]
>Message-ID: <[email-masked]>
>Content-Type: text/plain; charset="us-ascii"; format=flowed
>
>At 09:26 AM 10/13/2009, Henk Uijterwaal wrote:
>>Al,
>>
>>>At 06:17 AM 10/13/2009, Henk Uijterwaal wrote:
>>>>...PMOL deals with higher layers, besides that (AFAIK) the PMOL
>>>>WG is in some form of hybernation.
>>>Expect to see more action in PMOL when the revised draft of the
>>>framework and guidelines is published (new editor).
>>
>>I wasn't aware of this activity.  That said, do you agree that this
>>work belongs in IPPM rather than PMOL?
>>
>>Henk
>
>I think that TCP performance has always been in IPPM's charter.
>
>But like a few others, I have doubts that TCP-based throughput
>measurements will produce useful results, starting with the
>issue of what TCP version and other configuration options to use.
>However, Matt Mathis' suggestion deserves attention.
>
>I should also add that Ethernet-layer throughput testing has
>been proposed in ITU-T SG12 Q17, and contributions are expected on
>this topic at the November meeting (this is scheduled at the
>same time as IETF, so I won't be in Hiroshima in person).
>
>Al

---

Loki Jorgenson
Apparent Networks
t   604 433 2333 ext 105
m   604 250-4642

---

From: "Barry Constantine" <[email-masked]>
Date: Tue, 13 Oct 2009 11:24:55 -0700

Hi Loki, 

You mentioned MOS scores of VoIP and that is a very good analogy to what providers face today.

In a VoIP environment, IP layer packet metrics can be correlated to VoIP quality, but network providers obviously find MOS scores a much more meaningful metric of the VoIP application running over the network.

Similarly, network providers can run IP layer tests (as they do today with RFC-2544) and infer TCP / application layer performance.  But network providers seek a more meaningful and realistic measure of potential application performance and would benefit from TCP layer throughput measurements.

Thanks,

Barry

Principal Member of Technical Staff

JDSU Communication Test (formerly Acterna)

Emerging Markets and Technology Research         

One Milestone Center Court                              

Germantown, MD 20876                                         

(W) 240-404-2227                                                

(C) 301-325-7069

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Loki Jorgenson
Sent: Tuesday, October 13, 2009 1:38 PM
To: [email-masked]
Subject: [ippm] questions of TCP metrics

Refs:

---

Loki Jorgenson
Apparent Networks
t   604 433 2333 ext 105
m   604 250-4642

_______________________________________________
ippm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ippm

---

From: <[email-masked]>
Date: Wed, 14 Oct 2009 09:29:01 +0200

Hi Loki,

I like your statement, especially as you point out to Padhyes/Mathis et al.'s work. 

PMOL's charter states

"Thus there is a gap in the currently chartered coverage of IETF WGs:
development of performance metrics for IP-based protocols and
applications that operate over UDP, TCP, SCTP, DCCP, Forward Error
Correction (FEC) and other robust transport protocols, and that can be
used to characterize traffic on live networks."

That's why I thought, PMOL is chartered to deal with layers above TCP only. Maybe 
work on HTTP over TCP would fit to PMOL. 

While I'm not interested in work on TCP, I also don't object to it. As the IPPM charter
isn't explicitely mentioning the layers which are in scope, I formulated my questions 
in a way, that Barry hopefully could find a WG, where his proposal fits for the case 
that IPPM isn't the right WG. I don't object however, if IPPM is the WG to work on 
the subject (as expressed by several work group members).

Regards,

Ruediger

Deutsche Telekom Netzproduktion GmbH 
Zentrum Technik Einführung 
Technik Internet Backbone, TE142-19
Rüdiger Geib
Heinrich Hertz Str. 3-7
64297 Darmstadt
Tel.: 06151/6282747
Fax: 0251/7985109

Deutsche Telekom Netzproduktion GmbH 
Aufsichtsrat: Dr. Steffen Roehn (Vorsitzender) 
Geschäftsführung: Dr. Bruno Jacobfeuerborn (Vorsitzender), Albert Matheis, Klaus Peren 
Handelsregister: Amtsgericht Bonn HRB 14190 
Sitz der Gesellschaft: Bonn 
USt-IdNr.: DE 814645262

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Loki Jorgenson
Sent: Tuesday, October 13, 2009 7:38 PM
To: [email-masked]
Subject: [ippm] questions of TCP metrics

Refs:

---

Loki Jorgenson
Apparent Networks
t   604 433 2333 ext 105
m   604 250-4642

---

From: "Loki Jorgenson" <[email-masked]>
Date: Wed, 14 Oct 2009 12:15:11 -0700

Ruediger - thanks for your response.

It probably does then come down to philosophical leanings (and
historical precedence).  For myself, PMOL's hiatus notwithstanding, I
still see TCP as an additional layer on top of IP packets behavior -
access to network resources is being actively gated at the end-hosts
(not part of IP mid-path implementation), potentially in accordance with
application parameterization.  And especially in light of there being
different stacks.

That said, I have no objection to IPPM pursuing a TCP metric development
though I still consider PMOL the better fit.

I have some interest in participating depending on some clarification of
what "this" is.

Loki Jorgenson
Apparent Networks
t   604 433 2333 ext 105
m   604 250-4642

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Wednesday, October 14, 2009 00:29
To: Loki Jorgenson
Cc: [email-masked]
Subject: RE: [ippm] questions of TCP metrics

Hi Loki,

I like your statement, especially as you point out to Padhyes/Mathis et
al.'s work. 

PMOL's charter states

That's why I thought, PMOL is chartered to deal with layers above TCP
only. Maybe 
work on HTTP over TCP would fit to PMOL. 

While I'm not interested in work on TCP, I also don't object to it. As
the IPPM charter
isn't explicitely mentioning the layers which are in scope, I formulated
my questions 
in a way, that Barry hopefully could find a WG, where his proposal fits
for the case 
that IPPM isn't the right WG. I don't object however, if IPPM is the WG
to work on 
the subject (as expressed by several work group members).

---

From: Al Morton <[email-masked]>
Date: Wed, 14 Oct 2009 21:54:42 -0400

At 01:37 PM 10/13/2009, Loki Jorgenson wrote:
>The IPPM charter comes as close as referencing "bulk transport 
>capacity" but it is not trivial (for me) to conclude that a specific 
>mechanism for enabling bulk transport is necessarily within 
>scope.  Prior to PMOL, IPPM would seem the likeliest group to have 
>undertaken a TCP metric.  And, certainly, if another group such as 
>PMOL were undertaking a TCP metric, it would be obvious that IPPM 
>would be involved.
>
>This may well be simply a matter of philosophical leaning - but so 
>that I am clear - on what basis do you say, Al, that TCP performance 
>is unquestionably an IPPM matter?

I think the previous work on bulk transfer capacity
framework seals the deal:
http://www.ietf.org/rfc/rfc3148.txt

Al