Subject: [tcpm] Request for input regarding the initial window increase

From: "SCHARF, Michael" <[email-masked]>
Date: Wed, 13 Jul 2011 01:54:40 +0200

Dear all,

according to the TCPM charter, the working group has to determine the
intended publication track for the document on increasing the initial
window by Aug. 2011.

In addition, there are some further open issues regarding an increase of
TCP's permitted initial congestion window. I tried to narrow this down
to three key questions:

Question 1: Moving forward a fixed increase of the permitted TCP initial
congestion window (draft-ietf-tcpm-initcwnd-01)?

Answer 1A: Fixed upper limit to proposed standard
Answer 2B: Fixed upper limit to experimental
Answer 3C: Something else (e. g., some adaptive scheme, or no change at
all compared to RFC 3390); this would imply a substantial change of
draft-ietf-tcpm-initcwnd-01

Question 2: Maximum permitted initial congestion window?

Answer 2A: 10 MSS for an MTU of 1500 byte as suggested by
draft-ietf-tcpm-initcwnd-01 (other MTUs are still TBD)
Answer 2B: Another value (please suggest and justify)

Question 3: Adaptive solution as an alternative?

Answer 3A: Adoption of draft-touch-tcpm-automatic-iw-01 as WG item
heading towards experimental
Answer 3B: No adoption

These questions have partly been discussed so far (e. g.,
http://www.ietf.org/mail-archive/web/tcpm/current/msg06221.html), but in
order to determine the working group consensus, additional feedback and
opinions from the group would be very helpful. This also includes the
question whether there are further possibilities to be considered.

The chairs plan to allocate some meeting time in Quebec City for this
charter item, but prior input on the list would be highly welcome.

Thanks!

Michael Scharf and David Borman
(TCPM co-chairs)

---

From: Wesley Eddy <[email-masked]>
Date: Tue, 12 Jul 2011 20:53:00 -0400

My personal responses as a WG participant are below:

On 7/12/2011 7:54 PM, SCHARF, Michael wrote:
> ...

I think B (Experimental) is a conservative approach.  I'm happy with
that and scoping the "experiment" for general use on the Internet, with
intent to come back and go to Proposed Standard at some point.
Essentially, it already has close to that status ;).

> Answer 2B: Another value (please suggest and justify)

I think 2A is appropriate due to the volume of analysis that used the
same value (10MSS).  Anything greater hasn't been looked at as much,
and anything lesser sacrifices the intent.

> heading towards experimental

I favor 3A, sort-of.  I would rather see it generalized to explain how 
the same algorithm can be used for tuning SCTP and potentially a subset 
of CCIDs for DCCP.  Otherwise we're begging someone to write more
documents later in order to tell us how the same thing applies to those
other protocols.

-- 
Wes Eddy
MTI Systems

---

From: Jerry Chu <[email-masked]>
Date: Tue, 12 Jul 2011 18:47:32 -0700

> that and scoping the "experiment" for general use on the Internet, with

I'm curious about how to scope an "experiment" for "general use". I didn't
know one can have both at the same time...

> Essentially, it already has close to that status ;).

Yes and our "very large" ;-) experiment with IW=10 has been ongoing for
more than a year. I wonder how much more experiment is required for 1A,
which is my preference, unless you have IW > 10 in mind.

BTW, with Linux adopting IW=10 a few month back, the "experiment" has
been getting even larger everyday.

> and anything lesser sacrifices the intent.

Agreed.

> I favor 3A, sort-of.  I would rather see it generalized to explain how the same algorithm can be used for tuning SCTP and potentially a subset of CCIDs for DCCP.  Otherwise we're begging someone to write more

+1.

Jerry

>
> --
> Wes Eddy
> MTI Systems
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Wesley Eddy <[email-masked]>
Date: Wed, 13 Jul 2011 09:01:54 -0400

On 7/12/2011 9:47 PM, Jerry Chu wrote:
> On Tue, Jul 12, 2011 at 5:53 PM, Wesley Eddy<[email-masked]>  wrote:
>>

That means normal hosts on the Internet are intended to participate
in the experiment if desired.  In other words, it's not just to be
limited to some testbed settings, within a corporate network, etc.
I think that's appropriate, as Proposed Standard seems like it would
imply that IW3 would go Obsolete, which I'm not sure is right yet.
Or would IW3 and IW10 both coexist as PS?  That seems confusing;
IW10 as Experimental seems clear as it says we think it's perfectly
reasonable to try out at large scale, but we also think IW3 is still
a reasonable default.

That's just my feeling; I think it's close to what some others have
expressed, but am very curious what the consensus will be.

-- 
Wes Eddy
MTI Systems

---

From: Jerry Chu <[email-masked]>
Date: Wed, 13 Jul 2011 11:49:43 -0700

Hi Wes,

> limited to some testbed settings, within a corporate network, etc.

First, it has not been limited to just testbed or corporate network for more
than a year. Second, to participate in the experiment the OSes running on
the
endhosts must support a larger IW to begin with. Without a Standard status
I wonder how many OSes (with the exception of Linux) will provide the
support; and if the support is indeed added but requires some complex
configuration steps to turn it on, then I wonder how much of real
participation
it may draw. If the support is added and is also on by default will it still
be
compatible with the spirit of the "Experimental" status?

We have been closely monitoring our deployment for over a year and have
not noticed any issue (details can be found in the latest draft and paper
referenced). I think it's about time to take the step forward to make IW10
a PS otherwise it may get stuck in Experimental forever (unless OSes all
change their default to IW10 then it becomes moot).

> Or would IW3 and IW10 both coexist as PS?  That seems confusing;

Note that in both RFC3390 and draft-ietf-tcpm-initcwnd-01.txt the IW
number only serves as an upperbound and is OPTIONAL so the two
docs are not really in direct conflict.

Best,

Jerry

---

From: Joe Touch <[email-masked]>
Date: Wed, 13 Jul 2011 14:04:15 -0700

> Essentially, it already has close to that status ;).

+1

I would like to also see some discussion of a metric to know whether 
this is working or not (i.e., it's been deployed, but just because we 
don't see a problem doesn't mean one isn't happening).

> and anything lesser sacrifices the intent.

I think it should mirror how it was stated previously: M segments OR N 
bytes, whichever is *smaller*.

As to the M and N, there is clearly a goal of 10 1500-byte packets 
(15,000 bytes). Compared to RFC 3390, that's 3.4x more bytes *and* 5x 
more segments (at 1500 bytes, 3390 would yield 2 MSS).

The arguments in favor appear to be based on the 99% - or even 99.5% - 
case, and too much focused on current (and, IMO, likely ephemeral) 
transfer size measurements. I prefer to consider the increased variety 
of links and uses and trade safety for over-optimizing performance.

I personally would be more comfortable with doubling the numbers from 
RFC 3390, i.e., "4 segments or 8760 bytes whichever is smaller", as well 
as adding "SHOULD round down to an even number of segments" (to avoid 
creating compressed ACK stalls).

> documents later in order to tell us how the same thing applies to those

+1 on all points (FWIW, it was always intended to apply across all AIMD 
congestion control scenarios).

Joe

---

From: Jerry Chu <[email-masked]>
Date: Wed, 13 Jul 2011 14:52:51 -0700

Hi Joe,

On Wed, Jul 13, 2011 at 2:04 PM, Joe Touch <[email-masked]> wrote:
> I would like to also see some discussion of a metric to know whether this is
> working or not (i.e., it's been deployed, but just because we don't see a
> problem doesn't mean one isn't happening).
>
> As to the M and N, there is clearly a goal of 10 1500-byte packets (15,000
> bytes). Compared to RFC 3390, that's 3.4x more bytes *and* 5x more segments
> (at 1500 bytes, 3390 would yield 2 MSS).

I am confused. I thought all the numbers in RFC3390 are based on MSS, not
MTU, so it's 3.33x, # of bytes as well as segments.

"Equivalently, the upper bound for the initial window size is based on
   the MSS, as follows:

       If (MSS <= 1095 bytes)
           then win <= 4 * MSS;
       If (1095 bytes < MSS < 2190 bytes)
           then win <= 4380;
       If (2190 bytes <= MSS)
           then win <= 2 * MSS;
"

So IW=10 may not have been as aggressive as you might have thought. Will that
be enough to swing your vote :)?

>
> The arguments in favor appear to be based on the 99% - or even 99.5% - case,
> and too much focused on current (and, IMO, likely ephemeral) transfer size
> measurements. I prefer to consider the increased variety of links and uses
> and trade safety for over-optimizing performance.

draft-ietf-tcpm-initcwnd-01.txt already contains the following paragraph:
"8.  Mitigation of Negative Impact

   Much of the negative impact from an increase in the initial window is
   likely to be felt by users behind slow links with limited buffers.
   The negative impact can be mitigated by hosts directly connected to a
   low-speed link advertising a smaller initial receive window than 10
   segments. This can be achieved either through manual configuration by
   the users, or through the host stack auto-detecting the low bandwidth
   links.

   More suggestions to improve the end-to-end performance of slow links
   can be found in RFC 3150 [RFC3150].
"

It the above is the only road block to standard we can certainly try to expand
on it.

>
> I personally would be more comfortable with doubling the numbers from RFC
> 3390, i.e., "4 segments or 8760 bytes whichever is smaller", as well as
> adding "SHOULD round down to an even number of segments" (to avoid creating
> compressed ACK stalls).

Again it should be "6 segments or 8760 bytes whichever is smaller".

Jerry

---

From: Joe Touch <[email-masked]>
Date: Wed, 13 Jul 2011 15:46:14 -0700

Hi, Jerry,

On 7/13/2011 2:52 PM, Jerry Chu wrote:
> Hi Joe,
>
> On Wed, Jul 13, 2011 at 2:04 PM, Joe Touch<[email-masked]>  wrote:
...
> MTU, so it's 3.33x, # of bytes as well as segments.

I was assuming that 3390 corrected for an odd number of segments (which 
- you are correct, of course - it does not). So in the 1500-byte MTU 
case, 3390 results in an odd number of segments, resulting in an ACK 
compression stall. ;-(

...
> be enough to swing your vote :)?

Still 3.33x vs. 2x. I prefer 2x.

> It the above is the only road block to standard we can certainly try to expand

It would be preferable, IMO, to have users behind fast links be the ones 
that required manual configuration to get better performance, rather 
than expecting the have-nots to do manual configuration to avoid being 
punished.

> Again it should be "6 segments or 8760 bytes whichever is smaller".

RFC 3390 says:

(1)     min (4*MSS, max (2*MSS, 4380 bytes))

So it should be (or at least I would prefer):

(2)	min (8*MSS, max (4*MSS, 8760 bytes))

This will result in 6 segments for the 1500-byte MTU case anyway.

*IF* this is stated in terms of the current proposal, it should be more 
like the following:

(3)     min (10*MSS, 14600 bytes)

However, again, I prefer the more conservative version (2) above.

Joe

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 14 Jul 2011 12:05:37 +0200

> documents later in order to tell us how the same thing applies to those
I really would like to see the document covering TCP, SCTP, and DCCP. The problem covers
all transports having a TCP related CC, not only TCP.

Best regards
Michael

---

From: Mirja Kühlewind <[email-masked]>
Date: Wed, 20 Jul 2011 16:08:31 +0200

Hi Jerry,

first of all, sorry I'm late to response here... please see inline.

> been getting even larger everyday.
Okay, there have been large scale experiments with IW=10 but did you do any 
experiments on any other value? Maybe 9 or 11 is much better, but we never 
recognized..?

Mirja

---

From: Mirja Kuehlewind <[email-masked]>
Date: Wed, 20 Jul 2011 16:10:00 +0200

Hi again,

to sum up, here my option to the questions below

> draft-ietf-tcpm-initcwnd-01

2B

> Answer 2B: Another value (please suggest and justify)

2B, all least there should be done some investigation on other values than 10 
before we go for 10

> heading towards experimental

I guess 3B, mostly the senders which send a lot of data are servers of a 
company. Those servers can be changed more or less easily...

But I don't think there should be a fixed value for all flow which is not 
changeable. I guess there should be a socket option or something like this. 
Such that the application could change the IW for certain transmissions.

Mirja

---

From: Mirja Kuehlewind <[email-masked]>
Date: Wed, 20 Jul 2011 16:09:54 +0200

Hi again,

> Second, to participate in the experiment the OSes running on the
> participation it may draw. 
The initial (sending) window needs only needs to be changed at sender side, 
which usually is a server under control of a large company.
If you mean that the initial receive window needs to be increased as well, 
that's true. But that's a different story. I'm happy to increase the initial 
receiver window (if the access link of the receiver is large enough), even on 
standard track. That does not mean we have to increase the initial sending 
window in the same standard doc as well.

I would see the IW increase as experimental as well. Furthermore I would not 
make a general recommendation for ever transmission. For large flow there is 
basically no speed-up when increasing the window from 3 to 10. So why to risk 
to loss more packets or disturb other flows...? An IW of 10 gives most benefit 
to flow with have about 10 packets (if the link is large enough and not 
overly full)...

Mirja

---

From: Jerry Chu <[email-masked]>
Date: Wed, 20 Jul 2011 10:15:25 -0700

> experiments on any other value? Maybe 9 or 11 is much better, but we never

First an even number is much preferred to avoid delayed ack. Second, we did
try
different IW values. See our SIGCOMM CCR paper
"An Argument for Increasing TCP's Initial Congestion Window", Figure 2. From
our experiments IW16 seems to the optimal value but we decided to be
conservative
and picked 10.

Jerry

---

From: Jerry Chu <[email-masked]>
Date: Wed, 20 Jul 2011 10:28:15 -0700

> window in the same standard doc as well.

No, what I meant is some OSes hardwire IW3 in their code. Others require
some
configuration steps to change IW. Without a standard status it may be
difficult to
enable more IW10 experiment, a chicken&egg problem.

> basically no speed-up when increasing the window from 3 to 10. So why to

What's the risk here for large flows? The startup period is very small
compared to
the lifetime of the connection so whatever IW value is, the benefit is
negligible
and the impact is negligible too.

Jerry

to loss more packets or disturb other flows...? An IW of 10 gives most
> benefit

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 21 Jul 2011 10:54:39 +0200

Hi jerry,

see inline...

On Wednesday 20 July 2011 19:15:25 Jerry Chu wrote:
> > Okay, there have been large scale experiments with IW=10 but did you do
> > any experiments on any other value? Maybe 9 or 11 is much better, but we
> > never recognized..?
>
> "An Argument for Increasing TCP's Initial Congestion Window", Figure 2.
> From our experiments IW16 seems to the optimal value but we decided to be
> conservative
> and picked 10.

I only took a brief look at the paper (will read it more deeply later on). As 
far as I understand the IW of 16 seems to be the best fit for Google search. 
Which makes sense because a Google search response has usually less than 16 
packets and thus all data can be transmitted at once. 

A student of mine is doing some investigation about the IW. For now I would 
say that the right value depends very much on the traffic characteristic/flow 
size and also on the scenario we are looking at e.g. buffersizes. 

I believe that IW of 10 is a very  good value for Google search responses (in 
most scenarios). But I would wanne recommend IW of 10 for every kind of 
transmission in very scenario in a standard.

Mirja

---

From: Mark Allman <[email-masked]>
Date: Thu, 21 Jul 2011 08:20:24 -0400

> A student of mine is doing some investigation about the IW. For now I
> would say that the right value depends very much on the traffic
> characteristic/flow size and also on the scenario we are looking at
> e.g. buffersizes.

I think it is **well understood** that the "right value" depends on many
things and changes over time.

> I believe that IW of 10 is a very good value for Google search
> responses (in most scenarios). But I would wanne recommend IW of 10
> for every kind of transmission in very scenario in a standard.

I can't parse this.

Two things:

  - The spec doesn't say everyone *has* to use an IW of 10.  It says you
    can use an IW up to 10.  If you find that 7 works best for you, use
    it.  You're always free to be more conservative.

  - There is no global "optimal" IW.  One network might find 10 works
    best, another 15, another 8, etc.  TCP's cwnd adapts to observed
    network conditions.  The IW is just a place to start.  We'd like
    that place to start to work and not cause problems in the vast
    majority of the cases.  But, given the heterogeneous nature of the
    network there is no global optimal.

This issue has been studied a whole bunch.  The IETF proceedings are
full of presentations.  Those presentations point to reports and
papers.  We do not have complete knowledge here.  We never will.  There
is always "one more study" that can be done.  At some point we have to
decide that the marginal utility of another study is low and so we don't
need to wait any longer.

IMHO.

allman