Subject: Re: [tcpm] removing TCP's initial window

From: Mirja Kühlewind <[email-masked]>
Date: Wed, 6 Jan 2016 14:36:37 +0100

Hi,

there is actually nothing in an RFC that forbids to use a larger window. RFC3390 even says:

"This increased initial window is optional: a TCP MAY start with a
   larger initial window.“

Mirja

> Am 02.12.2015 um 13:34 schrieb Karen Elisabeth Egede Nielsen <[email-masked]>:
> 
> Hi,
> 
> Agree to do this also for SCTP.
> 
> Even more so I would like "the after idle case" to be covered as well AND
> I really like what
> was written in one of the previous postings in this thread:
> 
> 
>>> In the vein of giving an inch and they will ask to take a mile, if
>>> such a paced, large IW is OK for the start of a connection, would it
>>> also be OK for "slow start after idle?"  Or perhaps the lesser of that
>>> IW and cwnd when the connection went idle.
> 
>> Perhaps.  The difference is that in this case the host has *some* notion
> of the path it is traversing.  But, at some point that notion is so dated
> that indeed treating it like a brand new connection from a >CC standpoint
> seems OK to me.  But, there are probably more considerations here than at
> the start of a connection.
> 
>> allman
> 
> 
> This sounds as if we finally could get to handle the after idle situation,
> which is a significant issue for
> semi static long living connections/associations and critical for the
> re-usability of a path in multi-path situations.
> AND here I would like to see "refresh" of ssthresh come into the picture
> as well as the IW it self.
> 
> E.g., in the unfortunate case where the CWND/flightsize of the connection
> was low when the timeout occurred, then
> the set SSthresh would block slow start when traffic resumes after
> idleness.
> 
> All of this fist very nicely with our thoughts for RFC4960bis !
> 
> BR, Karen
> 
> 
> 
>> -----Original Message-----
>> From: tcpm [mailto:[email-masked]] On Behalf Of Michael Tuexen
>> Sent: 25. november 2015 20:59
>> To: [email-masked]
>> Cc: [email-masked]
>> Subject: Re: [tcpm] removing TCP's initial window
>> 
>>> On 24 Nov 2015, at 18:43, Mark Allman <[email-masked]> wrote:
>>> 
>>> 
>>> A proposal for allowing end hosts to set their initial cwnd to
>>> whatever they want ... in the canonical form that we use to propose
>>> such things ...
>> Hi Marc,
>> 
>> I really like and support this document. Wouldn't is make sense
>> not to restrict this document to TCP, but also cover SCTP?
>> 
>> Best regards
>> Michael
>>> 
>>> draft-allman-tcpm-no-initwin-00.txt
>>> 
>>> allman
>>> 
>>> 
>>> --
>>> http://www.icir.org/mallman/
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> tcpm mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/tcpm
>> 
>> _______________________________________________
>> tcpm mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/tcpm
> 
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Mark Allman <[email-masked]>
Date: Wed, 06 Jan 2016 08:49:08 -0500

> "This increased initial window is optional: a TCP MAY start with a

I think that "larger" is supposed to be "smaller".  I.e., the intent
of the document (and the WG) was not to allow the use of arbitrarily
large initial cwnds.

(Note: I now suggest it is likely fine to use arbitrary initial
cwnds, but that should be conflated with the intent when we wrote
RFC 2414/3390.)

allman

---

From: Mirja Kühlewind <[email-masked]>
Date: Wed, 6 Jan 2016 15:02:57 +0100

No, there is no normative language that forbids to use a larger window. An IW of 3 or 10 is effectively just recommended and what’s considered to be safe by the IETF.

I think we still need to give a recommendation here and especially if you have a large transmission that anyway will take several RTTs (or even minutes) to be transmitted, I don’t see an advantage in a larger window.

I would find it very dangerous to not give any recommendation here, because there are cases in the wild already where you see huge IWs which do cause burst loss.

The part that I support is that we give a recommendation that if you use a larger window than 10, you must be aware that there is a larger risk of burst loss and that pacing might be a potentially solution for it. (Not sure if ‚MUST use pacing if the IW is larger than 10‘ is the right thing to say…. I guess at least this probably is a ‚SHOULD‘). Another recommendation is that if you see burst loss in your IW, it's likely that you’ve hid the limit of your own access link buffer and that this might give you an indication how to set your IW in future for this link (namely IW_old - num_of_losses)

Mirja

> Am 06.01.2016 um 14:49 schrieb Mark Allman <[email-masked]>:
> 
>

---

From: Mark Allman <[email-masked]>
Date: Wed, 06 Jan 2016 09:29:26 -0500

I believe you are completely mis-interpreting the situation.  You
are hanging your hat on a mistake.

> No, there is no normative language that forbids to use a larger
> window.

Not true.

The most recent standards track document to specify the initial
window is RFC 5681 (a Draft Standard) that says:

   IW, the initial value of cwnd, MUST be set using the following
   guidelines as an upper bound.

   If SMSS > 2190 bytes:
       IW = 2 * SMSS bytes and MUST NOT be more than 2 segments
   If (SMSS > 1095 bytes) and (SMSS <= 2190 bytes):
       IW = 3 * SMSS bytes and MUST NOT be more than 3 segments
   if SMSS <= 1095 bytes:
       IW = 4 * SMSS bytes and MUST NOT be more than 4 segments

   As specified in [RFC3390], the SYN/ACK and the acknowledgment of the
   SYN/ACK MUST NOT increase the size of the congestion window.
   Further, if the SYN or SYN/ACK is lost, the initial window used by a
   sender after a correctly transmitted SYN MUST be one segment
   consisting of at most SMSS bytes.

That is a bunch of normative language that precisely sets the limits
of the initial cwnd.

RFC 6928 is an experimental RFC that says TCPs can use 10 segment
initial windows.  In fact, the second sentence of the RFC says:
"This increase is optional: a TCP MAY start with an initial window
that is smaller than 10 segments."  I.e., they got it right where we
botched it.

That doesn't mean all TCP implementations follow the
specifications.  Or, even that they all should.  As with lots of
things there may well be fine reasons to not follow the spec (e.g.,
certain LAN communication).  But, to frame the standards as merely a
recommendation and not a constraint is IMHO a completely wrong
reading of the documents and history.

A perfectly fine and reasonable position to take/discuss would be
whether we should move to a model where we give an precise initial
cwnd value and *explicitly* say it is a recommendation and hence
allow folks to use whatever they want and be within the spec.  But,
thinking that is the state of the current spec is simply wrong.

allman

---

From: Hagen Paul Pfeifer <[email-masked]>
Date: Wed, 6 Jan 2016 17:12:45 +0100

>  draft-allman-tcpm-no-initwin-00.txt

Hey Mark, regarding 1. (2):

    For initial windows of more than 10 segments, the initial
		window of segments MUST be paced evenly across the first
		round-trip time (as measured during the 3WHS).

For our setups with a bandwidth of ~3000byte/sec pacing will not work
as expceted:

Measured RTT will be roughly 0.033 sec (because of small, non-MTU sized
SYN/ACK packets). Pacing n (e.g. IW 50) full MTU sized packets will lead to
massive packet drop (for ~3000byte/sec pipes it is still a burst). Just to
much data in the pipe. Subsequently it will lead to congested (nearly
standing) forwarding queue which results in TCP timeouts because the control
loop is defacto broken.

Give it a try: take mininet or whatver you prefer, configure a low bandwidth
and increase the IW.

Hagen

---

From: Mark Allman <[email-masked]>
Date: Wed, 06 Jan 2016 11:16:20 -0500

> Measured RTT will be roughly 0.033 sec (because of small, non-MTU
> sized SYN/ACK packets). Pacing n (e.g. IW 50) full MTU sized
> packets will lead to massive packet drop (for ~3000byte/sec pipes
> it is still a burst). Just to much data in the pipe. Subsequently
> it will lead to congested (nearly standing) forwarding queue which
> results in TCP timeouts because the control loop is defacto
> broken.
> 
> Give it a try: take mininet or whatver you prefer, configure a low
> bandwidth and increase the IW.

Of course.  So, why would you do that?

allman

---

From: Hagen Paul Pfeifer <[email-masked]>
Date: Wed, 6 Jan 2016 18:05:22 +0100

>Of course.  So, why would you do that?

Just to get a sensitive feeling about increased IW in an low bandwidth
environment. A environment where TCP is used as well. At the lower end of low
bandwidth environments a pacing based on the min-mtu-sized packet RTT
(SYN/ACK) will differ from the full-mtu-sized RTT (DATA).

Hagen

---

From: Hagen Paul Pfeifer <[email-masked]>
Date: Wed, 6 Jan 2016 18:12:10 +0100

>Of course.  So, why would you do that?

One note: you will see that at a given bandwidth (i.e. 3000byte/sec) you will
encounter TCP timeouts if you raise IW to a certain level. "Somewhere" there
is a correlation between bandwidth and absorbable IW. The lower the bandwidth
the higher the risk that an increased IW has an negative effect on connection
stability.

Hagen

---

From: Joe Touch <[email-masked]>
Date: Wed, 6 Jan 2016 10:55:59 -0800

FWIW, I submitted an errata on this. It's clearly an error.

JOe

---

From: Mark Allman <[email-masked]>
Date: Wed, 06 Jan 2016 14:05:12 -0500

> One note: you will see that at a given bandwidth
> (i.e. 3000byte/sec) you will encounter TCP timeouts if you raise
> IW to a certain level. "Somewhere" there is a correlation between
> bandwidth and absorbable IW. The lower the bandwidth the higher
> the risk that an increased IW has an negative effect on connection
> stability.

Of course.  This is all well known stuff.  You could, for instance,
look at RFC 2416.  I think there is a general appreciation for the
impact of the IW in excessively low bandwidth environments.

And, your bringing this up underscores the basic reason why I think
no-initwin is the way to go: there is not any sort of one-size-fits-
all IW we can use.  Rather than pretending there is, we should get
out of the business of picking one answer and let implementations
handle this particular detail in a prudent fashion.

allman

---

From: Joe Touch <[email-masked]>
Date: Wed, 6 Jan 2016 11:12:49 -0800

> handle this particular detail in a prudent fashion.

Well, I can see two reasons not to do that:

1. tragedy of the commons
	There were a few companies that played that card in the
	late 90's as I recall, and we see it happening now with
	bigger players doing what's better for their bottom line,
	not necessarily for the Internet as a whole.

2. implementers don't know who their TCPs will be talking to (now or in
the future)

It's always been permissible to change any of TCP's parameters in closed
environments. The key question is whether it's safe to ignore the
broader impacts of widescale deployment of this recommendation. I still
don't think it is - IMO, the recommendation needs to come with a
feedback mechanism that prevents persistent use of window sizes that
could affect others badly (e.g., auto-iw).

Joe

---

From: Yuchung Cheng <[email-masked]>
Date: Wed, 6 Jan 2016 11:23:23 -0800

>> handle this particular detail in a prudent fashion.
so perhaps having some guideline on this would be useful since people
always want to and have already up IW to 100+ (no kidding).

> could affect others badly (e.g., auto-iw).
+1

---

From: SALLANTIN Renaud <[email-masked]>
Date: Thu, 7 Jan 2016 10:12:04 +0100

[@@ THALES ALENIA SPACE INTERNAL @@]

-----Message d'origine-----
De : tcpm [mailto:[email-masked]] De la part de Yuchung Cheng
Envoyé : mercredi 6 janvier 2016 20:23
À : Joe Touch
Cc : [email-masked] Extensions; [email-masked]
Objet : Re: [tcpm] removing TCP's initial window

>>>> One note: you will see that at a given bandwidth (i.e. 3000byte/sec) 
>>>> you will encounter TCP timeouts if you raise IW to a certain level. 
>>>>"Somewhere" there is a correlation between bandwidth and absorbable 
>>>> IW. The lower the bandwidth the higher the risk that an increased IW 
>>>> has an negative effect on connection stability.
>>>
so perhaps having some guideline on this would be useful since people always want to and have already up IW to 100+ (no kidding).

>> 2. implementers don't know who their TCPs will be talking to (now or 
>> in the future)
>>
>> It's always been permissible to change any of TCP's parameters in 
>> closed environments. The key question is whether it's safe to ignore 
>> the broader impacts of widescale deployment of this recommendation. I 
>> still don't think it is - IMO, the recommendation needs to come with a 

+1.
 A smart auto-IW could indeed limit unsuitable IW and reduce the risk of network collapse, but also enable some outstanding improvements in some other cases (such as satellite communication). 
In fact, our preliminary studies shown that a large IW  (higher than 50 segments) "smartly" paced in a large RTT environment (>500ms) improves the individual performance and has less incidence for the others that an un-paced IW of 10 segments.