Subject: [tcpm] Setting ssthresh after idle

From: Andrew Knutsen <[email-masked]>
Date: Fri, 12 Jul 2013 14:08:39 -0700

I have a question about idle-restarts.  All the congestion RFC's 
require shutting CWND after a 1RTO idle period, however (as far as I 
know) only 2861 addresses ssthresh for that case: "If RTO has elapsed,  
ssthresh is set to the maximum of 3/4 CWND and the current value of 
ssthresh" (section 3.1). This is implemented in the latest Linux stack, 
but not in many BSD-based stacks.

    The problem is that if ssthresh gets reduced due to sporadic packet 
loss, it is very slow to recover.  In TCP's which do not implement the 
rfc2861 reset, bursty traffic can end up in CA all the time, resulting 
in inappropriately bad performance.  Even with that reset, performance 
can be inhibited long after drops have stopped.

    An argument can also be made that after an idle period, ssthresh 
should be reset to its initial state, so a regular SS can be done. Or, 
perhaps the value could be adjusted depending on the length of the idle 
period, as is the result of saving ssthresh in the host-cache for a 
series of short-lived connections (sort of ;-). Another alternative is 
to set it to the full CWND value...

    So, given the silence of rfc5681 on this, I'm hoping to get a sense 
of current thinking from this list...

Andrew

---

From: Yuchung Cheng <[email-masked]>
Date: Fri, 12 Jul 2013 17:06:17 -0700

On Fri, Jul 12, 2013 at 2:08 PM, Andrew Knutsen
<[email-masked]> wrote:
>
>    I have a question about idle-restarts.  All the congestion RFC's require
> shutting CWND after a 1RTO idle period, however (as far as I know) only 2861
Not all. RFC2861 halves cwnd every RTO interval.

> addresses ssthresh for that case: "If RTO has elapsed,  ssthresh is set to
> the maximum of 3/4 CWND and the current value of ssthresh" (section 3.1).
> This is implemented in the latest Linux stack, but not in many BSD-based
> stacks.
>
>    The problem is that if ssthresh gets reduced due to sporadic packet loss,
> it is very slow to recover.  In TCP's which do not implement the rfc2861
> reset, bursty traffic can end up in CA all the time, resulting in
> inappropriately bad performance.  Even with that reset, performance can be
> inhibited long after drops have stopped.
after a lengthy idle cwnd is usually set to IW and it will likely slow
start because cwnd < ssthresh
if the loss is sporadic, ssthresh is probably not reduced too often

>
>    An argument can also be made that after an idle period, ssthresh should
> be reset to its initial state, so a regular SS can be done. Or, perhaps the
> value could be adjusted depending on the length of the idle period, as is
> the result of saving ssthresh in the host-cache for a series of short-lived
> connections (sort of ;-). Another alternative is to set it to the full CWND
> value...
>
>    So, given the silence of rfc5681 on this, I'm hoping to get a sense of
> current thinking from this list...
>
> Andrew
>
>
>
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: [email-masked]
Date: Sat, 13 Jul 2013 08:59:57 +0100

I agree that ssthresh needs to be increased when a TCP connection
validates a higher cwnd and then later reduces this. This is one of the
mechanisms proposed in:
http://datatracker.ietf.org/doc/draft-ietf-tcpm-newcwv

See section 4.4.2

 " If the sender exits the non-validated phase after this
   period, it MUST update the ssthresh:

         ssthresh = max(ssthresh, 3*cwnd/4). "

This adjustment of ssthresh ensures that the sender records that it
has safely sustained the present rate.  The change is beneficial to
rate-limited flows that encounter occasional congestion, and could
otherwise suffer an unwanted additional delay in recovering the
sending rate.

- We'd be interested to hear thoughts on this.

Gorry

>> require
>> loss,
>> should

---

From: [email-masked]
Date: Sun, 14 Jul 2013 10:50:54 +0100

I think I agree.This is one of the issues we are trying to resolve in the
new-cwv update. Are you arguing that we should also reset ssthresh to the
initial value after some (much) longer period of no loss?

Or do you think a reset to 3/4*cwnd before window reduction is sufficient?

One thing that could be useful perhaps is to specify some guidance on
implementing an ssthresh hostcache. What do people think we should say?

Gorry

>
> The nasty problem is with long lived persistent high data rate WAN
> connections. A congestion event or some other event can cause ssthresh to
> get ratcheted down to a very low level and if ssthresh only gets updated
> on packet drops, it may never recover until the connection is closed and
> re-established. If an ssthresh hostcache is in use, it may never recover
> because the re-established connection will assume the ssthresh from the
> previous connection.
>
> Recovery requires a sufficiently long transfer to ramp the flight-size up
> (at a CA ramp rate) to cause another packet loss. At that point ssthresh
> will get set to the proper level for congestion avoidance at the current
> congestion level. But on a high data persistent WAN connection, it may
> never transfer a sufficiently large file to cause another packet drop. So
> the connection gets stuck in a very low performance mode. Tests with
> setting ssthresh = max(ssthresh, 3*cwnd/4) during an idle period showed
> that it recovered until the transfers were too short to push CWND
> sufficiently above ssthresh so the 3/4 CWND was higher than ssthresh.
>
> As written, it looks to me like RFC5681 was intended for the slow-start,
> congestion avoidance, fast retransmit, and fast recovery mechanisms to be
> applied to each transfer or set of transfers that occur without a
> sufficiently long idle period (i.e. 1 RTO or more). But, after an idle
> period (which could be 1 RTO or a couple days on a long lived connection),
> slow-start is intended to be used to re-acquire "ACK clock" and determine
> the available bandwidth at the current congestion level. Although not
> explicitly specified, the below statement in section 3.1 should be
> re-applied after each sufficiently long idle period. Otherwise, only the
> first transfer(s) of a long-lived connection will adjust to the current
> network conditions. The rest will only adjust to the current network
> conditions as long as ssthresh happens to be sufficiently high.
>
> "The initial value of ssthresh SHOULD be set arbitrarily high (e.g.,
>    to the size of the largest possible advertised window), but ssthresh
>    MUST be reduced in response to congestion.  Setting ssthresh as high
>    as possible allows the network conditions, rather than some arbitrary
>    host limit, to dictate the sending rate.
> "
>
> Gary
>
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Saturday, July 13, 2013 1:00 AM
> To: Yuchung Cheng
> Cc: Knutsen, Andrew; [email-masked] Extensions; McAlpine, Gary
> Subject: Re: [tcpm] Setting ssthresh after idle
>
> This adjustment of ssthresh ensures that the sender records that it has
> safely sustained the present rate.  The change is beneficial to
> otherwise suffer an unwanted additional delay in recovering the sending
> rate.
>
>>> know) only
>>> 2861
>>> addresses ssthresh for that case: "If RTO has elapsed,  ssthresh is
>>> set to the maximum of 3/4 CWND and the current value of ssthresh"
>>> (section 3.1).
>>> This is implemented in the latest Linux stack, but not in many
>>> BSD-based stacks.
>>>
>>>    The problem is that if ssthresh gets reduced due to sporadic
>>> packet loss, it is very slow to recover.  In TCP's which do not
>>> implement the rfc2861 reset, bursty traffic can end up in CA all the
>>> time, resulting in inappropriately bad performance.  Even with that
>>> reset, performance can be inhibited long after drops have stopped.
>> start because cwnd < ssthresh if the loss is sporadic, ssthresh is
>> probably not reduced too often
>>
>>>
>>> should be reset to its initial state, so a regular SS can be done.
>>> Or, perhaps the value could be adjusted depending on the length of
>>> the idle period, as is the result of saving ssthresh in the
>>> host-cache for a series of short-lived connections (sort of ;-).
>>> Another alternative is to set it to the full CWND value...
>>>
>>>    So, given the silence of rfc5681 on this, I'm hoping to get a
>>> sense of current thinking from this list...
>>>
>>> Andrew
>>>
>>>
>>>

---

From: "McAlpine, Gary" <[email-masked]>
Date: Mon, 15 Jul 2013 02:28:37 +0000

I've been testing a version where we set ssthresh to the receivers advertised window initially and after each idle period so that it does a full slow start at the beginning of each round of transfers. The results look really good. After each idle period, the throughput ramps up at the slow start rate until it either starts being flow controlled by the receiver's advertised window or it detects a packet drop. If it detects a packet drop, then it goes into congestion control mode until the next idle period. Then CWND and ssthresh get re-initialized for the next round of transfers. This version properly adjusted each round of transfers to the current network conditions. To me, it looks like it does what RFC 5681 intended.

I also tested versions where we set ssthresh to either 3/4*cwnd or to cwnd, but both of those were still very slow to recover (if at all). The version that used 3/4*cwnd tended to quit recovering fairly quickly unless we kept ramping up the sizes of the transfers.

I am not a fan of ssthresh hostcache. If ssthresh can get into an inappropriately low state, why would you want to remember it and keep perpetuating it from connection to connection? I think you want to let each new connection re-initialize cwnd and ssthresh to allow the connection to adjust to the current network conditions.

Thanks,
Gary

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Sunday, July 14, 2013 2:51 AM
To: McAlpine, Gary
Cc: [email-masked]; Yuchung Cheng; Knutsen, Andrew; [email-masked] Extensions
Subject: RE: [tcpm] Setting ssthresh after idle (ssthresh hostcache)

I think I agree.This is one of the issues we are trying to resolve in the new-cwv update. Are you arguing that we should also reset ssthresh to the initial value after some (much) longer period of no loss?

One thing that could be useful perhaps is to specify some guidance on implementing an ssthresh hostcache. What do people think we should say?

Gorry

> connections. A congestion event or some other event can cause ssthresh 
> to get ratcheted down to a very low level and if ssthresh only gets 
> updated on packet drops, it may never recover until the connection is 
> closed and re-established. If an ssthresh hostcache is in use, it may 
> never recover because the re-established connection will assume the 
> ssthresh from the previous connection.
>
> Recovery requires a sufficiently long transfer to ramp the flight-size 
> up (at a CA ramp rate) to cause another packet loss. At that point 
> ssthresh will get set to the proper level for congestion avoidance at 
> the current congestion level. But on a high data persistent WAN 
> connection, it may never transfer a sufficiently large file to cause 
> another packet drop. So the connection gets stuck in a very low 
> performance mode. Tests with setting ssthresh = max(ssthresh, 
> 3*cwnd/4) during an idle period showed that it recovered until the 
> transfers were too short to push CWND sufficiently above ssthresh so the 3/4 CWND was higher than ssthresh.
>
> As written, it looks to me like RFC5681 was intended for the 
> slow-start, congestion avoidance, fast retransmit, and fast recovery 
> mechanisms to be applied to each transfer or set of transfers that 
> occur without a sufficiently long idle period (i.e. 1 RTO or more). 
> But, after an idle period (which could be 1 RTO or a couple days on a 
> long lived connection), slow-start is intended to be used to 
> re-acquire "ACK clock" and determine the available bandwidth at the 
> current congestion level. Although not explicitly specified, the below 
> statement in section 3.1 should be re-applied after each sufficiently 
> long idle period. Otherwise, only the first transfer(s) of a 
> long-lived connection will adjust to the current network conditions. 
> The rest will only adjust to the current network conditions as long as ssthresh happens to be sufficiently high.
>
> validates a higher cwnd and then later reduces this. This is one of 
> the mechanisms proposed in:
>> after a lengthy idle cwnd is usually set to IW and it will likely 
>> slow start because cwnd < ssthresh if the loss is sporadic, ssthresh 
>> is probably not reduced too often
>>
>>>

---

From: "McAlpine, Gary" <[email-masked]>
Date: Mon, 15 Jul 2013 15:44:01 +0000

With regard to ssthresh hostcache, the only problem I see with it is that ssthresh can get set to an inappropriately low value with no mechanism to recover reasonably fast. If re-initializing "ssthresh = rwnd" during each idle period is too aggressive, then something less aggressive could be used. I would like to see something time based or something like a binary search based on rwnd or a combination thereof. A binary search would be simple; i.e. during each idle, set ssthresh = ssthresh(current) + (rwnd - sshtresh(current))/2. Or, if the idle time was tracked, then make a similar adjustment for each increment (say 1 RTO) of idle time.

Thanks,
Gary

-----Original Message-----
From: McAlpine, Gary 
Sent: Sunday, July 14, 2013 7:29 PM
To: [email-masked]'
Cc: Yuchung Cheng; Knutsen, Andrew; [email-masked] Extensions
Subject: RE: [tcpm] Setting ssthresh after idle (ssthresh hostcache)

Thanks,
Gary

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Sunday, July 14, 2013 2:51 AM
To: McAlpine, Gary
Cc: [email-masked]; Yuchung Cheng; Knutsen, Andrew; [email-masked] Extensions
Subject: RE: [tcpm] Setting ssthresh after idle (ssthresh hostcache)

Gorry