Subject: [tcpm] Comments on draft-ietf-tcpm-rtorestart-01

From: Per Hurtig <[email-masked]>
Date: Wed, 18 Dec 2013 11:29:45 +0100

Hi all,

Following the discussions in Vancouver, there are two outstanding issues related
to the RTO restart draft that we would like feedback on:

(i) Should we have a section in the draft describing a socket option?

This was suggested during the tcpm meeting, but we also got comments in the
hallway that maybe this was not needed. Controlling RTO restart on a per socket
basis may be useful, but it is partly an implementation decision. Our current
Linux patch has both a sysctl and a socket option to aid in experimentation.
Looking at other RFCs the SCTP RFCs are quite careful to define socket options
whereas the TCP RFCs often leave it out. We don't have any strong opinions
regarding this. We are more than happy to add an API section if the working
group wants one, but we are not sure that it is necessary.

(ii) Should we apply RTO restart for all segments or only when the amount of
outstanding data is too small to support fast retransmit?

This discussion has been going for a long time, both on and off the list. Our
first version of the draft did the modified restart for all segments. However,
in discussions with various people it soon became apparent that from a
performance perspective it would be rather bad to do it this way, as it
increases the risk of having RTOs (with deep cuts in the cwnd and so on). As
long as you technically can recover a loss using fast retransmit you should do
that. Thus, in scenarios where it is possible to use fast retransmit, it would
only be contra productive to lower the value of the retransmission timer. On the
other hand, when it's impossible to use fast retransmit then it's a good thing :)
This is the rationale behind the current draft.
The suggestion by some for applying RTO restart to all segments is based on the
rationale that this would simplify the algorithm, and would avoid further
complicating an already complex RTO management.

The authors' suggestion is to leave the algorithm as it currently is, as we
believe this to be the right approach. The additional condition is by itself not
very complex. We believe any general issues with and needed reworking of the RTO
management belongs as a separate work item.

Cheers,
Per

---

From: Joe Touch <[email-masked]>
Date: Wed, 18 Dec 2013 08:42:23 -0800

> group wants one, but we are not sure that it is necessary.

IMO, you should definitely indicate whether this variant should be 
default, whether it should apply to all connections, and whether it 
should be configurable to apply to specific connections.

That is, IMO, required as part of the API of this protocol - and yes, as 
with RFC 793, I think such APIs are mandatory for all protocols (despite 
IETF confusion on this point, esp. in the past).

However, describing it as a "socket option" or specifying the Unix-style 
interface should be at best in an appendix as one example, but I would 
prefer to not include that level of detail at all.

I definitely feel strongly that RFC2119 keywords should never be used to 
describe such APIs in the context of a specific OS.

Joe

---

From: Yuchung Cheng <[email-masked]>
Date: Wed, 18 Dec 2013 10:58:24 -0800

> basis may be useful, but it is partly an implementation decision. Our current
What're the useful cases in your mind?

> very complex. We believe any general issues with and needed reworking of the RTO

Fine if you want to be conservative (and complicate things).

Part of these compilation comes from resetting cwnd on timeout. It's
like a time-ticking bomb for TCP performance. Consider the case where
the entire cwnd is delivered except the last one. There is no point to
re-slow-start to restore the ack clock, because you already have
cwnd-1 acks last round.

I plan to implement rto-restart idea in Linux as base of a new loss
recovery, but probably not the cwnd checking part. When an RTO fires
it's time to retransmit. The need to check cwnd implies my timer is
problematic.

> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 19 Dec 2013 09:24:16 +0000

Hi Per,

> What're the useful cases in your mind?

I don't think that a granularity on a per-session level would really do any good; having a global sysctl (or similar) to influence the overall behavior of the stack is fine. But from a troubleshooting point of view, I doubt that such a fine level of control will do any good.

>> (ii) Should we apply RTO restart for all segments or only when the
>> amount of outstanding data is too small to support fast retransmit?
>
> Part of these compilation comes from resetting cwnd on timeout. It's like
> a time-ticking bomb for TCP performance. Consider the case where the
> entire cwnd is delivered except the last one. There is no point to re-
> slow-start to restore the ack clock, because you already have
> cwnd-1 acks last round.
> 
> recovery, but probably not the cwnd checking part. When an RTO fires it's
> time to retransmit. The need to check cwnd implies my timer is
> problematic.

Provided that you have other means (ie TS) to detect if the ACK was due to an spurious retransmit (RTO) or genuinely needed, I'm not sure if that complexity of checking the currently outstanding segments really is needed.

As Yuchung points out, removing this would reduce the complexity of the algorithm. If spurious retransmits are a problem, and they often are, the stack requires some other way of dealing with those anyway, because "fixing" only rto-restart won't address all the other (currently happening) instances of spurious rtos...

Seasons greetings,

Richard Scheffenegger

---

From: Michael Welzl <[email-masked]>
Date: Thu, 19 Dec 2013 10:34:53 +0100

> I don't think that a granularity on a per-session level would really do any good; having a global sysctl (or similar) to influence the overall behavior of the stack is fine. But from a troubleshoot…

+1

Cheers,
Michael

---

From: Anna Brunström <[email-masked]>
Date: Thu, 19 Dec 2013 11:27:15 +0100

Hi,

> What're the useful cases in your mind?

The one case we had in mind was if an application with a bursty send 
pattern worried about spuriou RTOs would benefit from the possibility to 
turn it off.

But based on the feedback on the list so far and some further internal 
discussions, all authors agree that the added complexity will not be 
worthwhile and just having it on as a default is our suggestion.

> re-slow-start to restore the ack clock, because you already have

Agreed. The time-ticking bomb is why we suggest the conservative approach.

Anna

---

From: Anna Brunström <[email-masked]>
Date: Thu, 19 Dec 2013 11:41:56 +0100

Hi,

> As Yuchung points out, removing this would reduce the complexity of the algorithm. If spurious retransmits are a problem, and they often are, the stack requires some other way of dealing with those…

This is not only about spurious RTOs, it is also about how you recover 
lost packets. You don't want an RTO for that if a fast retransmit can be 
done.

Also for spurious RTOs, TS isn't always used, and anyway all mechanisms 
for spurious timeout detection detect and repair the problem *after* it 
happened ... this is better than nothing, but it's even better to keep 
the spurious timeout from happening in the first place. It is a simple 
check, so I think that one line of code is worthwhile.

Anna

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 19 Dec 2013 12:00:26 +0000

Hi Anna,

>> As Yuchung points out, removing this would reduce the complexity of the
>> algorithm. If spurious retransmits are a problem, and they often are, the
>> stack requires some other way of dealing with those anyway, because
>> "fixing" only rto-restart won't address all the other (currently
>> happening) instances of spurious rtos...
>>
> 

But how do you positively know in advance, if a fast retransmit can be done? Looking at cwnd alone is not sufficient, as you also need a model of the receiver (delayed ACK - phase and frequency) and return path (ACK loss, delay)... Just because there may exist a chance to do fast retransmits doesn't imply that fast retransmits will be the most timely loss recovery...
 
> happened ... this is better than nothing, but it's even better to keep the
> spurious timeout from happening in the first place. It is a simple check,
> so I think that one line of code is worthwhile.

Again, without precognition on the sender side, you'll always need some reactive mechanisms; trying to be proactive alone will not always help...

Overall, I think making rto-restart dependent on cwnd should be optional, not necessarily normative. Implementers who have repair mechanisms after spurious RTOs happened may choose to implement a more simple rto restart; or they may choose to make a better model to trigger RTO (ie. improving the timeout calculation utilizing additional information ) 

Richard Scheffenegger

---

From: Anna Brunström <[email-masked]>
Date: Thu, 19 Dec 2013 18:16:25 +0100

Hi Richard,

> But how do you positively know in advance, if a fast retransmit can be done? Looking at cwnd alone is not sufficient, as you also need a model of the receiver (delayed ACK - phase and frequency) an…

It is true that you can not be sure that a fast retransmit will always 
be triggered, but as it is much preferred you want to give it as good of 
a chance as possible.

As a side note, it is not the cwnd, but the number of outstanding 
segments that is used.

> Again, without precognition on the sender side, you'll always need some reactive mechanisms; trying to be proactive alone will not always help...

True, but this does not mean that you should not use proactive methods 
when possible! Preventing the problem is good even if you cannot prevent 
every instance. In this case the prevention is also very simple...

> Overall, I think making rto-restart dependent on cwnd should be optional, not necessarily normative. Implementers who have repair mechanisms after spurious RTOs happened may choose to implement a m…

If you create your own RTO calculation you are not following RFC 6298 
anyway and you can do as you wish also with RTO restart I guess? As RTO 
restart updates a RECOMMENDED algorithm I guess you also have some 
freedom from that.

Anna

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Fri, 20 Dec 2013 13:26:37 +0000

Hi,

Two questions related to this discussion:

- Wouldn't it make sense to use RFC 2119 language in the algorithm, given that different options seem to exist?

- An implementation question: Is there any real data on the complexity of the condition under discussion? For instance, could determining the number of outstanding packets for each ACK require traversing the whole SACK scoreboard, in cases where it would not be required today? That probably depends on what state the TCP sender keeps already today. Still, if this condition had significant side effects in certain stacks, it would be relevant to understand them IMHO.

Thanks

Michael

________________________________________
Von: tcpm [[email-masked]]&quot; im Auftrag von &quot;Anna Brunström [[email-masked]]
Gesendet: Donnerstag, 19. Dezember 2013 18:16
Bis: Scheffenegger, Richard
Cc: [email-masked]
Betreff: Re: [tcpm] Comments on draft-ietf-tcpm-rtorestart-01

Hi Richard,

Anna

---

From: Yuchung Cheng <[email-masked]>
Date: Fri, 20 Dec 2013 08:36:54 -0800

> - An implementation question: Is there any real data on the complexity of the condition under discussion? For instance, could determining the number of outstanding packets for each ACK require trav…
Not for Linux. it keeps these counters when it updates sack
scoreboard, and other stacks probably do that cause it's fairly
simple.