Subject: Re: [tcpm] spurious timeout reaction

From: Pasi Sarolahti <[email-masked]>
Date: Thu, 04 Jan 2007 18:40:34 +0200

On Fri, 2006-12-15 at 15:00 -0500, ext Mark Allman wrote:
> We think the document is in decent shape and that it might make a good
> standards track mechanism for reacting to spurious timeouts.  The basic
> idea is that we say a TCP MAY make the RTO estimate more conservative in
> the face of spurious timeouts with the hopes of avoiding future
> timeouts.  In particular, we believe that since RFC2988 explicitly
> allows a TCP to be more conservative than the estimator given in that
> RFC that this reaction is already allowed and a stack using it would be
> considered compliant.

Wouldn't it be better to make this Experimental at first, like the other
related RFCs are?

I have to say I'm (still) somewhat unconvinced about this scheme. If one
link on the path is having delay spikes, it does not make the other
links any less likely to have loss patterns that cause legitimate
timeouts. Also, a wireless host that suffers from delay spikes can also
suffer from wireless packet losses that may cause legitimate timeouts
(that are not an indication of congestion) -- for example in a case
where a wireless host is able to use both Wireless LAN (RTTs of about
few milliseconds) and GPRS (RTTs in the range of 500 - 1000 ms +
occasional delay spikes) links, which is a fairly common case in the
recent mobile terminals. When a host is getting to the edge of the
wireless LAN coverage, it starts suffering from packet losses that could
easily cause a timeout before the host gets a new access interface up
and running. On the other hand, while using the GPRS interface the same
TCP connection can suffer from delay spikes.

So I don't think that delay spikes and legitimate timeouts during the
same TCP connection are necessarily as unlikely phenomenon as Section 4
on the disadvantages indicates (with a note that a legitimate RTO is not
necessarily a signal of bad congestion, like the text says -- it could
be an indication of loss of entire window on a WLAN link that just got
out of coverage, or loss of a retransmission).

I was also wondering about the dynamics of K' in the above case, because
hand-off to a high-latency link can also cause a spurious timeout.
Consider a host using WLAN for a while, so that the SRTT becomes around
5-10 ms and RTTVAR a few ms. Then the host moves to use GPRS where the
RTT is around 500-1000 ms and RTTVAR would maybe be in the ballpark of
100 ms. If the first RTT sample on the GPRS link is a longer one (as it
often is), the following scenario could be possible

K' = ceil ((R' - SRTT_prev) / RTTVAR_prev)
could make
ceil ((1000 - 10) / 3)  =  330

and after a while on the GPRS link the RTO would be around
RTO <- SRTT + max (G, K*RTTVAR), that could make (with an example SRTT
of 700 ms)
700 ms + max (G, 330 * 100 ms)  =  33700 ms

(did I get everything right above?)

So I believe there are possible real-world scenarios where this scheme
could kill the RTO performance, which is another reason why I would
prefer this becoming Experimental rather than Standards Track.

- Pasi

_______________________________________________
tcpm mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tcpm

---

From: Mark Allman <[email-masked]>
Date: Mon, 08 Jan 2007 10:28:57 -0500

Pasi-

Thanks for the note.  Some thoughts ...

> > We think the document is in decent shape and that it might make a
> > good standards track mechanism for reacting to spurious timeouts.
> > The basic idea is that we say a TCP MAY make the RTO estimate more
> > conservative in the face of spurious timeouts with the hopes of
> > avoiding future timeouts.  In particular, we believe that since
> > RFC2988 explicitly allows a TCP to be more conservative than the
> > estimator given in that RFC that this reaction is already allowed
> > and a stack using it would be considered compliant.
> 
> Wouldn't it be better to make this Experimental at first, like the
> other related RFCs are?

IMO, no.

I have a few reasons for that opinion:

(1) As noted above, nothing we suggest in this document would make a TCP
    non-conformant with RFC 2988.  So, as far as I am concerned RFC or
    not this change could be implemented by a stack today and nobody
    could complain about it.

(2) This change would not be a MUST.  That is, implementers could use
    this scheme or not.

(3) I think we are way too conservative with the "PS" designation.  We
    seem to have this feeling that everything needs a run at
    experimental first without any real reason.  Some things certainly
    should take a spin there.  And, where the line should be is clearly
    fuzzy.  I would personally draw it differently than the community
    has drawn it.

(4) The change is not overall hurtful to the network.

> If one link on the path is having delay spikes, it does not make the
> other links any less likely to have loss patterns that cause
> legitimate timeouts. Also, a wireless host that suffers from delay
> spikes can also suffer from wireless packet losses that may cause
> legitimate timeouts (that are not an indication of congestion) -- for
> example in a case where a wireless host is able to use both Wireless
> LAN (RTTs of about few milliseconds) and GPRS (RTTs in the range of
> 500 - 1000 ms + occasional delay spikes) links, which is a fairly
> common case in the recent mobile terminals. When a host is getting to
> the edge of the wireless LAN coverage, it starts suffering from packet
> losses that could easily cause a timeout before the host gets a new
> access interface up and running. On the other hand, while using the
> GPRS interface the same TCP connection can suffer from delay spikes.

I understand that we can think about scenarios where anything can cause
problems.  And, it is not my goal to cause problems here, of course.
But, I really have to wonder about all this and how much of it is *real*
and how much of it is just *possible*.  (And, even for things that are
real one also wonders about the prevalence.  Seemingly every corner case
happens sometimes or somewhere, but I am not sure that should drive our
engineering.) 

We explicitly added a bit to the document that says one could not use K'
when the connection would be for sure in the regime where it would be
using the RTO for all loss recovery.  I guess it is not clear to me what
these other scenarios are that really dictate the use of the RTO.  We
know how to do pretty robust loss recovery without relying on the RTO.

And, recall that by increasing the K we are avoiding spurious timeouts
that ultimately collapse the window, cause some go-back-N and generally
do not help connection performance at all.

> same TCP connection are necessarily as unlikely phenomenon as Section
> 4 on the disadvantages indicates (with a note that a legitimate RTO is
> not necessarily a signal of bad congestion, like the text says -- it
> could be an indication of loss of entire window on a WLAN link that
> just got out of coverage, or loss of a retransmission).

Losing an entire window is bad congestion.

I understand that is not what you meant to convey.  But, the thing is
.... loss == congestion is the assumption built into TCP.  So, when the
assumption is busted things break.  I am not sure that our particular
technique can or should be held up because the underlying assumption
sometimes does not hold.

(Also note that I am not saying it is ideal that we assume loss ==
congestion.  Clearly it is not.  And, clearly work on changing this is
useful.  I am just saying that is the current situation and that I do
not buy 'the assumption is busted' as a really great argument against
what we said in our document.)

> I was also wondering about the dynamics of K' in the above case,
> because hand-off to a high-latency link can also cause a spurious
> timeout.  Consider a host using WLAN for a while, so that the SRTT
> becomes around 5-10 ms and RTTVAR a few ms. Then the host moves to use
> GPRS where the RTT is around 500-1000 ms and RTTVAR would maybe be in
> the ballpark of 100 ms. If the first RTT sample on the GPRS link is a
> longer one (as it often is), the following scenario could be possible
> 

Before coffee this looks mostly right.  If one thinks the RTO min is
being used then the numbers above might not cause a spurious RTO.  But,
if one increased the spike a little then I think the argument makes
sense. 

One point is that if you ever dropped back to the WLAN you'd be
protected.  

(And, per the above, I am not sure I completely buy this case ...)

A couple of thoughts ...

  + The draft could say that a way to decide to use K' would be to first
    assess the number of RTOs that would be impacted.  So, e.g., if a
    TCP connection never RTOed and then had a spurious RTO because of a
    delay spike then it might make fine sense to use the K'.  On the
    other hand, if the spurious RTO was one 1 out of 100 RTOs and the K'
    ended up big (as above) then the TCP might decide to just not use
    the K'.  The problem is how to do the fuzzy middle of this space.

    I am not saying that we'd have to work out some scheme to do this.
    I am just wondering if, since this is not presented to be mandatory,
    whether we can add a little bit of explicit wiggle room for
    implementers.

  + We could keep the fundamental notion that the RTO estimation has
    failed, but change the particulars of how we cope.  

    Right now we make the argument that delay spikes that cause spurious
    RTOs indicate that the variance estimator is not coping with the
    variance in the RTT all that well.  I think that is a reasonable
    line of thinking.  But, what if we decided that the variance
    component was not supposed to catch delay spikes and added another
    term to catch these things if we experienced any that caused
    spurious timeouts?  So, if we have an R' from a spurious timeout and
    an RTO_prev, we could do something like this:

        DS = R' - RTO
        RTO = SRTT + max (G, K*RTTVAR) + DS
            (which is the formula from RFC2988 with "+ DS" added)

    Would that give us the same amount of protection without giving a
    multiplicative effect that balloons the RTO to ridiculousness?

  + Also, there are schemes floating around that would essentially
    re-init all the CC parameters on network change.  It seems that it
    might be useful to include any K' in that re-initialization such
    that the above case does not happen.  That is, when we can explain
    the delay spike and take a different approach to ensure that
    spurious RTOs are not repeated then we don't have to kludge up the
    RTO to try to cope.  It is only when we cannot explain the delay
    spike that we try to cope using the RTO.  Make any sense?

Further thoughts?

Thanks,
allman

---

From: Pasi Sarolahti <[email-masked]>
Date: Wed, 10 Jan 2007 14:41:06 +0200

(I'm chopping off the original mail here and there)

>     fuzzy.  I would personally draw it differently than the community

Yep, you can say that again... And sometimes it takes time to get even
Experimentals through. Speaking of PS, shouldn't the normative
references in a PS document also be standards track documents? That
would seem logical, at least. In particular, we are missing a spurious
timeout detection PS.

[...]
> happens sometimes or somewhere, but I am not sure that should drive our

A corner case for you might not be a corner case for me. The WLAN-GPRS
example is an existing case from real world, so for some people it is
mote than just a corner case. It is quite hard to determine what are the
globally typical path characteristics nowadays.

> know how to do pretty robust loss recovery without relying on the RTO.

Because TCP sender is able to detect spurious RTO, it also knows how
often genuine RTOs occur, i.e., how typical it is that the normal loss
recovery does not work as expected. One possibility could be that when
TCP sender detects a genuine RTO, it would reset K' back to the original
value.

> technique can or should be held up because the underlying assumption

Yep. But even if we assume that genuine RTO is an indication of bad
congestion, I'm not sure if we can justify K' as an additional
congestion control mechanism. Retransmission timer already has the
backoff mechanism that is quite conservative. If the initial RTO
estimate was several seconds, with the normal RTO backoff and some bad
luck the TCP sender would need to wait for its turn for quite a while.
If your neighbor didn't happen to have spurious timeouts, you are in
quite bad position in terms of fairness.

>     the K'.  The problem is how to do the fuzzy middle of this space.

The problem with the statistical methods is that they don't help the
short connections. And often the short connections are those where the
user can sense and get irritated by the relative effect of an RTO more
than in the long transfers.

>     multiplicative effect that balloons the RTO to ridiculousness?

This seems better to me. I think it is appropriate to treat a spurious
timeout as an exceptional event, rather than a component of the normal
RTT variance.

>     spike that we try to cope using the RTO.  Make any sense?

Yes, it does. And it could be useful to refer to, for example, the RLCI
draft here, maybe with a note that since spurious timeouts have been
identified to be a characteristic of some wireless links, it could be
worthwhile to consider implementing also RLCI, if this is implemented.

- Pasi

---

From: Mark Allman <[email-masked]>
Date: Wed, 10 Jan 2007 08:46:35 -0500

> Speaking of PS, shouldn't the normative
> references in a PS document also be standards track documents? 

Yes.

> That would seem logical, at least. In particular, we are missing a
> spurious timeout detection PS.

Yes.

> mote than just a corner case. It is quite hard to determine what are
> the globally typical path characteristics nowadays.

Of course you are right.

So, then how do we engineer things for a network that includes as much
heterogeneity as we currently have?

> > We explicitly added a bit to the document that says one could not
> > use K' when the connection would be for sure in the regime where it
> > would be using the RTO for all loss recovery.  I guess it is not
> > clear to me what these other scenarios are that really dictate the
> > use of the RTO.  We know how to do pretty robust loss recovery
> > without relying on the RTO.
> 
> TCP sender detects a genuine RTO, it would reset K' back to the
> original value.

I don't like this idea or really any idea that collapses K' back down.
The issue is that these delay spikes are *unforeseen*.  If we could
somehow predict that the RTO estimate was going to be too low then we'd
clearly do something about it or change the RTO calculation to
accommodate the fact that we could see a delay spike coming.  But, in
the absence of such information all we can do is look to our history
about the magnitude of delay spikes.  So, after collecting the history,
throwing it away seems like a bad idea to me.

> quite bad position in terms of fairness.

I am not quite sure I follow this.  K' is not a congestion control
mechanism.  The RTO is the congestion control mechanism.  If you have an
increased K (over the standard 4) and are still losing packets and have
to backoff the RTO further, you're going to have a hard time convincing
me that the increased K is really a big problem.  I.e., we're in a bad
situation and clearly K' is not a factor.

The point of K' is to try to avoid needless timeouts, not to in some
what shape a connection's rate or fair hare or whatever.  Certainly it
may take longer for legit RTOs.  But, a connection also may prevent the
ill-effects of a needless RTO.

> than in the long transfers.

Yep.  This technique is clearly for longer-term transfers.

(On the other hand, short connections might also always be in the regime
the document declares as off-limits for a K'.  I.e., a small cwnd.)

Thanks!

allman