Subject: Re: [tcpm] timestamps usage (was Re: 1323bis - preparing for WGLC)

From: Yuchung Cheng <[email-masked]>
Date: Mon, 1 Apr 2013 12:26:05 -0700

On Fri, Mar 29, 2013 at 11:10 PM, Scheffenegger, Richard <[email-masked]> wrote:
>
> Hi Yuchung,
>
> In the following scenario,
>
> what should the left side (sender) do, when the last ACK arrives?
>
>     <A, TSval=1> --------------------->
>
>    <---------------- <ACK(A), TSecr=1>
>
>     <B, TSval=2> --------------------->
>
>      X-------------- <ACK(B), TSecr=2>
>
>     <C, TSval=3> -------------------X
>
>
>
>     <D, TSval=5> --------------------->
>
>    <------- <ACK(B), SACK(D), TSecr=2>
>
>
> If RTT is the quantity to be measured, updating with that last ACK (which, for the sender, updates the left edge of the window - thus is acceptable in 1323, when ignoring SACK - as most stacks do) would inflate sRTT and RTTvar, increasing the RTO. And sRTT is *not* the RTT of the network.
>
> My point being, if the mechanism is to measure RTT, it should do this (with the current semantics, the only sensible approach would be that the last ACK must be ignored).

If I am understanding correctly you (and I) prefer to have accurate
RTT estimation instead of the inflated ones. But RFC1323 explicitly
prefers the latter.

"""

  3.4  Which Timestamp to Echo

      (B)  A hole in the sequence space (segment(s) have been lost).

           The sender will continue sending until the window is filled,
           and the receiver may be generating ACKs as these out-of-order
           segments arrive (e.g., to aid "fast retransmit").

           The lost segment is probably a sign of congestion, and in
           that situation the sender should be conservative about
           retransmission.  Furthermore, it is better to overestimate
           than underestimate the RTT.  An ACK for an out-of-order
           segment should therefore contain the timestamp from the most
           recent segment that advanced the window.

           The same situation occurs if segments are re-ordered by the
           network.
"""

since the newest -08
(http://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-1323bis-08) does not
update the paragraph above, I found the new rule of not taking
(inflated) RTT samples on ACK with (D)SACK contradicting.

Personally I would prefer the receiver to echo latest timestamps but
it's moot b/c it can not be deployed :(

I started to see Mark's point on TS RTTM.

>
> If the point is to only come up with some conservative estimate for RTO (and an even more conservative estimate in this case), the current text and implementations are fine even though that last ACK does indicate loss...
>
> However, the variables sRTT and RTTvar need to have a connotation that they are *only* intermediary values just used for RTO purposes and nothing else, and should be renamed that they are NOT to be used for anything but RTO estimation. Other (future, experimental) mechanisms need to derive their own state then.
>
> Best regards,
>
>
> Richard Scheffenegger
>
>
>
>> -----Original Message-----
>> From: Yuchung Cheng [mailto:[email-masked]]
>> Sent: Freitag, 29. März 2013 20:48
>> To: Scheffenegger, Richard
>> Cc: John Leslie; [email-masked]
>> Subject: Re: [tcpm] timestamps usage (was Re: 1323bis - preparing for
>> WGLC)
>>
>> On Thu, Mar 28, 2013 at 4:13 AM, Scheffenegger, Richard <[email-masked]>
>> wrote:
>> > Hi John,
>> >
>> >> From: John Leslie [mailto:[email-masked]]
>> >> >    A TCP MAY send the Timestamp option (TSopt) in an initial <SYN>
>> >> >    segment (i.e., segment containing a SYN bit and no ACK bit), and
>> MAY
>> >> >    send a TSopt in other segments only if it received a TSopt in the
>> >> >    initial <SYN> or <SYN,ACK> segment for the connection.
>> >>
>> >>    That much is the original RFC 1323 text, right?
>> >
>> > With minor editorial changes and an update to RFC2119 wording, yes.
>> >
>> >
>> >> >    Once TSopt has been successfully negotiated during the <SYN>,
>> >> >    <SYN,ACK> exchange, TSopt MUST be sent in every non-<RST> segment
>> for
>> >> >    the duration of the connection.
>> >>
>> >>    I didn't find this MUST anywhere in RFC 1323. Did I miss something?
>> >
>> > No; as mentioned on this list, certain mechanisms assume the permanent
>> use of Tsopt right after the successful negotiation. This text was added
>> by David in 2007 ( http://tools.ietf.org/html/draft-borman-1323bis-00 ):
>> >
>> >          Once a TSopt has been sent or received in a
>> >          non <SYN> segment, it must be sent in all segments.  Once a
>> >          TSopt has been received in a non <SYN> segment, then any
>> >          successive segment that is received without the RST bit and
>> >          without a TSopt may be ACKed and dropped without further
>> >          processing.
>> >
>> > After the discussions on this list, it seemed to be better to not leave
>> any potential gap between the 3WHS negotiation, and the actual use of the
>> TS option. All the implementations that I know of effectively work that
>> way.
>> >
>> >
>> >> >    If a non-<RST> segment is received without a TSopt, a TCP MAY drop
>> >> >    the segment and MAY send an <ACK> for the last in-sequence
>> segment.
>> >>
>> >>    This text confuses me. I don't particularly want to argue it, but
>> I'm
>> >> at a loss to understand how it helps. (There are four cases, right? And
>> >> there seem to be a couple of unnamed variables as well.)
>> >
>> > Well, the 2nd MAY is superfluous, I guess at this reading. But I'm open
>> for better wording suggestions.
>> >
>> >
>> >> >    A TCP MUST NOT abort a TCP connection if a non-<RST> segment is
>> >> >    received without a TSopt.
>> >>
>> >>    This sounds reasonable at first blush, but I don't see how anyone
>> could
>> >> depend on it.
>> >>
>> >> >    If a TSopt is received on a connection where TSopt was not
>> negotiated
>> >> >    in the initial three-way handshake, the TSopt MUST be ignored and
>> the
>> >> >    packet processed normally.
>> >>
>> >>    This would have been good to include in RFC 1323, but I wonder what
>> it
>> >> accomplishes to add it now.
>> >
>> > Just to make this behavior explicit. There is another example in the
>> dreded RTTM mechanism. 1323 just stated that any ACK that indicates
>> loss/reordering should be excluded. Most ACKs that contain SACK do
>> indicate some loss/reordering, but when I investigated, these ACKs are
>> still used to update RTO under certain circumstances. Thus the added text
>> there to explicitly exclude ACK+SACK for the RTTM mechanism.
>>
>> This caught my attention; I am stunned. I can't believe it :(
>> """
>> However, the following rule prevents a resulting inflation of the measured
>> RTT:
>> ...
>>       b)  the segment does not indicate any loss or reordering, i.e.
>>           contains SACK options
>> """
>>
>> The RTT of the ack contains a SACK reflects the delay when the network
>> is in trouble, and we want to toss it and not update the RTO. It's
>> still 3 days before April 1st.
>>
>>
>> >
>> >
>> >> >    In the case of crossing <SYN> segments where one <SYN> contains a
>> >> >    TSopt and the other doesn't, both sides SHOULD put a TSopt in the
>> >> >    <SYN,ACK> segment.
>> >>
>> >>    This sounds funny: very likely the one that didn't send TSopt
>> doesn't
>> >> want to dedicate N bytes of option space in every packet.
>> >
>> > This is new text; it was added in 2009 in
>> http://tools.ietf.org/html/draft-ietf-tcpm-1323bis-01
>> > We could leave that case undefined, as in 1323...
>> >
>> >
>> >> >    TSopt is required for the two mechanisms described in sections 3.3
>> >> >    and 4.2.  There are also other mechanisms that rely on the
>> presence
>> >> >    of the TSopt, e.g.  [RFC3522].  If a TCP stopped sending TSopt at
>> any
>> >> >    time during an established session, it interferes with these
>> >> >    mechanisms.  This update to [RFC1323] describes explicitly the
>> >> >    previous implicit assumption, that each TCP segment must have
>> TSopt,
>> >> >    once negotiated.
>> >>
>> >>    No doubt some of us believe such an assumption was "implicit"; and
>> >> quite possibly there are implementations which won't interoperate
>> without
>> >> that assumption being valid.
>> >>
>> >>    Nonetheless, I find it hard to believe that any implementations
>> break
>> >> horribly if the assumption fails.
>> >>
>> >>    Writing a 1323-bis, the question isn't simply, "What should we have
>> >> said the first time?" We must include, "How do we get here from there?"
>> >> This is hard, IMHO: which is why I suggested a new aption instead.
>> >
>> > I agree, things will usually not go terribly wrong if the assumptions
>> don't hold, but the implementations seen behave as described in the latest
>> text. I think it would be save to clearly state, that this is how to
>> actually use TSopt...
>> >
>> > Richard
>> >
>> > _______________________________________________
>> > tcpm mailing list
>> > [email-masked]
>> > https://www.ietf.org/mailman/listinfo/tcpm

---

From: Yoshifumi Nishida <[email-masked]>
Date: Mon, 1 Apr 2013 20:55:55 -0700

Hello,

On Fri, Mar 29, 2013 at 11:10 PM, Scheffenegger, Richard <[email-masked]>wrote:

> If RTT is the quantity to be measured, updating with that last ACK (which,
> for the sender, updates the left edge of the window - thus is acceptable in
> 1323, when ignoring SACK - as most stacks do) would inflate sRTT and
> RTTvar, increasing the RTO. And sRTT is *not* the RTT of the network.
>
> My point being, if the mechanism is to measure RTT, it should do this
> (with the current semantics, the only sensible approach would be that the
> last ACK must be ignored).
>

Hmm. I have a naive question here.
If it must be ignored, why the receiver have to put the TS?

--
Yoshifumi

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 2 Apr 2013 04:56:15 +0000

Ts is defined with symetric signalling. Also, the receiver can not know, if and which packet might be lost.

The current semantics minimize inflated rttm samples, but the interaction with sack was never (afaik) considered.

>From my reading of 1323, acks with sack should be ignored (for the purpose of rttm, not paws), to keep with the spirit of the mechanism. However, in the scenario described (this particular loss pattern and similar events) many stacks are not refraining from updating (bloating) rttm (srtt / rttvar)

But this is admittedly a minuscle issue (and not the big fish some here want to fry ;) )

Richard

Von meinem iPhone gesendet

Am 02.04.2013 um 05:56 schrieb "Yoshifumi Nishida" <[email-masked]<mailto:[email-masked]>>:

Hello,

On Fri, Mar 29, 2013 at 11:10 PM, Scheffenegger, Richard <[email-masked]<mailto:[email-masked]>> wrote:

Hi Yuchung,

--
Yoshifumi

---

From: Yuchung Cheng <[email-masked]>
Date: Mon, 1 Apr 2013 22:18:02 -0700

On Mon, Apr 1, 2013 at 9:56 PM, Scheffenegger, Richard <[email-masked]> wrote:
> From my reading of 1323, acks with sack should be ignored (for the purpose
> of rttm, not paws), to keep with the spirit of the mechanism. However, in
> the scenario described (this particular loss pattern and similar events)
> many stacks are not refraining from updating (bloating) rttm (srtt / rttvar)
>

Let me repeat my concern:
"""
section 3.4
   (B)  A hole in the sequence space (segment(s) have been lost).

        The sender will continue sending until the window is filled, and
        the receiver may be generating <ACK>s as these out-of-order
        segments arrive (e.g., to aid "fast retransmit").

        The lost segment is probably a sign of congestion, and in that
        situation the sender should be conservative about
        retransmission.  Furthermore, it is better to overestimate than
        underestimate the RTT.  An <ACK> for an out-of-order segment
        SHOULD therefore contain the timestamp from the most recent
        segment that advanced the window.

"""
directly contacts
"""
section 3.3
      b)  the segment does not indicate any potential loss or
          reordering, i.e. contains SACK [RFC2018] or D-SACK [RFC2883]
          options
"""

Please clarify.

Any data to support this is a problem?

If this is minor issue as you said why bother with this change? I
don't think this change is small.

> Am 02.04.2013 um 05:56 schrieb "Yoshifumi Nishida" <[email-masked]>:
>
> Hello,
>
>> 1323, when ignoring SACK - as most stacks do) would inflate sRTT and RTTvar,
>> increasing the RTO. And sRTT is *not* the RTT of the network.
>>

---

From: Yuchung Cheng <[email-masked]>
Date: Mon, 1 Apr 2013 22:20:06 -0700

>         The same situation occurs if segments are re-ordered by the
typo: directly contradicts

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 2 Apr 2013 12:39:20 +0000

Hi Yuchung,

Section 3.4 describes the receiver's behavior (ie. when to update TS.recent).

Section 3.3 describes the sender behavior (what to do when TSecr is received).

So, they don't contradict each other, they describe different mechanisms at different points...

In the scenario I have depicted, there is loss in the forward (sender->receiver), and backwards (receiver->sender) path.

And no, I don't have any data to show how prevalent this particular issue is. As that requires a very specific set of circumstances, it's really a nit. The text should make it clear that an implementer not simply ignores the presence of a SACK option when processing TSopt...

I only found this (some time ago) while testing SACK+TS interactions while checking the sources of a number of stacks.

Richard Scheffenegger

> Sent: Dienstag, 02. April 2013 07:18
> Cc: Yoshifumi Nishida; tcpm ([email-masked])
> > The current semantics minimize inflated rttm samples, but the
> > interaction with sack was never (afaik) considered.
> >
> > From my reading of 1323, acks with sack should be ignored (for the
> > purpose of rttm, not paws), to keep with the spirit of the mechanism.
> > However, in the scenario described (this particular loss pattern and
> > similar events) many stacks are not refraining from updating
> > (bloating) rttm (srtt / rttvar)
> >
> If this is minor issue as you said why bother with this change? I don't
> think this change is small.
> 
> >
> > Richard
> >
> > On Fri, Mar 29, 2013 at 11:10 PM, Scheffenegger, Richard
> > <[email-masked]>
> > wrote:
> >>
> >>
> >> Hi Yuchung,
> >>
> >> If RTT is the quantity to be measured, updating with that last ACK
> >> (which, for the sender, updates the left edge of the window - thus is
> >> acceptable in 1323, when ignoring SACK - as most stacks do) would
> >> inflate sRTT and RTTvar, increasing the RTO. And sRTT is *not* the RTT
> of the network.
> >>
> >> (with the current semantics, the only sensible approach would be that
> >> the last ACK must be ignored).
> >
> >

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 2 Apr 2013 23:43:14 -0700

Hi Richard,

On Mon, Apr 1, 2013 at 9:56 PM, Scheffenegger, Richard <[email-masked]>
wrote:
> Ts is defined with symetric signalling. Also, the receiver can not know,
if
> and which packet might be lost.

This is just my personal preference. I'd rather make an exception if we can
safely simplify the logic.
But, this might be a different draft if we pursue.
--
Yoshifumi

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 3 Apr 2013 09:03:24 +0000

Hi Yoshifumi,

do you propose to remove the text so that the sender ignores any present SACK information while processing TSopt (RTTM updates)?

Again, this is a very minor issue, but is in-line with the spirit of 1323 RTTM (don't update RTTM / RTO, when the ACK indicates any loss/reordering).

However, any stack that wants to use RTTM for more advanced purposes than a (very) conservative bound on RTO, needs to have its own logic apart from the 1323 logic to deal with that. (Using Linux, this is less of a concern due to the per-segment state used for RTTM updates rather than TSopt).

But if no consensus can be found with that text, I revert it to the original 1323 text.

Best regards,

Richard Scheffenegger

From: Yoshifumi Nishida [mailto:[email-masked]]
Sent: Mittwoch, 03. April 2013 08:43
To: Scheffenegger, Richard
Cc: tcpm ([email-masked])
Subject: Re: [tcpm] timestamps usage (was Re: 1323bis - preparing for WGLC)

Hi Richard,

> and which packet might be lost.

This is just my personal preference. I'd rather make an exception if we can safely simplify the logic.
But, this might be a different draft if we pursue.
--
Yoshifumi

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 3 Apr 2013 23:16:24 -0700

Hi Richard,

On Wed, Apr 3, 2013 at 2:03 AM, Scheffenegger, Richard <[email-masked]>wrote:

>  Hi Yoshifumi,****
>
> ** **
>
> do you propose to remove the text so that the sender ignores any present
> SACK information while processing TSopt (RTTM updates)?
>
 **
>
> Again, this is a very minor issue, but is in-line with the spirit of 1323
> RTTM (don’t update RTTM / RTO, when the ACK indicates any loss/reordering).
>

Hmm. Which text are you referring to? It sounds good thing, but I'm not
proposing yet..
I just prefer not to send TSecr if it's no use for RTTM.

Thanks,
--
Yoshifumi

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 4 Apr 2013 09:00:51 +0000

Hi Yoshifumi,

>> do you propose to remove the text so that the 
>> sender ignores any present SACK information 
>> while processing TSopt (RTTM updates)?
>>  
>> Again, this is a very minor issue, but is 
>> in-line with the spirit of 1323 RTTM (don't 
>> update RTTM / RTO, when the ACK indicates any 
>> loss/reordering).
>
> Hmm. Which text are you referring to? 

I was referring to the (sender side) mechanism, which TSecr may be used to update RTT, specifically the added point b) in 1323bis-08

      RTTM Rule: A TSecr value received in a segment is used to update
      the averaged RTT measurement only if

      a)  the segment acknowledges some new data, i.e., only if it
          advances the left edge of the send window, and

(a) excludes DupACKs (provided the initial ACK was not lost).
(b) excludes DupACKs when SACK is negotiated, and when the initial ACK of the sequence of DupACKs was lost. It doesn't help non-SACK sessions. 

> It sounds good thing, but I'm not proposing yet..

Well, that would require a special handling of TSecr=0 on the sender side (linux generally ignored TSopt when TSecr=0, which also wasn't proper, before relying solely on sender-side per-segment state for RTTM due to experience with Cubic). Alternatively it requires a change in the basic signaling (using a shortened TSopt without the TSecr part), if I understand you correctly. Finally, this would be a receiver side change, while updating a sender to ignore TSecr when a SACK opt is present, doesn't break any existing stacks and interactions, only improves RTTM/RTO handling of the updated sender stack. The receiver side change would have unpredictable results as many stacks don't have any special logic to determine if TSecr=0 signals an invalid TSecr value, or a valid reflected TSval that just happens to be zero. Under some circumstances, negative RTT could be calculated, and that would not be a good thing...

>>>     <A, TSval=-8> --------------------->
>>> tik
>>>    <---------------- <ACK(A), TSecr=-8>
>>>
>>>     <B, TSval=-7> --------------------->
>>> tik
>>>      X-------------- <ACK(B), TSecr=-7>
>>>
>>>     <C, TSval=-6> -------------------X
>>> tik
>>>
>>>
>>>     <D, TSval=-5> --------------------->
>>> tik
>>>    <------- <ACK(B), SACK(D), TSecr=0>   <== *

*) current TSecr (as in 1323) is declared to be TSecr=-7; A 1323 stack measures a RTT of (-4 - -7)=3 in this scenario, rather than ignoring the sample (RTT is assumed to be truly 1 without loss here, e.g. TS clock ticks after sending of the segment, before receiving the ACKs "tik").

A legacy 1323 sender (without special logic for the case TSecr=0, e.g. most stacks) with a modified receiver would measure a RTT of (-4 - 0) = -4 here (or a potentially large positive value).

Again, if the WG thinks that this minuscule improvement delivered by point (b) above is not deemed worthwhile, I strongly believe that removing that text is less problematic than suggesting major changes in the TSopt semantics.

Regards,

Richard Scheffenegger

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 4 Apr 2013 09:07:00 +0000

On an additional glance, perhaps this text reflects better what an implementer should be looking at:

      RTTM Rule: A TSecr value received in a segment MAY be used to 
      update the averaged RTT measurement only if

Adding a RFC2119 MAY to underline Mark's point that per-segment RTT sampling might not be the wisest way to update RTO...

Regards,

Richard Scheffenegger

---

From: Yuchung Cheng <[email-masked]>
Date: Thu, 4 Apr 2013 10:21:37 -0700

I would prefer not changing RTTM section at all, unless data indicate
it's worth the change. The existing rule is great to help implementor
avoid a common pitfall. The implementor can decide on other smaller
cases and RFCs do not have to be so rigid.

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 9 Apr 2013 15:46:50 +0000

Hi Yuchung, Yoshifumi,

as the interactions between different options aren't documented really anywhere I think this update to 1323 would be a good opportunity to at least hint an implementer to the possibility of utilizing other signals as well.

These are the sections in the older versions:

Original (1323):
 
      that is inflated by gaps in sending data.  However, the following
      rule prevents a resulting inflation of the measured RTT:

           A TSecr value received in a segment is used to update the
           averaged RTT measurement only if the segment acknowledges
           some new data, i.e., only if it advances the left edge of the
           send window.

Debated Text (bis-08):

   inflated by gaps in sending data.  However, the following rule
   prevents a resulting inflation of the measured RTT:

I propose the following compromise in the text.

Updated (bis-09):

   inflated by gaps in sending data.  However, the following rule
   prevents a resulting inflation of the measured RTT:

      RTTM Rule: A TSecr value received in a segment MAY be used to
      update the averaged RTT measurement only if the segment
      acknowledges some new data, i.e. only if it advances the left
      edge of the send window, and does not indicate any potential loss,
      i.e. no TCP SACK option is present.

I repeat, this is a minor corner case, doing little harm if not fixed (inflating RTO and delaying loss recovery by timeout slightly). However, I stongly feel that this is the correct way to treat RTT updates on the sender side, and if not added in this iteration, it probably won't be added ever...

Anyone having strong objections to this text segment?

Richard Scheffenegger

> Sent: Donnerstag, 04. April 2013 19:22
> I would prefer not changing RTTM section at all, unless data indicate it's
> worth the change. The existing rule is great to help implementor avoid a
> common pitfall. The implementor can decide on other smaller cases and RFCs
> do not have to be so rigid.
> 
> > Adding a RFC2119 MAY to underline Mark's point that per-segment RTT
> sampling might not be the wisest way to update RTO...
> >
> > Regards,
> >
> > Richard Scheffenegger
> >

---

From: Yuchung Cheng <[email-masked]>
Date: Tue, 9 Apr 2013 09:37:11 -0700

On Tue, Apr 9, 2013 at 8:46 AM, Scheffenegger, Richard <[email-masked]>wrote:

>
> Hi Yuchung, Yoshifumi,
>
>
> as the interactions between different options aren't documented really
> anywhere I think this update to 1323 would be a good opportunity to at
> least hint an implementer to the possibility of utilizing other signals as
> well.
>
>
It's a rewording of bis-08 but logically it's the same.

> I repeat, this is a minor corner case, doing little harm if not fixed
> (inflating RTO and delaying loss recovery by timeout slightly). However, I
> stongly feel that this is the correct way to treat RTT updates on the
> sender side, and if not added in this iteration, it probably won't be added
> ever...
>
I repeat this will nullify much benefits TS-RTTM provides over regular
RTTM. The advantage of TS-RTTM is to sample RTT on ACKs of retransmitted
sequences. It's common for these ACKs to both advance SND.UNA and still
contain, sometimes new, SACK blocks.

Unless you can provide some data to prove this is a bug needs to be
addressed, we should not revise RFC by personal feelings.

> Anyone having strong objections to this text segment?
I strongly object.

> it's

---

From: Matt Mathis <[email-masked]>
Date: Tue, 9 Apr 2013 09:38:20 -0700

>       i.e. no TCP SACK option is present.

I would like to point out that this is over constrained.  In particular
"acknowledges some new data" explicitly include when snd.fack advances, or
even when SACK reports any never retransmitted data, even though disallowed
later.

There are a number of algorithms (such as this one) that could be improved
with: gs/snd.una/snd.fack/g

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

Privacy matters!  We know from recent events that people are using our
services to speak in defiance of unjust governments.   We treat privacy and
security as matters of life and death, because for some users, they are.

---

From: Joe Touch <[email-masked]>
Date: Tue, 09 Apr 2013 10:13:07 -0700

+1

---

From: David Borman <[email-masked]>
Date: Tue, 9 Apr 2013 12:38:26 -0500

Let's take a step back.  The goal is that if you are using TSecr to
compute the RTT, for use in the RTO, you want a reasonable assurance
that it'll be somewhat accurate.  That happens when TSecr is from a packet
that was recently received on the remote end.  Delayed ACKs add to latency,
so you want to account for that when calculating the RTT.  But ACK packets
can be generated for other reasons, and they can't be relied on for calculating
a reasonable RTT, since not all ACKs are generated in direct response to an
incoming packet.

Hence, the rules in 1323 for which RTT to echo are intended to include
delayed ACKs in the RTT measurement, and the rules for which TSecr can
be used for RTO calculations are meant to exclude TSecr values that
we know can cause wildly inflated RTT values.

The "acknowledges some new data" bit is based on the fact that when the
left edge of the window advances, it means that a packet arrived that
filled in the hole, and the ACK is being generated in response to that
packet.

The change from 1323 to exclude RTT calculations if there is potential
loss is, in my opinion, overly restrictive.  The issue is that corner
cases have been identified where due to loss you can get an inflated RTT
measurement.  But that's not necessarily a bad thing.  For purposes of
updating the RTO, one inflated RTT isn't going to throw things out of whack.
And if you are hitting this multiple times, maybe that should be considered
part of the latency and included in the RTO, just as 1323 already includes
the latency of delayed ACKs in the RTT measurement.

One advantage of TS is that it *can* get reasonable RTT measurements even
when there is some packet loss.  Excluding RTT measurements if there is
packet loss losses that advantage.

So, I see this as a grey area, not absolute.  We want to include the
situations that are most likely to generate a reasonable RTT measurement,
and exclude the situations that are most likely to generate inflated
RTT measurements.  We don't need protection for every corner case that
might cause an inflated RTT measurement, because the RTO should be
somewhat resilient against that.

			-David Borman

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 9 Apr 2013 18:14:08 +0000

Hi Yuchung,

> It's a rewording of bis-08 but logically it's the same.

Guilty as charged. But formatted differently to look less imposing.

>> I repeat, this is a minor corner case, doing little harm 
>> if not fixed (inflating RTO and delaying loss recovery 
>> by timeout slightly). However, I stongly feel that this 
>> is the correct way to treat RTT updates on the sender 
>> side, and if not added in this iteration, it probably 
>> won't be added ever...
>
> I repeat this will nullify much benefits TS-RTTM provides 
> over regular RTTM. The advantage of TS-RTTM is to sample 
> RTT on ACKs of retransmitted sequences. It's common for 
> these ACKs to both advance SND.UNA and still contain, 
> sometimes new, SACK blocks.

Ah, there might be a misconception. This change concerns what the sender is doing with received TSecr.

The receiver, however, is already - according to both 1323 and 1323bis - keeping an "outdated" TSrecent, and reflecting that as TSecr, for the *entire* duration of a loss/reordering event.

You simply won't get a proper RTTM sampling using the current TS semantics during such an episode. I'm merely proposing to limit the impact to the sender these semantics can already have, under a specific set of circumstances.

Please have another look at the graph I posted earlier, 
http://www.ietf.org/mail-archive/web/tcpm/current/msg07810.html
and you'll note that TS RTT *will not* deliver any benefits over what linux is already doing... Actually, the linux stack matching the SACK updates to the sent segments will most likely sample *accurate* network RTT even during loss/reordering using the per-segment state. (I've not looked into that stack deep enough to say this is so, but it's definitely possible using all the state kept).

"Proper" TS RTTM would require that last ACK to have a TSecr=5, not TSecr=2.

In that case stated (loss of the last in-sequence ACK), 1323 will bloat RTT/RTO.
 
Best regards,

Richard Scheffenegger

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 9 Apr 2013 18:25:14 +0000

Thank you David and Matt,

Then the text should than actually be

  RTTM Rule: A TSecr value received in a segment MAY be used to
  update the averaged RTT measurement only if the segment
  advances the left edge of the send window (e.g. SND.UNA is 
  increased). 

As Matt pointed out, the old text would apply when SND.FACK advances (or the number of bytes marked missing in the scoreboard decreases). 

But Dave asserts that we don't want to take a RTT Measurement then as this would definitely inflate the RTT due to the semantics of TSrecent on the receiver side. 

This must be one of the few cases where you don't want s/snd.una/snd.fack/g

Does that now make sense?

Richard Scheffenegger

> From: David Borman [mailto:[email-masked]]
> Sent: Dienstag, 09. April 2013 19:38
> To: Matt Mathis; tcpm ([email-masked])
> Cc: Scheffenegger, Richard
> compute the RTT, for use in the RTO, you want a reasonable assurance that
> it'll be somewhat accurate.  That happens when TSecr is from a packet that
> was recently received on the remote end.  Delayed ACKs add to latency, so
> you want to account for that when calculating the RTT.  But ACK packets
> can be generated for other reasons, and they can't be relied on for
> calculating a reasonable RTT, since not all ACKs are generated in direct
> response to an incoming packet.
> 
> delayed ACKs in the RTT measurement, and the rules for which TSecr can be
> used for RTO calculations are meant to exclude TSecr values that we know
> can cause wildly inflated RTT values.
> 
> 
> And if you are hitting this multiple times, maybe that should be
> considered part of the latency and included in the RTO, just as 1323
> already includes the latency of delayed ACKs in the RTT measurement.
> 
> and exclude the situations that are most likely to generate inflated RTT
> measurements.  We don't need protection for every corner case that might
> cause an inflated RTT measurement, because the RTO should be somewhat
> resilient against that.
> 
> 			-David Borman
> 
> > I would like to point out that this is over constrained.  In
> > particular "acknowledges some new data" explicitly include when
> > snd.fack advances, or even when SACK reports any never retransmitted
> > data, even though disallowed later.
> >

---

From: Matt Mathis <[email-masked]>
Date: Tue, 9 Apr 2013 14:44:02 -0700

Serves me for jumping in late, without fully grocking the thread.

Actually if you change both ends s/*.una/*.fack/g would be a good thing.
(E.g. have the receiver update TSrecent when it advances seg.fack) However,
this does have a bit of a legacy deployment problem...

Thanks,
--MM--

On Tue, Apr 9, 2013 at 11:25 AM, Scheffenegger, Richard <[email-masked]>wrote:

> As Matt pointed out, the old text would apply when SND.FACK advances (or
> the number of bytes marked missing in the scoreboard decreases).
>
> But Dave asserts that we don't want to take a RTT Measurement then as this
> would definitely inflate the RTT due to the semantics of TSrecent on the
> receiver side.
>

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 9 Apr 2013 22:09:53 +0000

Hi Matt,

> From: Matt Mathis [mailto:[email-masked]] 
> Sent: Dienstag, 09. April 2013 23:44
>
> Actually if you change both ends s/*.una/*.fack/g would be a 
> good thing.   (E.g. have the receiver update TSrecent when it 
> advances seg.fack) However, this does have a bit of a legacy 
> deployment problem...

well, that change in the TS semantics to make that change was one of the details in my draft that got me to this 1323bissness, I think...

http://tools.ietf.org/html/draft-scheffenegger-tcpm-timestamp-negotiation-05

(but that draft is too convoluted, working with Brian and Mirja on a better set of documents).

Thanks!

Richard Scheffenegger

---

From: Yuchung Cheng <[email-masked]>
Date: Tue, 9 Apr 2013 16:33:05 -0700

On Tue, Apr 9, 2013 at 11:14 AM, Scheffenegger, Richard <[email-masked]>wrote:

> The receiver, however, is already - according to both 1323 and 1323bis -
> keeping an "outdated" TSrecent, and reflecting that as TSecr, for the
> *entire* duration of a loss/reordering event.
>
> You simply won't get a proper RTTM sampling using the current TS semantics
> during such an episode. I'm merely proposing to limit the impact to the
> sender these semantics can already have, under a specific set of
> circumstances.
>

> and you'll note that TS RTT *will not* deliver any benefits over what
> linux is already doing... Actually, the linux stack matching the SACK
> updates to the sent segments will most likely sample *accurate* network RTT
> even during loss/reordering using the per-segment state. (I've not looked
> into that stack deep enough to say this is so, but it's definitely possible
> using all the state kept).
>
I understand the problem of potential RTT overestimation (before you came
up that case) so are the original authors of RFC1323 even it was published
before SACK.

The "flawed" algorithm has ran years w/ SACK widely deployed w/o obvious
problems reported. Now we want to change that b/c some (one?) has strong
feels. It's not like major stacks have all fixed the flaw so RFC is
outdated/wrong. There are no data no experiments for a non-trivial change.

I simply don't buy the approach.

---

From: Yuchung Cheng <[email-masked]>
Date: Tue, 9 Apr 2013 21:05:58 -0700

On Tue, Apr 9, 2013 at 4:33 PM, Yuchung Cheng <[email-masked]> wrote:

>> You simply won't get a proper RTTM sampling using the current TS
>> semantics during such an episode. I'm merely proposing to limit the impact
>> to the sender these semantics can already have, under a specific set of
>> circumstances.
>>
>
To compare with the ack-lost example that Richard provided, let's consider
a typical fast recovery.

Send 10 packets and the first 7 packets are dropped
Got three DUPACKs of SACK 8,8-9,8-10
Fast retransmit 1 plus maybe a few more depending on the algorithm you use.
On receiving retransmit of packet 1 (R1), the receiver returns an ACK of 2
with SACK {8-10} and ECR = ts.recent = R1's TS-val.
(Similar acks for other fast retransmits.)

The latest bis-08 forbids taking TS-RTT samples of these ACKs. Why bother
to have TS-RTTM then?

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 10 Apr 2013 07:36:59 +0000

Hi Yuchung,

> To compare with the ack-lost example that Richard provided, let's
> consider a typical fast recovery.
>
> On receiving retransmit of packet 1 (R1), the receiver returns an ACK
> of 2 with SACK {8-10} and ECR = ts.recent = R1's TS-val.
> The latest bis-08 forbids taking TS-RTT samples of these ACKs. Why 
> bother to have TS-RTTM then?

I see your point; the -08 wording would break the TS RTT sampling when the loss recovery properly closes the outstanding segments from the left. Checking that a new ACK only acknowledges one segment (including SACKed data) when SACK is present would be less trivial...

Do you (and everyone else) agree with the last iteration for the RTTM rule, if so then I'd submit the latest text.

Best regards,

Richard Scheffenegger

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 10 Apr 2013 00:39:42 -0700

> (Similar acks for other fast retransmits.)

With this logic, I think the sender will do RTTM without the effect of
Delayed ACK.
I'm not sure if this is good thing.
--
Yoshifumi

---

From: Yuchung Cheng <[email-masked]>
Date: Wed, 10 Apr 2013 10:26:45 -0700

On Wed, Apr 10, 2013 at 12:36 AM, Scheffenegger, Richard <[email-masked]>wrote:

> I see your point; the -08 wording would break the TS RTT sampling when the
> loss recovery properly closes the outstanding segments from the left.
> Checking that a new ACK only acknowledges one segment (including SACKed
> data) when SACK is present would be less trivial...
>
> Do you (and everyone else) agree with the last iteration for the RTTM
> rule, if so then I'd submit the latest text.
>
What's wrong with the original text?

"A TSecr value received in a segment is used to update the
averaged RTT measurement only if the segment acknowledges
some new data, i.e., only if it advances the left edge of the
send window."

If you must make a change (for reasons I am not sure), I suggest take up
Matt's comments:

"
A TSecr value received in a segment is used to update the
averaged RTT measurement only if the segment acknowledges
some new data including selective acknowledged ones.
"

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 10 Apr 2013 17:52:58 +0000

Hi Yuchung,

> If you must make a change (for reasons I am not sure), I suggest take up Matt's comments:
>
> "

The way the receiver handles TSecr... Only when the left window advances, the receiver updates TS.recent.

The wording the the original 1323 and Matt's suggested text would allow that sample, which is then bloated:

    <A, TSval=1> ---------------------> (advance left window edge, update TS.recent=1)

    <B, TSval=2> -------------------X

    <C, TSval=3> ---------------------> (don't advance left window edge, don't update TS.recent)

   <------- <ACK(A), SACK(C), TSecr=1>

    <D, TSval=5> ---------------------> (don't advance left window edge, don't update TS.recent)

   <----- <ACK(B), SACK(C,D), TSecr=1>

The two last ACKs both acknowledge some new data, but the left window edge is not updated - on the receiver side. If these ACKs are used for RTTM, the samples are inflated due to the receiver side handling when TS.recent is updated...

So, I think the proper wording should only point to an update of SND.UNA, not "new data".

Regards,

Richard Scheffenegger

---

From: Yuchung Cheng <[email-masked]>
Date: Wed, 10 Apr 2013 16:39:20 -0700

On Wed, Apr 10, 2013 at 10:52 AM, Scheffenegger, Richard <[email-masked]>wrote:

> The two last ACKs both acknowledge some new data, but the left window edge
> is not updated - on the receiver side. If these ACKs are used for RTTM, the
> samples are inflated due to the receiver side handling when TS.recent is
> updated...
>
Good point. I prefer to keep original text to your new text but the latter
is OK. I am not sure what's new in the new text tho.

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 11 Apr 2013 07:20:34 +0000

Hi Yuchung,

> Good point. I prefer to keep original text to your new text but the latter is OK. I am not sure what's new in the new text tho.

Matt made me aware of a potential misinterpretation.

The original text had "only if the segment acknowledges some new data", which could be interpreted - in conjunction with SACK (FACK) - that ACKs with new SACK blocks would qualify.

My suggestion has "only if the segment advances the left edge of the send window" explicitly, not only as an example, thus ruling out ACKs with new SACK blocks.

And we have the MAY, to indicate that not every possible RTT sample must be used to update sRTT / RTTvar, as Marks research clearly indicated the limited benefit of per-packet RTT samples.

Best regards,
   Richard

On Wed, Apr 10, 2013 at 10:52 AM, Scheffenegger, Richard <[email-masked]> wrote:
Hi Yuchung,

Regards,

Richard Scheffenegger