Subject: [tcpm] Re: RFC 1323: Timestamps option

From: David Borman <[email-masked]>
Date: Fri, 25 May 2007 13:09:48 -0500

Hi,

I've been having some off-line discussion about how to turn on  
Timestamps mid-stream, and I'm coming to a clearer picture.  I'm now  
now leaning towards changing to allow this to happen in a standard  
way.  I think I can now more clearly enumerate the specific issues  
about enumerating them mid-stream.

First, do you or do you not include Timestamps in the SYN exchange,  
and how much do you care about interoperability with existing 1323  
implementations?  If you do not include Timestamps negotiation, then  
I would expect that mid-stream enabling of Timestamps would only work  
on modified hosts that understand the new rules.

I've come up with a variation on Option #2

Option 2a:
----------
The Timestamps option is sent only in the SYN and SYN,ACK packet.  If  
both sides understand deferred enabling of Timestamps, then the  
connection will continue without Timestamps until one of them  
initiates Timestamps by including it in a packet.  Once Timestamps  
have been sent or received after the initial SYN,ACK, they are then  
sent in all packets for the duration of the connection.

This has the advantage of knowing up front that the other side  
understands the Timestamps option, so when you later decide to enable  
them, you just turn them on and start sending them, you don't have to  
try to figure out midstream whether or not the other side understands  
Timestamps.  When talking to an RFC-1323 implementation, since the  
other side will include Timestamps in all packets, the option will be  
enabled right away.  However, my assumption is that if you have a  
deferred Timestamps implementation talking to an RFC-1323  
implementation, that the desired behavior is that Timestamps will be  
enabled for the entire connection.  If the preference is that talking  
to an RFC-1323 implementation will leave Timestamps disabled for the  
entire connection, then Timestamps shouldn't be negotiated in the SYN  
and SYN,ACK.  But if you don't negotiate Timestamps in the SYN  
exchange, then when you try to enable them mid-stream you have to  
include a mechanism to determine that the other side doesn't support  
them, so that you can stop sending Timestamps to hosts that don't  
support them.

The other issue is how to actually start using Timestamps mid- 
stream.  The information in Timestamps are used for two purposes:  
calculating RTT and for PAWS.  With PAWS, the main issue is to make  
sure that you are doing a valid check.  But RFC 1323 already contains  
a requirement that TS.Recent be able to be invalidated when a  
connection is idle for more than 24 days.  With deferred timestamps,  
you'd just need to ensure that when the first Timestamps option is  
received, that the code for dealing with an invalidated TS.Recent is  
executed.

The other issue is of RTT calculation.  RTT measurements can only be  
taken from ACKs that acknowledge new data.  If the receiving side  
decides it wants to turn on Timestamps and includes one in an ACK  
that acknowledges new data, the TSecr value (which is used to  
calculate RTT) won't be valid, even though it acknowledges new data.   
Probably the simplest way to address this problem is to restrict the  
enabling of Timestamps mid stream to data packets.  That makes a  
certain amount of sense since enabling Timestamps midstream will most  
likely be based on passing some threshold for the amount of data sent.

However, I do have an idea for a deterministic way to know when  
Timestamps have been fully enabled.  When you first start sending  
Timestamps (either initiating, or in response to receiving a  
Timestamps), you fill in the exact same value for TSval in every  
packet that you send, until you receive a packet with a TSecr equal  
to that value.  At that point you know that Timestamps are now  
enabled on both sides, and you start sending new TSval values.  You  
don't do any RTT calculations until you get a packet that  
acknowledges new data, and has a TSecr value that is beyond the  
initial TSval used.  This should address all three scenarios: the  
data receiver initiating enabling Timestamps, the data sender  
initiating enabling Timestamps, and both the sender and receiver  
enabling Timestamps at the same time.  PAWS will work just fine,  
because it doesn't require the clock to tick for each packet, just at  
least once for each sequence space.

So, when a deferred Timestamps implementation connects to an RFC-1323  
implementation, what should be the desired behavior?
	a) Timestamps are enabled for the entire connection
	b) Timestamps are disabled for the entire connection
I think it should be "a".

			-David Borman

On Jan 26, 2007, at 12:33 PM, Borman, David wrote:

> One of the topics that has been discussed in the past for the
> revision of RFC 1323 is to relax the requirement on when to send the
> Timestamps option.  I'd like to come to some resolution on this issue.
>
> Current:
> The Timestamps option is negotiated during the initial SYN exchange.
> If both sides support it, then every packet in the connection has to
> have the Timestamps option.
>
> Option #1
> ---------
> The proposed change is to relax the requirement that Timestamps have
> to be negotiated in the initial SYN, and instead allow them to be
> enabled later on in the connection.  At the time 1323 was originally
> written, there was a big concern for legacy TCP stacks that didn't
> handle unknown options in the middle of the connection, but were able
> to properly ignore unknown options during the SYN exchange.  But that
> was a long time ago, and unknown options in the middle of a TCP
> connection *shouldn't* be as big of an issue anymore.
>
> The rules for enabling Timestamps mid-stream would be:
> 1) If you receive a Timestamps option mid-stream, you can enable
> Timestamps.  From then on, every packet sent will need to contain a
> Timestamps option.
> 2) If you want to enable Timestamps mid-stream, you can only do so on
> a data packet.  You record the sequence number of that data packet,
> and include the Timestamps option on every packet from then on, until
> you receive an ACK for that sequence number.  If at any time you
> receive a packet with the Timestamps option, then they are enabled
> for the rest of the connection.  If you do not receive a packet with
> a Timestamps option by the time you get the ACK for that initial
> sequence number, then the other side has not enabled Timestamps and
> you must stop sending Timestamps for the duration of the connection.
>
> The restriction on only being able to initiate Timestamps on a data
> packet is that they are the only packets that are delivered reliably,
> so determining whether or not the other side supports them is
> deterministic by getting the ACK for that data.
>
>
> Option #2
> ---------
> This is similar to Option #1, but not quite the same.  In this case,
> the Timestamps would be negotiated in the initial SYN as before, but
> then no additional packets would have Timestamps options.  At any
> point during the connection, either side can enable the use of
> Timestamps just by starting to send them.  Since we already know that
> both sides support them, once you start sending them you never stop.
> And if you every receive one, you are then required to also send them
> for the duration of the connection.
>
> The question then is, how do you negotiate Timestamps without turning
> them on?  My suggestion would be to send a Timestamps option with the
> special format TSval=0,TSecr=0.  If you receive this in a SYN, then
> the request is to negotiate knowledge of the Timestamps option.  The
> returning SYN,ACK would also need to contain TSval=0,TSecr=0 to
> complete the negotiation.  If you are connecting to a site that
> doesn't support this feature but does support Timestamps, it would
> respond with TSval=xxx,TSecr=0.  At that point, Timestamps are now
> enabled.  The downside of this is that now the originator has started
> their Timestamps values at 0, and must continue from there with their
> timestamps; they can't just suddenly jump to some arbitrary value
> because that could break PAWs.
>
>
> Option #3
> ---------
> Do nothing, and leave Timestamps alone as they are currently
> defined.  You negotiate them in the SYN, and then include them in
> every packet from then on.
>
> One of the issues with Timestamps is that the TSecr value is defined
> to be valid anytime the ACK bit is set.  This works well with the
> initial SYN negotiation, but when you enable Timestamps midstream,
> the initiator has to send a Timestamps option with an invalid TSecr
> value, but the ACK bit will be set.  That's the case for both of the
> previous options.
>
> So why should we change things?  My recollection was that the primary
> motivation was to conserve option space, to avoid having to put in
> the extra 12 bytes of TCP options on every packet until we get to the
> point where the sequence space might wrap, and then turn them on so
> that we can get PAWS to protect the rest of the connection.  But I'm
> not sure I buy that argument.  One of the things that you lose by
> deferring enabling Timestamps is that you don't get the RTT
> measurements that Timestamps provide.  Without Timestamps, using the
> typical BSD algorithm of timing one segment and waiting for an ACK,
> and adding in the Karn algorithm that you have to invalidate any
> measurements if you had to retransmit in that window, means that at
> best, you'll get one measurement per RTT.  At worst, you can get in a
> state where you never get a valid RTT measurement.  When you use the
> Timestamps option, the worst case is that you'll only get one
> measurement per RTT, and the best case is that you'll get multiple
> valid samples,
>
> My viewpoint
> ------------
> Right now, I'm inclined to leave the Timestamps option alone; you
> negotiate it during the SYN exchange, and then include it on every
> packet.  I'm willing to be convinced otherwise, but now I've placed a
> stake in the ground, and the mailing list will have to decide if
> there is sufficient reason to change the behavior.
>
>                         -David Borman
>
>

_______________________________________________
tcpm mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tcpm

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 May 2007 03:40:40 -0700

Hi, David,

I had some questions below...

David Borman wrote:
> Timestamps mid-stream, and I'm coming to a clearer picture.  I'm now now
> leaning towards changing to allow this to happen in a standard way.  I
> think I can now more clearly enumerate the specific issues about
> enumerating them mid-stream.
> 
> First, do you or do you not include Timestamps in the SYN exchange, and
> how much do you care about interoperability with existing 1323
> implementations?  If you do not include Timestamps negotiation, then I
> would expect that mid-stream enabling of Timestamps would only work on
> modified hosts that understand the new rules.
> 
> connection will continue without Timestamps until one of them initiates
> Timestamps by including it in a packet.  Once Timestamps have been sent
> or received after the initial SYN,ACK, they are then sent in all packets
> Timestamps.  When talking to an RFC-1323 implementation, since the other
> side will include Timestamps in all packets, the option will be enabled
> right away

How is reordering handled? I.e., all new packets may be stamped, but
reordering could affect how that's seen at the receiver. (see below...)

...
> Timestamps), you fill in the exact same value for TSval in every packet
> that you send, until you receive a packet with a TSecr equal to that
> value.  At that point you know that Timestamps are now enabled on both
> sides, and you start sending new TSval values.  You don't do any RTT
> calculations until you get a packet that acknowledges new data, and has
> a TSecr value that is beyond the initial TSval used.  This should
> address all three scenarios: the data receiver initiating enabling
> Timestamps, the data sender initiating enabling Timestamps, and both the
> sender and receiver enabling Timestamps at the same time.  PAWS will
> work just fine, because it doesn't require the clock to tick for each
> packet, just at least once for each sequence space.

The problem with enabling options mid-stream is that you haven't cleared
the packets from the system; this is not the case during SYN exchange.

I'm wondering whether there's a case where you could have started
sending new TSvals, and you get a new data ACK (and start calculating
RTTs), and then a packet comes in out of order (e.g., duplicated or
SACK). Could this corrupt the RTT? Could this ever happen?

> I think it should be "a".

This is asking whether deferred timestamps disables legacy 1323 use,
IMO, and I agree that the only conclusion is (a).

Joe

-- 
----------------------------------------
Joe Touch
Sr. Network Engineer, USAF TSAT Space Segment

---

From: David Borman <[email-masked]>
Date: Wed, 30 May 2007 09:16:18 -0500

Hi Joe,

On May 30, 2007, at 5:40 AM, Joe Touch wrote:

>> initiates
>> other

The receiver won't have fully turned on timestamp support until it  
has sent a TSval and received a TSecr with the same value.  The only  
issue is if you turn on full PAWS support when you receive the first  
packet with a timestamp option, and then start dropping any packets  
that don't contain a timestamp option.  So, you don't turn on PAWS  
until you've gotten a TSecr with your initial TSval.  And even if you  
should happen to drop reordered packets without the timestamp option,  
TCP retransmissions should take care of any dropped data.

>> packet

That why you keep sending the the same timestamp until you see it in  
at least one TSecr, and you don't use that timestamp for any RTT  
calculations.
> SACK). Could this corrupt the RTT? Could this ever happen?

Besides being an ACK for new data, it also has to have a TSecr beyond  
the initial timestamp you sent.  By the time both of those things are  
true, you should be into normal RFC-1323 processing.

---

From: Matt Mathis <[email-masked]>
Date: Wed, 30 May 2007 11:59:45 -0400 (EDT)

I had assumed that you can "fake" paws as follows: once you receive a late
timestamp negotiation (in either role), you treat any later untimestamped
segments as though they bear the timstamp just before the earliest time stamp
seen.   I believe that this is optimal, given other assumptions.

I think it also works well enough to treat any later untimestamped
segments as though they bear the timestamp just before the the most recently
received timestamp.  This is actually the same if the timestamps do not
advance during the negotiation RTT.   (Well enough means that the failures
are neither frequent nor serious).

Thanks,
--MM--
-------------------------------------------
Matt Mathis      http://www.psc.edu/~mathis
Work:412.268.3319    Home/Cell:412.654.7529
-------------------------------------------
Evil is defined by mortals who think they know
"The Truth" and use force to apply it to others.

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 May 2007 10:41:38 -0700

David Borman wrote:
...
> true, you should be into normal RFC-1323 processing.

So two checks for every incoming packet, not just during 'sync', right?
(just checking; that's not obvious to me from the description)

Thanks, BTW,

Joe

---

From: David Borman <[email-masked]>
Date: Wed, 30 May 2007 15:56:19 -0500

Hi Joe,

> (just checking; that's not obvious to me from the description)

Good question.  That'll be part of coming up with a good description.

The blind way of doing this is to just believe the TSecr value  
whenever it arrives in a packet that ACKs new data.  You'll get some  
bad samples during the transition when Timestamps are being enabled,  
but they should be averaged out after a little bit.  That might work  
ok most of the time, but that's not a good way to design a protocol;  
I'm looking for a deterministic way to enable timestamps mid-stream,  
and not get erroneous RTT measurements.

One way to simplify things is to restrict enabling timestamp mid- 
stream to data packets only, and go ahead and always put in the  
correct TSval value.  That means that the receiver in a one-way data  
transfer can't start sending timestamps.  This solves the problem of  
having to validate the TSecr values for RTT measurement for one-way  
transfers, but it still leaves ambiguity in the case of a bi- 
directional transfer, if both sides decide to enable Timestamps at  
the same time.  In that case, you can still construct scenarios where  
you get ACK only packets that acknowledge new data, and don't have a  
valid TSecr value.  So, you have the same problem, how to identify  
whether or not the TSecr is valid for RTT measurement.  Hence, I  
didn't propose restricting starting timestamps to data-only packets,  
and came up with an alternate way to identify invalid TSecr values.

But even with the TSecr validation, I'm sure there are clever ways of  
doing implementations to avoid doing extra checks once Timestamps are  
started.  I can some up with a few ideas myself. :-)

If anyone has a better idea of how to identify the invalid TSecr  
values, I'm open to all suggestions.

			-David

---

From: David Borman <[email-masked]>
Date: Wed, 30 May 2007 16:13:38 -0500

Hi Matt,

> seen.   I believe that this is optimal, given other assumptions.

Yes, for PAWS you only need the clock to tick at least once per  
sequence wrap, so as long as Timestamps are enabled before the first  
wrap, PAWS should work just fine this way.

> are neither frequent nor serious).

The problem is, how do you tell that it is a "later untimestamped"  
segment, and not an old segment from before we enabled timestamps?   
This would allow old packets from before we enabled timestamps to  
arrive after the sequence wrap and be treated as OK, the very thing  
that PAWS is trying to prevent.  That's one of the reasons why once  
you enable timestamps, you have to then include them in every packet  
for the duration of the connection.

			-David Borman

---

From: Matt Mathis <[email-masked]>
Date: Wed, 30 May 2007 18:23:16 -0400 (EDT)

You are being more conservative than necessary:   If I start out w/o
timestamps, then I am assuming that I will be at low rate (or perhaps a short
flow), and I don't need PAWS.  I just need to be sure that PAWS is fully
enabled before I would send the same sequence number within one MSL.

As long as I meet this criteria, all prior segments are protected by
either the MSL or as the "minus first" clock tick.

Which raises another point, if I start without timestamps and just a few
seconds later I am already 25% through the sequence space (think 10 Gigabit
Ethernet), and I then try to turn on timestamps and the other end balks, what
do I do?

One answer would be to force the window down to RTT*4294967296/MSL, but people
might think that is just a bit draconian.  Furthermore, this solution has the
nasty marketing problem that a correct highspeed stack is vastly slower than
an incorrect implementation in some situations. Other suggestions?  Ignore the
problem?

What happens today if you fail to request timestamps, and the path turns out
to be a much faster than 100 Mb/s?

Although permitting "late timestamps" has been on my todo list for years, I
have come to think that a safer solution is to unconditionally request
timestamps as long as the route of choice does not point to a directly
connected compressing modem on a voice grade line.  There are now several
secondary control algorithms (such as the receiver side autotuning in Linux)
that require timestamps for optimal operation, and thus provide motivation for
(almost) unconditionally turning them on.

Thanks,
--MM--

---

From: Joe Touch <[email-masked]>
Date: Thu, 31 May 2007 12:58:17 -0700

Matt Mathis wrote:
...
> Although permitting "late timestamps" has been on my todo list for years,

Not so much for me...
> I
> connected compressing modem on a voice grade line. 

I agree with this, though ;-)

Joe