Subject: [tcpm] some questions related to PAWS

From: Yoshifumi Nishida <[email-masked]>
Date: Sun, 2 Jun 2013 07:02:07 -0700

Hi,
I start wondering the following points on PAWS.
Please let me know if I miss something.

1: Do we always need PAWS check?
    If TCP uses enough length of MSL and it hasn't sent 2**32 data
yet,  I personally don't see strong reason to drop the packet without
TS-opt.

   Also,  when TCP has sent more than 2**32 data and if it has good
knowledge that more than MSL time has passed since a packet with the
seqno has sent, do we really need to perform PAWS check?

2: Do we always need to enable both RTTM and PAWS when TS is negotiated?
    I'm wondering there might be cases where sender only wants RTTM or PAWS.
   Since the sender's choice doesn't affect the receiver's logic,
implementing a simple parameter at the sender might be enough for
this. Or, does it cause serious problems?

3: Action for ACK without TS-opt
    I'm guessing we don't need to send an ACK for the last in-sequence
segment when a packet without TS-opt is ACK. But, I think the draft is
not very clear on this point.

Thanks,
--
Yoshifumi

---

From: Jakob Heitz <[email-masked]>
Date: Sun, 2 Jun 2013 17:00:02 +0000

Does anyone have statistics to show how many packets PAWS has dropped?

--

Jakob Heitz.

________________________________
From: [email-masked] [[email-masked]] on behalf of Yoshifumi Nishida [[email-masked]]
Sent: Sunday, 02 June 2013 7:02 AM
To: [email-masked]
Subject: [tcpm] some questions related to PAWS

Hi,

Thanks,
--
Yoshifumi
_______________________________________________
tcpm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tcpm

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Sun, 2 Jun 2013 21:16:36 +0000

Hi Yoshifumi,

ad 1) - I would think that PAWS is not only about correct, cooperative implementations; it is also about protecting the session from outside interference. With WS, the window could be up to 2^30 bytes in size. So, if a segment without (proper) TSopt or no TSopt may be accepted, a malicious interferer can have a chance of up to 1 in 4 to corrupt the data stream, just knowing (guessing) the 4-tuple. Requiring that acceptable packets always have an acceptable TSopt reduces this again to sane levels. (Blind in-window attack). However, as the timestamp clock is shared between sessions in may implementations, the attacker only needs to find out the TSopt of the sender, to bypass this. There is some small text in the examples, asking for a TSoffset that is destination-specific (only shared among sessions towards the same destination), to avoid this issue. But it's not mentioned very prominent either.

Ad 2) PAWS is receiver-only, while RTTM is sender side; A TSopt enabled sender has currently (1323 / 1323bis) no way of signaling the receiver, that the TSopt should NOT be used for PAWS (because, for example, it may sporadically stop sending TSopt). The sender has to assume, that a receiver will always enable PAWS (even if it doesn't) once TSopt was negotiated.

Best regards,

Richard Scheffenegger

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Yoshifumi Nishida
> Sent: Sonntag, 02. Juni 2013 16:02
> To: [email-masked]
>     If TCP uses enough length of MSL and it hasn't sent 2**32 data yet,  I
> personally don't see strong reason to drop the packet without TS-opt.
> 
> knowledge that more than MSL time has passed since a packet with the seqno
> has sent, do we really need to perform PAWS check?
> 
> implementing a simple parameter at the sender might be enough for this.
> Or, does it cause serious problems?
> 
> segment when a packet without TS-opt is ACK. But, I think the draft is not
> very clear on this point.
> 
> Thanks,
> --
> Yoshifumi

---

From: Tim Shepard <[email-masked]>
Date: Sun, 02 Jun 2013 18:20:54 -0400

> Ad 2) PAWS is receiver-only, while RTTM is sender side; A TSopt
> enabled sender has currently (1323 / 1323bis) no way of signaling
> the receiver, that the TSopt should NOT be used for PAWS (because,
> for example, it may sporadically stop sending TSopt).

I don't believe sporadically stop sending TSopt is ever allowed (by
either the current spec, or future spec).

Also another thought... PAWS protects not only TCP connections which
send more than 2^32 bytes in one TCP connection, but also protects
against the case where the initial sequence number selection wraps the
sequence space and multiple TCP connections use the same IP addresses
and port numbers over time.  You don't have to be sending a lot of
data to need PAWS.

To not need PAWS, you have to know there has not been a previous TCP
connection using the same port numbers between the same IP address
that might still have some packets stuck somewhere in the net.

As for whether PAWS is doing any good...    a linux machine of mine at
home from which I do many things but probably mostly web browsing
(using both a mozilla-based web browser and a chromium-based browser)
that has been up 147 days shows this:

$ netstat -s | egrep -e 'segments received|timestamp'
    367985020 segments received
    0 bad segments received.
    1375 packets rejects in established connections because of timestamp

I was a little surprised by how high that timestamp rejects number is,
but not very surprised that number is non-zero.

Wondering if this was an unusual quirk of my situation, I just checked
on the group git/track/web server of a project and it has been up 67
days and shows this:

$ netstat -s | egrep -e 'segments received|timestamp'
    751477138 segments received
    326 bad segments received.
    49130 packets rejects in established connections because of timestamp

Anyone have any idea why these numbers are as high as they are?

Even my Linux laptop with an uptime of 15 days shows this:

$ netstat -s | egrep -e 'segments received|timestamp'
    2083891 segments received
    352 bad segments received.
    54 packets rejects in established connections because of timestamp

I haven't audited the linux code to verify what exactly these counters
are counting, but I assume the "bad segments received" is due to TCP
checksum errors, and the "rejects in established connections because of
timestamp" is due to PAWS.

I am surprised that in the first two cases above rejects because of
timestamp were significantly more common than "bad segments received".

I went looking for those numbers hoping that non-zero counts there
would help illustrate that needing timestamp option turned on to
provide PAWS was not merely theoretical.  But the counts I found are
so high that now I wonder what is really going on.

What counts do you see on your machines?

			-Tim Shepard
			 [email-masked]

---

From: Rick Jones <[email-masked]>
Date: Sun, 02 Jun 2013 15:51:06 -0700

On Jun 2, 2013, at 3:20 PM, Tim Shepard <[email-masked]> wrote:

> netstat -s | egrep -e 'segments received|timestamp'

for netperf.org:

$ netstat -s | egrep -e 'segments received|timestamp'
        timestamp request: 1
        timestamp replies: 1
    2304112 segments received
    47 bad segments received.
    2333 packets rejects in established connections because of timestamp

uptime of 16 days,  5:18

rick jones

---

From: Jakob Heitz <[email-masked]>
Date: Mon, 3 Jun 2013 00:14:16 +0000

I believe the statistic

 "packets rejects in established connections because of timestamp'

is LINUX_MIB_PAWSESTABREJECTED.

This will increment if the received segment is disordered, but !tcp_disordered_ack.

This is apparently quite common, but not a PAWS error: would be caught in sequence number check.

Here is the code from tcp_input.c

static inline int tcp_paws_check(const struct tcp_options_received *rx_opt,
     int paws_win)
{
 if ((s32)(rx_opt->ts_recent - rx_opt->rcv_tsval) <= paws_win)
  return 1;
 if (unlikely(get_seconds() >= rx_opt->ts_recent_stamp + TCP_PAWS_24DAYS))
  return 1;
 /*
  * Some OSes send SYN and SYNACK messages with tsval=0 tsecr=0,
  * then following tcp messages have valid values. Ignore 0 value,
  * or else 'negative' tsval might forbid us to accept their packets.
  */
 if (!rx_opt->ts_recent)
  return 1;
 return 0;
}

...

/* Sorry, PAWS as specified is broken wrt. pure-ACKs -DaveM
 *
 * It is not fatal. If this ACK does _not_ change critical state (seqs, window)
 * it can pass through stack. So, the following predicate verifies that
 * this segment is not used for anything but congestion avoidance or
 * fast retransmit. Moreover, we even are able to eliminate most of such
 * second order effects, if we apply some small "replay" window (~RTO)
 * to timestamp space.
 *
 * All these measures still do not guarantee that we reject wrapped ACKs
 * on networks with high bandwidth, when sequence space is recycled fastly,
 * but it guarantees that such events will be very rare and do not affect
 * connection seriously. This doesn't look nice, but alas, PAWS is really
 * buggy extension.
 *
 * [ Later note. Even worse! It is buggy for segments _with_ data. RFC
 * states that events when retransmit arrives after original data are rare.
 * It is a blatant lie. VJ forgot about fast retransmit! 8)8) It is
 * the biggest problem on large power networks even with minor reordering.
 * OK, let's give it small replay window. If peer clock is even 1hz, it is safe
 * up to bandwidth of 18Gigabit/sec. 8) ]
 */

static int tcp_disordered_ack(const struct sock *sk, const struct sk_buff *skb)
{
 struct tcp_sock *tp = tcp_sk(sk);
 struct tcphdr *th = tcp_hdr(skb);
 u32 seq = TCP_SKB_CB(skb)->seq;
 u32 ack = TCP_SKB_CB(skb)->ack_seq;

 return (/* 1. Pure ACK with correct sequence number. */
  (th->ack && seq == TCP_SKB_CB(skb)->end_seq && seq == tp->rcv_nxt) &&

  /* 2. ... and duplicate ACK. */
  ack == tp->snd_una &&

  /* 3. ... and does not update window. */
  !tcp_may_update_window(tp, ack, seq, ntohs(th->window) << tp->rx_opt.snd_wscale) &&

  /* 4. ... and sits in replay window. */
  (s32)(tp->rx_opt.ts_recent - tp->rx_opt.rcv_tsval) <= (inet_csk(sk)->icsk_rto * 1024) / HZ);
}

static inline int tcp_paws_discard(const struct sock *sk,
       const struct sk_buff *skb)
{
 const struct tcp_sock *tp = tcp_sk(sk);

 return !tcp_paws_check(&tp->rx_opt, TCP_PAWS_WINDOW) &&
        !tcp_disordered_ack(sk, skb);
}

...

static int tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
         struct tcphdr *th, int syn_inerr)
{
 u8 *hash_location;
 struct tcp_sock *tp = tcp_sk(sk);

 /* RFC1323: H1. Apply PAWS check first. */
 if (tcp_fast_parse_options(skb, th, tp, &hash_location) &&
     tp->rx_opt.saw_tstamp &&
     tcp_paws_discard(sk, skb)) {
  if (!th->rst) {
   NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_PAWSESTABREJECTED);
   tcp_send_dupack(sk, skb);
   goto discard;
  }
  /* Reset is accepted even if it did not pass PAWS. */
 }

 /* Step 1: check sequence number */
 if (!tcp_sequence(tp, TCP_SKB_CB(skb)->seq, TCP_SKB_CB(skb)->end_seq)) {
...

--

Jakob Heitz.

________________________________
From: [email-masked] [[email-masked]] on behalf of Tim Shepard [[email-masked]]
Sent: Sunday, 02 June 2013 3:20 PM
To: Scheffenegger, Richard
Cc: [email-masked]
Subject: Re: [tcpm] some questions related to PAWS

                        -Tim Shepard
                         [email-masked]

---

From: Joe Touch <[email-masked]>
Date: Mon, 03 Jun 2013 10:29:46 -0700

> seqno has sent, do we really need to perform PAWS check?

The trouble is that we don't have a way to say "use TSopt to measure 
RTT" vs. "use TSopt for PAWS".

Right now, AFAICT it means "use TSopt for PAWS and to measure RTT".

However, sending more than 4GB isn't as absurd or exceptional as it used 
to be, and lots of devices do lots of things with TCP packets that keep 
them around a while. It'd be nice if "MSL" was relevant here, but 
without TSopt, AFAICT MSL isn't involved except voluntarily at the 
source; if a source wants to send more than 4GB in 2 minutes (a transfer 
rate over 267Mbps, which is easy even on a laptop), then it can, will, 
and *does*.

> this. Or, does it cause serious problems?

I'm not sure how to do that (see above).

> not very clear on this point.

It should be more clear; IMO, all segments in a TSopt-negotiated session 
without TSopt should be silently ignored, but that's me ;-)

Joe

---

From: Yoshifumi Nishida <[email-masked]>
Date: Mon, 3 Jun 2013 22:35:29 -0700

Hi Richard,

On Sun, Jun 2, 2013 at 2:16 PM, Scheffenegger, Richard <[email-masked]>
wrote:

> ad 1) - I would think that PAWS is not only about correct, cooperative
implementations; it is also about protecting the session from outside
interference. With WS, the window could be up to 2^30 bytes in size. So, if
a segment without (proper) TSopt or no TSopt may be accepted, a malicious
interferer can have a chance of up to 1 in 4 to corrupt the data stream,
just knowing (guessing) the 4-tuple. Requiring that acceptable packets
always have an acceptable TSopt reduces this again to sane levels. (Blind
in-window attack). However, as the timestamp clock is shared between
sessions in may implementations, the attacker only needs to find out the
TSopt of the sender, to bypass this. There is some small text in the
examples, asking for a TSoffset that is destination-specific (only shared
among sessions towards the same destination), to avoid this issue. But it's
not mentioned very prominent either.

I agree with that it can be used to protect the session from outside
interference to some extents. But, I'm still wondering how it is effective.
In my understanding, PAWS is checking if SEG.TSval < TS.Recent. Is it
enough to protect from malicious users?

>
> Ad 2) PAWS is receiver-only, while RTTM is sender side; A TSopt enabled
sender has currently (1323 / 1323bis) no way of signaling the receiver,
that the TSopt should NOT be used for PAWS (because, for example, it may
sporadically stop sending TSopt). The sender has to assume, that a receiver
will always enable PAWS (even if it doesn't) once TSopt was negotiated.

Sorry. As you mentioned, PAWS is receiver logic. What I meant was disabling
PAWS check.
Something like, "I want to have TS for RTTM, but I don't want to drop
packets without TS unless the sender sends data 4GB in less than 2MSL". In
this case, the node needs to measure receiving rate to check if PAWS check
is required.
It might be tricky, but I still think it's one possibility. one might
prefer it rather than dropping packets without TS blindly.

Thanks,
--
Yoshifumi

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 4 Jun 2013 12:05:06 +0000

Hi Yoshifumi,

> I agree with that it can be used to protect the session 
> from outside interference to some extents. But, I'm still 
> wondering how it is effective.
>
> In my understanding, PAWS is checking if SEG.TSval < 
> TS.Recent. Is it enough to protect from malicious users?

It at least increases the bar for a successful data injection (corruption), while Window Scale lowers the bar. Provided that the sender is using different timestamp clock offsets to different clients (destination IP addresses), of course, if this is to be seen as a security mechanism (deliberate data corruption)

But in the current implementations, by allowing a receiver to simply accept non-TSopt data packets (which are otherwise in window), data corruption or malicious interference can happen much more easily.

>>
>> Ad 2) PAWS is receiver-only, while RTTM is sender side; A 
>> TSopt enabled sender has currently (1323 / 1323bis) no way 
>> of signaling the receiver, that the TSopt should NOT be 
>> used for PAWS (because, for example, it may sporadically 
>> stop sending TSopt). The sender has to assume, that a 
>> receiver will always enable PAWS (even if it doesn't) once
>> TSopt was negotiated.
>
> Sorry. As you mentioned, PAWS is receiver logic. What 
> I meant was disabling PAWS check. Something like, "I want 
> to have TS for RTTM, but I don't want to drop packets
> without TS unless the sender sends data 4GB in less than
> 2MSL". In this case, the node needs to measure receiving
> rate to check if PAWS check is required. 
>
> It might be tricky, but I still think it's one possibility.
> one might prefer it rather than dropping packets without
> TS blindly.

Probably. The sender MUST still provide a TSopt in every segment, assuming that the receiver is strictly enforcing PAWS. If a receiver chooses to open a broader window for misuse (as apparently two dominant stacks currently do), that's their business. Specifying a resiliency mechanism and then stipulating it is completely optional to use it, is not the best approach for an RFC IMHO. That's an implementers choice.

I need to find some time to summarize the discussion around this topic that the WG had in the last few days, and pour it into some text for the 1323bis draft. (I'm always happy if someone want's to donate text..:)

Best regards,
   Richard

---

From: Pasi Sarolahti <[email-masked]>
Date: Tue, 4 Jun 2013 19:27:30 +0300

> Probably. The sender MUST still provide a TSopt in every segment, assuming that the receiver is strictly enforcing PAWS. If a receiver chooses to open a broader window for misuse (as apparently two…

Right. But we should also remember PAWS failure should be very rare case, where two uncommon events coincide:
* there is a old segment with wrapped sequence number that happens to hit the window
* the segment has timestamps option removed for some reason

> I need to find some time to summarize the discussion around this topic that the WG had in the last few days, and pour it into some text for the 1323bis draft. (I'm always happy if someone want's to…

I don't think there has been much disagreement about the fact that to ensure reliable operation of PAWS, all segments must have timestamps. I tend to agree with you that standardizing an algorithm that works unreliably is not elegant, and majority of mailing list feedback seems to support this, so this seems clear.

The trickier thing is, what language to add about the current state, but I think we should point out what we know: there are implementations that accept segments without timestamp after its successful negotiation (in fact no one so far has reported implementation that would discard segments without timestamps), and changing to follow the "MUST discard" rule may bring up some communication problems as a result of faulty network behavior (and there was some reported evidence of this actually happening). I'm not suggesting this as a text to be used, but hopefully it is an acceptable summary to everyone.

Not to be said in the document, but just wanted to say it out: even if segments without timestamps were accepted, PAWS could still be able to detect most cases of sequence wrapping, assuming that timestamps removal is a rare case. If user runs into communication problems that can be solved by turning timestamps off entirely, the natural reaction is to do so. After that there is no PAWS protection at all.

- Pasi