Subject: [tcpm] Does this fix sufficiently de-ossify overstrict ECN negotiation by Linux TCP servers?

From: Bob Briscoe <[email-masked]>
Date: Sun, 20 Jan 2019 23:31:20 +0000

TCPM folks,

RFC8311 and draft-ietf-tcpm-generalized-ecn enable ECN-capable SYNs. 
However, if a client sends an ECN-capable SYN, existing Linux servers 
disable ECN for the rest of the connection.

To fix as many existing Linux servers as possible, as soon as possible, 
I would recommend short-term solution #2 (see 'Three Fixes' below). An 
alternative would be short-term solution #3.

Anyone got a better fix? Anyone disagree with recommending #2? If not, 
pls confirm your agreement.

      Summary of Pros and Cons

#2 pro: No longer ossifies non-ECN SYN
#2 pro: If a path does "unbleach" IP-ECN, protects the connection by 
disabling ECN
#2 con: A residual degree of ossification remains

#3 pro: No longer ossifies non-ECN SYN
#3 pro: No ossification at all
#3 con: If a path does "unbleach" IP-ECN, connection still attempts to 
use ECN even though the path is mangling the ECN field.

#1 is the full solution, but it should proceed in parallel as a solution 
for the future, not to fix the past.

      Background

To negotiate ECN for a connection the client sets flag bits 8-9 in the 
TCP header to 11 [RFC3168 Section 6.1.1] 
<https://tools.ietf.org/html/rfc3168#section-6.1.1>. For your 
convenience, I've included a picture of the TCP header flags at the end.

In 2012 an additional test was patched into Linux servers. It disables 
ECN for the connection unless the IP-ECN field of the SYN is also zero 
(I'll call this the "overstrict ECN test").

This was because Section 6.1.1. of RFC3168 also said "A host MUST NOT 
set ECT on SYN or SYN-ACK packets." Unfortunately, the overstrict ECN 
test ossifies this requirement into TCP.

The Linux patch was intended to protect a connection against the network 
mangling the IP-ECN field to a non-zero value. Unfortunately, it also 
disables ECN completely if a client tries to set ECT in the IP-ECN field 
of the SYN.

RFC8311 has now updated RFC3168 to allow ECT on a SYN, in conjunction 
with the ECN++ experiment. But few developers are going to set ECT on a 
SYN if it causes about 84% of servers to completely disable ECN (see 
Section 4.2.2. of the ECN++ draft 
<https://tools.ietf.org/html/draft-ietf-tcpm-generalized-ecn-03#section-4.2.2.2> 
for further details and the source of these numbers).

      Three Fixes

I can think of three ways to fix existing Linux servers:

 1. Deploy Accurate ECN
    <https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn> on
    existing Linux servers.
      * AccECN inherently replaces the overstrict ECN test with a pair
        of proper two-ended tests for network mangling, in which each
        end feeds back the arriving IP ECN field in its TCP feedback. In
        contrast, the overstrict ECN test is one-ended, with the server
        only assuming what the client set.
      * Altho AccECN will solve the problem, it should not be relied on
        to fix the last six years of Linux server deployment. AccECN is
        a non-trivial update so it is unlikely to get back-ported and
        auto-deployed to most existing Linux servers any time soon.
 2. Make the overstrict ECN test more specific to RFC3168, and add logging.
      * Currently, the only proposal for allowing ECT on a SYN requires
        the client to also request Accurate ECN feedback, which requires
        TCP header bits 7-9 to be 111.
      * Therefore, the overstrict ECN test in a Linux server could be
        altered to solely apply if bits 7-9 == 011 (rather than just
        testing if bits 8-9 == 11).
      * However, I am concerned that this ossifies around the solution
        based on ECN++ and AccECN. What if ECN++ needs to be superseded
        by some future protocol that allows ECT on a SYN when some other
        field is different (e.g. one of TCP's three remaining reserved
        flags, or a new TCP option)?
      * Unfortunately, we cannot test for the absence of a
        yet-to-be-invented TCP option. Nonetheless, I suggest *the
        overstrict ECN test in a Linux server should at least be altered
        to solely apply if TCP header bits 4-9 == 000011*.
      * I suggest Linux also *logs whenever it disables ECN due to this
        test*, so that broken paths can be identified and fixed.
      * An extremely simple patch like this would be much more likely
        than #1 (AccECN) to be back-ported and deployed to most existing
        servers via regular Linux auto-updates.
 3. As #2, but do not disable ECN when the test fails, just log those
    paths suspected of mangling ECN.
      * Our most recent measurements of millions of Internet paths found
        no occurrences where zero IP-ECN on a SYN was changed to
        non-zero ("unbleaching"). So we could just remove the test.
      * However, no occurrences found does not mean none exist. Altho we
        tested millions of paths, we only tested from a few dozen
        vantage points, albeit we did test both mobile and fixed
        networks (see Section 4.2.2. of the ECN++ draft
        <https://tools.ietf.org/html/draft-ietf-tcpm-generalized-ecn-03#section-4.2.2.2>
        for summary of tests).
      * If the number of problematic paths is small, it would still make
        sense to run the ECN test as proposed under item #2, but not
        disable ECN if it fails. Instead, just log any negative results,
        so the broken network elements can be identified and fixed.

Cheers

Bob

PS. For convenience, here's the TCP header flags (assuming the 
experimental AccECN protocol is approved as RFC):

        0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
      +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
      |               |           | A | C | E | U | A | P | R | S | F |
      | Header Length | Reserved  | E | W | C | R | C | S | S | Y | I |
      |               |           |   | R | E | G | K | H | T | N | N |
      +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/