Subject: Re: TCP Problems with Path MTU Discovery

From: Joe Touch <[email-masked]>
Date: Thu, 16 Sep 1999 10:18:42 -0700 (PDT)

> From: Geert Jan de Groot <[email-masked]>
> Subject: Re: TCP Problems with Path MTU Discovery 
> Date: Thu, 16 Sep 1999 04:39:42 +0200
...
> First, a quicky at the end of the document:
> 
> >3.3.   Determining MSS from PMTU
> ...
> >     The MSS advertised at the start of a connection should be based on
>       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >     the MTU of the interfaces on the system.  Some systems use PMTUD
>       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >     determined values to determine the MSS to advertise.
> >     This results in an advertised MSS that is smaller than the largest
> >     MTU the system can receive.
> 
> I would like to change the marked sentence with:
> The MSS advertised at the start of a connection should be based on
> the largest MTU of all interfaces on the system.
> 
> Rationale: assume a TCP connect happens over an ethernet interface.
> While the TCP session is alive, routing changes and the session is
> now routed over an FDDI interface. If mss would have been 1460 (ethernet),
> we would see micrograms on the FDDI interface unneccesary.
> I just want to make the original text more explicit, that's all.

Advertising the largest of all interfaces at the _start_ of
a connection won't help a connection that changes interfaces
while it is running.

The advertised MSS need only be as large as the MTU of
the outgoing interface during connection start.

(if there is reason to re-check or re-try larger MTUs
later, that needs to be addressed separately).

> >     TCP should notice that the connection is timing out.  After several
> >     timeouts, TCP should attempt to send smaller packets, perhaps turn-
> >     ing off the DF flag for each packet.  If this succeeds, it should
> >     continue to turn off PMTUD for the connection for some reasonable
> >     period of time, after which it should probe again to try to deter-
> >     mine if the path has changed.
> 
> I am not sure if I agree with this. First, there is a huge performance
> impact; second, there is ambiguity between packets dropped because of
> MTU problems and packets dropped because of congestion. The side effect
> of the suggestion above is that congestion might lead to sending
> micrograms unneccesary.
> Second, this requires changes in the web server config, while it's
> the _client_ path that's broken. I see no big incentive for webserver
> maintainers to install this workaround if it becomes available.

Interesting idea - that backoff includes packet size, as well
as frequency. Whether that is useful depends not on whether there
is congestion, but on what is congested.

If there is output port contention at a router, smaller packet sizes
may be a good way to go here.

If the router queue slots grab fixed chunks of memory, or if
the problem is per-packet processing (i.e., if the problem is
per-packet, not per-byte) then only time-based backoff will help.

Very interesting reseach - might be worthwhile to mention
as a possibility, but until someone specs it more precisely and
implements it, it may be premature to reccommend.

Joe

---

From: David Borman <[email-masked]>
Date: Thu, 16 Sep 1999 14:16:44 -0500 (CDT)

> From: Joe Touch <[email-masked]>
> Date: Thu, 16 Sep 1999 10:18:42 -0700 (PDT)

Sure it will.

> the outgoing interface during connection start.

You've never heard of asymetric routes?

Just because the packets go out an ethernet interface, doesn't
mean that the return traffic can't come in the FDDI interface.
When you advertise only 1500 bytes in the MSS, then return
traffic through the FDDI interface is limited to 1500 bytes.

> later, that needs to be addressed separately).

PMTUD already does rechecks for larger MTUs.

BTW, if you look at RFC 1191, pg. 6, it says:

   ....  The MSS option should be 40 octets less than the
   size of the largest datagram the host is able to reassemble (MMS_R, 
   as defined in [1]); in many cases, this will be the architectural  
   limit of 65495 (65535 - 40) octets.  A host MAY send an MSS value 
   derived from the MTU of its connected network (the maximum MTU over 
   its connected networks, for a multi-homed host); this should not
   cause problems for PMTU Discovery, and may dissuade a broken peer
   from sending enormous datagrams.

          Note: At the moment, we see no reason to send an MSS greater
          than the maximum MTU of the connected networks, and we
          recommend that hosts do not use 65495.  It is quite possible
          that some IP implementations have sign-bit bugs that would be
          tickled by unnecessary use of such a large MSS.

Note that it doesn't say anything about using the MTU of the
outgoing interface, it says use the "maximum MTU of the connected
networks".  That sounds to me like using the largest MTU across
all interfaces.

			-David Borman, [email-masked]

---

From: Joe Touch <[email-masked]>
Date: Thu, 16 Sep 1999 13:22:58 -0700 (PDT)

> From [email-masked]  Thu Sep 16 12:17:38 1999
> Delivery-Date: Thu, 16 Sep 1999 12:17:38 -0700
> Return-Path: [email-masked]
> Date: Thu, 16 Sep 1999 14:16:44 -0500 (CDT)
> From: David Borman <[email-masked]>
> To: [email-masked], [email-masked], [email-masked]
> Cc: [email-masked], [email-masked], [email-masked]
> 
...
> You've never heard of asymetric routes?

Granted. In that case, then the discussion should read:

The MSS advertised for a connection, whether at the start
or during PMTU rediscovery, should be the largest incoming
MTU of all interfaces on the system.

(add the word incoming - not only are routes asymmetric, 
so may be MTUs).

Joe

---

From: Bill Fenner <[email-masked]>
Date: Thu, 16 Sep 1999 15:02:58 -0700

>a connection won't help a connection that changes interfaces

Since the MSS can only be advertised at the start of a connection,
this is the only time to communicate this information.

>later, that needs to be addressed separately).

RFC1191 already addresses trying larger MTUs, in sections 6.3 and 7.1 .
Section 6.3 explicitly mentions using a different interface on a
multi-homed host.

  Bill