Subject: Re: TCP Problems with Path MTU Discovery

From: Vernon Schryver <[email-masked]>
Date: Tue, 14 Sep 1999 08:15:58 -0600 (MDT)

> From: Mark Allman <[email-masked]>

> Vern and I believe the pmtud I-D is nearing the point where we can
> issue a last call and boot it up to the IESG.  However, there has
> been very little discussion recently on the draft.  So, we'd like to
> encourage folks to take a look at the draft and send any comments
> you might have.  Unless there is a large amount of discussion we'll
> probably issue the last call very soon. ...

I have a nit you might wisely ignore.  Some versions of `ping` can also
set the DF bit to probe for broken routers that do not generate the ICMP
message or that filter ICMP messages.  The `ping` in IRIX is one.  I think
the `ping` in NetBSD is another.  Perhaps someday the `ping` in BSD/OS
will be still another.   In other words, I've been flogging the hacks in
ftp.rhyolite.com:src/ping.tar.Z.  I suggest you *not* mention my vanity
domain.

There is a specific case of the first problem in the draft that needs to
be explicitly mentioned, because idiots are too stupid to understand the
implications of their brain-dead actions.  A significant number of people
who consider themselves expert administrators configure their routers to
discard all ICMP packets because ICMP packets are a security risk.  Even
after you point out the effects such nonsense has on systems that ahve
PMTUD, turned on by default, they don't or refuse to understand.  This
RFC seems like a good place to discuss the worst case security risks of
the Fragmentation Message ICMP message (reduced TCP throughput), the likely
results of discarding them ("Non-interoperation -- connectivity failure"),
and a better way to deal with the minor security problem (hosts should
refuse to shrink the MTU to less than a reasonable limit, such as 500).

This ICMP-is-a-security-risk nonsense predates the "smurf attack."  Years
ago I encountered an "internet security expert" who had "fixed" a
commercial version of Gauntlet to discard all ICMP because of the dangers
of Redirects received by typical firewalls, (then) singled homed and behind
(potentially filtering) routers.

There is another activity that is related to discarding ICMP packets, but
not exactly stupid.  Some major retail ISP's use something like NAT in
their routers to redirect all client SMTP traffic from their end-user
customers to their own SMTP servers that then filter for spam and
potentially other things.  Since they redirect only some packets based on
TCP port number, I doubt they do the right thing if there happens to be
a narrow pipe in the path, such as the ever popular misconconfigured PPP
link with an MTU of 500.   (That people insist on spending money on
"tuning" products that misconfigure Windows 95/98 PPP to use an MTU of
500 based on rumors of ancient computations of the effects of 2400 bit/sec
modems is another sad tale of woe that ought to be mentioned somewhere.)

Also, the document should mention the effects of the many NAT boxes in
the Intenret today on PMTUD.   Those that manage to forwad the ICMP 
messages are not a problem, but judging from the many complaints about
`ping` not working, some do not.

Yes, I realize that three of these are merely specific cases of the first
explicit problem in the draft.  Still, the median willingness and ability
to understand protocols and the effects of "improvements" is so low that
without explicit words in RFCs, most of the experts Won't Get It.

Vernon Schryver    [email-masked]

---

From: Mike Fisk <[email-masked]>
Date: Tue, 14 Sep 1999 14:11:43 -0600 (MDT)

> ftp.rhyolite.com:src/ping.tar.Z.  I suggest you *not* mention my vanity

As long as we're mentioning tools....

The LBL traceroute also has an option (-F) to set the DF bit.  When used
with small and large packets, this can be used to find PMTUD blackholes.

First, there is a bug in the code for little endian machines that prevents
that bit from actually being set.  The second attachment to this message
is the patch for that.

Second, when it receives an ICMP Need Fragmentation reponse, it doesn't
report what MTU the router will accept.  As an aid for traversing networks
with various MTUs, I also made a patch that not only prints that value,
but will then continue the trace with the smaller MTU. That patch is the
first attachment to this message.  

I sent these patches back to LBL in May, but never got a response.  I
don't know if anyone is listening to [email-masked] anymore.

Here's a sample of the output:

$ ./traceroute -F www.unm.edu 4352
traceroute to musca.unm.edu (129.24.57.2), 30 hops max, 4352 byte packets
 1  lanl-gw.lanl.gov (192.16.1.1)  1.311 ms  1.166 ms  1.262 ms
 2  esnet-rt1.lanl.gov (192.16.1.241)  2.003 ms  1.926 ms  2.029 ms
 3  lanl-technet.lanl.gov (192.16.1.246)  3.176 ms  8.990 ms  2.981 ms
 4  lanl-technet.lanl.gov (192.16.1.246)  3.098 ms !F<=1479 Resending with 1479 byte packets:
    206.206.125.17 (206.206.125.17)  90.907 ms  69.811 ms
 5  rtn.nm.org (129.121.1.1)  81.085 ms  48.157 ms  90.253 ms
 6  198.59.130.49 (198.59.130.49)  94.900 ms  103.146 ms  95.326 ms
 7  198.83.5.2 (198.83.5.2)  73.320 ms  73.084 ms  73.236 ms
 8  198.83.226.33 (198.83.226.33)  124.882 ms  74.908 ms  74.436 ms
 9  musca.unm.edu (129.24.57.2)  76.494 ms  75.609 ms  81.826 ms

=====================================================================
Mike Fisk                   | (505)667-5119 | MS B255
Network Engineering (CIC-5) |               | Los Alamos National Lab
[email-masked]              | FAX: 665-7793 | Los Alamos, NM  87545
diff -U3 -r traceroute-1.4a5.orig/traceroute.c traceroute-1.4a5/traceroute.c
--- traceroute-1.4a5.orig/traceroute.c	Mon May 10 12:41:39 1999
+++ traceroute-1.4a5/traceroute.c	Mon May 10 12:41:01 1999
@@ -325,6 +325,8 @@
 void	tvsub(struct timeval *, struct timeval *);
 __dead	void usage(void);
 int	wait_for_reply(int, struct sockaddr_in *, struct timeval *);
+void	adjustPMTU(const u_char *);
+
 
 int
 main(int argc, char **argv)
@@ -796,8 +798,8 @@
 					break;
 
 				case ICMP_UNREACH_NEEDFRAG:
+					adjustPMTU(packet);
 					++unreachable;
-					Printf(" !F");
 					break;
 
 				case ICMP_UNREACH_SRCFAIL:
@@ -1306,3 +1308,33 @@
 	    prog);
 	exit(1);
 }
+
+void 
+adjustPMTU (const u_char * packet) 
+{
+	struct ip * ip = (struct ip*)packet;
+	struct icmp* icp = (struct icmp*)(packet + (ip->ip_hl << 2));
+	int mtu = (ntohl(icp->icmp_void) & 0xffff);
+
+	Printf(" !F<=%d ", mtu);
+	if (mtu > packlen) {	
+		Fprintf(stderr, "\nPacket already smaller than %d!\n", mtu);
+		exit(1);
+	}
+	if (mtu < minpacket) {
+		Fprintf(stderr, "\nRequested MTU too small!\n");
+		exit(1);
+	}
+				
+	packlen = mtu;
+	Printf("Resending with %d byte packets:\n   ", packlen);
+#ifdef BYTESWAP_IP_LEN
+	outip->ip_len = htons(packlen);
+#else
+	outip->ip_len = packlen;
+#endif
+	if (!useicmp) 
+	      outudp->uh_ulen = htons((u_short)(packlen - (sizeof(*outip) + optlen)));
+}
+
+
diff -r -U5 traceroute-1.4a5.orig/traceroute.c traceroute-1.4a5/traceroute.c
--- traceroute-1.4a5.orig/traceroute.c	Fri Jun 13 03:30:27 1997
+++ traceroute-1.4a5/traceroute.c	Mon May 10 12:31:18 1999
@@ -504,11 +506,11 @@
 #ifdef BYTESWAP_IP_LEN
 	outip->ip_len = htons(packlen);
 #else
 	outip->ip_len = packlen;
 #endif
-	outip->ip_off = off;
+	outip->ip_off = htons(off);
 	outp = (u_char *)(outip + 1);
 #ifdef HAVE_RAW_OPTIONS
 	if (lsrr > 0) {
 		register u_char *optlist;