Subject: [tcpm] draft-minshall-nagle

From: Wesley Eddy <[email-masked]>
Date: Mon, 02 Jun 2014 21:58:36 -0400

I was working on an issue with Nagle in a space system, and
noticed that the Linux kernel actually implements the "Minshall"
version of the Nagle algorithm, described nicely in:
http://tools.ietf.org/html/draft-minshall-nagle-01

There are a couple later papers I was able to find also that
describe this.

To me, this looks like clearly a good thing that should be
recommended in the RFCs where we currently just have discussion
of "standard Nagle".  Apparently it was adopted in Linux, but I
wonder if it was picked up or not in other OSes as well.

Does anyone have history on why this draft never progressed?
Maybe someone has current contact information for Greg Minshall?

-- 
Wes Eddy
MTI Systems

---

From: "Eggert, Lars" <[email-masked]>
Date: Tue, 3 Jun 2014 06:05:19 +0000

CC'ing Greg at one address that used to work for him.

On 2014-6-3, at 3:58, Wesley Eddy <[email-masked]> wrote:

> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Joe Touch <[email-masked]>
Date: Tue, 03 Jun 2014 08:04:10 -0700

Hi, Wes,

Although I don't recall this and only scanned it briefly, I don't quite 
understand the problem.

I thought it was widely known that Nagle was useful for single-character 
interactive traffic, but also widely known as something that should be 
disabled for any multi-character interactive traffic - including 
multi-character encodings, HTTP, etc.

If Nagle is off - as it should be for interactive web traffic - then the 
optimizations in this draft won't have any impact.

So what kind of traffic does this actually help? Or is this an 
optimization of a path that either isn't or shouldn't be used?

Joe

---

From: Neal Cardwell <[email-masked]>
Date: Tue, 3 Jun 2014 11:15:21 -0400

On Tue, Jun 3, 2014 at 11:04 AM, Joe Touch <[email-masked]> wrote:
> So what kind of traffic does this actually help? Or is this an optimization
> of a path that either isn't or shouldn't be used?

The Minshall algorithm is a nice win for some workloads other than web traffic:

  Application performance pitfalls and TCP's Nagle algorithm
  http://dl.acm.org/citation.cfm?id=346012

neal

---

From: Joe Touch <[email-masked]>
Date: Tue, 03 Jun 2014 10:07:37 -0700

>    http://dl.acm.org/citation.cfm?id=346012

My point is that even by the time of this paper, it was well-understood 
that Nagle should not be used in these environments. The paper cites 
1999 personal communication with Jim Gettys for this, but it was 
well-known long before it was recited as an issue in this 1996 paper:
http://ccr.sigcomm.org/archive/1997/apr97/ccr-9704-heidemann.pdf

The use case cited is quite dated - network news has been largely 
abandoned. Is there a currently relevant use case for this approach, and 
given that interactive apps already should be disabling Nagle, how much 
will it help?

Joe

---

From: Wesley Eddy <[email-masked]>
Date: Tue, 03 Jun 2014 23:20:52 -0400

On 6/3/2014 1:07 PM, Joe Touch wrote:
>>> optimization

Even before that!  RFC 1122 (from 1989) says:

   Some applications (e.g., real-time display window
   updates) require that the Nagle algorithm be turned
   off, so small data segments can be streamed out at the
   maximum rate.

> given that interactive apps already should be disabling Nagle, how much

I agree that disabling Nagle for "interactive" flows should be a
recognized best current practice.  (assuming they're aiming for
a packets/segments per second rate that isn't absurd)

I think this would be a part of the "TCP Usage Guidelines"
analogue to RFC 5405 (for UDP) that Lars once thought about.

Since Nagle is on by default in many OSes, and deciding to turn
it off means a developer has to know what it is, and that it's
even there to be turned off, the Minshall tweak to Nagle does
seem to make it "more friendly" default to a diversity of
application behaviors (small writes, or odd patterns of writes),
while retaining the benefit of conserving packet count.

I've noticed some people seem to have a tendency to disable Nagle
automatically, whether or not that's really necessary, since
maybe it bit them once in the distant past, and was difficult to
debug or understand.  The Minshall flavor stands a better chance
of not indoctrinating people against Nagle this way.

That said, this is not a burning problem.

-- 
Wes Eddy
MTI Systems

---

From: [email-masked]
Date: Wed, 4 Jun 2014 09:37:16 +0100

And... In the old days programmers knew an app would nearly always send
small packets and Nagle needed to be disabled. For many modern apps that
is far less obvious - Nagle may be disabled simply because people think it
is better to do so without really understanding the requirements, or they
may simply not know what the network traffic will look like - Some TCP
connections vary their traffic patterns over time.

Gorry

---

From: Greg Minshall <[email-masked]>
Date: Tue, 03 Jun 2014 22:40:41 +0300

Lars, et al.,

i had heard that the Linux kernel implemented my proposed mods to
Nagle's algorithm, but i haven't looked at that implementation.  i don't
have any memory of any similar modifications being released in, e.g.,
*BSD.  (i did make some mods to FreeBSD, but i don't think i ever
released them.)  i also don't remember why the draft didn't progress,
though probably i dropped the ball somewhere.  hope that helps.

cheers, Greg

---

From: Joe Touch <[email-masked]>
Date: Wed, 04 Jun 2014 09:55:32 -0700

> On 6/3/2014 1:07 PM, Joe Touch wrote:
...
>     off, so small data segments can be streamed out at the

I saw that, but it's more than for displays; it's actually anything 
*interactive* where the data unit is longer than one byte (e.g., Unicode 
telnet!).

> even there to be turned off,

I didn't think O_NODELAY was all that cryptic...

> the Minshall tweak to Nagle does

Nagle is useful in a very small use case that no longer exists - it's 
rare that we have interactive traffic that we know can be split at a 
byte boundary.

> That said, this is not a burning problem.

Agreed.

Joe

---

From: Jim Gettys <[email-masked]>
Date: Wed, 4 Jun 2014 13:33:42 -0400

> *interactive* where the data unit is longer than one byte (e.g., Unicode

​The history of the reference to me is the following:

1) Bob Scheifler and I had developed early versions of the X Window System.
2) when a new BSD release came out (or it may have been a new snapshot of
the network stack, as there was the BSD vs. BBN TCP stack wars going on),
performance of rubber band tracking got dramatically worse.

We tracked it down to the addition of Nagle: and shortly thereafter,
TCP_NODELAY was born: I don't remember whether Bob coded it up, or Mike
Karels.

Specifically, the delay of sending mouse event packets to applications over
the X protocol was terrible to interactive performance.  These mouse events
were around 24 bytes at the time, IIRC (X11 the events are 32 bytes).  As
the X server and client library do various forms of buffering, having the
OS get in the way is not good.  And you *never* want to delay interactive
traffic by anything on the timescale required to merge keystrokes, which,
as I remember, was the original intent of the optimization.

Unfortunately, I believe the default (being on) is wrong.  But it's far too
late to make that change.
​

> I didn't think O_NODELAY was all that cryptic...

​And many poor developers have been bit with it being on over the decades.
They naively think that data will be transmitted when you ask the operating
system.  It surprises them badly when it does not...
                                      - Jim

​

> Nagle is useful in a very small use case that no longer exists - it's rare
> that we have interactive traffic that we know can be split at a byte
> boundary.
>
>

---

From: Joe Touch <[email-masked]>
Date: Wed, 04 Jun 2014 10:40:48 -0700

On 6/4/2014 10:33 AM, Jim Gettys wrote:
> Unfortunately, I believe the default (being on) is wrong.  But it's far
> too late to make that change.

We can try ;-)

That might be a useful place to start.

Joe

---

From: David Borman <[email-masked]>
Date: Wed, 4 Jun 2014 18:07:06 +0000

So, since I’m in the acknowledgement section of the draft, I did some digging to see if I could discover what I contributed.  One message I sent at the time shows the BSD/OS did implement some of these ideas.  You can find it at:
	http://roland.grc.nasa.gov/tcp-impl/list/archive/1498.html

All the previous discussion is in the tcp-impl archives, there were two threads, starting at:

	http://roland.grc.nasa.gov/tcp-impl/list/archive/1475.html
	http://roland.grc.nasa.gov/tcp-impl/list/archive/1743.html

It’s probably worth looking over the previous discussions before rehashing it all over again. :-)

			-David Borman

> though probably i dropped the ball somewhere.  hope that helps.

----------------------------------------------------------------------
The information contained in this transmission may be confidential. Any disclosure, copying, or further distribution of confidential information is not permitted unless such privilege is explicitly granted in writing by Quantum. Quantum reserves the right to have electronic communications, including email and attachments, sent across its networks filtered through anti virus and spam software programs and retain such messages in order to comply with applicable data security and retention requirements. Quantum is not responsible for the proper and complete transmission of the substance of this communication or for any delay in its receipt.

---

From: Joe Touch <[email-masked]>
Date: Thu, 05 Jun 2014 13:32:44 -0700

Hi, David,

Yes - those points raised before are probably still as relevant, but the 
broader point that Jim G. raised was never considered before - why not 
just push to change the default? We've long since evolved past a world 
where we can make assumptions that a TCP stream can safely be stalled at 
an arbitrary byte boundary...

Joe