Subject: Re: revised internet draft on suggested mod to the Nagle algorithm

From: Vernon Schryver <[email-masked]>
Date: Fri, 26 Feb 1999 14:55:18 -0700 (MST)

I have the distinct impression that the consensus of this mailinglist
is that:

  - as much as possible, turning off the Nagle algorithm should be avoided.
    However, no one has reported any harm done to any network by
    applications that have set TCP_NODELAY.

  - an explicit flush mechanism might be useful, but it would be
    functionally identical to how toggling TCP_NODELAY should work.

  - modifying the Nagle algorithm as proposed would not be profitable.
     + The proposal requires additional CPU cycles, code, and state in
        every TCP implemetation, albeit not a lot.
     + There are no existing applications or programs that would benefit
	from the change.  All existing programs that would benefit already
	set TCP_NODELAY, have not wrecked the Internet or even caused any
	reported harm, and in most cases would stll be forced to set
	TCP_NODELAY and so could not benefit from the proposed change
        or avoid doing hypothetical harm to networks.

> ...
>    applications.  However, for a certain class of applications
>    (notably, request-response protocols) the Nagle algorithm interacts
>    poorly with delayed acknowledgements to give these applications
>    poorer performance.

That's not quite right, is it?  Many request-response applications do
not suffer any bad Nagle effects.  The canonical Nagle application, telnet,
is a request-response protocol.  Its requests are keystrokes and the
responses are echos.

Doesn't the class of applications are suffer from the Nagle algorithm
consist of:
  1. poorly written applications that do send()-send()-receive()
      instead of writev()-receive().
  2. applications that necessarily do send()-send() of small blocks.
  3. applications that send blocks larger than 1 MSS but not a multiple
      of the MSS on systems that do not block based on send() requests.

As far as I can see, only #3 might benefit from the proposal.  #1 and #2
would still be forced to set TCP_NODELAY.

>    This draft is NOT suggesting that these applications should disable
>    the Nagle algorithm.

Why not?  Exactly what is the harm done to the network or anything else
when type #3 applications disable the Nagle algorithm?  The only harm I've
heard mentioned can be best summarized as ""application writers are idiots
and will write junk code that will trash the network if we give them the
slightest opportunity.  Therefore, we must make the API so complicated and
obscure so that only the smartest programmers will figure out how to do
what they need to do."  I like the Nagle algorithm, but I am offended by
some of the implicit attitudes offered in its defense.  Some programmers
are idiots, but some others are not.

>    This draft suggests a fairly small and simple modification to the
>    Nagle algorithm which preserves the Nagle algorithm as a means of
>    protecting the internet while at the same time giving better
>    performance to a wider class of applications.

> ...
>    The current Nagle algorithm can be described as follows:

> ...
> 	if ((packet.size < Eff.snd.MSS) && (snd.nxt > snd.una)) {
> 		do not send the packet;
> 	}

My copy of 4.4BSD-Lite uses

 	if ((packet.size < Eff.snd.MSS) && (snd.nxt != snd.una)) {

That avoids hassles with computing (snd.nxt > snd.una)

> ...
> 	if ((packet.size < Eff.snd.MSS) && (snd.sml > snd.una)) {
> 		do not send the packet;
> 	} else {
> 		snd.sml = snd.nxt+packet.size;
> 		send the packet;
> 	}

Do you have a favorite way of dealing with the problems behind computing 
(snd.sml > snd.una) ?

> ...
>    To implement the modified Nagle algorithm in such a system, we
>    replace snd.sml with two variables: ``snd.sml.add'' is a TCP
> ...

Why modify the Nagle algorithm in such a system?  Exactly how do any of
the "certain class of applications" benefit?

> ...
>    explicit direction of the application.  For example, a system based
>    on BSD might implement a socket option (using setsockopt(2))
>    SO_EXPLICITPUSH, as well as a flag to sendto(2) (possibly
>    overloading the semantics of an existing flag, such as MSG_EOF).

Why would any system put the SO_EXPLICITPUSH into setsockopt()?
Or exactly what would be the difference in such a system between
setting SO_EXPLICITPUSH and setting TCP_NODELAY?

And as we all seem to agree, note that turning TCP_NODELAY on and
off around a send() request is or should be functionally identical
to putting SO_EXPLICITPUSH in the send request.

>    In this scenario, an application would set a socket into
>    SO_EXPLICITPUSH mode, then enter a mode of writing data to the
>    socket and, at the last write, using send(2) with the MSG_EOF flag.
>    The underlying TCP would recognize the MSG_EOF flag as an indicator
>    to transmit the (possibly) small packet.

So SO_EXPLICITPUSH as a setsockopt() is not an action, but an
enabler for MSG_EOF?  Why is an enabler needed?  Why not just honer
MSG_EOF() (assuming you don't instead turn TCP_NODELAY on and off).

> ..
>    If a system were to implement this interface, it would be important
>    to NOT disable Nagle when using this interface.  In other words,
>    when using this interface, the default mode for TCP would be to NOT
>    transmit a small packet (even in the presence of MSG_EOF) if a
>    previously transmitted small packet was as yet unacknowledged.

If there were a packet stuck from a previous application of the Nagle
algorithm, would you not want to push it out along with the new data that
has been explicitly marked with MSG_EOF as needing to go out?  Why delay
the new, urgent data because there is old, non-urgent data that is waiting?
Regardless of what any RFC said, such behavior would generate plenty of
bug reports.

> ...
> Application scenarios that will not be helped by this modification
>
>    The proposed modification helps applications which do not need to
>    transmit more than one small packet in a single round-trip time.
>    This characterizes one way file transfer applications (such as FTP
>    [RFC959]) and request/response protocols (such as NNTP [RFC977] and
>    HTTP [RFC2068] without pipelining).

In practice, that paragraph is false.  
  - Doesn't the new NNTP include pipelining?  And what would be the harm
      should an NNTP implementation set TCP_NODELAY?
  - When was the last time someone complained about FTP performance
      effects of Nagle?
  - since all HTTP implementations set TCP_NODELAY, modifcations to
      Nagle on their behalf are at best moot and irrelevant.

>    However, applications that need to transmit more than one small
>    packet in a single round-trip time are not served by this
>    modification.  An example of such an application is HTTP [RFC2068]
>    using ``pipelining'', in which multiple requests (responses) are
>    transmitted asynchronously.

When I last asked about concrete applications that would benefit from the
proposal, only new HTTP was nominated, but seemed a better fit with
a flush bit or TCP_NODELAY toggling.

Vernon Schryver    [email-masked]

---

From: Mohit Aron <[email-masked]>
Date: Fri, 26 Feb 1999 18:04:11 -0600 (CST)

> would still be forced to set TCP_NODELAY.

Even #3 might not benefit under some cases. An example is associated with the
idiosyncrasies of the way application data is copied into mbufs in BSD based
systems. Data is copied in 4K or 1K chunks (whichever is larger) unless
the data is less than 1K when it is copied as a whole. Suppose the application
tries to write 1600 bytes of data (assuming 1MSS = 1500 bytes). Then the 
mbuf layer would make TCP send 1K first, and the remaining 576 bytes would
be held according to the Nagle's algorithm (even under the new proposal).

Due to the above, applications still might keep using TCP_NODELAY even if
the underlying TCP implements the proposed mod to Nagle's algorithm.

- Mohit