Subject: Re: [PATCH] Using TCP_NODELAY unconditionally

From: Markus Friedl <[email-masked]>
Date: Mon, 21 Jan 2002 10:26:15 +0100

On Mon, Jan 21, 2002 at 10:24:28AM +0100, Markus Friedl wrote:
> On Mon, Jan 21, 2002 at 12:41:37AM -0500, Nicolas Williams wrote:
> > Since the protocol allows a variety of kinds of traffic to be exchanged
> > over a TCP connection, some which should be used with Nagle, and some
> > which shouldn't, the protocol should really be capable of implementing
> > Nagle on a per-channel basis and always run with TCP_NODELAY, but that's
> > a lot easier said than done.
> 
> ietf-ssh is the place for this.

[email-masked]

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 22 Jan 2002 12:15:59 -0500

Brief summary of this thread for the IETF SECSH WG list readers:

 - two patches were contributed implementing overlapping reads/writes
   for OpenSSH's sftp client and server to improve its performance
 - the use of TCP_NODELAY was suggested
 - it turns out that OpenSSH's sftp implementation triggers the Nagle
   algorithm; this can probably be avoided, thereby avoiding the need to
   turn off Nagle on SSHv2 connections, at least in the case of SFTP

 - but what about X11? X11 normally runs over TCP with Nagle turned
   off and it stands to reason that a port forwarder ought to forward
   traffic with Nagle turned off if the traffic being forwarded
   requires that Nagle be turned off

 - which brings up the question of what the SSHv2 specs should say about
   Nagle and whether SSHv2 should have its own per-channel Nagle
   algorithm and run TCP connections with Nagle turned off

The three threads in the OpenSSH list where SFTP performance, Nagle and
X11 have been discussed are:

http://marc.theaimsgroup.com/?t=101143740600002&r=1&w=2
http://marc.theaimsgroup.com/?t=101010189400002&r=1&w=2
http://marc.theaimsgroup.com/?t=101034855100005&r=1&w=2

A quick search through the IETF SECSH WG mailing list archive yielded no
previous mention of Nagle or TCP_NODELAY.

It's not clear yet that forwarding X11 traffic with SSHv2 and Nagle
turned on is a problem. But this issue may yet come up again. So it's
probably worth addressing, even if just to state that Nagle is to remain
on at all times...

Here's some technical approaches to dealing with Nagle that I can think
of right now:

 - specify heuristics by which SSHv2 clients/servers should unilaterally
   decide to turn off Nagle. E.g., no pty sessions + X11 forwarding ->
   turn off Nagle.

 - add a global request type for requesting the remote side to turn
   Nagle off or on.

 - add a per-channel Nagle algorithm protocol, which would probably
   require new channel request message for turning the algorithm on or
   off per-channel - an acknowledgment message would also be needed, but
   gratuitous window adjustments would probably do just fine.

I'm not an expert on SSHv2. Take my comments with some salt.

Cheers,

Nico

--
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

Visit our website at http://www.ubswarburg.com

This message contains confidential information and is intended only 
for the individual named.  If you are not the named addressee you 
should not disseminate, distribute or copy this e-mail.  Please 
notify the sender immediately by e-mail if you have received this 
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free 
as information could be intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain viruses.  The sender therefore 
does not accept liability for any errors or omissions in the contents 
of this message which arise as a result of e-mail transmission.  If 
verification is required please request a hard-copy version.  This 
message is provided for informational purposes and should not be 
construed as a solicitation or offer to buy or sell any securities or 
related financial instruments.

---

From: Bill Sommerfeld <[email-masked]>
Date: Tue, 22 Jan 2002 12:23:53 -0500

[wg chair hat off]

> turned on is a problem. 

In my experience, it is; most ssh implementations turn off nagle on
"interactive" connections.

> But this issue may yet come up again. So it's probably worth
> addressing, even if just to state that Nagle is to remain on at all
> times...

Applications which attempt to track mouse movement (e.g., scroll bar
sliders) don't work very well with nagle turned on.  my understanding
is that "animated" widgest typically require 24-30 updates per second
before it begins to look smooth, while the interaction between delayed
acks and nagle cause very bursty behavior.

					- Bill

---

From: [email-masked] (Niels Möller)
Date: 22 Jan 2002 19:08:47 +0100

> acks and nagle cause very bursty behavior.

I'm not familiar with this issue, and I'm mostly ignorant about what
tcp does below the sockets interface. Can anybody briefly explain what
"nagle" is, and how and when to turn it off? Or point me to the
appropriate manual.

Regards,
/Niels

---

From: Bill Sommerfeld <[email-masked]>
Date: Tue, 22 Jan 2002 13:14:29 -0500

> "nagle" is, and how and when to turn it off? Or point me to the

See RFC896; on BSD-derived systems, the TCP_NDELAY socket option
allows it to be turned on and off.

					- Bill

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 22 Jan 2002 13:46:11 -0500

> "interactive" connections.

OpenSSH doesn't. Which implementations do?

> acks and nagle cause very bursty behavior.

Yes, this is my understanding as well. I'll do a simple test of X11
forwarding with OpenSSH with and without Nagle and with some intensive
application - nothing scientific, mind you, just a simple test to get a
feel of how much of a difference turning off Nagle makes with X11.

Rick Jones makes the argument, on the OpenSSH list, that forwarding of
other protocols should be done with Nagle turned off, that the
forwarded traffic's characteristics would thus mirror the
characteristics of the original traffic. I agree. I just don't know
whether it would be problematic to run interactive character-based
sessions without Nagle.

If SSH runs best with Nagle turned off then the draft RFCs should state
it, possibly  with "SHOULD" language.

Cheers,

Nico

---

From: Rick Jones <[email-masked]>
Date: Tue, 22 Jan 2002 11:25:12 -0800

> If SSH runs best with Nagle turned off then the draft RFCs should
> state it, possibly  with "SHOULD" language.

However, I would hope that any such language woud include something to
the effect that nagle in and of itself is not bad, and that disabling it
in ssh is done when acting as a pipe (man in the middle) and so it is
simply preserving (or at least trying to) the behaviour of the
applications on either end, and that any implementation of ssh SHOULD
NOT gratuitously "break-up" the data it recieves as a pipe etc etc
etc...

rick jones
normally a rather zealous opponent of arbitrarily setting TCP_NODELAY

-- 
Wisdom Teeth are impacted, people are affected by the effects of events.
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to raj in cup.hp.com  but NOT BOTH...

---

From: Markus Friedl <[email-masked]>
Date: Tue, 22 Jan 2002 22:19:14 +0100

> OpenSSH doesn't. Which implementations do?

OpenSSH does, currently.

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 22 Jan 2002 16:37:02 -0500

> > OpenSSH doesn't. Which implementations do?

Ooo, I see that, yes, and only for interactive sessions.

If interactive character-based sessions are the one place where Nagle
definitely belongs and OpenSSH turns it off there, why not turn off
Nagle altogether for all SSHv2 sessions?

Nico

PS: Apologies to the list for the automatically appended disclaimer that
    follows. The first bit is my disclaiming it and assigning rights to
    the list owner; the second bit is added blindly.
--
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

---

From: Tobias Ringstrom <[email-masked]>
Date: Tue, 22 Jan 2002 23:58:56 +0100 (CET)

> Nagle altogether for all SSHv2 sessions?

This is moving away from the scope of ietf-ssh, and back into 
openssh-unix-dev...

I'm not familiar with the SSHv2 spec, but would it be possible, when
opening a new channel, to specify whether the channel should be nodelay or
not.  Is there a standard-compliant way to do this?  A backwards
compatible way?  If such a facility existed, the implementation could set
TCP_NODELAY if one or more channels were nodelay channels.

/Tobias