Subject: Re: Speaking of implementation quirks...

From: [email-masked] (Peter Gutmann)
Date: Fri, 02 Apr 2004 01:50:10 +1200

der Mouse <[email-masked]> writes:

>>Based on this, it'd be useful if the text could emphasise that the
>>value refers to the total packet size by adding the "(including
>>length, padding length, payload, padding, and MAC)" in -transport...
>
>Except the text that needs changing isn't in -transport; it's in -connect.
>The description of channel opening just says it's "the maximum size of an
>individual data packet that can be sent to the sender".  In context, this
>could reasonably be taken to refer to packets at the SSH Connection Protocol
>layer, though whether it refers to data payload or payload plus nine bytes of
>CHANNEL_DATA header is still ambiguous.

The reason I asked for the disambiguation in -transport is that I've only ever
found one implementation that, if asked for a max.packet size of nK, sends
significantly more than nK.  This would imply that most implementations regard
the packet size as "(including length, padding length, payload, padding, and
MAC)" rather than excluding all that, thus leading to over-sized packets when
you add all the other bits.

(Either that or they all use very small amounts of padding, I allow some slop
 in lengths, but not that much.  In any case it needs clarification, otherwise
 there's not much point in being able to specify a length if there are several
 different interpretations of what it means.  What do other implementations
 do?).

>First, -connect 5.1's limit applies only to channel data packets; it has no
>bearing whatever on other packets and thus cannot usefully be used as a limit
>on other packets.

That's fine, channel data packets are the only ones that are likely to get
very big.

>As for your own issues, nothing says that you have to have a 32K buffer in
>order to receive and process packets 32K in length.  You're entirely free to
>process packets byte by byte, chewing your way through that 32K packet
>piecemeal - though you have to be careful that you can back out any global
>data structure changes in case the MAC turns out to be wrong.

You're also entirely free to push a pea up a mountain with your nose, however
that's not a very good way of getting it there.  What critical
interoperability requirement is being served by the MUST 32K?  It just
appeared from nowhere in one of the early drafts (-02 I think).  Why not 64K?
2MB?  2GB?  (or going the other way, 2K?).  The point is that since it's a
negotiable parameter, there's no need to have a MUST tying it to a single
value.

(It's interesting to note that SSL has introduced extensions to allow the use
 of shorter packets in resource-constrained devices, while SSH, which is often
 used as the CLI admin interface on such devices, mandates 32K buffers for an
 interface that usually transmits single keystrokes).

Peter.

---

From: [email-masked] (Niels Möller)
Date: 01 Apr 2004 22:54:23 +0200

[email-masked] (Peter Gutmann) writes:

> significantly more than nK.

I think it would be good if you spelled out exactly which
implementation(s) you are talking about. Mine (lsh) applies the
channel max packet size to the application data portion of DATA and
EXTENDED_DATA packets. I don't know what other implementors do.

> What critical interoperability requirement is being served by the
> MUST 32K? It just appeared from nowhere in one of the early drafts
> (-02 I think). Why not 64K? 2MB? 2GB? (or going the other way, 2K?).
> The point is that since it's a negotiable parameter, there's no need
> to have a MUST tying it to a single value.

It's not *explicitly* negotiated, but as long as the largest packets
are the channel data packets, one can make sure one doesn't get very
large packets by advertising a small channel max packet size. So it
would probably work with a required buffer size of just a few thousand
bytes.

The packets that may potentially hit this limit, besides channel data
(which is of negotiated size), are key exchange messages including
certificate chains, as well as debug and ignore messages, which can
potentially be large.

In practice, I think you could manage with small buffers and still be
99% compliant to the spec, by advertising a smallish max packet size on
all channels, and have special code that can skip and discard messages
of type DEBUG and IGNORE without buffering the complete messages.

(You would fail to interoperate with implementations that put
ridiculously long algorithm or language lists into their KEXINIT
message, but that should not be common. Only example I'm aware of is
SSH-2.0-Sun_SSH_1.0 which dumps its complete locale list into the ssh
language list, resulting in a KEXINIT message of some 1400 bytes plus
header, and that's a known bug which I hope will be fixed in the next
release).

/nisse

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 1 Apr 2004 22:25:01 -0600

> header, and that's a known bug which I hope will be fixed in the next

The known bug is that it uses locales rather than RFC3066 language tags.

The next release will use RFC3066 language tags and will send its entire
list of installed locales (mapped to langtags).  Do you propose that we
do something different?

I posted sometime last year on inconsistencies in the spec wrt language
tags.  I got precious few comments...  This is the first that I hear of
any issue with the size of our server's KEXINIT packets.  If our current
approach causes problems for anyone then perhaps you might like to
review those comments and make some suggestions.

IIRC I mentioned the possibility of the server not advertising any
langtags and instead relying on the langtag fields of various packets to
convey the negotiated language to the client.

Since then I'm also finding that, while RFC3066 lang tags are really
nice for negotiating a language for localization (L10N), they are not
appropriate for negotiating other I18N matters, such as what encoding to
use in a given channel.  I'm afraid that SSHv2 tried valiantly to get
I18N/L10N right but missed a few things (though perhaps nothing that
can't be corrected with new channel requests).

Nico
--

---

From: [email-masked] (Niels Möller)
Date: 02 Apr 2004 09:34:25 +0200

> list of installed locales (mapped to langtags).  Do you propose that we

No, if you actually support all those languages, then including the
full list seems to be the right thing to do. About how many are there?

(My code does impose some arbitrary limits on the lenght of language
and algorithm lists, independent of the maximum packet size, just to
let me use simpler code that would waste cpu or memory for large
inputs. I'll have to reconsider those limits if/when there's a
reasonable use of large lists out there).

> This is the first that I hear of any issue with the size of our
> server's KEXINIT packets.

I don't think there's any problem with the current protocol. There may
be a concern with a hypothetical protocol that changes the
SSH_MAX_PACKET (the packet size that implementations MUST handle) from
today's 32K to a significantly smaller value, say 1K, 

Best regards,
/Niels

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 2 Apr 2004 11:28:57 -0600

> full list seems to be the right thing to do. About how many are there?

Er, many?  The most locales I've seen installed was ~280.  Of course,
there was much overlap since, for example, "en_US" and "en_US.UTF-8"
both map to the same langtag, "en-us".

So say, maybe, 150 5-charater-long langtags (plus commas), so... close
to 1KB.

Of course, few install so many locales on a server, but it does happen.

> reasonable use of large lists out there).

See above.  One way to figure out many langtags might be sent is to
figure out how many langtags of the form

<language tag>-<regional variation sub-tag>

are possible given the current registrations.

> today's 32K to a significantly smaller value, say 1K, 

Clients have [almost[1]] no reason to send large KEXINITs; servers do,
if they use certs, for example, or have many locales installed.

[1]  In the case of GSS-API keyex w/ optimistic keyex it is possible
     that client could send a very large packet following its KEXINIT.

At least for the initial key exchange I think that we need to mandate
that implementions be able to deal with large packets; after the initial
kex there's no need, I think, to have a large packet support
requirement.

Cheers,

Nico
--

---

From: der Mouse <[email-masked]>
Date: Fri, 2 Apr 2004 12:34:28 -0500 (EST)

> that implementions be able to deal with large packets; after the
> initial kex there's no need, I think, to have a large packet support
> requirement.

Also note that "deal with" can mean "recognize that no kex packet that
can lead this implementation to a successful exchange can be over (say)
2000 bytes, so, if it's larger, read and ignore it in many small pieces
and fail the protocol".

This means permitting an implementation to fail kex just because the
other side offers (eg) too many languages.  I'd have to read the spec
carefully to be sure whether there's enough latitude there for that at
present....

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       [email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 2 Apr 2004 15:20:15 -0600

> carefully to be sure whether there's enough latitude there for that at

I was thinking of how servers respond to incorrect optimistic selection
of first kex packets that are too large.

Nico
--

---

From: [email-masked] (Niels Möller)
Date: 02 Apr 2004 23:25:53 +0200

> carefully to be sure whether there's enough latitude there for that at

I'm pretty sure the spec doesn't allow such behaviour. What you could
do is to parse the large KEXINIT packet incrementally, so you can
ignore large language lists without ever storing the full list.

Similarly, you MUST be able to read and ignore DEBUG and IGNORE
messages of size up to 32K.

(My current code imposes the arbitrary limit of 47 elements on all
lists in the KEXINIT message. I realize that I have to make that more
flexible).

/Niels

---

From: [email-masked] (Peter Gutmann)
Date: Sun, 04 Apr 2004 16:39:06 +1200

[email-masked] (=?iso-8859-1?q?Niels_M=F6ller?=) writes:
>I think it would be good if you spelled out exactly which implementation(s)
>you are talking about. Mine (lsh) applies the channel max packet size to the
>application data portion of DATA and EXTENDED_DATA packets. I don't know what
>other implementors do.

Well I can't list all of them because I don't know what users are
communicating with, only the ones that they complain about, which in this case
was WS_FTP for Windows.  I've since worked around it by increasing the amount
of slop when defining upper bounds buffer sizes, but that's probably not a
long-term solution.

>It's not *explicitly* negotiated, but as long as the largest packets are the
>channel data packets, one can make sure one doesn't get very large packets by
>advertising a small channel max packet size. So it would probably work with a
>required buffer size of just a few thousand bytes.

Right, and that works fine.  The problem is that putting in a MUST 32K means
that someone can come along in the future and claim that something is non-
compliant because it can't handle a 32K packet even when though explicitly
asked for an 8K packet size.

Peter.

---

From: [email-masked] (Peter Gutmann)
Date: Sun, 04 Apr 2004 16:42:24 +1200

der Mouse <[email-masked]> writes:

>Also note that "deal with" can mean "recognize that no kex packet that can
>lead this implementation to a successful exchange can be over (say) 2000
>bytes, so, if it's larger, read and ignore it in many small pieces and fail
>the protocol".

I can just see the code comment for this, "Encountering this behaviour is the
equivalent of seeing someone wearing his underpants on his head coming down
the street towards you.  Best practice is to cross the read to avoid them.  We
now close the connection and exit..." :-).

Peter.