Subject: Re: New drafts

From: [email-masked] (Niels Möller)
Date: 04 Nov 2004 17:31:05 +0100

Chris Lonvick <[email-masked]> writes:

> Just to summarize the discussion, I am planning on the following:
> 
> =====
> For SSH_MSG_CHANNEL_OPEN_FAILURE we will have
> 
> 0x0000 0000 - 0xFDFF FFFF        IETF CONSENSUS
> 0xFE00 0000 - 0xFEFF FFFF        PRIVATE USE - channel-type specific
> 0xFF00 0000 - 0xFFFF FFFF        PRIVATE USE - (experimental stuff)
> 
> with accompanying notes about
> 
> - The range starting with 0xFE should be used when localized names are
> used for opening a channel (e.g. SSH_MSG_CHANNEL_OPEN message with a
> 'channel type' of "[email-masked]").  Interoperability is
> assumed by a "gentleman's agreement" that IF you accept the localized
> channel open type THEN you will understand the failure 'reason code'
> values associated with that localized name.  Also, the IANA assigned
> 'reason code' values are still valid even when opening a channel using a
> localized name.

I don't think "localized" is the right word for referring to names of
the "name@domain" type. What are they called where this naming scheme
is introduced? "locally defined"?

>    Note that the 'plaintext password' value is encoded in ISO-10646
>    UTF-8.  It is up to the server how it interprets the password and
>    validates it against the password database.  However, if the client
>    reads the password in some other encoding (e.g., ISO 8859-1 - ISO
>    Latin1), it MUST convert the password to ISO-10646 UTF-8 before
>    transmitting, and the server MUST convert the password to the
>    encoding used on that system for passwords.

I strongly prefer that we don't change that. The problem is the server
side. If a some unix system uses non-ascii characters in passwords,
there really ought to be a configuration somewhere on the system that
says what character set that is. Without that, I don't think there's a
sane way to get passwords right when more than one character set is
used (consider a unix server with latin-1 passwords, and a windows-ce
client using utf8).

I have one suggestion that may or may not satisfy der Mouse:

  If a server really has no clue about what character set is used in
  its password database, it SHOULD default to iso-8859-1.

Motivation: If you really need to pass an arbitrary octet string as a
password, you can do that by encoding the octet string into utf8. I
choose iso-8859-1 primarily because transformation between
utf8/unicode and iso-8859-1 is particularly easy.

Consider (a slight variation of) the given example: The user's
password is the octet string 0x72 0xc3 0xeb. The user likes 8859-7,
and thinks about the password as r, capital gamma, lowercase lambda,
but the sysadmin and the ssh server have no clue about that.

Now, the user sits at an utf8-only system (say, the windows-ce
client), and wants to login. Typing r, capital gamma, lowercase lambda
won't work (and it wouldn't work if the protocol treated passwords as
opaque octet strings either). However, if the server interprets the
utf8 it receives as latin 1, the user can login if he types "rÃë".

I think that der Mouse's point is that *if* the user is sitting at an
iso-8859-7-only device, then this won't work (there's no way to input
"rÃë" to the client), while things would work if passwords were octet
strings in the protocol. I can think of some kludges, but I see no
really good way to avoid the problem.

Sure, this way of logging in is obscure and not particularly friendly,
but I don't think it is worse than the given example.

To really support arbitrary character sets in passwords, different for
each user, the ssh server somehow needs to figure out the user's
preferred locale before trying to verify the password (say, storing it
in /etc/passwd, or in $HOME/.locale, whatever). I'm sorry that doesn't
seem very practical, but I really don't see any sane way to support
network login, from devices using different character sets, without
doing this.

Conversion on the client side is easier, since all per-user's locale
information should be available in the environment.

(Another issue is that servers that use utf8 passwords natively *must*
normalize the passwords in one way or the other, since no
normalization is required on the wire. Same applies to usernames. But
I really *hope* that is obvious for everybody who actually implements
utf8 for usernames and/or passwords).

Best regards,
/Niels

---

From: der Mouse <[email-masked]>
Date: Thu, 4 Nov 2004 22:13:13 -0500 (EST)

>>    UTF-8.  [...]
> I strongly prefer that we don't change that.  [...]

Me too - as you may have gathered. :-)

>   its password database, it SHOULD default to iso-8859-1.

Better than the proposed silence, but not entirely satisfactory, since
it leaves nothing for a server to do for octets in the 0x80-0x9f range
(which do not correspond to 8859-1 characters, as I understand 8859-1).

> password, you can do that by encoding the octet string into utf8.

...hmm?  UTF-8 is a way of encoding 10646 codepoints as an octet
stream; it's not clear how you're proposing to get from "an arbitrary
octet string" to a string of 10646 codepoints to apply the UTF-8
algorithm to.  The closest thing to a sane method I can think of is to
zero-pad each octet to 16 bits and treat the resulting numbers as 10646
codepoints.  If this is what you're proposing, fine, but I think it
should be spelled out explicitly.

> The user's password is the octet string 0x72 0xc3 0xeb.  The user
> likes 8859-7, and thinks about the password as r, capital gamma,
> lowercase lambda, but the sysadmin and the ssh server have no clue
> about that.

> client), and wants to login.  Typing r, capital gamma, lowercase
> lambda won't work ([...]).  However, if the server interprets the

Yes...though if the server interprets received UTF-8 as Latin-1, it's
not clear what the server should do with that gamma and lambda, or any
other received 10646 not present in Latin-1.

> To really support arbitrary character sets in passwords, different
> for each user, the ssh server somehow needs to figure out the user's
> preferred locale before trying to verify the password [...].

Yes.  The difficulty of implementation and ugliness of that are why I
think passwords shouldn't be character strings, but octet strings, with
character sets brought in only when it's time to type a password.  (Or
display them, if that's ever done, though I'd hope it wouldn't be.)

This would mean that an ssh client system needs a way for the user to
enter an arbitrary octet sequence when prompted for a password.  I
consider that a less serious problem than requiring every Unix
derivative supporting ssh to either (a) redesign their password systems
so that passwords become character strings or (b) not support password
authentication.  If it came down to that choice, I'd unhesitatingly
pick (b), though more likely I'd implement passwords as arbitrary octet
strings and ignore the conformance issue.

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       [email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: [email-masked] (Niels Möller)
Date: 05 Nov 2004 20:58:36 +0100

> (which do not correspond to 8859-1 characters, as I understand 8859-1).

I admit I haven't actually read the iso-8859-1 standard. I was
thinking about the 0x80 - 0xff range as defined by the "C1 controls
and latin1" page of the unicode book, which defines 0x80-0x9f.

> octet string"

Under the assumption that the server will convert the utf-8 it
receives to iso-8859-1 (including control characters, i.e. the range
0x00 - 0xff of unicode, and equvivalents), for each octet x there
exists one (sometimes more than one) sequence of utf-8 octets that
server will convert back into the octet x you started with.

> I consider that a less serious problem than requiring every Unix
> derivative supporting ssh to either (a) redesign their password
> systems so that passwords become character strings or (b) not
> support password authentication.

I'm not sure which alternative you are comparing utf-8 passwords to
here. I just want to add that unix systems also have the choice of
supporting only ascii-passwords.

In practice, I'd expect the majority of unix systems to

  * use ascii-only passwords, or

  * use a single eight-bit character set for (almost) all user's
    passwords, or

  * use utf-8 passwords. And then we have to pray to $DEITY that
    passwords are normalized in some consistent way before being
    passed to crypt(3).

About the same applies to user names.

Regards,
/Niels

---

From: Joseph Galbraith <[email-masked]>
Date: Fri, 05 Nov 2004 14:36:27 -0700

> strings and ignore the conformance issue.

I would prefer (c), every ssh implementation must either
know (from configuration or natively), or guess, the charset in
use for the password database on the system.

The alternative, is (d), every SSH client on every operating
system, talking to every server on any OS guesses what charset
is in use for passwords on the server, which it knows nothing
about.

This sounds like a recipe for disaster to me.  And just
because a client doesn't explicitly do a translation
doesn't mean it isn't guessing... it is just guessing
the server is the same as it is.... which might work
pretty good in a homogeneous environment, but won't
necessarily be so hot in a heterogeneous environment.

And I'm sorry, but I don't buy that passwords aren't
character strings.  Humans generate and enter passwords,
and humans work in character strings.  (Now I do buy
that some, maybe most, OSes aren't aware of the charset
property of passwords.)

- Joseph