Subject: [scim] case preserving vs case insensitive

From: Dale Olds <[email-masked]>
Date: Tue, 11 Sep 2012 18:27:51 -0700

from my reading, the scim specs are very clear about case-sensitivity 
issues. For example, there is this paragraph in the API doc, section 
3.2.2.1:

"String type attributes are case insensitive by default unless the 
attribute type is defined as a caseExact string. Attribute operators 
'eq', 'co', and 'sw' MUST perform caseIgnore matching for all string 
attributes unless the attribute is defined as caseExact. By default all 
string attributes are caseIgnore."

However, I have not been able to find anything which indicates whether 
attribute values should be case-preserving -- unless perhaps there is 
something relevant in the XML Schema Datatypes Specification referred to 
in the schema doc. I would think there would be something about it in 
the API doc for creating or updating resources.

Personally, I don't remember using a case-insensitive, 
non-case-preserving system since DOS 8.3 file names, but it has come up 
in our implementation.

Has this been discussed and does the WG have rough consensus about this?

My preference would be to specify case-preservation for case-insensitive 
attributes. I think it's the most useful option, and then we could rely 
on it as we cross system domains. It would be inconvenient for a SCIM 
service to magically modify data passed into it.

--Dale

---

From: Samuel Erdtman <[email-masked]>
Date: Wed, 12 Sep 2012 08:01:50 +0200

As far as I can remember we have not discussed case-preservation.

I agree that it should be specified how to handle it. have I understod
you correctly with these two statements:
* Case-sensitive attributes must preserve case.
* Case-insensitive attributes may not preserve case if explicitly
stated, in similarly fashion as case-sensitivity is specified,
otherwise case must be preserved.

Regards
//Samuel

On Wed, Sep 12, 2012 at 3:27 AM, Dale Olds <[email-masked]> wrote:
> "String type attributes are case insensitive by default unless the attribute
> type is defined as a caseExact string. Attribute operators 'eq', 'co', and
> 'sw' MUST perform caseIgnore matching for all string attributes unless the
> attribute is defined as caseExact. By default all string attributes are
> caseIgnore."
>
> something relevant in the XML Schema Datatypes Specification referred to in
> the schema doc. I would think there would be something about it in the API
> doc for creating or updating resources.
>
> Personally, I don't remember using a case-insensitive, non-case-preserving
> system since DOS 8.3 file names, but it has come up in our implementation.
>
> attributes. I think it's the most useful option, and then we could rely on
> it as we cross system domains. It would be inconvenient for a SCIM service
> to magically modify data passed into it.
>
> --Dale
>
>
> _______________________________________________
> scim mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/scim
>

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 12 Sep 2012 08:12:59 -0600

I hate to say the dreaded word "internationalization", but if we're
talking about comparison of strings then there's much more involved
here than case. If we avoid comparison and just pass strings around as
a series of octets then we're probably safe...

On 9/12/12 12:01 AM, Samuel Erdtman wrote:
> I agree that it should be specified how to handle it. have I
> understod you correctly with these two statements: * Case-sensitive
> attributes must preserve case. * Case-insensitive attributes may
> not preserve case if explicitly stated, in similarly fashion as
> case-sensitivity is specified, otherwise case must be preserved.
> 
> Regards //Samuel
> 
> 
>> from my reading, the scim specs are very clear about
>> case-sensitivity issues. For example, there is this paragraph in
>> the API doc, section 3.2.2.1:
>> 
>> "String type attributes are case insensitive by default unless
>> the attribute type is defined as a caseExact string. Attribute
>> operators 'eq', 'co', and 'sw' MUST perform caseIgnore matching
>> for all string attributes unless the attribute is defined as
>> caseExact. By default all string attributes are caseIgnore."
>> 
>> However, I have not been able to find anything which indicates
>> whether attribute values should be case-preserving -- unless
>> perhaps there is something relevant in the XML Schema Datatypes
>> Specification referred to in the schema doc. I would think there
>> would be something about it in the API doc for creating or
>> updating resources.
>> 
>> non-case-preserving system since DOS 8.3 file names, but it has
>> come up in our implementation.
>> 
>> My preference would be to specify case-preservation for
>> case-insensitive attributes. I think it's the most useful option,
>> and then we could rely on it as we cross system domains. It would
>> be inconvenient for a SCIM service to magically modify data
>> passed into it.
>> 
>> --Dale
>> 
>> 
>> _______________________________________________ scim mailing
>> list [email-masked] https://www.ietf.org/mailman/listinfo/scim
>>

---

From: Igor Faynberg <[email-masked]>
Date: Wed, 12 Sep 2012 12:15:06 -0400

A very good point!

Igor

On 9/12/2012 10:12 AM, Peter Saint-Andre wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>> On Wed, Sep 12, 2012 at 3:27 AM, Dale Olds<[email-masked]>  wrote:
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
>
> iEYEARECAAYFAlBQmGsACgkQNL8k5A2w/vzp3ACgvEaWhyhKb7bAEc8z+f2m8aJd
> JBIAoOmDm6YXp1JC0rWd4AFqgtLFvyLI
> =BJel
> -----END PGP SIGNATURE-----

---

From: Kelly Grizzle <[email-masked]>
Date: Wed, 12 Sep 2012 17:14:15 +0000

+1 for always preserving case.

JSON (which seems likely to become the only representation for SCIM) uses Unicode characters for strings.  This would handle the i18n requirements.

--Kelly

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Peter Saint-Andre
Sent: Wednesday, September 12, 2012 9:13 AM
To: Samuel Erdtman
Cc: Dale Olds; [email-masked]
Subject: Re: [scim] case preserving vs case insensitive

I hate to say the dreaded word "internationalization", but if we're talking about comparison of strings then there's much more involved here than case. If we avoid comparison and just pass strings around as a series of octets then we're probably safe...

On 9/12/12 12:01 AM, Samuel Erdtman wrote:
> you correctly with these two statements: * Case-sensitive attributes 
> must preserve case. * Case-insensitive attributes may not preserve 
> case if explicitly stated, in similarly fashion as case-sensitivity is 
> specified, otherwise case must be preserved.
> 
> Regards //Samuel
> 
> 
>> attributes unless the attribute is defined as caseExact. By default 
>> all string attributes are caseIgnore."
>> 
>> whether attribute values should be case-preserving -- unless perhaps 
>> there is something relevant in the XML Schema Datatypes Specification 
>> referred to in the schema doc. I would think there would be something 
>> about it in the API doc for creating or updating resources.
>> 
>> non-case-preserving system since DOS 8.3 file names, but it has come 
>> up in our implementation.
>> 
>> case-insensitive attributes. I think it's the most useful option, and 
>> then we could rely on it as we cross system domains. It would be 
>> inconvenient for a SCIM service to magically modify data passed into 
>> it.
>> 
>> --Dale
>> 
>> 
>> _______________________________________________ scim mailing list 
>> [email-masked] https://www.ietf.org/mailman/listinfo/scim
>>

---

From: Dale Olds <[email-masked]>
Date: Wed, 12 Sep 2012 10:50:00 -0700

Peter, thanks for the comment, but in this case I'm not actually asking 
for clarification on anything involving string comparison -- I think the 
existing spec covers case-insensitive comparisons and filtering. My 
question involved whether the spec should also indicate case-preserving.

Another way of asking my question would be:

Is a scim server free to change the data of an attribute value as long 
as the new value would produce the same matching result in a filter?

--Dale

---

From: Erik Wahlström <[email-masked]>
Date: Wed, 12 Sep 2012 17:57:34 +0000

IMHO it's possible to change anything on the server side. Ignoring attributes, adding attributes, changing cases and so on. The client gets the created resource back in the response and can inspect it to look at what was created on the server side. Should the servers change a lot of things in the creation? No I don't think so, but it should be possible to do to enable more back end storage types.
/ Erik

On Sep 12, 2012, at 7:50 PM, Dale Olds wrote:

> Peter, thanks for the comment, but in this case I'm not actually asking for clarification on anything involving string comparison -- I think the existing spec covers case-insensitive comparisons and filtering. My question involved whether the spec should also indicate case-preserving.
> 
> Is a scim server free to change the data of an attribute value as long as the new value would produce the same matching result in a filter?
> 
> --Dale
>

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 12 Sep 2012 14:18:24 -0600

On 9/12/12 11:14 AM, Kelly Grizzle wrote:
> JSON (which seems likely to become the only representation for
> SCIM) uses Unicode characters for strings.  This would handle the
> i18n requirements.

/me steps onto his soap-box...

Saying "Unicode handles all i18n requirements" is like saying "TLS
handles all security requirements". If you made the latter statement
in a room full of security experts, you'd get howls of laughter or
shouts of objection.

As to i18n, yes it is more complex than "just use Unicode" (or, to be
precise, UTF-8-encoded Unicode code points). To choose a simple
example, "AEIOU" compares to "aeiou" in a case-insensitive comparison,
right? Well, not necessarily. In Turkish locales, "AEIOU" actually
compares to "aeıou" (where ı = U+0131, "LATIN SMALL LETTER DOTLESS
I"). And that's not even to get started on things like canonical vs.
compatibility equivalence, Unicode normlization, bidirectional text,
full-width and half-width characters, confusables, and so on. I gave a
two-hour tutorial about these topics at an IETF meeting last year, and
the slides are here (unfortunately, there are a few errors in the
slides and they need to be updated to reflect more recent work, but I
lost the originals and have only the PDF now, so I need to reconstruct
the whole thing):

https://stpeter.im/files/i18n-intro.pdf

If we have internationalization requirements, we need to clear about
them up front and understand what the security implications might be
(e.g., incorrectly compared usernames could result in escalation of
privilege attacks or denial of service). Depending on the nature of
those requirements, saying "just use Unicode" is probably not enough.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Barry Leiba <[email-masked]>
Date: Wed, 12 Sep 2012 16:31:56 -0400

> /me steps onto his soap-box...

What Peter said.

> those requirements, saying "just use Unicode" is probably not enough.

Indeed, absolutely.

And also, to make things more complicated, it's not as simple as using
octet comparisons.  Somewhere, some identity is going to get set up as
"Barry Leiba", and some administrator is going to search for that by
typing "leiba" (or even "lieba") into a search form.  Or an identity
will be set up for "Hélène Forêt" and an admin with an English
keyboard will search for "helene foret" (or even "helene forest").
The extent to which we want these searches to find the right
identities will be part of the internationalization requirements that
have to be satisfied.

Barry

---

From: Dale Olds <[email-masked]>
Date: Wed, 12 Sep 2012 15:13:37 -0700

(changed subject line to distinguish from evolving thread about i18n, 
which IMO is a fine thread, but goes way beyond the original question)

Erik,

I actually mostly agree with your point, but I'm also trying to complete 
an implementation...

Do you think it should be acceptable for a scim server to force all 
userNames to uppercase (for some definition of uppercase)?

--Dale

---

From: Erik Wahlström <[email-masked]>
Date: Thu, 13 Sep 2012 17:27:56 +0000

Hi,

In general yes. I don't think it's a very good behavior to change it, but I think we build our selfs in a corner if we don't let servers store it's resources in there own way. 

Best regards
Erik

On Sep 13, 2012, at 12:13 AM, Dale Olds wrote:

> (changed subject line to distinguish from evolving thread about i18n, which IMO is a fine thread, but goes way beyond the original question)
> 
> Erik,
> 
> I actually mostly agree with your point, but I'm also trying to complete an implementation...
> 
> Do you think it should be acceptable for a scim server to force all userNames to uppercase (for some definition of uppercase)?
> 
> --Dale
> 
>

---

From: Gil Kirkpatrick <[email-masked]>
Date: Fri, 14 Sep 2012 03:01:21 +0000

>> if we don't let servers store it's resources in there own way.

I disagree that something like the spelling of my last name is the "server's" resource. The storage medium is the server's resource, and attributes that the server creates for its own use are its own. I think that you can argue that a username is a server's resource, or is at least subject to the server's syntax rules. But I would say the spelling of my last name (e.g.) is mine and the server should not alter it. What sort of corner are we building ourselves into if we don't insist on case preservation, at least for attributes that are not "owned" by the server?

Cheers,

-gil 

-----Original Message-----
From: Erik Wahlström [mailto:[email-masked]] 
Sent: Friday, September 14, 2012 3:28 AM
To: Dale Olds
Cc: Samuel Erdtman; Peter Saint-Andre; [email-masked]
Subject: Re: [scim] case preserving

Hi,

Best regards
Erik

On Sep 13, 2012, at 12:13 AM, Dale Olds wrote:

>>>> here than case. If we avoid comparison and just pass strings around 
>>>> as a series of octets then we're probably safe...
>>>> 
>>>>> understod you correctly with these two statements: * 
>>>>> Case-sensitive attributes must preserve case. * Case-insensitive 
>>>>> attributes may not preserve case if explicitly stated, in 
>>>>> similarly fashion as case-sensitivity is specified, otherwise case must be preserved.
>>>>> 
>>>>> Regards //Samuel
>>>>> 
>>>>>

---

From: Erik Wahlström <[email-masked]>
Date: Fri, 14 Sep 2012 13:13:54 +0000

Hi,

First of all I think is a corner and that we should not put the burden of what attributes are case preserving and who is not on a standard implementor. And regarding a specific corner the last name example you mentioned is great. My last name is Wahlström and it includes the Swedish character ö (sound confused and you got the pronunciation correct :)). Every now and then when I register to non swedish systems they complain about the strange looking character ö and I'm force to use o instead, and sometime the servers just change it to some random texts. It could be over ambitious validation rules anywhere down the call stack to the storage and wrong encodings. To enable all legacy server to use SCIM we will run into one of these over ambitious validation rules and the server must then be able to own all attributes.

/ Erik

---

From: Dale Olds <[email-masked]>
Date: Fri, 14 Sep 2012 13:54:06 -0700

Erik, as I understand your reply to Gil, you'd prefer that a scim server 
not munge your name due to "over ambitious validation rules" or "wrong 
encodings," but you'd like the scim spec to allow such munging to ease 
compliance and increase adoption.

I'd like to increase adoption too, but we also need to have some agreed 
level of usefulness -- hopefully a somewhat higher level than currently 
in the name munging wild. I can see situations that, if I could rely on 
a scim server not munging a name, I would be more likely to choose a 
scim server instead of some other service (or rolling my own).

Also, Pamela Dingle has pointed out an attribute that I think is 
relevant to this issue: externalId. Mostly we've talked about human 
names, but externalId is significant also. I may be wrong, but I have 
not been able to find anything in the scim specs that says externalId is 
case-exact. If that's true, it defaults to case-ignore. I would 
certainly not want a scim server to be able to change case or other 
characters in an externalId.

--Dale

---

From: Erik Wahlström <[email-masked]>
Date: Mon, 17 Sep 2012 18:26:08 +0000

See below.

On Sep 14, 2012, at 10:54 PM, Dale Olds wrote:

> Erik, as I understand your reply to Gil, you'd prefer that a scim server not munge your name due to "over ambitious validation rules" or "wrong encodings," but you'd like the scim spec to allow such munging to ease compliance and increase adoption.

Correct. 

> 
> I'd like to increase adoption too, but we also need to have some agreed level of usefulness -- hopefully a somewhat higher level than currently in the name munging wild. I can see situations that, if I could rely on a scim server not munging a name, I would be more likely to choose a scim server instead of some other service (or rolling my own).

But should we build that into the protocol? If you do a bad implementation of anything you will have a hard time getting new customers anyway. Provisioning users in a big company using many different systems usually means that some of them stores data in a database that don't support for example, same example again, Swedish characters. If I where the admin responsible for the project I would prefer getting the users in the system with a couple of strange characters instead of making the legacy system change database models.

> 
> Also, Pamela Dingle has pointed out an attribute that I think is relevant to this issue: externalId. Mostly we've talked about human names, but externalId is significant also. I may be wrong, but I have not been able to find anything in the scim specs that says externalId is case-exact. If that's true, it defaults to case-ignore. I would certainly not want a scim server to be able to change case or other characters in an externalId.

Yes, that's very true. That's a special thing indeed. The spec states the following. "The value of the externalId attribute is always issued be the Service Consumer and can never be specified by the Service Provider." You could interpret that as case preserving, but I think that we could probably explicitly specify it in more detail.

Cheers
Erik