Subject: Re: [scim] Proprietary/custom properties

From: Ganesh and Sashi Prasad <[email-masked]>
Date: Fri, 21 Sep 2012 12:01:11 +1000

I've found that a simple naming convention (e.g., that all metadata element
names begin with an underscore) makes extensions much easier. This only
means the inclusion of a metadata element called "_value" to represent the
value of an attribute, and this then makes the treatment of metadata
uniform.

{  "abc" : 123 }

is semantically equivalent to

{
    "abc" : {
        "_value" : 123
    }
}

This is the same element expressed in terms of metadata.

Now, we can extend this to add more metadata elements (custom properties),
woven into the resource structure itself, without breaking the original.

{
    "abc" : {
        "_value" : 123,
        "_created" : "2011-10-25T13:45",
        "_updated" : "2012-03-18T08:05"
    }
}

If a system doesn't understand a custom metadata tag, it just ignores it.

Nesting is also possible.

{
    "abc" : { "def" : 456, "ghi" : 789 }
}

in expanded notation could be

{
    "abc" : {
        "_value" : {
            "def" : {
                "_value" : 456,
                "_replica" : true
            },
            "ghi" : 789
        },
        "_created" : "2011-10-25T13:45",
        "_updated" : "2012-03-18T08:05"
    }
}

The "_created" and "_updated" metadata tags refer to resource element "abc"
(same level as "_value" of "abc").
The "_replica" tag refers to resource element "def" (same level as "_value"
of "def").
Note that not all elements need be expanded (e.g., "ghi").

Regards,
Ganesh

On 21 September 2012 10:59, <[email-masked]> wrote:

> If you have received this digest without all the individual message
> attachments you will need to update your digest options in your list
> subscription.  To do so, go to
>
> https://www.ietf.org/mailman/listinfo/scim
>
> Click the 'Unsubscribe or edit options' button, log in, and set "Get
> MIME or Plain Text Digests?" to MIME.  You can set this option
> globally for all the list digests you receive at this point.
>
>
>
> Send scim mailing list submissions to
>         [email-masked]
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://www.ietf.org/mailman/listinfo/scim
> or, via email, send a message with subject or body 'help' to
>         [email-masked]
>
> You can reach the person managing the list at
>         [email-masked]
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of scim digest..."
>
> Today's Topics:
>
>    1. Re: Proprietary/custom properties (Emmanuel Dreux)
>    2. Re: Proprietary/custom properties (Andrew Badera)
>    3. Re: evaluating filters: true, false, and undefined (Dale Olds)
>
>
> ---------- Forwarded message ----------
> From: Emmanuel Dreux <[email-masked]>
> To: Andrew Badera <[email-masked]>, "[email-masked]" <[email-masked]>
> Cc:
> Date: Thu, 20 Sep 2012 19:18:55 +0000
> Subject: Re: [scim] Proprietary/custom properties
>
> Our own feedback :****
>
> We’re integrating SCIM on 2 SAAS provider’s platforms.****
>
> For both (= 100%), we’re extending the core user schema because we have to
> handle non standard defined attributes.****
>
> ** **
>
> --****
>
> Cordialement / Regards,****
>
> Emmanuel Dreux****
>
> http://www.cloudiway.com****
>
> Tel: +33 4 26 78 17 58****
>
> Mobile: +33 6 47 81 26 70****
>
> skype: Emmanuel.Dreux****
>
> ** **
>
> *De :* Andrew Badera [mailto:[email-masked]]
> *Envoyé :* jeudi 20 septembre 2012 20:00
> *À :* [email-masked]
> *Objet :* [scim] Proprietary/custom properties****
>
> ** **
>
> Hello all-****
>
> ** **
>
> If an organization is looking at early adoption of SCIM and has a number
> of additional properties associated with users and groups that aren't part
> of core schema or existing/proposed extensions, is there a preferred or
> recommended way of handling those values? My thoughts to date are:****
>
> ** **
>
> 1. Treat it like an unofficial extension, (but then wonder about returning
> supporting schema and configs, though we may not support that at this point
> anyhow)****
>
> 2. Bolt-on a users and groups attribute system, which can easily get ugly,
> bloated, onerous and non-intuitive, fast. (Defining and supporting
> attributes and their metadata, supporting retrieval and modification on a
> per-attribute basis, etc.)****
>
> 3. Something I haven't considered, or missed being discussed on-list
> previously?****
>
> ** **
>
> These are not (strictly) LDAP values, though many are currently stored in
> otherwise-unused, sometimes repurposed LDAP fields. These run the gamut
> from users being "enabled" or "disabled" to user's roles (beyond usertype)
> to user's customer details, etc.****
>
> ** **
>
> Thanks in advance for any insights-
> ∞ Andy Badera
> ∞ Technical Evangelist, MPS Partners, Chicago****
>
> ∞ This email is: [ ] bloggable [x] ask first [ ] private
> ∞ Google me: http://www.google.com/search?q=andrew%20badera
>
> ****
>
>
> ---------- Forwarded message ----------
> From: Andrew Badera <[email-masked]>
> To: "[email-masked]" <[email-masked]>
> Cc:
> Date: Thu, 20 Sep 2012 14:39:15 -0500
> Subject: Re: [scim] Proprietary/custom properties
> Thanks Emmanuel.
>
> With JSON, as long as JSON remains schema-less, with most JSON
> serialization frameworks being fairly loose and lax, that's probably pretty
> pain-free. What about those consuming or serving XML however? Or what
> happens if JSON Schema is adopted? Won't extending the core schema open up
> the possibility, create the likelihood even, of breakage due to validation
> issues? Or issues with anyone who's a very strict consumer?
>
> --ab
>
>
> On Thu, Sep 20, 2012 at 2:18 PM, Emmanuel Dreux <[email-masked]>wrote:
>
>>  Our own feedback :****
>>
>> We’re integrating SCIM on 2 SAAS provider’s platforms.****
>>
>> For both (= 100%), we’re extending the core user schema because we have
>> to handle non standard defined attributes.****
>>
>> ** **
>>
>> --****
>>
>> Cordialement / Regards,****
>>
>> Emmanuel Dreux****
>>
>> http://www.cloudiway.com****
>>
>> Tel: +33 4 26 78 17 58****
>>
>> Mobile: +33 6 47 81 26 70****
>>
>> skype: Emmanuel.Dreux****
>>
>> ** **
>>
>> *De :* Andrew Badera [mailto:[email-masked]]
>> *Envoyé :* jeudi 20 septembre 2012 20:00
>> *À :* [email-masked]
>> *Objet :* [scim] Proprietary/custom properties****
>>
>> ** **
>>
>> Hello all-****
>>
>> ** **
>>
>> If an organization is looking at early adoption of SCIM and has a number
>> of additional properties associated with users and groups that aren't part
>> of core schema or existing/proposed extensions, is there a preferred or
>> recommended way of handling those values? My thoughts to date are:****
>>
>> ** **
>>
>> 1. Treat it like an unofficial extension, (but then wonder about
>> returning supporting schema and configs, though we may not support that at
>> this point anyhow)****
>>
>> 2. Bolt-on a users and groups attribute system, which can easily get
>> ugly, bloated, onerous and non-intuitive, fast. (Defining and supporting
>> attributes and their metadata, supporting retrieval and modification on a
>> per-attribute basis, etc.)****
>>
>> 3. Something I haven't considered, or missed being discussed on-list
>> previously?****
>>
>> ** **
>>
>> These are not (strictly) LDAP values, though many are currently stored in
>> otherwise-unused, sometimes repurposed LDAP fields. These run the gamut
>> from users being "enabled" or "disabled" to user's roles (beyond usertype)
>> to user's customer details, etc.****
>>
>> ** **
>>
>> Thanks in advance for any insights-
>> ∞ Andy Badera
>> ∞ Technical Evangelist, MPS Partners, Chicago****
>>
>> ∞ This email is: [ ] bloggable [x] ask first [ ] private
>> ∞ Google me: http://www.google.com/search?q=andrew%20badera
>>
>> ****
>>
>
>
>
> ---------- Forwarded message ----------
> From: Dale Olds <[email-masked]>
> To: Kelly Grizzle <[email-masked]>
> Cc: Samuel Erdtman <[email-masked]>, "[email-masked]" <[email-masked]>
> Date: Thu, 20 Sep 2012 17:59:27 -0700
> Subject: Re: [scim] evaluating filters: true, false, and undefined
> Kelly, thanks for the reply. I'm more interested in what is compatible
> with other implementations than a particular outcome, and your position is
> certainly implementable. Just in case there are others that might agree
> with my reasoning, I will reply inline below, but I'm more interested in
> determining if the wg feels this issue should be clarified.
>
> replies inline:
>
> On 09/17/2012 09:53 AM, Kelly Grizzle wrote:
>
>> "Filters that contain references to attributes unknown to the server do
>>> not
>>> cause an error, but are evaluated such that those attributes are not
>>> present
>>> on each resource."
>>>
>> I would not support this.  IMO filters that reference attributes that are
>> unknown by the server should return an error, otherwise the client may get
>> unexpected results.
>>
>
> I think it depends on the client and how much the scim schema is extended
> in practice. If clients are tightly coupled to the server then they can
> expect specific attributes to be known and anything else is an error. If
> the scim server supports extensions (as our implementation does, and as do
> others represented on this list), then IMO it's easier with fewer
> unexpected results if they can make queries that work in a reasonable
> fashion to different servers.
>
>
>>
>>  we'd like to be able to write queries so that they work whether a
>>> particular
>>> scim server supports that extension or not.
>>>
>> The client should be able to use the /Schemas resource to determine
>> whether the server supports the extension.
>>
>
> Of course, but that means that an application needs to query the server
> first and will need to support multiple queries based on what extensions
> are supported. As I said above, it depends on the client application and
> how often there are variations in the server schema, but in my experience
> it's crucial that schema extensions can be added without requiring clients
> and servers to be in sync.
>
>
>>
>> In general, I am in favor of making things simpler for the client, but
>> this seems like a case where it would be beneficial for the server to be
>> more strict.
>>
>
> Complete agreement on keeping things simpler for the client, I just think
> it's simpler to make logical queries rather than querying for schema
> support.
>
> I prefer: does joe have a hat or a foobar?
>
> rather than: if (server supports the foobar extension) then (does joe have
> a hat or a foobar?) else (does joe have a hat?)
>
> --Dale
>
>
>> --Kelly
>>
>>
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of
>> Dale Olds
>> Sent: Friday, September 14, 2012 8:18 PM
>> To: Samuel Erdtman
>> Cc: [email-masked]
>> Subject: Re: [scim] evaluating filters: true, false, and undefined
>>
>> I would expect that if "PR foobar" on a resource evaluates to false it
>> means "this resource has no value for attribute foobar", which is not quite
>> the same as "this server does not know what foobar means".
>> Nevertheless, if the server does not know what an attribute means no
>> resource will have a value for that attribute, so perhaps it's close
>> enough. I can't think of an example that's not overly contrived where the
>> difference would matter.
>>
>> One of the reasons I brought this issue up is because there are places
>> we'd like to add our own scim extensions, and we'd like to be able to write
>> queries so that they work whether a particular scim server supports that
>> extension or not. I also need to know how to handle undefined attributes in
>> our own server.
>>
>> Would the working group support this additional sentence to the first
>> paragraph of section 3.2.2.1?
>>
>> "Filters that contain references to attributes unknown to the server do
>> not cause an error, but are evaluated such that those attributes are not
>> present on each resource."
>>
>> --Dale
>>
>> On 09/11/2012 11:21 PM, Samuel Erdtman wrote:
>>
>>> In the scimproxy we will probably do it by interpreting undefined as
>>> false i.e. the statements would be evaluated as you describe it.
>>> However currently we only support one filter operation i.e. no 'and'
>>> 'or' stuff yet. Have actually not tested much filtering during the
>>> interops.
>>>
>>> If you would want to require foobar to be precent before evaluating
>>> rest of the expression you could state it as folows:
>>> ?filter=PR foobar AND (foobar GT 7 OR shirtColor EG red) (PR means
>>> present i.e. not undefined)
>>>
>>> Regards
>>> //Samuel
>>>
>>> On Wed, Sep 12, 2012 at 3:51 AM, Dale Olds <[email-masked]> wrote:
>>>
>>>> section 3.2.2.1 of the API doc describes filtering expressions and
>>>> bindings.
>>>> It is very clear about what should happen if an operator is not
>>>> defined (the example is 'regex'), but I do not see anything which
>>>> explicitly states what happens if an attribute in the expression is
>>>> not defined. The closest statement is in the first paragraph: "The
>>>> expression language that is used in the filter parameter supports
>>>> references to attributes and literals." I suppose that could mean a
>>>> reference to an unknown attribute is not supported, but it doesn't
>>>> really say that, and it doesn't say undefined attributes are an error.
>>>>
>>>> In filtering code for similar systems I've seen implementations that
>>>> did not require attributes to be defined. This can be quite natural
>>>> but does make the implementation more interesting. Consider this filter
>>>> of rooms:
>>>>
>>>> True if contains someone with hat size > 7 or wearing a read shirt.
>>>>
>>>> There may not be anyone wearing a hat in any room, but it would still
>>>> find rooms with red shirts. IOW, it only takes one true component of an
>>>> 'or'
>>>> expression to make it true, even if one of the components is undefined.
>>>>
>>>> "True if someone with foobar > 7 or wearing a read shirt" works just
>>>> as well.
>>>>
>>>> This is not a big issue for us, we're just checking out
>>>> implementation options. What do most implementations currently do?
>>>>
>>>> --Dale
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ______________________________**_________________
>>>> scim mailing list
>>>> [email-masked]
>>>> https://www.ietf.org/mailman/**listinfo/scim<https://www.ietf.org/mailman/listinfo/scim>
>>>>
>>>>  ______________________________**_________________
>> scim mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/**listinfo/scim<https://www.ietf.org/mailman/listinfo/scim>
>>
>>
>>
>
>
> _______________________________________________
> scim mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/scim
>
>

---

From: Andrew Badera <[email-masked]>
Date: Fri, 21 Sep 2012 09:16:15 -0500

Thanks Ganesh, putting some thought into this.

The issue with "custom extensions" as it were though still comes down to
compatibility and validation issues. I feel like if I'm adopting a
standard, that it should meet my needs without all sorts of caveats and
kludges and hacks ... otherwise, what's the benefit to adopting a standard
that doesn't fully serve its purpose? Somewhat philosophical, but the
validation/compatibility stuff is purely technical.

I can't imagine we're the only ones who have had such a need ... there must
be a way to achieve this without risking broken validation down the line,
and without a weighty, obtuse attributes bolt-on. Maybe something as simple
as a property bag in the core user definition, simple name/value pairs ...

--ab

On Thu, Sep 20, 2012 at 9:01 PM, Ganesh and Sashi Prasad <
[email-masked]> wrote:

> I've found that a simple naming convention (e.g., that all metadata
> element names begin with an underscore) makes extensions much easier. This
> only means the inclusion of a metadata element called "_value" to represent
> the value of an attribute, and this then makes the treatment of metadata
> uniform.
>
> {  "abc" : 123 }
>
> The "_created" and "_updated" metadata tags refer to resource element
> "abc" (same level as "_value" of "abc").
> The "_replica" tag refers to resource element "def" (same level as
> "_value" of "def").
>>> These are not (strictly) LDAP values, though many are currently stored
>>> in otherwise-unused, sometimes repurposed LDAP fields. These run the gamut
>>> I would not support this.  IMO filters that reference attributes that
>>> are unknown by the server should return an error, otherwise the client may
>>> get unexpected results.
>>>
>>
>> rather than: if (server supports the foobar extension) then (does joe
>> have a hat or a foobar?) else (does joe have a hat?)
>>
>> --Dale
>>
>>
>>> --Kelly
>>>
>>>
>>>>> but does make the implementation more interesting. Consider this
>>>>> filter of rooms:
>>>>>
>>>>> find rooms with red shirts. IOW, it only takes one true component of
>>>>> an 'or'