Subject: Re: [scim] SCIM Attribute Validation

From: Shelley <[email-masked]>
Date: Mon, 3 Nov 2014 13:00:16 -0600

The following includes some initial proposed wording to reflect some of
these discussions.

The text is still somewhat loosely worded so as to not impose strict
requirements on consumers or service providers, but to attempt to encourage
standardized behavior. Some of the wording has been tweaked slightly, but
the primary modifications are represented in green.

*2.2 Multi-valued Attributes*
>
> When returning multi-valued attributes, service providers SHOULD
> canonicalize the value returned, if appropriate (e.g. for e-mail addresses
> and URLs). Service providers MAY return the canonicalized value using the
> "display" sub-attribute and return the original value using the "value"
> attribute.
>
> Service providers MAY return the same value more than once with different
> types (e.g. the same e-mail address may used for work and home), but SHOULD
> NOT return the same (type, value) combination more than once per Attribute,
> as this complicates processing by the Consumer.
>
> *3.10. HTTP Status and Error Response Handling*
>
> *invalidValue*
> A required value was missing, or the value specified was not compatible
> with the operation, attribute type (see Section 2.1
> [I-D.ietf-scim-core-schema]), or schema (see Section 4
> [I-D.ietf-scim-core-schema])
>
> *4.1.2. Multi-valued Attributes*
>
> *phoneNumbers*
> Phone numbers for the user. The value SHOULD be specified according to
> format in [RFC3966], e.g. 'tel:+1-201-555-0123'. Service providers SHOULD
> canonicalize the value according to [RFC3966] format, when appropriate.
> The 'display' sub-attribute MAY be used to return the canonicalized
> representation of the phone number value. Canonical type values of
> "work", "home", "mobile", "fax", "pager", and "other".
>
> *emails*
> E-mail addresses for the user. The value SHOULD be specified according to
> [RFC5321]. Service providers SHOULD canonicalize the value according to
> [RFC5321], e.g. "[email-masked]" instead of "[email-masked]". The
> 'display' sub-attribute MAY be used to return the canonicalized
> representation of the phone number value. Canonical type values of
> "work", "home", and "other".
>

Regarding the canonicalization of RFC 3966 phone numbers into separate
"display" and "type" values; e.g.

"type":"internal",
"display":"7042",
"value":"tel:7042;phone-context=internal"

I'd suggest omitting this type of behavior. Canonicalization to RFC 3966
alone should be sufficient, and consumers reading data are welcome to
further parse the RFC phone number. Overloading the use of "type" with
non-canonical values may add unnecessary complexity to parsing for both SPs
and consumers.

For example, I'd merely suggest the following type of canonicalizations:

Request:
"value":"tel:7042;phone-context=internal"
"type":"work"

Response:
"value":"tel:7042;phone-context=internal"
"display":"tel:7042;phone-context=internal"
"type":"work"

Request:
"value":"+1 (201) 555-0123"
"type":"work"

Response:
"value":"+1 (201) 555-0123"
"display":"tel:+1-201-555-0123"
"type":"work"

On Wed, Oct 22, 2014 at 2:46 PM, Phil Hunt <[email-masked]> wrote:

> Comments inline…
>
> Phil
>
> @independentid
> www.independentid.com
> [email-masked]
>
>
>
> On Oct 22, 2014, at 12:13 PM, Shelley <[email-masked]> wrote:
>
> Before any specific text is proposed, I'd like to get a general
> understanding for what the agreement/direction is. It seems there are still
> some major open questions that I'd appreciate some input on.
>
>    - Should consumers be required to provide values in specific formats?
>       - For example, for phone numbers, must consumers provide RFC 3966
>       format, an international format, any of multiple common/pre-defined
>       formats, or is any value allowed? If not in RFC/international format, is
>       the country/area code assumed?
>    - Should service providers perform attribute validation or accept any
>    values?
>       - If service providers perform validation, what is the expected
>       behavior when validation fails? (400 / ignore the value / store the value
>       as-is?)
>    - Particularly if there are no requirements for data input and/or no
>    validation is performed:
>       - What are the assumptions that service providers can make during
>       canonicalization?
>       - What should service providers / consumers do if canonicalization
>       is not possible?
>       - Should the use of both the "value" and "display" attributes be
>       recommended?
>
> There are minimally two major variations of possible changes on the table,
> with various flavors in between:
>
>    - Require consumers to provide values in specified formats and SPs
>    perform validation, rejecting invalid values
>       - SCIM serves as a repository of well-formed, valid data. This
>       results in the most usable data for SCIM consumers *reading *data,
>       but puts some burden on SCIM consumers *providing *the data to
>       conform.
>
> *-OR-*
>
>    - Allow consumers to provide values in any format, SP performs no
>    validation though it may *attempt *perform canonicalization
>    - SCIM serves as a repository of data from third parties, data may not
>       be consistent, valid, or normalized. This results in the least effort for
>       SCIM consumers *providing *data, but puts some burden on SCIM
>       consumers *reading *the data to make sense of the data.
>       - SCIM SPs may *attempt *to normalize some of the data, but there
>       may be no guarantees with respect to the consistency or accuracy of these
>       normalizations.
>
> [PH - <individualHat/>] I prefer this because it aligns with Postel’s Law.
>
> I recommend stipulating that any modification to the supplied values (e.g.
> canonicalization) by the service providers MUST be reflected in the HTTP
> Response JSON representation (e.g. after a POST request).
>
> I think we also need to be clear about how 3966 ABNF elements should map
> to SCIM's multi-valued attribute representation for phoneNumbers. The
> earlier suggestion you made is on the right track (mapping to display,
> type, value, etc).  This might get a bit complex if for example the
> telephone number includes embedded parameters *and* the client already
> specifies type, value, and display representations.  My feeling is that the
> SP should always honour any assigned “type” value by the client.  However,
> if the client does not assign a “type”, the SP could assign one based on
> parsing parameters. For example, say a client provides a value of "
> tel:7042;phone-context=internal”.  The server might canonicalize/map as:
>
> “type”:”internal”,
> “display”:”7042”,
> “value”:"tel:7042;phone-context=internal”
>
> More generically, here is one possible mapping based on the ABNF from
> RFC3966 to phoneNumbers attribute:
>
> “value”: “*{telephone-uri}*”
> “display”:”{global-number-digits / local-number-digits}”
> “type”:”{descriptor}”
>
> SP’s may attempt to canonicalize “value” and normalize it. SPs may assign
> values to “display” and “type” if the client has not asserted values for
> these sub-attribues.
>
> Regarding mapping of context and descriptor to SCIM type, we would be
> profiling 3966 to say that the ABNF  “context” component is used and
> interpreted as a type. (can anybody confirm I have interpreted use of
> context in 3966 correctly?)
>
>
> On Wed, Oct 22, 2014 at 12:37 PM, Phil Hunt <[email-masked]> wrote:
>
>> Shelley,
>>
>> I think we are pretty much in agreement. Would you mind providing some
>> new text for the definition of phoneNumbers?  I like where you are going
>> with value vs. displayName etc.
>>
>> It would be good to see how the group feels on a some new proposed text.
>>
>> Thanks,
>>
>> Phil
>>
>> @independentid
>> www.independentid.com
>> [email-masked]
>>
>>
>>
>> On Oct 22, 2014, at 10:22 AM, Shelley <[email-masked]> wrote:
>>
>> Thanks for the response, Phil.
>>
>>  If the SP can understand the value, it may accept it.
>>>
>>
>> Right, but what values can/should SPs attempt to "understand" since these
>> aren't described in the spec? What assumptions, if any, can be made about
>> the format provided by consumers? Further, what happens if the SP doesn't
>> "accept" the value: 400 response? value is silently ignored? value is
>> stored, but not canonicalized?
>>
>> Further it may modify the value to conform to RFC3966 and return the
>>> modified value in the HTTP response.
>>
>>
>> The problem here is that modifying the value is not always possible or
>> may be done incorrectly depending on the format provided by the consumer.
>>
>> Many service providers would rather accept the POST and simply ignore the
>>> value than to loose the transaction.
>>>
>>
>> Silently ignoring unknown values seems a bit dangerous as well. I agree
>> that the corrective action may be difficult to determine automatically if
>> the transaction fails fast, but at least it brings some form of visibility
>> to the problem.
>>
>>
>> On Wed, Oct 22, 2014 at 11:39 AM, Phil Hunt <[email-masked]> wrote:
>>
>>> Shelley,
>>>
>>> Thanks for your comments and bringing this to the groups attention.
>>>
>>> Overall, I would like to strike a balance in favour of being flexible in
>>> what is accepted.  If the SP can understand the value, it may accept it.
>>> Further it may modify the value to conform to RFC3966 and return the
>>> modified value in the HTTP response. For example a service provider
>>> should feel free to correct something in telephone-subscriber format to be
>>> returned as a URI instead.
>>> —> maybe Postel’s Law is one of the general principles of SCIM that we
>>> should make more clearer?
>>>
>>> I think one of the problems with 3966 is that it seems quite broadly
>>> defined. Enforcement may be quite valid. But will it improve quality of
>>> information given the complication it will cause for clients (e.g. when a
>>> request fails due to format, what is the corrective action)?  Many service
>>> providers would rather accept the POST and simply ignore the value than to
>>> loose the transaction.
>>>
>>> I agree we should adjust the language a bit. For example, are we
>>> expecting a telephone-uri or a telephone-subscriber as the value?  The spec
>>> is not clear.
>>>
>>> There are also some other parts to 3966 which duplicate what we are
>>> doing in multi-valued attributes. For example the ABNF “parameter” enables
>>> the following example:
>>>   tel:7042;phone-context=example.com
>>> In this case, “phone-context” could be seen as duplicative of the
>>> function of “type”.
>>>
>>> Maybe it is not only that we want to following RFC3966, but we actually
>>> want to fully specify how 3966 is mapped to phoneNumber’s multi-valued JSON
>>> structure?
>>>
>>> Phil
>>>
>>> @independentid
>>> www.independentid.com
>>> [email-masked]
>>>
>>>
>>>
>>> On Oct 22, 2014, at 8:41 AM, Shelley <[email-masked]> wrote:
>>>
>>> I'd also like to clarify/highlight this problem/challenge with the
>>> current SCIM specification - the spec recommends that service providers
>>> canonicalize phone numbers to RFC 3966, yet, if there are no requirements
>>> for the data provided by the consumer, this may be impossible.
>>>
>>> In the extreme scenario, how can a SCIM SP canonicalize "not a phone
>>> number" to RFC 3966? In an incomplete scenario, how can an SP canonicalize
>>> "222-2222" without a country/area code? And in an ambiguous scenario, how
>>> can an SP canonicalize "201-5399 123" (where 123 is assumed to be an
>>> extension)?
>>>
>>> While it may be possible for the SP to canonicalize *some *values under
>>> some *assumptions *(e.g. country code may be assumed, and assumptions
>>> based on patterns, etc), it is still possible that the canonicalization
>>> would be incorrect and in some cases, impossible.
>>>
>>> How are other SPs handling this? Is anyone doing validation or
>>> canonicalization?
>>>
>>> Likewise, how are SCIM clients consuming phone numbers (especially those
>>> that may be read-only consumers of data and don't have visibility/control
>>> over the source data)? Is anyone doing any normalizing, validating,
>>> canonicalizing, or just leaving the data as-is regardless of the
>>> format/validity?
>>>
>>>
>>> On Wed, Oct 22, 2014 at 10:05 AM, Alex Redston <
>>> [email-masked]> wrote:
>>>
>>>>  Thanks - interesting link.
>>>>
>>>> Alex Redston
>>>>
>>>> CEO
>>>> Switch Identity Governance Ltd
>>>> 07973 320 795
>>>> 020 3434 3888
>>>>
>>>> On 22/10/2014 15:50, Shelley wrote:
>>>>
>>>>  The simplest approach would be to require SCIM consumers to provide
>>>> phone numbers in the RFC 3966 format, which at least helps to guarantee the
>>>> validity of the format and canonicalizes it into a standard global format
>>>> consumable by both external systems and users. This also accounts for some
>>>> of the scenarios you mentioned, such as sip calls.
>>>>
>>>>  As far as validating whether the phone number is actually a
>>>> potentially valid number (based on country, area code, etc.), this can be
>>>> discussed further/separately as well. I agree that validating whether a
>>>> phone number is actually in service is probably not feasible but there are
>>>> some standards and libraries that *could *potentially be used to deter
>>>> completely bogus numbers [1], but the value in this may be debatable.
>>>>
>>>>  The overall problem is just that without centralized data validation
>>>> in the service, SCIM consumers cannot rely on the syntax/validity of the
>>>> data and therefore the burden of validating and normalizing falls to them.
>>>>
>>>> [1] https://code.google.com/p/libphonenumber/
>>>>
>>>>
>>>> On Tue, Oct 21, 2014 at 2:53 PM, Alex Redston <
>>>> [email-masked]> wrote:
>>>>
>>>>>  Of course the challenge here comes down to the extent to which that
>>>>> particular data can be standardised and validated. Taking phone numbers as
>>>>> the example you have sip calls, invalid numbers, country specific
>>>>> shortcodes, this is potentially a minefield.
>>>>>
>>>>> Well formed and valid values are two distinct concepts and with
>>>>> multiple carrier technologies the anatomy of simple +4412345678901 style
>>>>> numbers is no longer fit for purpose. For this reason I do not think that
>>>>> it is feasible for this particular data type to be rejected as bad data
>>>>> because the only way to know if a phone number is valid is to dial it.
>>>>>
>>>>> What do you think?
>>>>>
>>>>> Alex
>>>>>
>>>>> Alex Redston
>>>>>
>>>>> CEO
>>>>> Switch Identity Governance Ltd
>>>>> 07973 320 795
>>>>> 020 3434 3888
>>>>>
>>>>>  On 21/10/2014 20:36, Shelley wrote:
>>>>>
>>>>>  The SCIM specification expects service providers should be able to
>>>>> canonicalize provided data into a common format (e.g. RFC 3966 for
>>>>> phoneNumbers), yet it does not explicitly describe input requirements
>>>>> or validation for attributes such as these.* Is it expected that
>>>>> values that cannot accurately be canonicalized are rejected by the SCIM
>>>>> service or that the SCIM service accepts this invalid data and merely
>>>>> persists and returns this bad data to consumers?*
>>>>>
>>>>> As a SCIM Service Provider, we are currently performing validation on
>>>>> attributes such as phoneNumbers and emails to ensure that they
>>>>> contain well-formed, valid values, and returning a 400 with a description
>>>>> of the error if validation fails. One of the benefits of SCIM is that data
>>>>> may come from multiple sources and may be consumed by multiple clients, and
>>>>> SCIM provides a repository of structured and well-formed data so that
>>>>> consumers can reliably derive meaning and use from this data. Without
>>>>> attribute validation and a guarantee of canonicalized values, this goal
>>>>> does not seem feasible and puts the burden on consumers to make sense of
>>>>> potentially invalid data.
>>>>>
>>>>> The downside to attribute validation is that it requires the SCIM
>>>>> consumers sending data to ensure valid data as well. As these consumers are
>>>>> closest to the source data, though, this seems like the most reasonable
>>>>> approach rather than putting the burden on the service provider and/or
>>>>> other SCIM consumers to make sense of the data. Aside from just "bad data"
>>>>> (e.g. sending "not a phone number" or "000-0000" for a phone number), for
>>>>> example, if the SCIM consumer sending source data sends phone numbers
>>>>> without area codes or sends only internal extensions, another SCIM consumer
>>>>> may not be able to make sense of this data. It seems reasonable to require
>>>>> the consumer sending the data to ensure that this data is well-formed and
>>>>> canonicalized/canonicalizable.
>>>>>
>>>>> A *good *example in the SCIM specification is the country attribute,
>>>>> which clearly indicates the attribute's requirements. A *bad *example
>>>>> is the phoneNumbers attribute, which doesn't describe the expected
>>>>> format to be provided by consumers, yet recommends that the service
>>>>> provider canonicalize the value even though the input may not be able to be
>>>>> canonicalized.
>>>>>
>>>>>  I'd be interested in hearing others' opinions on this matter, and
>>>>> whether the SCIM specification should clarify further restrictions on
>>>>> attribute values.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>  ------------------------------
>>>>>     <http://www.avast.com/>
>>>>>
>>>>> This email is free from viruses and malware because avast! Antivirus
>>>>> <http://www.avast.com/> protection is active.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>>    <http://www.avast.com/>
>>>>
>>>> This email is free from viruses and malware because avast! Antivirus
>>>> <http://www.avast.com/> protection is active.
>>>>
>>>>
>>> _______________________________________________
>>> scim mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/scim
>>>
>>>
>>>
>>
>>
> _______________________________________________
> scim mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/scim
>
>
>

---

From: Phil Hunt <[email-masked]>
Date: Mon, 3 Nov 2014 11:33:53 -0800

Shelley,

Thanks for putting this together. I suggest folks take a look. 

FYI...These issues were raised by Shelley during the WGLC process.

Phil

@independentid
www.independentid.com
[email-masked]

On Nov 3, 2014, at 11:00 AM, Shelley <[email-masked]> wrote:

> The following includes some initial proposed wording to reflect some of these discussions.
> 
> The text is still somewhat loosely worded so as to not impose strict requirements on consumers or service providers, but to attempt to encourage standardized behavior. Some of the wording has been tweaked slightly, but the primary modifications are represented in green.
> 
> 2.2 Multi-valued Attributes
> 
> When returning multi-valued attributes, service providers SHOULD canonicalize the value returned, if appropriate (e.g. for e-mail addresses and URLs). Service providers MAY return the canonicalized value using the "display" sub-attribute and return the original value using the "value" attribute.
> 
> Service providers MAY return the same value more than once with different types (e.g. the same e-mail address may used for work and home), but SHOULD NOT return the same (type, value) combination more than once per Attribute, as this complicates processing by the Consumer.
> 
> 3.10. HTTP Status and Error Response Handling
> 
> invalidValue
> A required value was missing, or the value specified was not compatible with the operation, attribute type (see Section 2.1 [I-D.ietf-scim-core-schema]), or schema (see Section 4 [I-D.ietf-scim-core-schema])
> 
> 4.1.2. Multi-valued Attributes
> 
> phoneNumbers
> Phone numbers for the user. The value SHOULD be specified according to format in [RFC3966], e.g. 'tel:+1-201-555-0123'. Service providers SHOULD canonicalize the value according to [RFC3966] format, when appropriate. The 'display' sub-attribute MAY be used to return the canonicalized representation of the phone number value. Canonical type values of "work", "home", "mobile", "fax", "pager", and "other".
> 
> emails
> E-mail addresses for the user. The value SHOULD be specified according to [RFC5321]. Service providers SHOULD canonicalize the value according to [RFC5321], e.g. "[email-masked]" instead of "[email-masked]". The 'display' sub-attribute MAY be used to return the canonicalized representation of the phone number value. Canonical type values of "work", "home", and "other".
> 
> 
> Regarding the canonicalization of RFC 3966 phone numbers into separate "display" and "type" values; e.g.
> 
> "type":"internal",
> "display":"7042",
> I'd suggest omitting this type of behavior. Canonicalization to RFC 3966 alone should be sufficient, and consumers reading data are welcome to further parse the RFC phone number. Overloading the use of "type" with non-canonical values may add unnecessary complexity to parsing for both SPs and consumers.
> 
>> Before any specific text is proposed, I'd like to get a general understanding for what the agreement/direction is. It seems there are still some major open questions that I'd appreciate some input on.
>> Should consumers be required to provide values in specific formats?
>> For example, for phone numbers, must consumers provide RFC 3966 format, an international format, any of multiple common/pre-defined formats, or is any value allowed? If not in RFC/international format, is the country/area code assumed?
>> Should service providers perform attribute validation or accept any values?
>> If service providers perform validation, what is the expected behavior when validation fails? (400 / ignore the value / store the value as-is?)
>> Particularly if there are no requirements for data input and/or no validation is performed:
>> What are the assumptions that service providers can make during canonicalization?
>> What should service providers / consumers do if canonicalization is not possible?
>> Should the use of both the "value" and "display" attributes be recommended?
>> There are minimally two major variations of possible changes on the table, with various flavors in between:
>> 
>> Require consumers to provide values in specified formats and SPs perform validation, rejecting invalid values
>> SCIM serves as a repository of well-formed, valid data. This results in the most usable data for SCIM consumers reading data, but puts some burden on SCIM consumers providing the data to conform.
>> -OR-
>> Allow consumers to provide values in any format, SP performs no validation though it may attempt perform canonicalization
>> SCIM serves as a repository of data from third parties, data may not be consistent, valid, or normalized. This results in the least effort for SCIM consumers providing data, but puts some burden on SCIM consumers reading the data to make sense of the data.
>> SCIM SPs may attempt to normalize some of the data, but there may be no guarantees with respect to the consistency or accuracy of these normalizations.
> I recommend stipulating that any modification to the supplied values (e.g. canonicalization) by the service providers MUST be reflected in the HTTP Response JSON representation (e.g. after a POST request). 
> 
> I think we also need to be clear about how 3966 ABNF elements should map to SCIM's multi-valued attribute representation for phoneNumbers. The earlier suggestion you made is on the right track (mapping to display, type, value, etc).  This might get a bit complex if for example the telephone number includes embedded parameters *and* the client already specifies type, value, and display representations.  My feeling is that the SP should always honour any assigned “type” value by the client.  However, if the client does not assign a “type”, the SP could assign one based on parsing parameters. For example, say a client provides a value of "tel:7042;phone-context=internal”.  The server might canonicalize/map as:
> 
> “type”:”internal”,
> “display”:”7042”,
> More generically, here is one possible mapping based on the ABNF from RFC3966 to phoneNumbers attribute:
> 
> “value”: “{telephone-uri}”
> SP’s may attempt to canonicalize “value” and normalize it. SPs may assign values to “display” and “type” if the client has not asserted values for these sub-attribues.
> 
> Regarding mapping of context and descriptor to SCIM type, we would be profiling 3966 to say that the ABNF  “context” component is used and interpreted as a type. (can anybody confirm I have interpreted use of context in 3966 correctly?)
> 
>> 
>> I think we are pretty much in agreement. Would you mind providing some new text for the definition of phoneNumbers?  I like where you are going with value vs. displayName etc.
>> 
>>> Right, but what values can/should SPs attempt to "understand" since these aren't described in the spec? What assumptions, if any, can be made about the format provided by consumers? Further, what happens if the SP doesn't "accept" the value: 400 response? value is silently ignored? value is stored, but not canonicalized?
>>> 
>>> Further it may modify the value to conform to RFC3966 and return the modified value in the HTTP response.
>>> 
>>> The problem here is that modifying the value is not always possible or may be done incorrectly depending on the format provided by the consumer.
>>> 
>>> Many service providers would rather accept the POST and simply ignore the value than to loose the transaction.
>>> 
>>> Silently ignoring unknown values seems a bit dangerous as well. I agree that the corrective action may be difficult to determine automatically if the transaction fails fast, but at least it brings some form of visibility to the problem.
>>>  
>>> 
>>> Overall, I would like to strike a balance in favour of being flexible in what is accepted.  If the SP can understand the value, it may accept it. Further it may modify the value to conform to RFC3966 and return the modified value in the HTTP response. For example a service provider should feel free to correct something in telephone-subscriber format to be returned as a URI instead.
>>> —> maybe Postel’s Law is one of the general principles of SCIM that we should make more clearer?
>>> 
>>> I think one of the problems with 3966 is that it seems quite broadly defined. Enforcement may be quite valid. But will it improve quality of information given the complication it will cause for clients (e.g. when a request fails due to format, what is the corrective action)?  Many service providers would rather accept the POST and simply ignore the value than to loose the transaction.
>>> 
>>> I agree we should adjust the language a bit. For example, are we expecting a telephone-uri or a telephone-subscriber as the value?  The spec is not clear.  
>>> 
>>> There are also some other parts to 3966 which duplicate what we are doing in multi-valued attributes. For example the ABNF “parameter” enables the following example: 
>>> In this case, “phone-context” could be seen as duplicative of the function of “type”.
>>> 
>>> Maybe it is not only that we want to following RFC3966, but we actually want to fully specify how 3966 is mapped to phoneNumber’s multi-valued JSON structure?
>>> 
>>> Phil
>>> 
>>> @independentid
>>> www.independentid.com
>>> [email-masked]
>>> 
>>> 
>>> 
>>>> I'd also like to clarify/highlight this problem/challenge with the current SCIM specification - the spec recommends that service providers canonicalize phone numbers to RFC 3966, yet, if there are no requirements for the data provided by the consumer, this may be impossible.
>>>> 
>>>> In the extreme scenario, how can a SCIM SP canonicalize "not a phone number" to RFC 3966? In an incomplete scenario, how can an SP canonicalize "222-2222" without a country/area code? And in an ambiguous scenario, how can an SP canonicalize "201-5399 123" (where 123 is assumed to be an extension)?
>>>> 
>>>> While it may be possible for the SP to canonicalize some values under some assumptions (e.g. country code may be assumed, and assumptions based on patterns, etc), it is still possible that the canonicalization would be incorrect and in some cases, impossible.
>>>> 
>>>> How are other SPs handling this? Is anyone doing validation or canonicalization? 
>>>> 
>>>> Likewise, how are SCIM clients consuming phone numbers (especially those that may be read-only consumers of data and don't have visibility/control over the source data)? Is anyone doing any normalizing, validating, canonicalizing, or just leaving the data as-is regardless of the format/validity?
>>>> 
>>>> 
>>>> On Wed, Oct 22, 2014 at 10:05 AM, Alex Redston <[email-masked]> wrote:
>>>>> The simplest approach would be to require SCIM consumers to provide phone numbers in the RFC 3966 format, which at least helps to guarantee the validity of the format and canonicalizes it into a standard global format consumable by both external systems and users. This also accounts for some of the scenarios you mentioned, such as sip calls.
>>>>> 
>>>>> As far as validating whether the phone number is actually a potentially valid number (based on country, area code, etc.), this can be discussed further/separately as well. I agree that validating whether a phone number is actually in service is probably not feasible but there are some standards and libraries that could potentially be used to deter completely bogus numbers [1], but the value in this may be debatable.
>>>>> 
>>>>> The overall problem is just that without centralized data validation in the service, SCIM consumers cannot rely on the syntax/validity of the data and therefore the burden of validating and normalizing falls to them.
>>>>> 
>>>>> On Tue, Oct 21, 2014 at 2:53 PM, Alex Redston <[email-masked]> wrote:
>>>>> Of course the challenge here comes down to the extent to which that particular data can be standardised and validated. Taking phone numbers as the example you have sip calls, invalid numbers, country specific shortcodes, this is potentially a minefield.
>>>>> 
>>>>> Well formed and valid values are two distinct concepts and with multiple carrier technologies the anatomy of simple +4412345678901 style numbers is no longer fit for purpose. For this reason I do not think that it is feasible for this particular data type to be rejected as bad data because the only way to know if a phone number is valid is to dial it.
>>>>> 
>>>>> What do you think?
>>>>> 
>>>>> Alex
>>>>> Alex Redston
>>>>> 
>>>>> CEO
>>>>>> The SCIM specification expects service providers should be able to canonicalize provided data into a common format (e.g. RFC 3966 for phoneNumbers), yet it does not explicitly describe input requirements or validation for attributes such as these. Is it expected that values that cannot accurately be canonicalized are rejected by the SCIM service or that the SCIM service accepts this invalid data and merely persists and returns this bad data to consumers?
>>>>>> 
>>>>>> As a SCIM Service Provider, we are currently performing validation on attributes such as phoneNumbers and emails to ensure that they contain well-formed, valid values, and returning a 400 with a description of the error if validation fails. One of the benefits of SCIM is that data may come from multiple sources and may be consumed by multiple clients, and SCIM provides a repository of structured and well-formed data so that consumers can reliably derive meaning and use from this data. Without attribute validation and a guarantee of canonicalized values, this goal does not seem feasible and puts the burden on consumers to make sense of potentially invalid data.
>>>>>> 
>>>>>> The downside to attribute validation is that it requires the SCIM consumers sending data to ensure valid data as well. As these consumers are closest to the source data, though, this seems like the most reasonable approach rather than putting the burden on the service provider and/or other SCIM consumers to make sense of the data. Aside from just "bad data" (e.g. sending "not a phone number" or "000-0000" for a phone number), for example, if the SCIM consumer sending source data sends phone numbers without area codes or sends only internal extensions, another SCIM consumer may not be able to make sense of this data. It seems reasonable to require the consumer sending the data to ensure that this data is well-formed and canonicalized/canonicalizable.
>>>>>> 
>>>>>> A good example in the SCIM specification is the country attribute, which clearly indicates the attribute's requirements. A bad example is the phoneNumbers attribute, which doesn't describe the expected format to be provided by consumers, yet recommends that the service provider canonicalize the value even though the input may not be able to be canonicalized.
>>>>>> 
>>>>>> I'd be interested in hearing others' opinions on this matter, and whether the SCIM specification should clarify further restrictions on attribute values.
>>>>> 
>>>>> 
>>>>> 
>>>>>  	
>>>>> This email is free from viruses and malware because avast! Antivirus protection is active. 
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>>  	
>>>> This email is free from viruses and malware because avast! Antivirus protection is active. 			
>>>> 
>>>> 
>>>>