Subject: [scim] Error responses

From: "Steven W. Moyer" <[email-masked]>
Date: Tue, 3 Sep 2013 11:40:33 -0400 (EDT)

After reviewing the latest drafts of the 2.0 specification, it dawned on me that we could do a better job of providing error responses.  I'll make several observations and a proposal at the bottom of this e-mail, but wanted to reference the appropriate sections of the API document.  The sections that refer to error handling are as follows:

- Section 3. API

   All requests to the Service Provider are made via HTTP operations on
   a URL derived from the Base URL.  Responses are returned in the body
   of the HTTP response, formatted as JSON.  Response and error codes
   SHOULD be transmitted via the HTTP status code of the response (if
   possible), and SHOULD also be specified in the body of the response.

- Section 3.1 Creating Resources

   If the Service Provider determines creation of the requested Resource
   conflicts with existing resources; e.g., a User Resource with a
   duplicate userName, the Service Provider MUST return a 409 error and
   SHOULD indicate the conflicting attribute(s) in the body of the
   response.

- 3.2.2.2. Filtering

   Providers MAY support additional filter operations if they choose.
   Providers MUST decline to filter results if the specified filter
   operation is not recognized and return a HTTP 400 error with an
   appropriate human readable response.  For example, if a Consumer
   specified an unsupported operator named 'regex' the Service Provider
   should specify an error response description identifying the Consumer
   error; e.g., 'The operator 'regex' is not supported.'

- 3.3.2. Modifying with PATCH

   A delete operation is ignored if the attribute's name
   is in the meta.attributes list.  If the requested value to delete
   does not match a unique value on the Resource the server MAY
   return a HTTP 400 error.

- 3.4. Deleting Resources

   Consumers request Resource removal via DELETE.  Service Providers MAY
   choose not to permanently delete the Resource, but MUST return a 404
   error code for all operations associated with the previously deleted
   Id.  Service Providers MUST also omit the Resource from future query
   results.  In addition the Service Provider MUST not consider the
   deleted resource in conflict calculation.  For example if a User
   resource is deleted, a CREATE request for a User resource with the
   same userName as the previously deleted resource should not fail with
   a 409 error due to userName conflict.

   {
     "Errors":[
       {
         "description":"Resource 2819c223-7f76-453a-919d-413861904646 not found",
         "code":"404"
       }
     ]
   }

3.5. Bulk

   If a bulk job is processed successfully the HTTP response code 200 OK
   MUST be returned, otherwise an appropriate HTTP error code MUST be
   returned.

   The Service Provider response MUST include the result of all
   processed operations.  A location attribute that includes the
   Resource's end point MUST be returned for all operations excluding
   failed POSTs.  The status attribute includes information about the
   success or failure of one operation within the bulk job.  The
   attribute status MUST include the code attribute that holds the HTTP
   response code that would have been returned if a single HTTP request
   would have been used.  If an error occurred the status MUST also
   include the description attribute containing a human readable
   explanation of the error.

   "status": {
     "code": "400",
     "description": "Request is unparseable, syntactically incorrect, or violates schema."
   }

   ...
     {
       "location": "https://example.com/v1/Users/b7c14771-226c-4d05-8860-134711653041",
       "method": "PUT",
       "status": {
         "code": "412",
         "description": "Failed to update as user changed on the server since you last retrieved it."
       }
     },
   ...

- 3.9. HTTP Response Codes

   The SCIM Protocol uses the response status codes defined in HTTP [6]
   to indicate operation success or failure.  In addition to returning a
   HTTP response code implementers MUST return the errors in the body of
   the response in the client requested format containing the error
   response and, per the HTTP specification, human-readable
   explanations.  Implementers SHOULD handle the identified errors as
   described below.

   {
     "Errors":[
       {
         "description":"Resource 2819c223-7f76-453a-919d-413861904646 not found",
         "code":"404"
       }
     ]
   }

My observations are as follows:

- Several of the sections refer to the fact that error messages might be plural (such as the phrase "conflicting attribute(s)" in section 3.1), so the error message structure should allow for plural descriptions.

- Several of the sections state that the error message should be human-readable, but the overall error message structure should also be machine parseable.

- It's stated that a service provider MUST return a status code, but SHOULD/MAY return a description of the error in the entity body.  Shouldn't the entity body be consistent if it's sent?

- The Bulk API operation states that an error code must be accompanied by a human-readable error description (which is inconsistent with the underlying single operations).

- When I see the "description" sub-attribute in the "status" attribute, I mentally expect it to contain the HTTP status message associated with the status codes (as found here: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html).

- Section 3.9 states that the provider "MUST return the errors in the body of the response in the client requested format containing the error response and, per the HTTP specification, human-readable explanations" but that's not consistent with the description of responses for some of the operations.  The array of "Errors" might return several messages (in the case of a conflict) but (except for the Bulk operations) there will only every be a single error code (and it should match the one returned from the server.

So my proposal would be to strengthen section 3.9 and to make it consistent with the format needed for the Bulk operations.  These needs could be met by a structure similar to the following fragment:

   {
     "location": "https://example.com/v1/Users/5d8d29d3-342c-4b5f-8683-a3cb6763ffcc",
     "method": "PUT",
     "status": {
       "code": 409,
       "description": "Conflict"
     },
     "reasons": [
       "The userName attribute may not be modified on this system",
       "The SSN provided with this request does not match the one stored in the user's record"
     ]
   }

In addition, it could be also be used within the BulkResponse's "Operations" array attribute (though I haven't thought through where to put the transient value of the "bulkId" sub-attribute).  It's my opinion that having a unified format for the responses is highly desireable, and that the format above reduces the inconsistencies stated above.  I also believe it meets the requirements of being both human-readable and machine-parseable.

I'm sure I've made some mistakes in my assertions and there are probably gaps in my proposal, but I wanted to present a straw-man to get the discussion started.  Thanks for putting up with my long-winded e-mail!

Steve

---

From: Samuel Erdtman <[email-masked]>
Date: Wed, 4 Sep 2013 11:42:32 -0500

Interesting, thanks for a very detailed suggestion.
When I read it I see that you have found inconsistencies that at least
needs to be fixed. It might also be desirable to have both description and
reason, however it was my belief the 409 meant conflict and that it
therefor did not needed to be spelled out.

Regarding the multiple reasons e.g. multiple conflicting attributes I´m not
so sure it is needed.

This is the reason that I do not think it is needed:
First, when I implemented for SCIM 1.1 I failed on first error and did not
proceed to find all potential errors. The reason that i did it that way was
that it was easier. When the client got an error it had to fix it try again
and get the next error, fix it and try again until it worked. I can of
course do it this way with your proposal to but it creates a more complex
error response and it might give the client the impression that all errors
are in the first response (which might not be possible).

Then it comes to when we would need multiple status codes e.g 400 bad
request for a request missing attributes and at the same time has
conflicting attributes, which status code should we chose? we could create
a more complex structure e.g.:
 {
     "location": "
https://example.com/v1/Users/5d8d29d3-342c-4b5f-8683-a3cb6763ffcc",
     "method": "PUT",
     },
     "errors": [
       {
         "code": 409,
         "description": "Conflict"
         "reason" : "The userName attribute may not be modified on this
system",
       },
       {
         "code": 400,
         "description": "Bad Request"
         "reason" : "userType is mandatory",
       }
     ]
   }

Finally the backend system might not support to return multiple error
messages, this is a minor if returning just one message is ok.

So as I wrote initially I would prefer the less complex object type since
the benefit of this is minor form my point of view.

Cheers
//Samuel

On Tue, Sep 3, 2013 at 5:40 PM, Steven W. Moyer <[email-masked]> wrote:

> After reviewing the latest drafts of the 2.0 specification, it dawned on
> me that we could do a better job of providing error responses.  I'll make
> several observations and a proposal at the bottom of this e-mail, but
> wanted to reference the appropriate sections of the API document.  The
> sections that refer to error handling are as follows:
>
> - Section 3. API
>
>        "location": "
> https://example.com/v1/Users/b7c14771-226c-4d05-8860-134711653041",
>        "method": "PUT",
>        "status": {
>          "code": "412",
> - Several of the sections refer to the fact that error messages might be
> plural (such as the phrase "conflicting attribute(s)" in section 3.1), so
> the error message structure should allow for plural descriptions.
>
> - Several of the sections state that the error message should be
> human-readable, but the overall error message structure should also be
> machine parseable.
>
> - It's stated that a service provider MUST return a status code, but
> SHOULD/MAY return a description of the error in the entity body.  Shouldn't
> the entity body be consistent if it's sent?
>
> - The Bulk API operation states that an error code must be accompanied by
> a human-readable error description (which is inconsistent with the
> underlying single operations).
>
> - When I see the "description" sub-attribute in the "status" attribute, I
> mentally expect it to contain the HTTP status message associated with the
> status codes (as found here:
> http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html).
>
> - Section 3.9 states that the provider "MUST return the errors in the body
> of the response in the client requested format containing the error
> response and, per the HTTP specification, human-readable explanations" but
> that's not consistent with the description of responses for some of the
> operations.  The array of "Errors" might return several messages (in the
> case of a conflict) but (except for the Bulk operations) there will only
> every be a single error code (and it should match the one returned from the
> server.
>
> So my proposal would be to strengthen section 3.9 and to make it
> consistent with the format needed for the Bulk operations.  These needs
> could be met by a structure similar to the following fragment:
>
>    {
>      "location": "
> https://example.com/v1/Users/5d8d29d3-342c-4b5f-8683-a3cb6763ffcc",
>      "method": "PUT",
>      "status": {
>        "code": 409,
> In addition, it could be also be used within the BulkResponse's
> "Operations" array attribute (though I haven't thought through where to put
> the transient value of the "bulkId" sub-attribute).  It's my opinion that
> having a unified format for the responses is highly desireable, and that
> the format above reduces the inconsistencies stated above.  I also believe
> it meets the requirements of being both human-readable and
> machine-parseable.
>
> I'm sure I've made some mistakes in my assertions and there are probably
> gaps in my proposal, but I wanted to present a straw-man to get the
> discussion started.  Thanks for putting up with my long-winded e-mail!
>
> Steve
>
> _______________________________________________
> scim mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/scim
>