Subject: [apps-discuss] draft-nottingham-http-problem-04 - details, remediation, links

From: "David J. Biesack" <[email-masked]>
Date: Fri, 18 Oct 2013 11:52:43 -0400

I just learned of this draft, and I have some suggestions related to "Problem Details for HTTP APIs draft-nottingham-http-problem-04", http://tools.ietf.org/html/draft-nottingham-http-problem-04

In our error response, the details is an array of strings instead of a single "detail" string, as there may be multiple details associated with a problem. (One example we have is when parsing a request; we can list multiple syntax errors from a parser.)

In addition, we separate problem details from problem remediation - i.e. the suggested means to remediate or correct the problem.

Finally, we use Atom links in the error response, so we can link to multiple resources (documentation, to clarify the details and/or remediation and/or error code, etc) with different link relations. (The draft's "problemInstance" could be just a link relation in the set of Atom links.)

{ "details" : [ "no viable alternative at character '@'" ],
  "errorCode" : 1732,
  "httpStatusCode" : 400,
  "message" : "The request parameter 'filter' with value '@' has invalid syntax",
  "remediation" : "Use correct filter syntax",
  "links" : [ { "rel" : "errorDoc", "href" : "http://www.example.com/errorDoc/1732", "method" : "GET", ..... },
              { "rel" : "synatx", "href" : "http://www.example.com/reference/synatx/media-types/vnd.sas.foo", ... },
              ... ]
  "version" : 1
}

Other minor points:

Our "message" corresponds to this draft's "title".

Our error code is an integer, not a string/enumeration. I can see that enumerations may be better.

The "version" is the media type/representation's schema version.

I saw some comments on draft -03 that seemed geared towards distinguishing between human consumers of an API (and its problem responses) and programmatic consumers. In our case, we do expect that in many cases, there is a human using the client application that is invoking the API, and the client needs to present problem reports - clients may not be able to cleanse or validate all user input, for example.

That's all for now, thanks for listening.

djb

-- 
David J. Biesack | Principal API Architect, SAS | @davidbiesack | 919-531-7771 | www.sas.com

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Oct 2013 18:33:35 +1100

Hi David,

> In our error response, the details is an array of strings instead of a single "detail" string, as there may be multiple details associated with a problem. (One example we have is when parsing a req…

Yes; this has come up in discussion a few times; I should add a FAQ.

The problem (ha!) is that people often try to lump multiple types of problems into one message, leading into a mismatch between the HTTP status code and the payload. This conundrum led to the Multistatus status code in WebDAV, which is widely agreed to be a Bad Thing.

OTOH if your problem type can encompass multiple details (e.g., "these records are bad"), you can easily encode that in a type-specific extension (which supports both arrays and objects).

> In addition, we separate problem details from problem remediation - i.e. the suggested means to remediate or correct the problem.

Interesting. Do you have variations between remediation strategies within a single error code?

> Finally, we use Atom links in the error response, so we can link to multiple resources (documentation, to clarify the details and/or remediation and/or error code, etc) with different link relation…

I think you mean link relations  -- see RFC5988 :)

It might be interesting to add a generic 'links' object, but without strong motivation I'm a bit wary of this, as some people will see it as a needless abstraction. Since a problem type can firmly identify all of the extensions in use, I'm not sure there is one, unless people want to be able to add typed links to problems arbitrarily (i.e., without them being nominated by the problem type).

I'm about to publish a (hopefully final) draft, where the only major change might be shortening the CamelCase to be lowercase (e.g., "problemType" to "type"), so as not to collide with different people's JSON conventions. Would that cause any problems (for you or anyone else reading)?

Cheers and thanks much for the feedback,

--
Mark Nottingham   http://www.mnot.net/

---

From: "David J. Biesack" <[email-masked]>
Date: Mon, 21 Oct 2013 11:24:36 -0400

> OTOH if your problem type can encompass multiple details (e.g., "these records are bad"), you can easily encode that in a type-specific extension (which supports both arrays and objects).

I can see the danger of getting t0o complex.

Given form-based requests or multiple query parameters, there may be errors (problems) in each. When I run a compiler on a source program, I want to get back all the errors, so I can fix them all rather than iterate and fix one at a time.  Since we can't trust all clients, we must be prepared to validate all input. We like the flexibility to report *all* problems with a request.  (This is generic as opposed to a type-specific problem report, i.e. a multi-record media type)

I'll have to investigate the use of type-specific extensions but of course that would be non-standard and thus create tighter coupling between clients and servers that adopt that extension. Of course, other APIs from different vendors might (will) implement the same concept differently.

I'm not well-versed in WebDAV and its history. Can you point out some of the criticism/consensus against multi-response, and what is recommended in its place to deal with multifarious request problems?

> Interesting. Do you have variations between remediation strategies within a single error code?

Not at this time.

> It might be interesting to add a generic 'links' object, but without strong motivation I'm a bit wary of this, as some people will see it as a needless abstraction. Since a problem type can firmly…

We simply find it more RESTful to include links (and link relations) in the error response.

We embed links objects in the JSON response using Atom link representation instead of using response headers. "problemInstance" could be a link relation in one of these link elements. We prefer embedded link objects over headers, because our representations often contain subobjects (each with their own links). Similarly, we can envision one of these Error responses itself being embedded in some other larger response (which itself may not be an error; embedded links are more natural for this scenario than pushing the links into the response header as per rfc5988. 

"problemType" could also be a link relation; this would allow multiple links with types in addition to text/html. But that too may be overly complex. I'm just trying to anticipate likely and uniform extensibility.

Perhaps your FAQ can elaborate on "a problem type can firmly identify all of the extensions in use", with some examples. I.e. how are extensions identifier?

(Note: section 3 has a typo: 'and "account" gives a link  where the account can be topped up.' The JSON shows an array, accounts, not an "account" scalar link)

> Cheers and thanks much for the feedback,

thanks!

djb

---

From: Julian Reschke <[email-masked]>
Date: Mon, 21 Oct 2013 18:21:13 +0200

> The problem (ha!) is that people often try to lump multiple types of problems into one message, leading into a mismatch between the HTTP status code and the payload. This conundrum led to the Multi…

It would be nice if you could back that up a bit.

> ...

Best regards, Julian

---

From: Erik Wilde <[email-masked]>
Date: Mon, 21 Oct 2013 11:36:35 -0700

hello david.

just honing in on this one comment of yours:

> "problemType" could also be a link relation; this would allow multiple links with types in addition to text/html. But that too may be overly complex. I'm just trying to anticipate likely and unifor…

and then there's something related that darrel was writing a little 
earlier today:

On 2013-10-21 06:48 , Darrel Miller wrote:
> On the positive side it would make it easier to consider registering
> "probleminstance" as link relation considering the requirement for
> link relation types to be lowercase.  Which actually leads me to the
> question, would simply "instance" be a reasonable link relation to
> describe the relation between the problem details and the specific
> instance of the problem?  I can certainly imagine other scenarios
> where the notion of an "instance" might be valuable.  Could this stand
> on it's own?

these are interesting and complementary thoughts. in the context of an 
HTTP problem report, type and instance indeed might be defined and 
reused based on more generic link relations. interestingly, there 
already is a "type" link relation: 
http://tools.ietf.org/html/rfc6903#section-6

afaict, there is no existing link relation corresponding to the 
"instance" concept proposed by darrel. and i am not 100% sure which link 
relation to best use here, if we wanted to reuse an existing one. if we 
wanted to use link relations, would anybody see a good candidate, or 
would it make more sense to mint and register a new link relation for 
what is now covered by "problemInstance"?

thanks and cheers,

dret.

-- 
erik wilde | mailto:[email-masked]  -  tel:+1-510-2061079 |
            | UC Berkeley  -  School of Information (ISchool) |
            | http://dret.net/netdret http://twitter.com/dret |

---

From: Darrel Miller <[email-masked]>
Date: Mon, 21 Oct 2013 14:43:50 -0400

Erik, David,

The only challenge I see with having problemType as a link relation,
or even using the existing 'type' link relation type is that currently
the problemType URI does not have to resolve to a representation.  The
spec says it is a should not a must.  If we use a link relation, are
we not forcing the dev's hand to effectively make it a must?

I wouldn't object to the change, but others might.

Darrel

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 22 Oct 2013 16:41:25 +1100

> It would be nice if you could back that up a bit.

Sure.

Much of the value of a HTTP status code is that it's visible, so that intermediaries, high-level (or low-level, depending upon perspective) HTTP stacks, etc., can use it to provide things like caching, automatic retries, etc. By stuffing several status codes into a payload and labelling it with one generic one, you require all of that generic software to understand a specific format, which generally doesn't work.

In a nutshell, it's tunnelling.

Yaron talks about it more in the WebDAV Book of Why:
  http://lists.w3.org/Archives/Public/w3c-dist-auth/1999JanMar/0128.html

He finishes up:

> While this logic is sound it ignores the bigger picture. Future generations
> are quite likely to use 207 and equally like to muck things up and use it in
> a context where someone receiving it may not have expected it. However they
> will expect to be able to rely on the response class and the response class
> fails them here because it will always be 207 even if everything inside of
> it is a failure.
> 
> In the end however, the mixed error case refused to admit a sensible
> solution. So rather than mucking up the protocol with a large number of
> response line codes with arcane rules specifying their use we decided to
> keep matters simple and return a single response line code.

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Oct 2013 18:37:24 +0200

> In a nutshell, it's tunnelling.

Well, it's at least a well-defined tunnel.

I wish that people who dismiss a WebDAV feature actually fully 
understood how it's used.

I've seen lots of complaints about the inner workings of WebDAV, and 
many discussions about how things *could* be better for some value of 
"better", but so far nobody has produced something that actually works 
and is inter-operable.

>> fails them here because it will always be 207 even if everything inside of

That's every very misleading; if "everything" failed you won't get a 207 
in the first place.

People who dismiss the 207/multistatus approach altogether are invited 
to come up with a "bad" example, and a concrete proposal what would have 
been a better design. (And yes, that includes Yaron :-)

Best regards, Julian