Subject: [OAUTH-WG] Language encoding in error_description

From: Eran Hammer-Lahav <[email-masked]>
Date: Wed, 18 May 2011 09:20:18 -0700

The error_description parameter and similar parameters in the MAC and Bearer specs do not specify the language or encoding used. This is a problem. Can someone offer a solution?

EHL

---

From: Julian Reschke <[email-masked]>
Date: Wed, 18 May 2011 19:16:56 +0200

On 2011-05-18 18:20, Eran Hammer-Lahav wrote:
> The error_description parameter and similar parameters in the MAC and
> Bearer specs do not specify the language or encoding used. This is a
> problem. Can someone offer a solution?

For parameters, you can use the encoding defined in RFCs 2231 and 5987.

See, for instance, 
<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-content-disp-latest.html#rfc.section.4.1> 
(filename-param).

Best regards, Julian

---

From: Kris Selden <[email-masked]>
Date: Wed, 18 May 2011 13:25:52 -0700

How about taking a page from URI spec since it is a query parameter value and follow this for the value of such human readable text params:

When a new URI scheme defines a component that represents textual
   data consisting of characters from the Universal Character Set [UCS],
   the data should first be encoded as octets according to the UTF-8
   character encoding [STD63]; then only those octets that do not
   correspond to characters in the unreserved set should be percent-
   encoded.

http://tools.ietf.org/html/rfc3986

That is also inline with JSON as the one OAuth response type, since it is already limited to UTF encodings.

On May 18, 2011, at 9:20 AM, Eran Hammer-Lahav wrote:

> _______________________________________________
> OAuth mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/oauth

---

From: Julian Reschke <[email-masked]>
Date: Wed, 18 May 2011 22:32:07 +0200

On 2011-05-18 22:25, Kris Selden wrote:
> ...

...which yields almost the same representation as RFC2231/5987 (except 
that in these, the string gets labeled with the charset name).

Best regards, Julian

---

From: Kris Selden <[email-masked]>
Date: Wed, 18 May 2011 16:24:58 -0700

Is there a problem with sticking to UTF-8? OAuth already mandates JSON which is Unicode only.

Would be nice to keep it simple.

I'm guessing without guidance, most would convert to UTF-8 and percent encode anyway.

On May 18, 2011, at 1:32 PM, Julian Reschke wrote:

> ...which yields almost the same representation as RFC2231/5987 (except that in these, the string gets labeled with the charset name).
> 
> Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 19 May 2011 07:48:26 +0200

> Is there a problem with sticking to UTF-8? OAuth already mandates JSON which is Unicode only.

Well, like it or not, the default for HTTP header fields is not UTF-8.

> I'm guessing without guidance, most would convert to UTF-8 and percent encode anyway.

Without guidance, people usually do not encode at all, and we'll see 
different encodings on the wire.

Best regards, Julian

---

From: Kris Selden <[email-masked]>
Date: Thu, 19 May 2011 02:14:32 -0700

> Well, like it or not, the default for HTTP header fields is not UTF-8.

Encoding in HTTP header fields is not the topic, error_description is already encoded into a URI before it is in the Location field. 

There are 3 spots where error_description appears:
http://tools.ietf.org/html/draft-ietf-oauth-v2-16#section-4.1.2.1
http://tools.ietf.org/html/draft-ietf-oauth-v2-16#section-4.2.2.1
http://tools.ietf.org/html/draft-ietf-oauth-v2-16#section-5.2

In section 4.1.2.1 and 4.2.2.1 the issue is about character encoding before application/x-www-form-urlencoded encoding (after that it is ASCII only). In section 4.2.2.1, the parameter is encoded in the fragment component which is only visible on the client side, and likely to be read by a script in Javascript (which is unicode only).

In section 5.2 the response type is JSON which already deals with character encoding (http://tools.ietf.org/html/rfc4627#section-3) and is Unicode only.  So there isn't anything to solve for error_description in section 5.2, except maybe to reference section 3 of rfc4627.

Proposal for sections 4.1.2.1 and 4.2.2.1:

error_description
         OPTIONAL.  A human-readable text providing additional
         information, used to assist in the understanding and resolution
         of the error occurred.  The text should first be encoded as
         octets according to the UTF-8 character encoding before being
         encoded using the "application/x-www-form-urlencoded" format.

Examples:
HTTP/1.1 302 Found
Location: https://client.example.com/cb?error=access_denied&error_description=Acc%C3%A8s+refus%C3%A9

HTTP/1.1 302 Found
Location: https://client.example.com/cb#error=access_denied&error_description=Acc%C3%A8s%20refus%C3%A9

Proposal for section 5.2:

error_description
         OPTIONAL.  A human-readable text providing additional
         information, used to assist in the understanding and resolution
         of the error occurred.  The text shall be encoded in Unicode
         as defined by [RFC4627].

Example:
HTTP/1.1 400 Bad Request
Content-Type: application/json
Cache-Control: no-store

{
  "error":"invalid_request",
  "error_description":"Accès refusé"
}

For query strings encoded with application/x-www-form-urlencoded the most common default is UTF-8, while a response body encoded with application/x-www-form-urlencoded should set a charset parameter in the Content-Type header.

Here are examples of dealing with query strings in a few languages and app frameworks:

Javascript (very relevant to section 4.2.2.1)
> decodeURIComponent("Acc%C3%A8s%20refus%C3%A9") // UTF-8
'Accès refusé'

http://www.ecmascript.org/docs.php
> The decodeURIComponent function computes a new version of a URI in which each escape
> sequence and UTF-8 encoding of the sort that might be introduced by the encodeURIComponent
> function is replaced with the character that it represents.

NodeJS
node
> var querystring = require('querystring')
> querystring.parse('error=access_denied&error_description=Acc%C3%A8s+refus%C3%A9') // UTF-8
{ error: 'access_denied', error_description: 'Accès refusé' }

.Net
Request.QueryString	// UTF-8
HttpUtility.ParseQueryString(String)  // UTF-8
HttpUtility.ParseQueryString(String, Encoding) // Need to know the encoding before the query string is parsed

Ruby
# Rack 3 only parses query strings as UTF-8 but older versions use binary strings
Rack::Request.params
URI.decode_www_form_component(str, enc=Encoding::UTF_8)

Python (binary string)
python
>>> from urlparse import parse_qs
>>> parse_qs("error=access_denied&error_description=Acc%C3%A8s+refus%C3%A9")
{'error_description': ['Acc\xc3\xa8s refus\xc3\xa9'], 'error': ['access_denied']}

PHP (binary string)
php -r 'parse_str("error=access_denied&error_description=Acc%C3%A8s+refus%C3%A9", $output); print_r($output);'
Array
(
    [error] => access_denied
    [error_description] => Accès refusé
)

Java
ServletRequest.getParameter(String name) // Tomcat has 2 settings which govern query string parsing URIEncoding which defaults to ISO-8859-1 and useBodyEncodingForURI which defaults to false
URLDecoder.decode(String s, String enc) // Need to know encoding before percent decoding

On May 18, 2011, at 10:48 PM, Julian Reschke wrote:

> Without guidance, people usually do not encode at all, and we'll see different encodings on the wire.
> 
> Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 19 May 2011 11:55:31 +0200

> In section 5.2 the response type is JSON which already deals with character encoding (http://tools.ietf.org/html/rfc4627#section-3) and is Unicode only.  So there isn't anything to solve for error_…

My comments applied to the proposal of returning error_description in 
the WWW-Authenticate header field (see 
<http://tools.ietf.org/html/draft-ietf-oauth-v2-bearer-04#section-2.4.1>).

Best regards, Julian

---

From: Kris Selden <[email-masked]>
Date: Thu, 19 May 2011 10:47:10 -0700

I totally missed the error_description in the WWW-Authenticate header in the bearer spec.  I'm not sure why the human readable error description is not in the response body on a 401 but I assume there is a reason.

Is what you were proposing only apply to error_description when in HTTP headers?

Eran, are you looking for a solution to apply to error_description wherever it appears?

In the oauth spec it appears in a url query string, a url fragment, and the json response body and in the bearer and mac spec it appears in a http header.

If it is possible for general guidance, I would start at the most limited case, the url fragment which is intended to be parsed by a client side script.  Javascript cannot percent decode into a string anything other than UTF-8.

On May 19, 2011, at 2:55 AM, Julian Reschke wrote:

> My comments applied to the proposal of returning error_description in the WWW-Authenticate header field (see <http://tools.ietf.org/html/draft-ietf-oauth-v2-bearer-04#section-2.4.1>).
> 
> Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 19 May 2011 19:52:13 +0200

> I totally missed the error_description in the WWW-Authenticate header in the bearer spec.  I'm not sure why the human readable error description is not in the response body on a 401 but I assume th…

Dunno.

> Is what you were proposing only apply to error_description when in HTTP headers?

Yes.

BR, Julian

---

From: Andrew Wooster <[email-masked]>
Date: Fri, 20 May 2011 20:41:31 -0700

>> Is what you were proposing only apply to error_description when in HTTP headers?

That seems bad. Human readable text isn't something that should be put into an HTTP header.

The MAC spec has the same problem, in section 4.1.

These should be brought in line with the OAuth v2 spec, which specifies the human readable error descriptions going in the body.

-Andrew

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Fri, 20 May 2011 22:42:38 -0700

I'm dropping it from MAC.

EHL

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Andrew Wooster
> Sent: Friday, May 20, 2011 8:42 PM
> To: Julian Reschke
> Cc: Kris Selden; OAuth WG
> Subject: Re: [OAUTH-WG] Language encoding in error_description
> 
> >> I totally missed the error_description in the WWW-Authenticate header in
> the bearer spec.  I'm not sure why the human readable error description is
> not in the response body on a 401 but I assume there is a reason.
> >
> > Dunno.
> >
> These should be brought in line with the OAuth v2 spec, which specifies the
> human readable error descriptions going in the body.
> 
> -Andrew