Subject: [OAUTH-WG] Reevaluating Assumptions (Important!)

From: Eran Hammer-Lahav <[email-masked]>
Date: Mon, 21 Sep 2009 18:41:42 -0700

OAuth has been designed over 2 years ago based on many assumptions that were true at that time. The most significant assumptions were:

* HTTPS cannot be required due to deployment limitations
* OAuth will be mostly used with proprietary APIs
* Many web platforms do not provide access to the wire HTTP request URI (either on the client or server side)

I strongly believe it would be a mistake to continue this work without questioning these assumptions and making sure we are not producing a lesser solution by trying to accommodate them.

The first two assumptions are easy:

* HTTPS cannot be required due to deployment limitations

This still holds true. OAuth security cannot be based solely on using HTTPS due to deployment limitations. However, the protocol must easily support deployments that have HTTPS easily available and not make them waste resources will timestamps, nonces, and other security elements. While PLAINTEXT provides such support, it doesn't go far enough and still requires the presence of such parameters.

* OAuth will be mostly used with proprietary APIs

This has proved to be false. With new open APIs there is a greater need to enable clients to interact with a protected resource that it has not encountered before or one it has been coded explicitly for. What this means is that we need more discovery features built into the core protocol or more required portions to promote interoperability. There should be a minimum that any client facing an OAuth 404 should be able to handle.

The third item:

* Many web platforms do not provide access to the wire HTTP request URI (either on the client or server side)

requires more research but it also holds the key to much of the protocol design, namely:

- The canonicalization of the HTTP request parameter and URI - this was done due to the fact that at the time, many popular web platforms did not provide easy access to the raw HTTP request URI and headers. If this is no longer the case, OAuth can be significantly simplified to remove the need to process the parameters and only treat the request URI.

- The inclusion of multiple parameter transmission methods - this was done due to lack of access to the Authorization header in some clients and server environment. If this is no longer a requirement or if we come to the conclusion that HTTP caching requires that we use the header in all requests, the need to support parameters in places other than the header will also go away.

If we come to the conclusion that the last assumption is either incorrect or irrelevant (the requirement to use HTTP auth headers), we will be able to significantly simplify the protocol which will also accomplish improving interoperability and security.

ACTION ITEM:

We need to conduct a quick survey of web platforms to figure out if they provide access to the RAW HTTP request (client and server), and if they provide access to the HTTP auth headers (client and server). If you know, please reply with the information for the platforms you use.

EHL

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Tue, 29 Sep 2009 09:34:54 -0700

Reminder:

Please help conduct the platform survey. Please answer this about the platforms you use for developing HTTP applications. Does it provide access to:

* the raw HTTP request URI
* HTTP authentication headers (WWW-Authenticate, Authorize)
* the Host HTTP header
* the raw HTTP request entity-body

If we have wide enough support (3 years later) for these items, we can significantly simplify the signature workflow. If no one is going to reply with information about the platforms they use I am going to assume that they can (both client and server side) access all of the above.

Please reply back by 10/6.

EHL

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Eran Hammer-Lahav
> Sent: Monday, September 21, 2009 6:42 PM
> To: [email-masked]
> Subject: [OAUTH-WG] Reevaluating Assumptions (Important!)
> 
> OAuth has been designed over 2 years ago based on many assumptions that
> were true at that time. The most significant assumptions were:
> 
> I strongly believe it would be a mistake to continue this work without
> questioning these assumptions and making sure we are not producing a
> lesser solution by trying to accommodate them.
> 
> This still holds true. OAuth security cannot be based solely on using
> HTTPS due to deployment limitations. However, the protocol must easily
> support deployments that have HTTPS easily available and not make them
> waste resources will timestamps, nonces, and other security elements.
> While PLAINTEXT provides such support, it doesn't go far enough and
> still requires the presence of such parameters.
> 
> This has proved to be false. With new open APIs there is a greater need
> to enable clients to interact with a protected resource that it has not
> encountered before or one it has been coded explicitly for. What this
> means is that we need more discovery features built into the core
> protocol or more required portions to promote interoperability. There
> should be a minimum that any client facing an OAuth 404 should be able
> to handle.
> 
> The third item:
> 
> - The canonicalization of the HTTP request parameter and URI - this was
> done due to the fact that at the time, many popular web platforms did
> not provide easy access to the raw HTTP request URI and headers. If
> this is no longer the case, OAuth can be significantly simplified to
> remove the need to process the parameters and only treat the request
> URI.
> 
> - The inclusion of multiple parameter transmission methods - this was
> done due to lack of access to the Authorization header in some clients
> and server environment. If this is no longer a requirement or if we
> come to the conclusion that HTTP caching requires that we use the
> header in all requests, the need to support parameters in places other
> than the header will also go away.
> 
> If we come to the conclusion that the last assumption is either
> incorrect or irrelevant (the requirement to use HTTP auth headers), we
> will be able to significantly simplify the protocol which will also
> accomplish improving interoperability and security.
> 
> ACTION ITEM:
> 
> We need to conduct a quick survey of web platforms to figure out if
> they provide access to the RAW HTTP request (client and server), and if
> they provide access to the HTTP auth headers (client and server). If
> you know, please reply with the information for the platforms you use.
> 
> EHL
> 
> 
> 
> _______________________________________________
> OAuth mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/oauth

---

From: Brian Eaton <[email-masked]>
Date: Tue, 29 Sep 2009 09:51:33 -0700

> * the raw HTTP request URI

Usually.

> * HTTP authentication headers (WWW-Authenticate, Authorize)

Almost always.  There are a few cases where OAuth signatures have been
used to sign URLs embedded in web pages, but those aren't core use
cases.  All the other use cases I've seen allow HTTP headers.

> * the Host HTTP header

Always.

> * the raw HTTP request entity-body

Rarely.

I don't think OAuth should attempt to provide bullet-proof security
for deployments that can't use https.  HTTP cookies are widely used in
such deployments.  So long as OAuth is as good as or better than HTTP
cookies, we're probably fine.

Cheers,
Brian

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Tue, 29 Sep 2009 09:55:29 -0700

Can you name the platforms this is applicable for? (i.e. PHP5, Java, etc.)

Not sure how the last comment relates to this thread...

EHL

> From: Brian Eaton [mailto:[email-masked]]
> Sent: Tuesday, September 29, 2009 9:52 AM
> To: Eran Hammer-Lahav
> Cc: [email-masked]
> Subject: Re: [OAUTH-WG] Reevaluating Assumptions (Important!)
>

---

From: Brian Eaton <[email-masked]>
Date: Tue, 29 Sep 2009 10:18:08 -0700

The SSL comment relates to your initial e-mail.  You wrote "HTTPS
cannot be required due to deployment limitations", and had some quite
sensible remarks about not basing OAuth security solely on https.  I
just wanted to add that most deployments that don't use https don't
have perfect security *anyway*, and that we shouldn't bend over
backwards trying to give it to them with OAuth.

The other comments are really not platform specific.

Raw HTTP request URI: this is almost always available, but is
sometimes a bit tricky to get ahold of.  For example, a reverse proxy
may rewrite the URL, or various thick frameworks might hide the URL.
I don't know of any platforms where this is out-and-out impossible,
but I do know of several where it is annoying.

Raw entity body: this is actually really hard to capture, on almost
any platform.  The problem is that most frameworks implement some form
of streaming for request bodies, for performance and scalability
reasons.  If one piece of code reads the request body, that might
cause problems for another piece of code down the line that also needs
to read the request body.  Buffering the entire request body is
feasible for small requests, but causes problems for larger ones.

It can be done in some circumstances, but developers need to shuck and
jive to do it.  And it might not be feasible at all in complicated
environments where proxies are used.

Cheers,
Brian

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Tue, 29 Sep 2009 10:26:38 -0700

> Sent: Tuesday, September 29, 2009 10:18 AM

> but I do know of several where it is annoying.

I am looking for ways to simplify the request normalization part which is where 99% of the problems people have are.

At a minimum, OAuth should guarantee the integrity of the request URI (which includes the Host request header) and provide some support for the entity-body.

There are generally three ways to do it:

1. Assume the raw data is available and just define how to put it together (i.e. URI&Host&Body-hash) and sign/hash/etc. that.
2. Assume the raw data is not available and:
  a. Repeat what is being signed, to be compared to the actual request by the server
  b. Normalize what is being signed by using the bits we know are always available

Today we have 2b. I would like to use 1.

EHL

---

From: Blaine Cook <[email-masked]>
Date: Tue, 29 Sep 2009 20:06:04 +0100

> This still holds true. OAuth security cannot be based solely on using HTTPS due to deployment limitations. However, the protocol must easily support deployments that have HTTPS easily available and…

I want to point out that it is not mandatory that deployments with
HTTPS use (or check) timestamps or nonces; it's not even mandatory for
those without HTTPS. If resource usage is a concern, the provider can
(and should) evaluate the security/resource trade-offs involved.

> This has proved to be false. With new open APIs there is a greater need to enable clients to interact with a protected resource that it has not encountered before or one it has been coded explicitl…

An action item here is to collect examples of APIs that rely on OAuth
that are hampered or complicated by the lack of discovery of endpoints
and consumer key negotiation. Examples of APIs that provide discovery
are particularly useful.

> * Many web platforms do not provide access to the wire HTTP request URI (either on the client or server side)

Server side request libraries are often more forgiving here. Ruby's
Net::HTTP library makes it almost impossible to get access to the raw
request that will be sent; it's possible to do some extensive surgery
to gain access to the outgoing request, but it's very seriously not
code that you'd want to put into production.

Higher level HTTP request libraries (e.g., Ruby's open-uri[1]) make it
nearly impossible to get at the outgoing request.

It's also very much not clear how one would intercept the composed
request from, for example, the Apache Java HttpClient library.

b.

[1] http://www.ruby-doc.org/stdlib/libdoc/open-uri/rdoc/

---

From: Hubert Le Van Gong <[email-masked]>
Date: Tue, 29 Sep 2009 23:24:28 +0200

We use Jersey [1] and you can see on the feature list on the right
of the front page that we've added OAuth support :)

All the features you mentioned below are supported in Jersey.

Cheers,
Hubert

[1] https://jersey.dev.java.net/

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Tue, 29 Sep 2009 21:45:01 -0700

> Of Blaine Cook
> Sent: Tuesday, September 29, 2009 12:06 PM
 
> request from, for example, the Apache Java HttpClient library.

Do they give access to the request URI and Host? That is, the information that is going to be included in:

GET /request_uri HTTP/1.1
Host: example.com

Or does Ruby do something stupid and changes the request URI, encode it, etc.?

EHL

---

From: Blaine Cook <[email-masked]>
Date: Wed, 30 Sep 2009 16:06:59 +0100

> Or does Ruby do something stupid and changes the request URI, encode it, etc.?

Well, generally (and this goes for client libs in all languages), a
request will be constructed like so:

===
request myRequest = MakeNewRequest(path, parameters, headers)
httpClient myClient = MakeNewClient(host, port)

result myResult = httpClient.sendRequest(myRequest)
===

or some variation on that, but the underlying libraries almost always
do it that way. The problem is that "parameters" and "headers" is
usually some kind of hash that gets translated into a query string
and/or request body. The translation normally happens with a private
method in the request class or worse in the httpClient; sometimes
(mostly) you can get the request data out of the request object, but
you have no guarantee that the httpClient is going to munge it (which
would be *hell* to debug).

Worse is that wrappers (like open-uri) often take the above code and
simplify it, like so:

===
simpleHttpClient.get(host, port, path, parameters, headers)
===

or even:

===
simpleHttpClient.post("http://example.com/path/", { "parameter" =>
"value" }, { "Accept" => "text/html" })
===

since everything happens internally, you have to add OAuth support to
simpleHttpClient, or choose a different client, in order to use OAuth.
The approach taken by the existing OAuth spec means that developers
can add OAuth support irrespective of the client they're using. It's
worth noting that this has enabled OAuth support in virtually every
major language --- this, despite the fact that I'm not aware of a
single "primary" HTTP client library supporting OAuth "natively"

I'm not sure what the right approach is here, but I do know that while
figuring out the Signature Base String is difficult, it's *much*
easier (and safer) than hacking the internals of your [least]
favourite HTTP client.

b.

---

From: Owen Evans <[email-masked]>
Date: Thu, 1 Oct 2009 08:23:51 +1300

.net provides access to all of the listed components.

O
On 30/09/2009, at 5:34 AM, Eran Hammer-Lahav wrote:

> Reminder:
>
> Please help conduct the platform survey. Please answer this about  
> the platforms you use for developing HTTP applications. Does it  
> provide access to:
>
> If we have wide enough support (3 years later) for these items, we  
> can significantly simplify the signature workflow. If no one is  
> going to reply with information about the platforms they use I am  
> going to assume that they can (both client and server side) access  
> all of the above.
>
>> * Many web platforms do not provide access to the wire HTTP request  
>> URI
>> (either on the client or server side)
>>
>> * Many web platforms do not provide access to the wire HTTP request  
>> URI
>> (either on the client or server side)
>>

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Wed, 30 Sep 2009 13:17:41 -0700

> From: Blaine Cook [mailto:[email-masked]]
> Sent: Wednesday, September 30, 2009 8:07 AM

> easier (and safer) than hacking the internals of your [least]

It made perfect sense to come up with all the different options in the 1.0 spec to enable quick adoption. Not needing native support from major platform was one of the main reasons OAuth got adopted quickly. However, now that we are looking at creating an internet standard with the attention of the entire web community, and a successful momentum, I think we can switch the burden to the platforms to provide native support for OAuth.

To make this work, we need to clearly separate the application-layer parts of the protocol from the transport-layer parts. We already started doing this with the two specs, but we need to take it further. The authentication spec should not include any application-layer dependencies and operate at the same level as HTTP Basic and Digest auth, both of which are supported by pretty much any web platform in their native HTTP clients. Applications should be able to make a simple HTTP request and provide their token credentials just like a username and password.

As for how they obtain a token - this is where the application-layer logic should be, and include the use of client credentials, exchanging usernames for tokens, managing token state, sessions, etc.

I am publishing OAuth 1.0 as an (informational) RFC so that we have a documented pattern for cases where platform limitations prevent adoption. I don't think we need to worry about these limitations right now and work with the assumption that this work will produce native support. What we MUST worry about, is that the architecture of this protocol is compatible with native support in the various layers we expect it to be added in.

EHL