Subject: [OAUTH-WG] Notes on draft-ietf-oauth-introspection-01

From: Thomas Broyer <[email-masked]>
Date: Tue, 02 Dec 2014 16:08:55 +0000

Hi,

Here are some notes about draft-ietf-oauth-introspection-01.
Background: I have implemented and deployed -00 (actually that was some
version of the individual draft, before it got adopted by the WG),
currently with only a couple "clients" (out of 20 or so OAuth 2.0 clients
currently, only a couple expose resources themselves and thus need the
introspection endpoint; we otherwise have many resources exposed by the
same piece of software as the AS so they use internal means of validating
the token without the need for the introspection endpoint).

   resource_id  OPTIONAL.  A service-specific string identifying the
      resource that the token is being used for.  This value allows the

      protected resource to convey to the authorization server the
      context in which the token is being used at the protected
      resource, allowing the authorization server to tailor its response
      accordingly if desired.

I think it should be noted somewhere that it's totally OK for the
introspection endpoint to tailor the response to the resource server making
the request too, independently of whether there's a resource_id or not.
With "tailoring the response" meaning that it could return active:false
even if the token is active but the AS doesn't want the RS to know about it
(because, for example, it knows that the token doesn't grant any scope that
the RS accepts, and therefore couldn't be used at the RS), or limiting the
returned list of scopes to the ones the AS knows the RS handles.

As far as resource_id is concerned, I really think an example would make
things clearer (what kind of value could be used in a real scenario, etc. –
there's been a mail earlier today assuming it would be a URL, which I
assume to mean the URL of the resource that received the token and needs to
introspect it to allow access or not; my interpretation of the draft
initially was that it would rather be identifiers as can be seen for
scopes, or a resource-set ID <
https://tools.ietf.org/html/draft-hardjono-oauth-resource-reg-03> )

   The methods of managing and
   validating these authentication credentials are out of scope of this
   specification, though it is RECOMMENDED that these credentials be
   distinct from those used at an authorization server's token endpoint.

and later in the Security Considerations section:

   The authorization server SHOULD issue credentials to any
   protected resources that need to access the introspection endpoint,
   SHOULD require protected resources to be specifically authorized to
   call the introspection endpoint, and SHOULD NOT allow a single piece
   of software acting as both a client and a protected resource to re-
   use the same credentials between the token endpoint and the
   introspection endpoint.

Could you expand on the RECOMMENDED and SHOULD NOT here?
What would be the problem with using the same credentials? What's the
trade-off?

   The response MAY be cached by the protected resource, and the
   authorization server SHOULD communicate appropriate cache controls
   using applicable HTTP headers.

Reading through https://tools.ietf.org/html/rfc7234 (and
https://tools.ietf.org/html/rfc7231), it's not clear to me how cache
headers would really help, given that the requests to the introspection
endpoint are mostly using the POST method ("optionally" a GET method, and
the Security Considerations section somehow discourages it).
You'd want to check with the HTTPWG but maybe this text should define what
the cache-key would be (it would at least include the token and resource_id
if provided, maybe also the token_type_hint), and that the response SHOULD
NOT have Cache-Control:public or even s-maxage (for the same reason that it
should be protected by authentication).
I'd actually rather say that the RS may cache the response (we're talking
about an "application-level cache" here, not an HTTP cache), and probably
should do it for a small amount of time; and possibly (not sure how well
that would fit here) hint that the AS could very well return an HTTP 429
(Too Many Requests) <https://tools.ietf.org/html/rfc6585> if it somehow
detects that the RS doesn't use a (application-level) cache (e.g. asks many
times for the same token in a very short time frame). This is the kind of
things I could very well add to my implementation later on if we ever see a
very high number of requests on our introspection endpoint (because looking
up a key-value store using the token as key is much faster than validating
the token – our tokens are base64url-encoded JSON structures containing an
ID and a salt, and we store the ID and a hash in our datastore; validating
a token thus involves decoding base64url, parsing JSON and computing a
hash, in addition to looking it up in the datastore and validating "iat"
and "exp").

   If the protected resource uses OAuth 2.0 client credentials to
   authenticate to the introspection endpoint and its credentials are
   invalid, the authorization server responds with an HTTP 400 (Bad
   Request) as described in section 5.2
<https://tools.ietf.org/html/draft-ietf-oauth-introspection-01#section-5.2>
of OAuth 2.0 [RFC6749 <https://tools.ietf.org/html/rfc6749>].

Either I don't understand what "OAuth 2.0 client credentials" actually
means, or that section should mention HTTP 401 (Unauthorized).
(we use HTTP Basic auth FWIW so, per the HTTP spec, we return a 401 for bad
credentials).

   If the protected resource uses an OAuth 2.0 bearer token to authorize
   its call to the introspection endpoint and the token used for
   authorization does not contain sufficient privileges or is otherwise
   invalid for this request, the authorization server responds with an
   HTTP 400 code as described in section 3
<https://tools.ietf.org/html/draft-ietf-oauth-introspection-01#section-3>
of OAuth 2.0 Bearer Token
   Usage [RFC6750 <https://tools.ietf.org/html/rfc6750>].

Same here; unless you use the "Form-Encoded Body Parameter" or "URL Query
Parameter" means of sending a Bearer token, the status code would be a 401.
BTW, if an introspection endpoint MAY support those means of authenticating
a RS, then it should be more clearly stated in the draft that it is allowed
and left at the discretion of the implementation. As an implementer, unless
I'm told that I could use access_token in the request body, I would assume
only the Authorization header is accepted.

---

From: "Richer, Justin P." <[email-masked]>
Date: Tue, 2 Dec 2014 18:53:25 +0000

Thomas, thanks for the review. Responses inline.

On Dec 2, 2014, at 11:08 AM, Thomas Broyer <[email-masked]<mailto:[email-masked]>> wrote:

Hi,

Here are some notes about draft-ietf-oauth-introspection-01.
Background: I have implemented and deployed -00 (actually that was some version of the individual draft, before it got adopted by the WG), currently with only a couple "clients" (out of 20 or so OAuth 2.0 clients currently, only a couple expose resources themselves and thus need the introspection endpoint; we otherwise have many resources exposed by the same piece of software as the AS so they use internal means of validating the token without the need for the introspection endpoint).

I think it should be noted somewhere that it's totally OK for the introspection endpoint to tailor the response to the resource server making the request too, independently of whether there's a resource_id or not. With "tailoring the response" meaning that it could return active:false even if the token is active but the AS doesn't want the RS to know about it (because, for example, it knows that the token doesn't grant any scope that the RS accepts, and therefore couldn't be used at the RS), or limiting the returned list of scopes to the ones the AS knows the RS handles.

This is very true, we should call that out explicitly in the description of the response. Thanks!

As far as resource_id is concerned, I really think an example would make things clearer (what kind of value could be used in a real scenario, etc. – there's been a mail earlier today assuming it would be a URL, which I assume to mean the URL of the resource that received the token and needs to introspect it to allow access or not; my interpretation of the draft initially was that it would rather be identifiers as can be seen for scopes, or a resource-set ID <https://tools.ietf.org/html/draft-hardjono-oauth-resource-reg-03> )

My thought on this was that it would be the URL in many cases, but I want to keep it as a generic string to allow for resource sets or other identifying mechanisms. This is going to have to be something that's agreed on between the AS and RS if it's going to mean anything. I agree on adding it to at least one of the examples.

Could you expand on the RECOMMENDED and SHOULD NOT here?
What would be the problem with using the same credentials? What's the trade-off?

Different credentials for different purposes, and it lets you manage things separately at the server. In other words, you've got one class of thing that *gets* tokens, and one class of thing that *accepts* tokens. The dynamic resource registration draft doesn't presume client credentials at all, since a resource might not (and in many cases is not) also an OAuth client. This draft even uses tokens to authorize its calls to the introspection endpoint, which was suggested as MTI in another thread.

Additionally, and this may be getting unnecessarily colored by our own implementation and deployment of pre-WG drafts: we have it currently implemented such that both are clients (and Ping does something similar with their own method of accomplishing the same thing), and we want to start to keep these classes separate. We've found that developers get confused about whether they're a client or a resource or whatnot as it is. This recommendation helps keep the roles separate logically, though servers are of course free to throw everyone in the same bucket if they so choose.

Reading through https://tools.ietf.org/html/rfc7234 (and https://tools.ietf.org/html/rfc7231), it's not clear to me how cache headers would really help, given that the requests to the introspection endpoint are mostly using the POST method ("optionally" a GET method, and the Security Considerations section somehow discourages it).
You'd want to check with the HTTPWG but maybe this text should define what the cache-key would be (it would at least include the token and resource_id if provided, maybe also the token_type_hint), and that the response SHOULD NOT have Cache-Control:public or even s-maxage (for the same reason that it should be protected by authentication).
I'd actually rather say that the RS may cache the response (we're talking about an "application-level cache" here, not an HTTP cache), and probably should do it for a small amount of time; and possibly (not sure how well that would fit here) hint that the AS could very well return an HTTP 429 (Too Many Requests) <https://tools.ietf.org/html/rfc6585> if it somehow detects that the RS doesn't use a (application-level) cache (e.g. asks many times for the same token in a very short time frame). This is the kind of things I could very well add to my implementation later on if we ever see a very high number of requests on our introspection endpoint (because looking up a key-value store using the token as key is much faster than validating the token – our tokens are base64url-encoded JSON structures containing an ID and a salt, and we store the ID and a hash in our datastore; validating a token thus involves decoding base64url, parsing JSON and computing a hash, in addition to looking it up in the datastore and validating "iat" and "exp").

All that we're really trying to say here is that the protected resource is allowed to cache the response if it wants to, and that the AS could give some hints as to how to do it. I can pull out the HTTP-cache-mechanism language if it's just confusing the matter (which I suspect it is). In one deployment profile I've written of this, we say that the RS can cache the introspection result for up to half the token lifetime, given by the 'exp' claim (which we also require in the profile).

   If the protected resource uses OAuth 2.0 client credentials to
   authenticate to the introspection endpoint and its credentials are
   invalid, the authorization server responds with an HTTP 400 (Bad
   Request) as described in section 5.2<https://tools.ietf.org/html/draft-ietf-oauth-introspection-01#section-5.2> of OAuth 2.0 [RFC6749<https://tools.ietf.org/html/rfc6749>].

Either I don't understand what "OAuth 2.0 client credentials" actually means, or that section should mention HTTP 401 (Unauthorized).
(we use HTTP Basic auth FWIW so, per the HTTP spec, we return a 401 for bad credentials).

   If the protected resource uses an OAuth 2.0 bearer token to authorize
   its call to the introspection endpoint and the token used for
   authorization does not contain sufficient privileges or is otherwise
   invalid for this request, the authorization server responds with an
   HTTP 400 code as described in section 3<https://tools.ietf.org/html/draft-ietf-oauth-introspection-01#section-3> of OAuth 2.0 Bearer Token
   Usage [RFC6750<https://tools.ietf.org/html/rfc6750>].

Same here; unless you use the "Form-Encoded Body Parameter" or "URL Query Parameter" means of sending a Bearer token, the status code would be a 401.
BTW, if an introspection endpoint MAY support those means of authenticating a RS, then it should be more clearly stated in the draft that it is allowed and left at the discretion of the implementation. As an implementer, unless I'm told that I could use access_token in the request body, I would assume only the Authorization header is accepted.

Noted, I'll change these to 401.

Thanks very much!
 -- Justin

---

From: Thomas Broyer <[email-masked]>
Date: Wed, 03 Dec 2014 14:03:49 +0000

On Tue Dec 02 2014 at 19:53:27 Richer, Justin P. <[email-masked]> wrote:

>  On Dec 2, 2014, at 11:08 AM, Thomas Broyer <[email-masked]> wrote:
>
>
>  Different credentials for different purposes, and it lets you manage
> things separately at the server. In other words, you've got one class of
> thing that *gets* tokens, and one class of thing that *accepts* tokens. The
> dynamic resource registration draft doesn't presume client credentials at
> all, since a resource might not (and in many cases is not) also an OAuth
> client. This draft even uses tokens to authorize its calls to the
> introspection endpoint, which was suggested as MTI in another thread.
>
>  Additionally, and this may be getting unnecessarily colored by our own
> implementation and deployment of pre-WG drafts: we have it currently
> implemented such that both are clients (and Ping does something similar
> with their own method of accomplishing the same thing), and we want to
> start to keep these classes separate. We've found that developers get
> confused about whether they're a client or a resource or whatnot as it is.
> This recommendation helps keep the roles separate logically, though servers
> are of course free to throw everyone in the same bucket if they so choose.
>

That explains why you *could* use different credentials, not why you should
do it.

Until June this year, we were issuing distinct credentials for the "client"
and "resource server" parts of applications (what we used to call "service
provider" vs. "data provider"), and people didn't understand what each
"part" meant (and that's knowing that most of them don't currently expose
APIs themselves!)
We thus moved to a single set of credentials that's shared between all the
clients (e.g. "front office" vs. "back office") and resource servers (TBH,
having distinct credentials for the distinct clients could be challenging
in some cases, so that "simplification" was also needed for other reasons).

>  You'd want to check with the HTTPWG but maybe this text should define
> what the cache-key would be (it would at least include the token and
> resource_id if provided, maybe also the token_type_hint), and that the
> response SHOULD NOT have Cache-Control:public or even s-maxage (for the
> same reason that it should be protected by authentication).
>  All that we're really trying to say here is that the protected resource
> is allowed to cache the response if it wants to, and that the AS could give
> some hints as to how to do it. I can pull out the HTTP-cache-mechanism
> language if it's just confusing the matter (which I suspect it is).
>

Yes please.

> In one deployment profile I've written of this, we say that the RS can
> cache the introspection result for up to half the token lifetime, given by
> the 'exp' claim (which we also require in the profile).
>

Caching the response on the RS is a trade-off between accuracy (and thus
security, as you might detect a revoked token long after it's been revoked)
and performance (HTTP roundtrips, putting too much pressure on the AS,
etc.).
This trade-off has to be considered on a per-RS basis, depending on the
kind (and sensitivity) of data being managed by the RS. For very sensitive
data, you'd likely have a very short cache TTL (or no cache at all, apart
from avoiding concurrent requests to the AS for the same token), whereas
for "semi-public" data you'd likely trade security for performance (e.g.
data is likely to be accessed in several –possibly many– requests, and/or
you're using authentication mostly to trace accesses –and possibly charge
for them–, or to check "who" has access, but you don't really care "when"
it has access).
This trade-off should probably be mentioned in the Security Considerations,
but I don't think giving guidance about how long to cache an introspection
response would be useful (could even be harmful as people could just follow
the suggestion without really thinking about the implications).

---

From: "Richer, Justin P." <[email-masked]>
Date: Wed, 3 Dec 2014 15:54:59 +0000

On Dec 3, 2014, at 9:03 AM, Thomas Broyer <[email-masked]<mailto:[email-masked]>> wrote:

On Tue Dec 02 2014 at 19:53:27 Richer, Justin P. <[email-masked]<mailto:[email-masked]>> wrote:
Thomas, thanks for the review. Responses inline.

That explains why you *could* use different credentials, not why you should do it.

Until June this year, we were issuing distinct credentials for the "client" and "resource server" parts of applications (what we used to call "service provider" vs. "data provider"), and people didn't understand what each "part" meant (and that's knowing that most of them don't currently expose APIs themselves!)
We thus moved to a single set of credentials that's shared between all the clients (e.g. "front office" vs. "back office") and resource servers (TBH, having distinct credentials for the distinct clients could be challenging in some cases, so that "simplification" was also needed for other reasons).

OK, that's fair. We can back off the first RECOMMENDED easily enough but I would like to keep some kind of note in the security considerations section. I don't want to necessarily encourage people to conflate clients and protected resources. Can you help with wording this so that it's not too prescriptive?

All that we're really trying to say here is that the protected resource is allowed to cache the response if it wants to, and that the AS could give some hints as to how to do it. I can pull out the HTTP-cache-mechanism language if it's just confusing the matter (which I suspect it is).

Yes please.

Will do, thanks.

In one deployment profile I've written of this, we say that the RS can cache the introspection result for up to half the token lifetime, given by the 'exp' claim (which we also require in the profile).

Caching the response on the RS is a trade-off between accuracy (and thus security, as you might detect a revoked token long after it's been revoked) and performance (HTTP roundtrips, putting too much pressure on the AS, etc.).
This trade-off has to be considered on a per-RS basis, depending on the kind (and sensitivity) of data being managed by the RS. For very sensitive data, you'd likely have a very short cache TTL (or no cache at all, apart from avoiding concurrent requests to the AS for the same token), whereas for "semi-public" data you'd likely trade security for performance (e.g. data is likely to be accessed in several –possibly many– requests, and/or you're using authentication mostly to trace accesses –and possibly charge for them–, or to check "who" has access, but you don't really care "when" it has access).
This trade-off should probably be mentioned in the Security Considerations, but I don't think giving guidance about how long to cache an introspection response would be useful (could even be harmful as people could just follow the suggestion without really thinking about the implications).

Yes, that's exactly the tradeoff. We can move the discussion to the security considerations, and just mention that the response MAY be cached up above. Would you be willing to contribute text for this item as well?

And, thanks for your thorough review. Feedback from real implementors is always best, and it helps ensure a spec doesn't devolve into hand-waving.

 -- Justin