Subject: [OAUTH-WG] OAuth 2.0 Introspection RFC Issues

From: Michael Ciarlillo <[email-masked]>
Date: Sun, 1 Nov 2015 13:17:07 -0500

Hello all,

I have a couple of comments/issues with the RFC at
https://tools.ietf.org/html/rfc7662.

According to Section 2.1 (Introspection Request) says that "To prevent
token scanning attacks, the endpoint MUST also require some form of
authorization to access this endpoint..." This might make sense for token
introspection requests only coming from Resource Servers, but to many
implementers it makes sense to allow Clients access to the introspection
endpoint since it would serve the same purpose (token validity checking,
particularly for refresh tokens, but also for access tokens, or identity
tokens in OpenID Connect, or possibly other token types in UMA and other
specs). Unfortunately, this means that public clients should be accounted
for in some way, as refresh tokens don't have an explicit requirement in
the OAuth 2.0 spec that they be issued only to Confidential Clients. As
such, many implementers don't feel like the Introspection spec fits well
for Client consumption because of the "MUST" requirement on authentication,
when public clients won't have a Client Secret to authenticate with.
Further, token scanning attacks would not be mitigated for authenticated
callers (be they Resource Servers or Clients).

An example of a real-world implementer discussion talking about this topic
is one I had with Kévin Chalet:
https://github.com/aspnet-contrib/AspNet.Security.OpenIdConnect.Server/pull/159

We are not the only ones who see this as a problem with the spec, and there
do not seem to be many concrete reasons for this endpoint ONLY being for
Resource Servers themselves. Can some insight into why this is the case?
Can the MUST be changed to a SHOULD with the Security Considerations
section expanded to talk about the issue? If not, is there another spec or
draft out there somewhere for OAuth token validity for which the intent is
explicitly client consumption? Any and all feedback on this would be
greatly appreciated as we develop the ASOS middleware project.

Another (small) comment on the spec: Section 2.1 (Introspection Request)
currently says that requests be HTTP POST method requests, however Section
4 (Security Considerations) mentions Authorization Servers explicitly
disallowing the GET method due to server logs. What is the requirement
here? POST with optional GET or explicitly only allowing POST requests?

Sincerely,
Michael Ciarlillo

---

From: Justin Richer <[email-masked]>
Date: Tue, 3 Nov 2015 01:42:17 +0900

Hi Michael,

Thanks for the comments. First off, the text of an RFC is fixed and cannot be changed. The spec can only be altered with a new document that obsoletes RFC7662, which would have to go through the working group process again from the very beginning.

Authentication is required but we’re open to how it happens. The introspection spec is targeted at resource servers, not clients, since a client can always just *use* the token in order to see if it’s valid or not. Why bother with the extra round trip to check if the token is good before trying it, since trying it will also tell you if the token’s good? The recovery process for a failed token is the same in either state. We initially had this written for either clients or protected resources, but that’s not how people were using it and restricting its focus helped clarify the spec immensely.

ID tokens are a slightly different case as they’re an assertion targeted at the client itself, but in those cases an ID token is really meant to be self-contained as a JWT, and generally has a short validity period. If you’re looking to introspect an old ID token to check an OIDC session status, then you might be better served implementing one of the OIDC session spec components for that purpose.

Also, we didn’t mandate a specific authentication technology. Most people are going to use either client authentication or a specific OAuth token designed for this purpose. The latter of these is technically available for public clients, but then you have the question of if you want to check the status of *that* token too. Still, I think that for any clients (including public ones) the right thing to do is just use the token and see if it works. 

To the other comment, HTTP POST is the only defined verb and we’re silent on the use of other verbs. That said, many HTTP implementation frameworks don’t differentiate between verbs for requests and so a POST with body form parameters and a GET with query parameters would come out the same. We don’t outright prohibit this behavior, but it’s off-label and there are security considerations for its use.

Hope this helps,
 — Justin

> On Nov 2, 2015, at 3:17 AM, Michael Ciarlillo <[email-masked]> wrote:
> 
> Hello all,
> 
> I have a couple of comments/issues with the RFC at https://tools.ietf.org/html/rfc7662 <https://tools.ietf.org/html/rfc7662>.
> 
> According to Section 2.1 (Introspection Request) says that "To prevent token scanning attacks, the endpoint MUST also require some form of authorization to access this endpoint..." This might make sense for token introspection requests only coming from Resource Servers, but to many implementers it makes sense to allow Clients access to the introspection endpoint since it would serve the same purpose (token validity checking, particularly for refresh tokens, but also for access tokens, or identity tokens in OpenID Connect, or possibly other token types in UMA and other specs). Unfortunately, this means that public clients should be accounted for in some way, as refresh tokens don't have an explicit requirement in the OAuth 2.0 spec that they be issued only to Confidential Clients. As such, many implementers don't feel like the Introspection spec fits well for Client consumption because of the "MUST" requirement on authentication, when public clients won't have a Client Secret to authenticate with. Further, token scanning attacks would not be mitigated for authenticated callers (be they Resource Servers or Clients).
> 
> An example of a real-world implementer discussion talking about this topic is one I had with Kévin Chalet: https://github.com/aspnet-contrib/AspNet.Security.OpenIdConnect.Server/pull/159 <https://github.com/aspnet-contrib/AspNet.Security.OpenIdConnect.Server/pull/159>
> 
> We are not the only ones who see this as a problem with the spec, and there do not seem to be many concrete reasons for this endpoint ONLY being for Resource Servers themselves. Can some insight into why this is the case? Can the MUST be changed to a SHOULD with the Security Considerations section expanded to talk about the issue? If not, is there another spec or draft out there somewhere for OAuth token validity for which the intent is explicitly client consumption? Any and all feedback on this would be greatly appreciated as we develop the ASOS middleware project.
> 
> Another (small) comment on the spec: Section 2.1 (Introspection Request) currently says that requests be HTTP POST method requests, however Section 4 (Security Considerations) mentions Authorization Servers explicitly disallowing the GET method due to server logs. What is the requirement here? POST with optional GET or explicitly only allowing POST requests?
> 
> Sincerely,
> Michael Ciarlillo
> _______________________________________________
> OAuth mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/oauth

---

From: William Denniss <[email-masked]>
Date: Mon, 02 Nov 2015 23:58:25 +0000

We had a debate on that MUST at the last IETF, but the spec was too far
along to change. The workaround is to treat the token you are introspecting
as the authentication. With that workaround, the spec is quite usable for
non-confidential clients, even if resource servers were the primary target.

Google currently runs an introspection endpoint of sorts for clients, named
"tokeninfo", and the latest version (v3) is close to the spec (coincidence
- it's a good design pattern). Aimed mostly at debugging use-cases or if
you don't have a JWT decoding library handy.
On Tue, Nov 3, 2015 at 1:43 AM Justin Richer <[email-masked]> wrote:

> Hi Michael,
>
> Thanks for the comments. First off, the text of an RFC is fixed and cannot
> be changed. The spec can only be altered with a new document that obsoletes
> RFC7662, which would have to go through the working group process again
> from the very beginning.
>
> Authentication is required but we’re open to how it happens. The
> introspection spec is targeted at resource servers, not clients, since a
> client can always just *use* the token in order to see if it’s valid or
> not. Why bother with the extra round trip to check if the token is good
> before trying it, since trying it will also tell you if the token’s good?
> The recovery process for a failed token is the same in either state. We
> initially had this written for either clients or protected resources, but
> that’s not how people were using it and restricting its focus helped
> clarify the spec immensely.
>
> ID tokens are a slightly different case as they’re an assertion targeted
> at the client itself, but in those cases an ID token is really meant to be
> self-contained as a JWT, and generally has a short validity period. If
> you’re looking to introspect an old ID token to check an OIDC session
> status, then you might be better served implementing one of the OIDC
> session spec components for that purpose.
>
> Also, we didn’t mandate a specific authentication technology. Most people
> are going to use either client authentication or a specific OAuth token
> designed for this purpose. The latter of these is technically available for
> public clients, but then you have the question of if you want to check the
> status of *that* token too. Still, I think that for any clients (including
> public ones) the right thing to do is just use the token and see if it
> works.
>
> To the other comment, HTTP POST is the only defined verb and we’re silent
> on the use of other verbs. That said, many HTTP implementation frameworks
> don’t differentiate between verbs for requests and so a POST with body form
> parameters and a GET with query parameters would come out the same. We
> don’t outright prohibit this behavior, but it’s off-label and there are
> security considerations for its use.
>
> Hope this helps,
>  — Justin
>
> We are not the only ones who see this as a problem with the spec, and
> there do not seem to be many concrete reasons for this endpoint ONLY being
> for Resource Servers themselves. Can some insight into why this is the
> case? Can the MUST be changed to a SHOULD with the Security Considerations

---

From: William Denniss <[email-masked]>
Date: Tue, 3 Nov 2015 09:45:03 +0900

I advocated a similar position as yours 4 months ago, in the thread
titled [OAUTH-WG]
Token introspection for public clients?
<https://mailarchive.ietf.org/arch/search/?email_list=oauth&gbt=1&index=FFhT6ZOH5kMe06YoKZ1YFsTNTMw>
.

My impression at the time is that the primary use-case of the spec was the
RS one, even though it could support these other use-cases but for that
MUST.  I agree that using the token for authentication as a workaround does
kind of break the point of the MUST, but as it is, the spec defines the
authentication as "The methods of managing and validating these
authentication credentials are out of scope of this specification.", so it
is open to interpretation.

On Tue, Nov 3, 2015 at 9:24 AM, Michael Ciarlillo <
[email-masked]> wrote:

> Thanks for the feedback.
>
> I agree that the spec is not badly designed over all. I just think that
> the MUST is a bit too restrictive, especially with how none of the OAuth
> grants requires authentication to begin with except when a client is
> Confidential or when using Client Credential flow (for obvious reasons).
>
> I don't believe that simply using a token to determine its validity is a
> good course of action. There may very well be unintended side-effects (like
> the issuance of a new refresh/access token when using a refresh token at
> the Token endpoint, or one-time use access tokens). One use case presented
> by Kévin Chalet is when a client might need to determine and what users it
> has been authorized access on behalf of and possibly remove those that are
> no longer valid. Another might be something like determining if an initial
> one-time use access token for registration requests is still active and
> available to use for client registration.
>
> Not having available resources or libraries to validate JWTs or other
> tokens on the client was another use case mentioned in discussions we had.
> I'm glad others have had similar discussions for this scenario.
>
> That being said, the idea of using the token itself for authentication
> intrigues me, though doesn't that inherently break the entire point of
> having the authentication be a MUST in the spec to begin with? How might an
> Authorization Server mitigate inherent security risks with this approach
> for any type of tokens?
> ------------------------------
> From: William Denniss <[email-masked]>
> Sent: ‎11/‎2/‎2015 6:58 PM
> To: Justin Richer <[email-masked]>; Michael Ciarlillo
> <[email-masked]>
> Cc: <[email-masked]> <[email-masked]>
> Subject: Re: [OAUTH-WG] OAuth 2.0 Introspection RFC Issues
>
> Google currently runs an introspection endpoint of sorts for clients,
> named "tokeninfo", and the latest version (v3) is close to the spec
> (coincidence - it's a good design pattern). Aimed mostly at debugging
> use-cases or if you don't have a JWT decoding library handy.
>> Thanks for the comments. First off, the text of an RFC is fixed and
>> cannot be changed. The spec can only be altered with a new document that
>> obsoletes RFC7662, which would have to go through the working group process
>> again from the very beginning.
>>
>> authorization to access this endpoint..." This might make sense for
>> token introspection requests only coming from Resource Servers, but to many
>> An example of a real-world implementer discussion talking about this
>> topic is one I had with Kévin Chalet:

---

From: Michael Ciarlillo <[email-masked]>
Date: Mon, 2 Nov 2015 19:24:17 -0500

Thanks for the feedback.

I agree that the spec is not badly designed over all. I just think that the MUST is a bit too restrictive, especially with how none of the OAuth grants requires authentication to begin with except when a client is Confidential or when using Client Credential flow (for obvious reasons).

I don't believe that simply using a token to determine its validity is a good course of action. There may very well be unintended side-effects (like the issuance of a new refresh/access token when using a refresh token at the Token endpoint, or one-time use access tokens). One use case presented by Kévin Chalet is when a client might need to determine and what users it has been authorized access on behalf of and possibly remove those that are no longer valid. Another might be something like determining if an initial one-time use access token for registration requests is still active and available to use for client registration.

Not having available resources or libraries to validate JWTs or other tokens on the client was another use case mentioned in discussions we had. I'm glad others have had similar discussions for this scenario.

That being said, the idea of using the token itself for authentication intrigues me, though doesn't that inherently break the entire point of having the authentication be a MUST in the spec to begin with? How might an Authorization Server mitigate inherent security risks with this approach for any type of tokens?

-----Original Message-----
From: "William Denniss" <[email-masked]>
Sent: ‎11/‎2/‎2015 6:58 PM
To: "Justin Richer" <[email-masked]>; "Michael Ciarlillo" <[email-masked]>
Cc: "<[email-masked]>" <[email-masked]>
Subject: Re: [OAUTH-WG] OAuth 2.0 Introspection RFC Issues

We had a debate on that MUST at the last IETF, but the spec was too far along to change. The workaround is to treat the token you are introspecting as the authentication. With that workaround, the spec is quite usable for non-confidential clients, even if resource servers were the primary target.

Google currently runs an introspection endpoint of sorts for clients, named "tokeninfo", and the latest version (v3) is close to the spec (coincidence - it's a good design pattern). Aimed mostly at debugging use-cases or if you don't have a JWT decoding library handy. 

Hi Michael,

Hope this helps,
 — Justin

Hello all,

I have a couple of comments/issues with the RFC at https://tools.ietf.org/html/rfc7662.

An example of a real-world implementer discussion talking about this topic is one I had with Kévin Chalet: https://github.com/aspnet-contrib/AspNet.Security.OpenIdConnect.Server/pull/159

Sincerely,
Michael Ciarlillo

---

From: Sergey Beryozkin <[email-masked]>
Date: Wed, 4 Nov 2015 16:01:16 +0000

Hi All

I'm having a discussion with my colleagues on the pros and cons of 
sharing a client_id.

For example, say we have N number of public mobile applications (the 
same application package, an application instance on an individual 
phone), and one approach is for each of these applications to have the 
same client_id.

I've been trying to analyze why it can be bad and the only thing I can 
come up with is that there will be no (easy) way to track which 
application instance actually accessed a given RS.

Can someone please explain what the pros and cons are of having the same 
client_id shared between public client applications.

And what about multiple confidential clients being set up with the same 
id/secret. I suspect it is a bad idea but what is main line why it is a 
bad idea, lets say it is all done in the protected network, no chance of 
the bad clients interfering...

Thanks, Sergey

---

From: Jim Manico <[email-masked]>
Date: Wed, 4 Nov 2015 08:04:44 -0800

> And what about multiple confidential clients being set up with the same id/secret. 

Bad idea. For security when you see one confidential client doing bad things you will need to revoke it individually. If multiple confidential clients have the same client secrets, thats no longer possible.

--
Jim Manico
@Manicode
Secure Coding Education
+1 (808) 652-3805

> On Nov 4, 2015, at 8:01 AM, Sergey Beryozkin <[email-masked]> wrote:
> 
> Hi All
> 
> I'm having a discussion with my colleagues on the pros and cons of sharing a client_id.
> 
> For example, say we have N number of public mobile applications (the same application package, an application instance on an individual phone), and one approach is for each of these applications to have the same client_id.
> 
> I've been trying to analyze why it can be bad and the only thing I can come up with is that there will be no (easy) way to track which application instance actually accessed a given RS.
> 
> Can someone please explain what the pros and cons are of having the same client_id shared between public client applications.
> 
> And what about multiple confidential clients being set up with the same id/secret. I suspect it is a bad idea but what is main line why it is a bad idea, lets say it is all done in the protected network, no chance of the bad clients interfering...
> 
> 
> 
> Thanks, Sergey
>

---

From: John Bradley <[email-masked]>
Date: Thu, 5 Nov 2015 07:20:42 +0900

For a native app you can have one clientID and no secret (same as having one secret for all of them) or you can use dynamic client registration to give each one a separate client_id and secret.

The middle ground is to use PKCE and no client secret.  

The client generates a pkce challenge in the authorization request and then presents the verifier.  You can then use that verifier to place a identifier for that app instance in the refresh/access tokens.   That would allow the RS to differentiate between two clients with the same user on different devices.  
That gets you more or less equivalent security, but binds the client instance to a user/device as part of the authorization flow,  rather than by creating separate client_id for each instance and managing that.

It is up to more of a deployer preference on what way to go.  

I prefer the PKCE route personally,  that requires a real user to be involved in the flow before the AS needs to store state.   In open dynamic client registration you can face a denial of service attack by creating unlimited clients, that is harder to manage.  

One thing that will be coming up at the WG meeting today is the possibility of extending PKCE verification to refresh tokens as well as there current use with code.  That would move the PKCE and client registration security even closer together.

John B.

> On Nov 5, 2015, at 1:01 AM, Sergey Beryozkin <[email-masked]> wrote:
> 
> Hi All
>

---

From: Sergey Beryozkin <[email-masked]>
Date: Thu, 5 Nov 2015 11:09:32 +0000

Hi John, and Jim

Thanks for the feedback
> It is up to more of a deployer preference on what way to go.
We have a code dealing with the PKCE approach available on the server 
side, but I've had a little clue so far :-) about how practical it can 
be, thanks for the explanation, I'll have a deeper look, might ask a 
couple of questions later on...
Thanks, Sergey

---

From: Sergey Beryozkin <[email-masked]>
Date: Thu, 5 Nov 2015 11:29:35 +0000

Hi John
On 05/11/15 11:09, Sergey Beryozkin wrote:
>> For a native app you can have one clientID and no secret (same as
>> having one secret for all of them) or you can use dynamic client
>> registration to give each one a separate client_id and secret.
>>
>> The client generates a pkce challenge in the authorization request and
>> then presents the verifier.  You can then use that verifier to place a
>> identifier for that app instance in the refresh/access tokens.   That
>> would allow the RS to differentiate between two clients with the same
>> user on different devices.
>> That gets you more or less equivalent security, but binds the client
>> instance to a user/device as part of the authorization flow,  rather
>> than by creating separate client_id for each instance and managing that.
I guess you meant 'client_secret' in this last sentence as I was just 
about to ask if client_id was not needed when PKCE was used, but
https://tools.ietf.org/html/rfc7636#page-9 clarified my confusion :-)

Thanks, Sergey
>> I prefer the PKCE route personally,  that requires a real user to be
>> involved in the flow before the AS needs to store state.   In open
>> dynamic client registration you can face a denial of service attack by
>> creating unlimited clients, that is harder to manage.
>>
>> One thing that will be coming up at the WG meeting today is the
>> possibility of extending PKCE verification to refresh tokens as well
>> as there current use with code.  That would move the PKCE and client
>> registration security even closer together.
>>
> Thanks, Sergey
>
>> John B.
>>
>>> phone), and one approach is for each of these applications to have
>>> the same client_id.
>>>
>>> I've been trying to analyze why it can be bad and the only thing I
>>> can come up with is that there will be no (easy) way to track which
>>> Can someone please explain what the pros and cons are of having the
>>> same client_id shared between public client applications.
>>>
>>> And what about multiple confidential clients being set up with the
>>> same id/secret. I suspect it is a bad idea but what is main line why
>>> it is a bad idea, lets say it is all done in the protected network,
>>> no chance of the bad clients interfering...
>>>
>>>
>>>
>>> Thanks, Sergey
>>>

-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/