oauth2 refreshing of tokens, default behaviour

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

When a refresh of oauth2 token occurs, existing tokens are invalidated. This can lead to race conditions in multithreaded / multi-process applications. This doesnt follow the practice of other git vendors such as github or bitbucket.

from your documentation https://docs.gitlab.com/ee/api/oauth2.html#authorization-code-flow

To retrieve a new access_token, use the refresh_token parameter. Refresh tokens may be used even after the access_token itself expires. This request:
Invalidates the existing access_token and refresh_token.
Sends new tokens in the response.

This leads to race conditions if you have an application that is making multiple requests to gitlab and any process triggers a refresh it will break any other processes are in flight with the existing token / refresh.

Actual behaviour

application with 2 processes, eg a build system

time process 1 process 2
time 0 Both threads have the same starting info
refresh token aaaa refresh token aaaa
access token bbbb access token bbbb
----- ------ ------
time 1 p1 performs a token refresh request
----- ------ ------
time 2 p2 requests data with old access token and fails
----- ------ ------
time 4 response from gitlab with new tokens
refresh 1111
token 2222

NB the refresh token is also invalid, see the curl statements below

Expected behaviour

The behaviour of oauth2 does not replicate other SCM services github / bitbucket implementing oauth2.

time process 1 process 2
time 0 Both threads have the same starting info
refresh token aaaa refresh token aaaa
access token bbbb access token bbbb
----- ------ ------
time 1 p1 performs a token refresh request
----- ------ ------
time 2 p2 requests data with old access token and succeeds
----- ------ ------
time 4 response from gitlab with new tokens
refresh 1111
token 2222

IE threads with the current access/refresh token are still able to succeed for the rest of their lifetime, even through a refresh has happened.

Real Example

So here is a set of token existing and after a refresh

old token, refresh, expires: 356ebc667372c2ec7aa1a093030ea34f38b032f17a1588ff4ed0558e3edcf671, 878e314c3de4110c4563fdd600640d21c2b47c59f3ba1a745bb95ac10eafe418, 1970-01-01 01:00:00 +0100 BST
new token, refresh, expires: 2b21ed2598828cbadf8a345e29c86a5997d18d8f1c2fc175922f8c588e029849, 5d08b948ad31341f4b36aacb1b88090b4b3a423bb17559f8fd6fdd2f717ab56e, 2022-09-01 13:27:44.975038173 +0100 BST m=+7394.400235734

old access token

curl --header "Authorization: Bearer 356ebc667372c2ec7aa1a093030ea34f38b032f17a1588ff4ed0558e3edcf671" "https://gitlab.com/api/v4/projects"
{"error":"invalid_token","error_description":"Token was revoked. You have to re-authorize from the user."}

new access token

curl --header "Authorization: Bearer 2b21ed2598828cbadf8a345e29c86a5997d18d8f1c2fc175922f8c588e029849" "https://gitlab.com/api/v4/projects"
[{"id":39057173,"description":null,"name":"WDC040-S01","name_with_namespace":"batch214 / WDC040-S01","path":"wdc040-s01","path_with_namespace":"batch2145/wdc04 ....

old refresh token

curl -i -v  -d 'client_id=secret&client_secret=secret&refresh_token=878e314c3de4110c4563fdd600640d21c2b47c59f3ba1a745bb95ac10eafe418&grant_type=refresh_token&redirect_uri=http://localhost:8080/login' -H 'cache-control: no-cache' -H 'content-type: application/x-www-form-urlencoded' -X POST  https://gitlab.com/oauth/token
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 172.65.251.78:443...
* TCP_NODELAY set
* Connected to gitlab.com (172.65.251.78) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=gitlab.com
*  start date: Jul  4 00:00:00 2022 GMT
*  expire date: Oct  2 23:59:59 2022 GMT
*  subjectAltName: host "gitlab.com" matched cert's "gitlab.com"
*  issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55f524fd38c0)
> POST /oauth/token HTTP/2
> Host: gitlab.com
> user-agent: curl/7.68.0
> accept: */*
> cache-control: no-cache
> content-type: application/x-www-form-urlencoded
> content-length: 298
>
* We are completely uploaded and fine
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 400
HTTP/2 400
< date: Thu, 01 Sep 2022 10:30:37 GMT
date: Thu, 01 Sep 2022 10:30:37 GMT
< content-type: application/json; charset=utf-8
content-type: application/json; charset=utf-8
< content-length: 213
content-length: 213
< cache-control: no-store
cache-control: no-store
< content-security-policy: base-uri 'self'; child-src https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://content.googleapis.com https://content-compute.googleapis.com https://content-cloudbilling.googleapis.com https://content-cloudresourcemanager.googleapis.com https://www.googletagmanager.com/ns.html https://gitlab.com/admin/ https://gitlab.com/assets/ https://gitlab.com/-/speedscope/index.html https://gitlab.com/-/sandbox/mermaid https://gitlab.com/assets/ blob: data:; connect-src 'self' https://gitlab.com wss://gitlab.com https://sentry.gitlab.net https://customers.gitlab.com https://snowplow.trx.gitlab.net https://sourcegraph.com; default-src 'self'; font-src 'self'; frame-ancestors 'self'; frame-src 'self' https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://content.googleapis.com https://content-cloudresourcemanager.googleapis.com https://content-compute.googleapis.com https://content-cloudbilling.googleapis.com https://*.codesandbox.io https://customers.gitlab.com; img-src * data: blob:; manifest-src 'self'; media-src 'self' data:; object-src 'none'; report-uri https://sentry.gitlab.net/api/105/security/?sentry_key=a42ea3adc19140d9a6424906e12fba86; script-src 'strict-dynamic' 'self' 'unsafe-inline' 'unsafe-eval' https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://www.recaptcha.net/ https://apis.google.com 'nonce-Wka64T09eanJCYm4ow9oJQ=='; style-src 'self' 'unsafe-inline'; worker-src https://gitlab.com blob: data:; form-action 'self' https: http: http:
content-security-policy: base-uri 'self'; child-src https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://content.googleapis.com https://content-compute.googleapis.com https://content-cloudbilling.googleapis.com https://content-cloudresourcemanager.googleapis.com https://www.googletagmanager.com/ns.html https://gitlab.com/admin/ https://gitlab.com/assets/ https://gitlab.com/-/speedscope/index.html https://gitlab.com/-/sandbox/mermaid https://gitlab.com/assets/ blob: data:; connect-src 'self' https://gitlab.com wss://gitlab.com https://sentry.gitlab.net https://customers.gitlab.com https://snowplow.trx.gitlab.net https://sourcegraph.com; default-src 'self'; font-src 'self'; frame-ancestors 'self'; frame-src 'self' https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://content.googleapis.com https://content-cloudresourcemanager.googleapis.com https://content-compute.googleapis.com https://content-cloudbilling.googleapis.com https://*.codesandbox.io https://customers.gitlab.com; img-src * data: blob:; manifest-src 'self'; media-src 'self' data:; object-src 'none'; report-uri https://sentry.gitlab.net/api/105/security/?sentry_key=a42ea3adc19140d9a6424906e12fba86; script-src 'strict-dynamic' 'self' 'unsafe-inline' 'unsafe-eval' https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://www.recaptcha.net/ https://apis.google.com 'nonce-Wka64T09eanJCYm4ow9oJQ=='; style-src 'self' 'unsafe-inline'; worker-src https://gitlab.com blob: data:; form-action 'self' https: http: http:
< pragma: no-cache
pragma: no-cache
< referrer-policy: strict-origin-when-cross-origin
referrer-policy: strict-origin-when-cross-origin
< vary: Accept, Origin
vary: Accept, Origin
< www-authenticate: Bearer realm="Doorkeeper", error="invalid_grant", error_description="The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client."
www-authenticate: Bearer realm="Doorkeeper", error="invalid_grant", error_description="The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client."
< x-content-type-options: nosniff
x-content-type-options: nosniff
< x-download-options: noopen
x-download-options: noopen
< x-frame-options: SAMEORIGIN
x-frame-options: SAMEORIGIN
< x-permitted-cross-domain-policies: none
x-permitted-cross-domain-policies: none
< x-request-id: 01GBW9MJWTCQNMT4E5F8G5HGE2
x-request-id: 01GBW9MJWTCQNMT4E5F8G5HGE2
< x-runtime: 0.042024
x-runtime: 0.042024
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< gitlab-lb: fe-14-lb-gprd
gitlab-lb: fe-14-lb-gprd
< gitlab-sv: web-gke-us-east1-d
gitlab-sv: web-gke-us-east1-d
< cf-cache-status: DYNAMIC
cf-cache-status: DYNAMIC
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=aV0Os9k%2B1iCQtjPLkz9LGcQ7LNw663JHB%2Bt7U5KiYjCoJGagUdi8ahiBxIJq4CMJo%2FAMlOs43KVe5DqrkVassxc1f8M3LuwSYr3Ax7qaUpmbiccIfZEIE%2Bpl%2BM8%3D"}],"group":"cf-nel","max_age":604800}
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=aV0Os9k%2B1iCQtjPLkz9LGcQ7LNw663JHB%2Bt7U5KiYjCoJGagUdi8ahiBxIJq4CMJo%2FAMlOs43KVe5DqrkVassxc1f8M3LuwSYr3Ax7qaUpmbiccIfZEIE%2Bpl%2BM8%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0.01,"report_to":"cf-nel","max_age":604800}
nel: {"success_fraction":0.01,"report_to":"cf-nel","max_age":604800}
< strict-transport-security: max-age=31536000
strict-transport-security: max-age=31536000
< server: cloudflare
server: cloudflare
< cf-ray: 743d54e44d9edd37-LHR
cf-ray: 743d54e44d9edd37-LHR

<
* Connection #0 to host gitlab.com left intact
{"error":"invalid_grant","error_description":"The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client."}%

new refresh token

 curl -i -v  -d 'client_id=secret&client_secret=secret&refresh_token=5d08b948ad31341f4b36aacb1b88090b4b3a423bb17559f8fd6fdd2f717ab56e&grant_type=refresh_token&redirect_uri=http://localhost:8080/login' -H 'cache-control: no-cache' -H 'content-type: application/x-www-form-urlencoded' -X POST  https://gitlab.com/oauth/token
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 172.65.251.78:443...
* TCP_NODELAY set
* Connected to gitlab.com (172.65.251.78) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=gitlab.com
*  start date: Jul  4 00:00:00 2022 GMT
*  expire date: Oct  2 23:59:59 2022 GMT
*  subjectAltName: host "gitlab.com" matched cert's "gitlab.com"
*  issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x564053b8c8c0)
> POST /oauth/token HTTP/2
> Host: gitlab.com
> user-agent: curl/7.68.0
> accept: */*
> cache-control: no-cache
> content-type: application/x-www-form-urlencoded
> content-length: 298
>
* We are completely uploaded and fine
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 200
HTTP/2 200
< date: Thu, 01 Sep 2022 10:32:11 GMT
date: Thu, 01 Sep 2022 10:32:11 GMT
< content-type: application/json; charset=utf-8
content-type: application/json; charset=utf-8
< content-length: 244
content-length: 244
< cache-control: no-store
cache-control: no-store
< content-security-policy: base-uri 'self'; child-src https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://content.googleapis.com https://content-compute.googleapis.com https://content-cloudbilling.googleapis.com https://content-cloudresourcemanager.googleapis.com https://www.googletagmanager.com/ns.html https://gitlab.com/admin/ https://gitlab.com/assets/ https://gitlab.com/-/speedscope/index.html https://gitlab.com/-/sandbox/mermaid https://gitlab.com/assets/ blob: data:; connect-src 'self' https://gitlab.com wss://gitlab.com https://sentry.gitlab.net https://customers.gitlab.com https://snowplow.trx.gitlab.net https://sourcegraph.com; default-src 'self'; font-src 'self'; form-action 'self' https: http:; frame-ancestors 'self'; frame-src 'self' https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://content.googleapis.com https://content-cloudresourcemanager.googleapis.com https://content-compute.googleapis.com https://content-cloudbilling.googleapis.com https://*.codesandbox.io https://customers.gitlab.com; img-src * data: blob:; manifest-src 'self'; media-src 'self' data:; object-src 'none'; report-uri https://sentry.gitlab.net/api/105/security/?sentry_key=a42ea3adc19140d9a6424906e12fba86; script-src 'strict-dynamic' 'self' 'unsafe-inline' 'unsafe-eval' https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://www.recaptcha.net/ https://apis.google.com 'nonce-pKkHTISY58VJa9cl1+ynlA=='; style-src 'self' 'unsafe-inline'; worker-src https://gitlab.com blob: data:
content-security-policy: base-uri 'self'; child-src https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://content.googleapis.com https://content-compute.googleapis.com https://content-cloudbilling.googleapis.com https://content-cloudresourcemanager.googleapis.com https://www.googletagmanager.com/ns.html https://gitlab.com/admin/ https://gitlab.com/assets/ https://gitlab.com/-/speedscope/index.html https://gitlab.com/-/sandbox/mermaid https://gitlab.com/assets/ blob: data:; connect-src 'self' https://gitlab.com wss://gitlab.com https://sentry.gitlab.net https://customers.gitlab.com https://snowplow.trx.gitlab.net https://sourcegraph.com; default-src 'self'; font-src 'self'; form-action 'self' https: http:; frame-ancestors 'self'; frame-src 'self' https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://content.googleapis.com https://content-cloudresourcemanager.googleapis.com https://content-compute.googleapis.com https://content-cloudbilling.googleapis.com https://*.codesandbox.io https://customers.gitlab.com; img-src * data: blob:; manifest-src 'self'; media-src 'self' data:; object-src 'none'; report-uri https://sentry.gitlab.net/api/105/security/?sentry_key=a42ea3adc19140d9a6424906e12fba86; script-src 'strict-dynamic' 'self' 'unsafe-inline' 'unsafe-eval' https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://www.recaptcha.net/ https://apis.google.com 'nonce-pKkHTISY58VJa9cl1+ynlA=='; style-src 'self' 'unsafe-inline'; worker-src https://gitlab.com blob: data:
< etag: W/"642961d48c73b1c14f438fcb1811a155"
etag: W/"642961d48c73b1c14f438fcb1811a155"
< pragma: no-cache
pragma: no-cache
< referrer-policy: strict-origin-when-cross-origin
referrer-policy: strict-origin-when-cross-origin
< vary: Accept, Origin
vary: Accept, Origin
< x-content-type-options: nosniff
x-content-type-options: nosniff
< x-download-options: noopen
x-download-options: noopen
< x-frame-options: SAMEORIGIN
x-frame-options: SAMEORIGIN
< x-permitted-cross-domain-policies: none
x-permitted-cross-domain-policies: none
< x-request-id: 01GBW9QEMP83S2P13BN230RHVR
x-request-id: 01GBW9QEMP83S2P13BN230RHVR
< x-runtime: 0.089480
x-runtime: 0.089480
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< gitlab-lb: fe-27-lb-gprd
gitlab-lb: fe-27-lb-gprd
< gitlab-sv: web-gke-us-east1-b
gitlab-sv: web-gke-us-east1-b
< cf-cache-status: DYNAMIC
cf-cache-status: DYNAMIC
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=s6lOmEQLzQGSoh69XgRl6bd0d7jBSbMypI2g4D0SX%2FwfokH5Egp5N3y0EuP0250KCoRcLzORBK%2BAGlZpC%2Fn7Ws%2BLo4v%2BVGiCd3ZlutoD79kGTxk6o5YAIj8aGw8%3D"}],"group":"cf-nel","max_age":604800}
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=s6lOmEQLzQGSoh69XgRl6bd0d7jBSbMypI2g4D0SX%2FwfokH5Egp5N3y0EuP0250KCoRcLzORBK%2BAGlZpC%2Fn7Ws%2BLo4v%2BVGiCd3ZlutoD79kGTxk6o5YAIj8aGw8%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0.01,"report_to":"cf-nel","max_age":604800}
nel: {"success_fraction":0.01,"report_to":"cf-nel","max_age":604800}
< strict-transport-security: max-age=31536000
strict-transport-security: max-age=31536000
< server: cloudflare
server: cloudflare
< cf-ray: 743d57305e63dc21-LHR
cf-ray: 743d57305e63dc21-LHR

<
* Connection #0 to host gitlab.com left intact
{"access_token":"5b7f2a5818f4b5bff259c6e9e8b28163e7a36aa05f402a0abc57dad9e40127ed","token_type":"Bearer","expires_in":7200,"refresh_token":"4347948353f67ddd4ea3771cef9f7597bb3269ff47f887ba89a55149ab9d4fc4","scope":"api","created_at":1662028331}
Edited by 🤖 GitLab Bot 🤖