API v4 does not return 401 Auth error when sending incorrect credentials but returns full public resource list instead for several endpoints

Summary

This is a serious regression from the API v3.

When querying API v4 endpoints like /projects and /groups, which can return both authed-user-scoped and public items, the endpoint will return the full public list when sending incorrect authorization credentials instead of returning a 401 Unauthorized response!

This leads to very large paginated result sets with thousands of items and there is no way of detecting that something went wrong on the side of an API client consumer.

The authentication method (OAuth vs. Personal Access Token) does not matter.

API v3 correctly returns a 401 Unauthorized response (see log output below).

Steps to reproduce

1. Run curl -I --header "Authorization: Bearer FALSE" https://gitlab.com/api/v4/projects
2. Note the 200 OK status code and the X-Total-Pages header (see log output below)

What is the current bug behavior?

API v4 endpoint appears to fall back to “Anonymous” access and returns all public items just as if no authorization info was sent.

What is the expected correct behavior?

API v4 endpoint sends a 401 Unauthorized HTTP status code and error.

Relevant logs and/or screenshots

API v4

$ curl -I --header "Authorization: Bearer FALSE" https://gitlab.com/api/v4/projects

HTTP/1.1 200 OK
Server: nginx
Date: Wed, 27 Sep 2017 07:33:54 GMT
Content-Type: application/json
Content-Length: 10668
Cache-Control: max-age=0, private, must-revalidate
Etag: W/"11ab44341ace677846a9bf136b37cea3"
Link: <https://gitlab.com/api/v4/projects?archived=false&membership=false&order_by=created_at&owned=false&page=2&per_page=20&simple=false&sort=desc&starred=false&statistics=false&with_issues_enabled=false&with_merge_requests_enabled=false>; rel="next", <https://gitlab.com/api/v4/projects?archived=false&membership=false&order_by=created_at&owned=false&page=1&per_page=20&simple=false&sort=desc&starred=false&statistics=false&with_issues_enabled=false&with_merge_requests_enabled=false>; rel="first", <https://gitlab.com/api/v4/projects?archived=false&membership=false&order_by=created_at&owned=false&page=15830&per_page=20&simple=false&sort=desc&starred=false&statistics=false&with_issues_enabled=false&with_merge_requests_enabled=false>; rel="last"
Vary: Origin
X-Frame-Options: SAMEORIGIN
X-Next-Page: 2
X-Page: 1
X-Per-Page: 20
X-Prev-Page: 
X-Request-Id: 18334aef-50dc-4ea8-972e-85e6d5207b1a
X-Runtime: 2.111566
X-Total: 316599
X-Total-Pages: 15830
Strict-Transport-Security: max-age=31536000
RateLimit-Limit: 600
RateLimit-Observed: 2
RateLimit-Remaining: 598

API v3

$ curl -I --header "Authorization: Bearer FALSE" https://gitlab.com/api/v3/projects

HTTP/1.1 401 Unauthorized
Server: nginx
Date: Wed, 27 Sep 2017 07:34:16 GMT
Content-Type: application/json
Content-Length: 30
Cache-Control: no-cache
Vary: Origin
X-Frame-Options: SAMEORIGIN
X-Request-Id: d35666a7-a31e-4997-91c2-d17dbe18c0ac
X-Runtime: 0.024779
RateLimit-Limit: 600
RateLimit-Observed: 3
RateLimit-Remaining: 597

Ping @DouweM