Consider displaying 404 pages for public projects without authentication
Background
We tried not to expose the existence of private pages projects anywhere. Consider the following scenarios:
- There is a private pages project
groupname.example.io/my-private-project
An attacker can access groupname.example.io/my-private-project
, and we'll redirect to the authentication workflow.
This way attacker can determine that groupname.example.io/my-private-project
exists as a private project because we redirected instead of rendering 404.
To avoid this we always force users through the auth workflow before rendering 404's. We do that even if we know that page belongs to the public project, because consider this:
- There is a public pages project for the namespace, e.g.
groupname.example.io
from https://docs.gitlab.com/ee/user/project/pages/getting_started_part_one.html#gitlab-pages-default-domain-names - There is also a private pages project
groupname.example.io/my-private-project
By visiting the groupname.example.io/non-existing-page
attacker can confirm that there is no groupname.example.io/non-existing-page
private project because he wasn't redirected. So even though groupname.example.io
is a public project, we force users through the auth workflow before showing them 404's.
Problem
These auth workflows are expensive, and it's just overkill for 404's on public pages projects.
It also caused an incident https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7126:
- there is a very popular pages site hosting assets used by some application
- this application very often requests one URL
- URL was deleted
- we forced the app through auth workflow each time
- it got redirected to gitlab.com/sign_in
- app didn't remember any cookies
- we created a new session for each request
- redis which stores sessions consumed more and more memory
Suggestion
Serve 404's for public projects without auth workflow.
But we need to:
- document that the combination of a public groupname project and private child project can expose the latter
- check how many projects on gitlab.com will be affected and warn their owners
See !263 (merged) for context