GitLab's robots.txt file is out of date

Problem

We need to verify entries in https://gitlab.com/robots.txt and potentially fix some paths to reflect the path changes /*/X to /*/-/X (see #215362 (closed)).

For example:

# Before
Disallow: /*/project_members

# After
Disallow: /*/-/project_members

Otherwise, we prevent crawlers from indexing projects named project_members on GitLab.com.

Same for Disallow: /*/settings and some other.

Proposed solution

Verify entries in https://gitlab.com/robots.txt and adjust if needed.