Fix robots.txt

robots.txt is generated by https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/content/robots.txt.erb.

With the way we deploy our versioned site, we end up with multiple robots.txt:

• https://docs.gitlab.com/robots.txt
• https://docs.gitlab.com/14.7/robots.txt
• https://docs.gitlab.com/14.6/robots.txt
• https://docs.gitlab.com/14.5/robots.txt
• https://docs.gitlab.com/13.12/robots.txt
• https://docs.gitlab.com/12.10/robots.txt

However, only the one under the root domain matters: docs.gitlab.com/robots.txt. So, it doesn't make any sense deploying it under the versions.

We want to exclude the versions from being indexed, and I see we had added this in !288 (diffs), but removed it in !1432 (diffs).

We should bring back !288 (diffs).