Prevent Googlebot from rendering badges
What does this MR do and why?
Googlebot somehow indexed a badge alongside a page title: #1705 (closed)
This is not a widespread issue, as we've only seen it on this one page, but it's definitely not desirable.
This MR moves badge code to its own file, and blocks that file from crawlers in robots.txt. Without the badges.js file, there is no badge text alongside headers, only empty <span>
tags.
This technique should not have negative SEO effects as long as the rest of the page still loads for the bot: https://www.searchenginejournal.com/google-blocking-external-resources/357974/
Screenshots, screen recordings, or links to review app
No visual changes expected, unless you're a crawler that reads robots.txt (like Googlebot).
How to set up and validate locally
This one isn't really possible to test whether or not it fixes the bug, but we can validate a few things:
-
Configure a local GitLab Docs environment: https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/doc/setup.md. -
Compile and view the site; verify badges still render as expected. -
As a way to (sort of) emulate Googlebot: Comment-out the include for badges.js
fromlayouts/default.html
and recompile; verify the page loads fine without the badges script (it will not show badges, but everything else works as normal).
Merge request acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this merge request.
Closes #1705 (closed)