Disallow profile route in robots.txt

What does this MR do?

This MR disallows crawlers from indexing the /-/profile route.

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Separation of EE specific content

Related to #272998 (closed)