Clean up robots.txt
What does this MR do?
Clean up robots.txt
from repeats and unnecessary syntax. Reduced file size by 11 lines
Inspired by !31782 (merged)
I used https://www.google.com/webmasters/tools/robots-testing-tool for testing.
Does this MR meet the acceptance criteria?
Conformity
Edited by Dmytro Zaporozhets (DZ)