Add noidex tag for stable branches and update robots.txt

Achilleas Pipinellisrequested to merge
noindex into master

For #915 (closed).

We'll need to backport this to the online versions, at least.

Edited by Achilleas Pipinellis

Merge request reports