Allow site.robots value in playbook to be read from a file
Currently, the contents of the robots.txt file must be defined in the playbook. The allowable values are "allow" to auto-generate an allow all file, "disallow" to auto-generate a disallow all file, or a custom string to use as the contents of the file. (See https://docs.antora.org/antora/2.3/playbook/site-robots/)
The custom string can be quite lengthy and thus take up a lot of space in the playbook file. This proposal is to allow a local file to be specified as an alternative, in the same way that local files are specified for other keys (such as cache or output directories).
Example:
site:
robots:
path: ./robots-prod.txt
or the path could be implied from a single-line custom string value based on the presence of the file extension:
site:
robots: ./robots-prod.txt
It should also be possible to specify this value via the CLI using the --robots
option.