Draft: [ENH] add new instructions dynamically to the robots.txt file
- [ENH] add new instructions dynamically to the robots.txt file
- Indeed, when I go to the url example/robots.txt I see the new instruction at the bottom of the file, but in the file there is no physical change.
I'm wondering whether it's necessary to physically modify the contents of this file.
Related task: https://avan.tech/item98070-ANSI-robots-txt-in-24x-Do-we-just-block-all
Edited by luci