Add a robots.txt / sitemap.xml

This best practices gives us a pretty good breakdown of how we can guide search engines more, including linking to a sitemap

https://www.searchenginewatch.com/2019/03/21/robotstxt-best-practice-guide/

https://www.advancedhtml.co.uk/robots-sitemaps.htm/

I suspect that we can gain a lot of power by setting this up. It will also be work looking into how Facebook publishes its "Public preview"