add robots.txt file
Description
Add robots.txt file
Examples
https://v2.postgres.ai/robots.txt
Checklist
-
the MR description has been reviewed -
this MR contains text changes and they have been reviewed OR there are no texts changes -
this MR contains GUI/CLI changes and they have been reviewed OR there are no GUI/CLI changes -
this MR contains API changes and they have been reviewed OR there are no API changes