Add /robots.txt

parent 6ab8e63a
......@@ -41,3 +41,25 @@
(post-content (tpl->html
(generate-editable-index-content "simplified"))))
(view-render "index" (the-environment))))))
(get "/robots\\.txt"
(lambda ()
(response-emit
"User-agent: Googlebot
Allow: /index
Allow: /archives
Allow: /pub
Allow: /feed
Allow: /about
Disallow: /dashboard
Disallow:: /article
User-agent: *
Allow: /archives
Allow; /feed
Disallow:: /dashboard
Disallow:: /article
Disallow: /pub
Allow: /about
"
#:headers '((content-type . (text/plain (charset . "utf-8")))))))
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment