add a robots.txt

Yahoo! has been pounding on ikiwiki.cgi again. While I'd prefer ikiwiki to
generate sites that avoided robots hitting it in other ways, I'm adding a
robots.txt, at least temporarily.

1 files changed, 2 insertions, 0 deletions

diff --git a/doc/robots.txt b/doc/robots.txt
new file mode 100644
index 000000000..7be87f9bd
--- /dev/null
+++ b/doc/robots.txt
@@ -0,0 +1,2 @@
+User-Agent: *
+Disallow: /ikiwiki.cgi