From 1ea66c3d3f0a33bc3f04d073457b525a70380c37 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Mon, 10 Nov 2008 18:08:09 -0500 Subject: add a robots.txt Yahoo! has been pounding on ikiwiki.cgi again. While I'd prefer ikiwiki to generate sites that avoided robots hitting it in other ways, I'm adding a robots.txt, at least temporarily. --- doc/robots.txt | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 doc/robots.txt diff --git a/doc/robots.txt b/doc/robots.txt new file mode 100644 index 000000000..7be87f9bd --- /dev/null +++ b/doc/robots.txt @@ -0,0 +1,2 @@ +User-Agent: * +Disallow: /ikiwiki.cgi -- cgit v1.2.3