diff options
| author | Christopher Baines <mail@cbaines.net> | 2025-01-15 11:18:52 +0000 |
|---|---|---|
| committer | Christopher Baines <mail@cbaines.net> | 2025-01-15 11:25:12 +0000 |
| commit | 4b1a0204eb340ea0b773de0d3eed918c3660af87 (patch) | |
| tree | 62c257157a07025d26bb1db352196bbdce21b5d7 | |
| parent | 7cce9ec5ee63e9844a70483f87d4a6750f08a886 (diff) | |
| download | qa-frontpage-4b1a0204eb340ea0b773de0d3eed918c3660af87.tar qa-frontpage-4b1a0204eb340ea0b773de0d3eed918c3660af87.tar.gz | |
Add a robots.txt file
To try and discourage scraping the pages that change often and aren't
worth indexing.
| -rw-r--r-- | guix-qa-frontpage/server.scm | 7 | ||||
| -rw-r--r-- | guix-qa-frontpage/view/util.scm | 4 |
2 files changed, 11 insertions, 0 deletions
diff --git a/guix-qa-frontpage/server.scm b/guix-qa-frontpage/server.scm index dd1a306..ba17cc3 100644 --- a/guix-qa-frontpage/server.scm +++ b/guix-qa-frontpage/server.scm @@ -791,6 +791,13 @@ has no patches or has been closed.") (render-html #:sxml (package-view package-data)))) + (('GET "robots") ; robots.txt + (render-text + "User-agent: * +Disallow: /patches +Disallow: /issue +")) + (('GET "README") (let ((filename (string-append doc-dir "/README.html"))) (if (file-exists? filename) diff --git a/guix-qa-frontpage/view/util.scm b/guix-qa-frontpage/view/util.scm index 1b0d945..575ab3c 100644 --- a/guix-qa-frontpage/view/util.scm +++ b/guix-qa-frontpage/view/util.scm @@ -46,6 +46,7 @@ render-html render-json + render-text general-not-found error-page @@ -421,6 +422,9 @@ main > header { (define render-json guix-data-service:render-json) +(define render-text + guix-data-service:render-text) + (define (general-not-found header-text body) (layout #:body |