aboutsummaryrefslogtreecommitdiff
path: root/guix/scripts/publish.scm
Commit message (Collapse)AuthorAge
* publish: Remove "Guix-Nar-Compression" header from responses.Ludovic Courtès2018-01-09
| | | | | | | | | | This was harmless but non-compliant and unnecessary. * guix/scripts/publish.scm <top level>: Rename "Guix-Nar-Compression" to "X-Nar-Compression" as should have always been. (render-nar, nar-response-port): Adjust accordingly. (strip-headers): New procedure. (sans-content-length, with-content-length): Use it.
* publish: Publish build logs.Ludovic Courtès2018-01-07
| | | | | | | | * guix/scripts/publish.scm (render-log-file): New procedure. (make-request-handler): Add "log" case. * tests/publish.scm ("/log/NAME") ("/log/NAME not found"): New tests. * doc/guix.texi (Invoking guix publish): Document /log URLs.
* publish: Use 'x-raw-file' internal response header.Ludovic Courtès2018-01-07
| | | | | | | | | | | | | This adjusts the workaround for <http://bugs.gnu.org/21093> so that it's not limited to a single content-type. * guix/scripts/publish.scm (render-nar/cached): Add the 'x-raw-file' header on the response. (render-content-addressed-file): Likewise. (with-content-length): Remove the 'x-raw-file' header. (http-write): Instead of dispatching on 'application/octet-stream', check whether 'x-raw-file' is set to determine whether to spawn a thread.
* publish: Save bandwidth on narinfo 404s.Ludovic Courtès2018-01-06
| | | | | | | | This saves 18 bytes on each 404 narinfo response. * guix/scripts/publish.scm (render-narinfo): Pass #:phrase to 'not-found'. (render-narinfo/cached): Likewise.
* publish: Cache uncompressed nars as well.Ludovic Courtès2017-10-05
| | | | | | | | | | | | Fixes <https://bugs.gnu.org/28664>. Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com>. * guix/scripts/publish.scm (bake-narinfo+nar): When COMPRESSION is 'none, write NAR instead of doing nothing. (make-request-handler): Use 'render-nar/cached' for /nar URLs with no compression. * tests/publish.scm ("with cache", "with cache, uncompressed"): Adjust expected result accordingly.
* publish: Make the cache eviction policy less aggressive.Ludovic Courtès2017-07-21
| | | | | | | | Suggested by Mark H Weaver <mhw@netris.org>. * guix/scripts/publish.scm (nar-expiration-time): New procedure. (render-narinfo/cached): Use it as the #:entry-expiration passed to 'maybe-remove-expired-cache-entries'.
* publish: Avoid 'valid-path?' RPC for non-existent items.Ludovic Courtès2017-07-21
| | | | | * guix/scripts/publish.scm (render-narinfo/cached): Call 'file-exists?' before calling 'valid-path?'. This makes the 404 path slightly faster.
* publish: Remove 'regexp-exec' call from the hot path.Ludovic Courtès2017-07-21
| | | | | * guix/scripts/publish.scm (extract-narinfo-hash): Rewrite without resorting to regexps.
* derivations: Introduce 'read-derivation-from-file'.Ludovic Courtès2017-06-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This avoids the open/fstat/close syscalls upon a cache hit that we had with the previous idiom: (call-with-input-file file read-derivation) where caching happened in 'read-derivation' itself. * guix/derivations.scm (%read-derivation): Rename to... (read-derivation): ... this. (read-derivation-from-file): New procedure. (derivation-prerequisites, substitution-oracle) (derivation-prerequisites-to-build): (derivation-path->output-path, derivation-path->output-paths): (derivation-path->base16-hash, map-derivation): Use 'read-derivation-from-file' instead of (call-with-input-file … read-derivation). * guix/grafts.scm (item->deriver): Likewise. * guix/scripts/build.scm (log-url, options->things-to-build): Likewise. * guix/scripts/graph.scm (file->derivation): Remove. (derivation-dependencies, %derivation-node-type): Use 'read-derivation-from-file' instead. * guix/scripts/offload.scm (guix-offload): Likewise. * guix/scripts/perform-download.scm (guix-perform-download): Likewise. * guix/scripts/publish.scm (load-derivation): Remove. (narinfo-string): Use 'read-derivation-from-file'.
* scripts: Set thread names.Ludovic Courtès2017-05-28
| | | | | | | | | | | | | This allows 'guix publish' threads as well as 'guix substitute' and 'guix offload' processes to be properly labeled in 'top', 'pstree', etc. * guix/workers.scm (worker-thunk): Add #:thread-name parameter and honor it. (make-pool): Likewise. * guix/scripts/publish.scm (http-write): Add calls to 'set-thread-name' in bodies of 'call-with-new-thread'. (guix-publish): Call 'set-thread-name'. Pass #:thread-name to 'make-pool'. * guix/scripts/offload.scm (guix-offload): Call 'set-thread-name'. * guix/scripts/substitute.scm (guix-substitute): Likewise.
* publish: Do not bake the same item several times.Ludovic Courtès2017-05-24
| | | | | | | | | | Before this patch, there was a possibility that we'd queue several baking tasks for the same item. The first one would bake the item but subsequent tasks wouldn't check whether the item had been baked in the meantime and would re-bake it. * guix/scripts/publish.scm (render-narinfo/cached): Don't call 'bake-narinfo+nar' when (file-exists? cached).
* publish: Fix narinfo rendering for already-compressed items.Ludovic Courtès2017-05-18
| | | | | | | | | | | Fixes <http://bugs.gnu.org/26975>. Reported by Mark H Weaver <mhw@netris.org>. * guix/scripts/publish.scm (bake-narinfo+nar): Pass #f as the 2nd argument to 'stat' and properly handle #f. * tests/publish.scm (wait-for-file): New procedure. ("with cache"): Remove 'wait-for-file' procedure. ("with cache, uncompressed"): New test.
* publish: Advertise a short TTL for "baking" 404s.Ludovic Courtès2017-05-11
| | | | | | | * guix/scripts/publish.scm (not-found): Add #:phrase and #:ttl parameters and honor them. * tests/publish.scm ("with cache"): Check the 'cache-control' header on of the 404 response.
* ui: Rename '_' to 'G_'.Ludovic Courtès2017-05-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This avoids collisions with '_' when the latter is used as a 'match' pattern for instance. See <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>. * guix/ui.scm: Rename '_' to 'G_'. * po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly. * build-aux/compile-all.scm (warnings): Remove 'format'. * gnu/packages.scm, gnu/services.scm, gnu/services/shepherd.scm, gnu/system.scm, gnu/system/shadow.scm, guix/gnupg.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/elpa.scm, guix/import/pypi.scm, guix/nar.scm, guix/scripts.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/build.scm, guix/scripts/challenge.scm, guix/scripts/container.scm, guix/scripts/container/exec.scm, guix/scripts/copy.scm, guix/scripts/download.scm, guix/scripts/edit.scm, guix/scripts/environment.scm, guix/scripts/gc.scm, guix/scripts/graph.scm, guix/scripts/hash.scm, guix/scripts/import.scm, guix/scripts/import/cpan.scm, guix/scripts/import/cran.scm, guix/scripts/import/crate.scm, guix/scripts/import/elpa.scm, guix/scripts/import/gem.scm, guix/scripts/import/gnu.scm, guix/scripts/import/hackage.scm, guix/scripts/import/nix.scm, guix/scripts/import/pypi.scm, guix/scripts/import/stackage.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/scripts/pack.scm, guix/scripts/package.scm, guix/scripts/perform-download.scm, guix/scripts/publish.scm, guix/scripts/pull.scm, guix/scripts/refresh.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/scripts/system.scm, guix/ssh.scm, guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`".
* publish: Use a larger zlib buffer for compression with '--cache'.Ludovic Courtès2017-05-02
| | | | | * guix/scripts/publish.scm (bake-narinfo+nar): Pass #:buffer-size to 'call-with-gzip-output-port'.
* publish: Produce a "FileSize" narinfo field when possible.Ludovic Courtès2017-05-01
| | | | | | | | | | * guix/scripts/publish.scm (narinfo-string): Add #:file-size parameter. Produce a "FileSize" field when COMPRESSION is eq? to '%no-compression' or when FILE-SIZE is true. (bake-narinfo+nar): Pass #:file-size. * tests/publish.scm ("/*.narinfo") ("/*.narinfo with properly encoded '+' sign") ("with cache"): Check for "FileSize".
* publish: Add a handler for / and /index.html.Ludovic Courtès2017-04-19
| | | | | | | | Suggested by Quiliro <quiliro@riseup.net> in <https://bugs.gnu.org/26567>. * guix/scripts/publish.scm (render-home-page): New procedure. (make-request-handler): Handle it.
* publish: Remove expired cache entries when '--ttl' is used.Ludovic Courtès2017-04-18
| | | | | | | | * guix/scripts/publish.scm (narinfo-files): New procedure. (render-narinfo/cached)[delete-file]: New procedure. Add call to 'maybe-remove-expired-cache-entries'. * doc/guix.texi (Invoking guix publish): Document the interation between --cache and --ttl.
* publish: Add '--cache' and '--workers'.Ludovic Courtès2017-04-18
| | | | | | | | | | | | | | | | | | | | | | | Fixes <http://bugs.gnu.org/26201>. Reported by <dian_cecht@zoho.com>. These options allow nars to be "baked" off-line and cached instead of being compressed on the fly. As a side-effect, this allows us to provide a 'Content-Length' header for nars. * guix/scripts/publish.scm (show-help, %options): Add '--cache' and '--workers'. (%default-options): Add 'workers'. (nar-cache-file, narinfo-cache-file, run-single-baker): New procedures. (single-baker): New macro. (render-narinfo/cached, bake-narinfo+nar) (render-nar/cached): New procedures. (make-request-handler): Add #:cache and #:pool parameters and honor them. (run-publish-server): Likewise. (guix-publish): Honor '--cache' and '--workers'. * tests/publish.scm ("with cache"): New test. * doc/guix.texi (Invoking guix publish): Document it.
* publish: Use 'sendfile' when possible.Ludovic Courtès2017-04-18
| | | | | | * guix/scripts/publish.scm (http-write): In the 'application/octet-stream' case, use 'sendfile' when OUTPUT is a file port.
* publish: Introduce 'actual-compression'.Ludovic Courtès2017-04-18
| | | | | * guix/scripts/publish.scm (actual-compression): New procedure. (narinfo-string): Use it.
* publish: Add '--nar-path'.Ludovic Courtès2017-03-22
| | | | | | * guix/scripts/publish.scm (show-help, %options): Add '--nar-path'. (%default-options): Add 'nar-path'. (guix-publish): Honor it.
* publish: Make the nar URL prefix a parameter.Ludovic Courtès2017-03-22
| | | | | | | | * guix/scripts/publish.scm (narinfo-string): Add #:nar-path and honor it. (render-narinfo): Likewise. (make-request-handler): Likewise. (run-publish-server): Likewise. * tests/publish.scm ("custom nar path"): New test.
* publish: Add '--public-key' and '--private-key'.Ludovic Courtès2017-03-22
| | | | | | * guix/scripts/publish.scm (show-help, %options): Add --public-key and --private-key. * doc/guix.texi (Invoking guix publish): Document it.
* publish: The public and private keys are now SRFI-39 parameters.Ludovic Courtès2017-03-22
| | | | | | | | | | | | * guix/scripts/publish.scm (%default-options): Add 'public-key-file' and 'private-key-file'. (lazy-read-file-sexp): Remove. (%private-key, %public-key): Turn into SRFI-39 parameters. (signed-string, render-narinfo): Adjust accordingly. (guix-publish): Honor 'public-key-file' and 'private-key-file' from OPTS. Use 'parameterize'. * guix/pk-crypto.scm (read-file-sexp): New procedure. * tests/publish.scm: Initialize '%public-key' and '%private-key'.
* publish: Factorize 'content-length' addition.Ludovic Courtès2016-12-04
| | | | | * guix/scripts/publish.scm (with-content-length): New procedure. (http-write) <application/octet-stream>: Use it.
* Use (ice-9 binary-ports) instead of (rnrs io ports).Ludovic Courtès2016-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reduces the closure of (guix ui) from 123 to 106 modules. * guix/derivations.scm: Use (ice-9 binary-ports) instead of (rnrs io ports). (map-derivation)[substitute-file]: Use 'read-string' instead of 'get-string-all'. * guix/ftp-client.scm: Likewise. * guix/hash.scm: Likewise. * guix/http-client.scm: Likewise. * guix/pki.scm (ensure-acl, current-acl): Likewise. * guix/scripts/archive.scm (authorize-key)[read-key]: Likewise. * guix/scripts/authenticate.scm (read-canonical-sexp) (read-hash-data): Likewise. * guix/scripts/download.scm: Likewise. * guix/scripts/offload.scm (register-gc-root, remove-gc-roots) (send-files): Likewise. * guix/scripts/publish.scm (lazy-read-file-sexp): Likewise. * guix/scripts/refresh.scm: Likewise. * guix/scripts/substitute.scm (check-acl-initialized): Likewise. * guix/serialization.scm (read-maybe-utf8-string): Likewise. * guix/scripts/hash.scm (guix-hash): Use 'force-output' instead of 'flush-output-port'. * guix/store.scm (process-stderr): Likewise. * guix/tests.scm: Likewise. * guix/utils.scm: Use (ice-9 binary-ports) and autoload (rnrs io ports) for 'make-custom-binary-input-port'.
* publish: Do not compress already-compressed files.Ludovic Courtès2016-08-01
| | | | | | | * guix/scripts/publish.scm (narinfo-string): Force %NO-COMPRESSION when STORE-PATH matches 'compressed-file?'. * guix/utils.scm (compressed-file?): New procedure. * tests/publish.scm ("/*.narinfo for a compressed file"): New test.
* publish: Handle '/file' URLs, for content-addressed files.Ludovic Courtès2016-07-20
| | | | | | | | | | * guix/scripts/publish.scm (render-content-addressed-file): New procedure. (http-write): Add 'application/octet-stream' case. (make-request-handler): Add /file/NAME/sha256/HASH URLs. * tests/publish.scm ("/file/NAME/sha256/HASH") ("/file/NAME/sha256/INVALID-NIX-BASE32-STRING") ("/file/NAME/sha256/INVALID-HASH"): New tests. * doc/guix.texi (Invoking guix publish): Mention the /file URLs.
* publish: Keep compression disabled when zlib is missing.Ludovic Courtès2016-07-19
| | | | | | | Reported by David Thompson. * guix/scripts/publish.scm (%options)[--compression]: Warn if LEVEL > 0 and zlib is missing, and return RESULT.
* publish: Add '--compression'.Ludovic Courtès2016-07-19
| | | | | | | | | | | | | | | | | | | | | | | * guix/scripts/publish.scm (show-help, %options): Add '--compression'. (<compression>): New record type. (%no-compression, %default-gzip-compression): New variables. (%default-options): Add 'compression' key. (narinfo-string): Add #:compression parameter and honor it. (render-narinfo): Likewise. (render-nar): Likewise. <top level>: Add call to 'declare-header!'. (swallow-zlib-error): New macro. (nar-response-port): New procedure. (http-write): Add call to 'force-output'. Use 'nar-response-port' instead of 'response-port'. Use 'swallow-zlib-error'. (make-request-handler): Add #:compression parameter and honor it. Add "nar/gzip" URL handler. (run-publish-server): Add #:compression parameter and honor it. (guix-publish): Honor --compression. * tests/publish.scm (http-get-port, wait-until-ready): New procedures. <top level>: Run main server with "-C0". Call 'wait-until-ready'. ("/nar/gzip/*", "/*.narinfo with compression"): New tests. * doc/guix.texi (Invoking guix publish): Document it.
* publish: Add '--ttl'.Ludovic Courtès2016-06-09
| | | | | | | | * guix/scripts/publish.scm (show-help, %options): Add --ttl. (render-narinfo): Add #:ttl and honor it. (make-request-handler): Add #:narinfo-ttl and honor it. (run-publish-server): Likewise. (guix-publish): Honor --ttl, pass it to 'run-publish-server'.
* publish: Encore URIs that appear in narinfos.Ludovic Courtès2016-06-08
| | | | | | | | | | Fixes <http://bugs.gnu.org/21888>. Reported by iyzsong@member.fsf.org (宋文武). * guix/scripts/publish.scm (narinfo-string): Use 'encode-and-join-uri-path' instead of 'string-append' to compute URL. * tests/publish.scm ("/*.narinfo with properly encoded '+' sign"): ("/nar/ with properly encoded '+' sign"): New tests.
* store: 'path-info-deriver' is #f when there is no deriver.Ludovic Courtès2016-02-26
| | | | | | | * guix/store.scm (read-path-info): Use #f when we get the empty string for DERIVER. * guix/scripts/publish.scm (narinfo-string): Adjust accordingly. * tests/store.scm ("path-info-deriver"): New test.
* publish: Do not publish nars for invalid store items.Ludovic Courtès2016-02-25
| | | | | | | | | | | Before that, /nar requests could succeed if the requested store item exists but is invalid (although such requests were unlikely because the corresponding narinfo request would have failed.) * guix/scripts/publish.scm (render-nar): Add 'store' parameter. Use 'valid-path?' instead of 'file-exists?'. (make-request-handler): Adjust 'render-nar' call accordingly. * tests/publish.scm ("/nar/invalid"): New test.
* publish: Move 'query-path-info' call where it belongs.Ludovic Courtès2016-02-24
| | | | | | * guix/scripts/publish.scm (narinfo-string): Remove 'path-info' parameter and add 'store' parameter. Call 'query-path-info'. (render-narinfo): Adjust accordingly.
* publish: Fix file descriptor leak.Ludovic Courtès2015-10-06
| | | | | | | | A client closing the connection while reading from a /nar URL would leave an open file descriptor in the server. This patch fixes it. * guix/scripts/publish.scm (swallow-EPIPE): New macro. (http-write): Use it around 'write-file' call.
* Add (guix scripts).Alex Kost2015-09-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * guix/ui.scm: Add missing copyright lines. (args-fold*, environment-build-options, %default-argument-handler, parse-command-line): Move to ... * guix/scripts.scm: ...here. New file. * guix/scripts/archive.scm: Use it. * guix/scripts/build.scm: Likewise. * guix/scripts/download.scm: Likewise. * guix/scripts/edit.scm: Likewise. * guix/scripts/environment.scm: Likewise. * guix/scripts/gc.scm: Likewise. * guix/scripts/graph.scm: Likewise. * guix/scripts/hash.scm: Likewise. * guix/scripts/import/cpan.scm: Likewise. * guix/scripts/import/cran.scm: Likewise. * guix/scripts/import/elpa.scm: Likewise. * guix/scripts/import/gem.scm: Likewise. * guix/scripts/import/gnu.scm: Likewise. * guix/scripts/import/hackage.scm: Likewise. * guix/scripts/import/nix.scm: Likewise. * guix/scripts/import/pypi.scm: Likewise. * guix/scripts/lint.scm: Likewise. * guix/scripts/package.scm: Likewise. * guix/scripts/publish.scm: Likewise. * guix/scripts/pull.scm: Likewise. * guix/scripts/refresh.scm: Likewise. * guix/scripts/size.scm: Likewise. * guix/scripts/system.scm: Likewise. * tests/ui.scm (with-environment-variable, "parse-command-line", "parse-command-line and --no options"): Move to ... * tests/scripts.scm: ...here. New file. * Makefile.am (MODULES): Add guix/scripts.scm. (SCM_TESTS): Add tests/scripts.scm. * po/guix/POTFILES.in: Add guix/scripts.scm.
* publish: Gracefully handle the lack of a deriver.Ludovic Courtès2015-09-04
| | | | | | | | * guix/scripts/publish.scm (narinfo-string): Catch 'system-error' around 'load-derivation' call; return BASE-INFO upon ENOENT. This allows us to return the narinfo even if DERIVER is missing. Before that, the exception would be uncaught, leading to 500 Internal Error on the client side.
* Fix typos in translatable strings.Ludovic Courtès2015-07-23
| | | | | | | | | Reported by Anders Jonsson <anders.jonsson@norsjovallen.se>. * gnu/packages/backup.scm, gnu/packages/databases.scm, gnu/packages/linux.scm, gnu/packages/perl.scm, gnu/packages/web.scm, guix/scripts/lint.scm, guix/scripts/publish.scm: Fix typos in translatable strings.
* publish: Do not load archive content in memory.Ludovic Courtès2015-07-20
| | | | | | | | | | | | Previously, before replying to a /nar/* request, 'guix publish' would first build up the whole nar into memory (as a consequence of <http://bugs.gnu.org/21093>), which obviously doesn't scale. * guix/scripts/publish.scm (render-nar): Return STORE-PATH instead of a procedure that calls 'write-file'. (sans-content-length): New procedure. (http-write): For 'x-nix-archive', don't call '%http-write'. Instead, call 'write-file' right from here, using BODY as the file name.
* publish: Serve /nar requests in a separate thread.Ludovic Courtès2015-07-19
| | | | | | | * guix/scripts/publish.scm (%http-write): New variable. (http-write): New procedure. (concurrent-http-server): New variable. (run-publish-server): Use it.
* publish: Write hashes in nix-base32 format.Ludovic Courtès2015-07-18
| | | | | * guix/scripts/publish.scm (narinfo-string): Use 'bytevector->nix-base32-string', not 'bytevector->base32-string'.
* publish: Add '--listen'.Ludovic Courtès2015-05-12
| | | | | | | | | | | * guix/scripts/publish.scm (show-help, %options): Add --listen. (getaddrinfo*): New procedure. (%default-options): Add 'address'. (open-server-socket): Replace 'addr' and 'port' with 'address', a sockaddr. (guix-publish): Adjust accordingly. Augment "publishing" message with the actual address. * doc/guix.texi (Invoking guix publish): Document it.
* publish: Add '--user' option.Ludovic Courtès2015-05-12
| | | | | | | | | | | | * guix/scripts/publish.scm (show-help): Add --user. (%options): Likewise. (run-publish-server): Change 'port' parameter to 'socket'. Pass #:socket instead of #:addr and #:port to 'run-server'. Update caller accordingly. (open-server-socket, gather-user-privileges): New procedures. (guix-publish): Use them. Force %PRIVATE-KEY and %PUBLIC-KEY early on. Warn when running as root. * doc/guix.texi (Invoking guix publish): Document --user.
* scripts: Add 'publish' command.David Thompson2015-04-04
* guix/scripts/publish.scm: New file. * po/guix/POTFILES.in: Add it. * tests/publish.scm: New file. * Makefile.am (MODULES): Add script module. (SCM_TESTS): Add test module. * doc/guix.texi ("Invoking guix publish"): New node.
generated by cgit v1.2.3 (git 2.45.0) at 2024-12-04 01:24:26 +0000