aboutsummaryrefslogtreecommitdiff
path: root/guix/store.scm
Commit message (Collapse)AuthorAge
* store: Add with-store/non-blocking.Christopher Baines2 days
| | | | | | | | | | | | | | | | | | | | | | | For some applications, it's important to establish a non-blocking connection rather than just making the socket non-blocking after the connection is established. This is because there is I/O on the socket that will block during the handshake. I've noticed this blocking during the handshake causing issues in the build coordinator for example. This commit adds a new with-store variant to avoid changing the behaviour of with-store/open-connection to ensure that this change can't break anything that depends on the blocking nature of the socket. * guix/store.scm (open-unix-domain-socket, open-inet-socket): Take #:non-blocking? and use SOCK_NONBLOCK when calling socket if appropriate. (connect-to-daemon, open-connection, call-with-store): Take #:non-blocking? and pass it on. (with-store/non-blocking): New syntax rule. * .dir-locals.el (scheme-mode): Add entry for with-store/non-blocking. Change-Id: I8225762b78448bc1f7b698c8de5d736e13f577bf
* Autoload (gcrypt hash).Ludovic Courtès2024-04-15
| | | | | | | * guix/derivations.scm: Autoload (guix utils) and (gcrypt hash). * guix/git.scm, guix/store.scm: Autoload (gcrypt hash). Change-Id: I6145231d41c61f2d8c36e28f29e91074910bdd15
* guix: Delay loading of (gnutls).Ludovic Courtès2024-04-15
| | | | | | | | | | | | | | (web …) modules pull in (gnutls) indirectly. Arrange to load them lazily, thereby reducing I/O and allocations when GnuTLS is not needed such as when running ‘guix describe’ or ‘guix shell’ on a cache hit. * guix/download.scm: Autoload (web uri). * guix/scripts/describe.scm: Likewise. * guix/store.scm: Likewise. (%default-substitute-urls): Remove ‘resolve-interface’ call and use https URLs unconditionally. Change-Id: Ide470c556a14866e8740966d25821df487a79859
* Switch order of the default substitute servers.Christopher Baines2024-04-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | The aim here is to improve the user experience. There's anecdotal evidence that the network performance for bordeaux is better compared to ci at least for some users, and I don't know of any issues with rate limiting or access restriction for bordeaux compared to ci. It also has IPv6 support. Additionally, bordeaux generally had more substitutes than ci, particularly for aarch64-linux and armhf-linux. This change will offer a very slight speedup for those substitutes that only bordeaux has. Bordeaux has been a default substitute server for nearly 3 years now and I think this change is overdue. I'm also hopeful that we'll be able to build on the testing regarding mirrors for bordeaux, and that'll allow potentially improving the hosting setup (through providing more redundancy) and further improving substitute fetching for users who currently have issues with substitute access. * config-daemon.ac: Switch substitute urls order. * doc/guix.texi: Ditto. * etc/guix-install.sh: Ditto. * gnu/installer/newt/network.scm (wait-service-online): Ditto. * guix/store.scm (%default-substitute-urls): Ditto. Change-Id: I4f6d93ae1fc8b03d80b47b18b5749a51f1fde17b Signed-off-by: Christopher Baines <mail@cbaines.net>
* daemon: Implement ‘substitute-urls’ RPC.Ludovic Courtès2023-12-11
| | | | | | | | | | | | | | * nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump. (WorkerOp): Add ‘wopSubstituteURLs’. * nix/nix-daemon/nix-daemon.cc (performOp): Implement it. * guix/store.scm (%protocol-version): Bump. (operation-id): Add ‘substitute-urls’. (substitute-urls): New procedure. * tests/store.scm ("substitute-urls, default") ("substitute-urls, client-specified URLs") ("substitute-urls, disabled"): New tests. Change-Id: I2c0119500c3a1eecfa5ebf32463ffb0f173161de
* guix: Strip #:use-module lists.Ludovic Courtès2023-03-13
| | | | | | | | | | | | This was obtained by setting up this environment: guix shell -D guix --with-input=guile@3.0.9=guile-next \ --with-commit=guile-next=e2ed33ef0445c867fe56c247054aa67e834861f2 -- make -j5 then adding 'unused-module' to (@@ (guix build compiler) %warnings), building, and checking all the "unused module" warnings and removing those that were definitely unused.
* grafts: Move '%graft?' and related bindings to (guix store).Ludovic Courtès2022-10-22
| | | | | | | | | The goal is to allow (guix grafts) to use (guix gexp) without introducing a cycle between these two modules. * guix/grafts.scm (%graft?, call-without-grafting, without-grafting) (set-grafting, grafting?): Move to... * guix/store.scm: ... here.
* store: Open daemon connections with SOCK_CLOEXEC.Ludovic Courtès2022-09-04
| | | | | | | | | Previously, 'guix shell' for example would leak the socket that's connected to the daemon. * guix/store.scm (open-unix-domain-socket, open-inet-socket): Pass SOCK_CLOEXEC to 'socket'. * tests/guix-shell.sh: Add test.
* store: Remove unused variable and 'socket' call.Ludovic Courtès2022-09-04
| | | | * guix/store.scm (open-inet-socket): Remove unused 'sock' variable.
* store: 'map/accumulate-builds' preserves caches.Ludovic Courtès2022-05-30
| | | | | | | | | | | | | | | | | | Fixes <https://issues.guix.gnu.org/55721>. Starting from the switch to functional object caches ca. 9e5812ac59b01ff011ec0c5b0f437dfe85d6fcc7, we would be losing accumulated caches when aborting to the build handler. This patch fixes that. In particular, by preserving '%reference-cache-id', we avoid redundant 'query-references' RPCs, which accounted for a large part of the extra processing time. * guix/store.scm (build-accumulator): When returning an <unresolved> node, call 'set-store-connection-caches!' before and after to preserve caches. (map/accumulate-builds): Pass STORE as the first argument to the <unresolved> continuation.
* store: Record cache lookups in 'references/cached'.Ludovic Courtès2022-05-29
| | | | | * guix/store.scm (references/cached): Add call to 'record-cache-lookup!'.
* guix: Remove duplicate definitions of dump-port.Ricardo Wurmus2022-05-20
| | | | | | | * guix/store.scm (dump-port): Remove procedure; autoload from (guix build utils) instead. * guix/store/deduplication.scm (dump-port): Remove procedure; unhide dump-port from (guix build utils).
* store: Use a decaying cutoff in 'map/accumulate-builds'.Ludovic Courtès2022-05-18
| | | | | | | | | | | | | This reduces the wall-clock time of: ./pre-inst-env guix system vm gnu/system/examples/desktop.tmpl -n from 2m13s to 53s (the timings depend on which derivations have already been built and are in store; in this case, many were missing). * guix/store.scm (default-cutoff): New variable. (map/accumulate-builds): Use it. Parameterize it in recursive calls to have decaying cutoff.
* store: 'mcached' users can specify a cache ID.Ludovic Courtès2022-05-18
| | | | | | | | | | | | | | | Users of 'mcached' can now specify a cache ID; furthermore, the cache hit rate is automatically recorded for all the caches accessed with 'mcached'. * guix/store.scm (%max-store-connection-caches) (%store-connection-cache-names): New variables. (recorder-for-cache): New procedure. (record-cache-lookup!): Add 'cache-id' parameter and rewrite in terms of 'recorder-for-cache'. (lookup-cached-object): Add 'cache-id' parameter and honor it. (%mcached): Add #:cache parameter and honor it. (mcached): Add '=>' keyword and corresponding clauses.
* deprecation: Refine the old daemon deprecation warning message.Maxim Cournoyer2022-02-08
| | | | | | | | | * guix/deprecation.scm (warn-about-old-daemon): Mention a direct consequence of failing to upgrade the daemon. * guix/store.scm (build-things): Add a comment explaining the rationale to emit a deprecation warning for a daemon older than version 0x163. Suggested-by: Ricardo Wurmus <rekado@elephly.net>
* store: Warn about daemon deprecation.Mathieu Othacehe2022-02-08
| | | | | * guix/deprecation.scm (warn-about-old-daemon): New procedure. * guix/store.scm (build-things): Use it to warn about old daemons.
* store: 'mapm/accumulate-builds' does not compare stores with 'eq?'.Ludovic Courtès2021-11-10
| | | | | | | | | | | | Fixes <https://issues.guix.gnu.org/51732>. Regression introduced in 2015d3f042870860efef10e801b93eacc0742d38. The (eq? store expected-store) comparison doesn't work on this branch where the functional cache, with 'cache-object-mapping', is used extensively, thereby changing the store's object identity. * guix/store.scm (build-accumulator): Compare the socket of STORE and EXPECTED-STORE rather than the store themselves.
* store: 'map/accumulate-builds' handler checks the store received.Ludovic Courtès2021-10-28
| | | | | | | | | | | This is a followup to b19250eec6f92308f237a09a43e8e3e2355345b9, providing a proper fix for <https://issues.guix.gnu.org/46756>. * guix/remote.scm (remote-eval): Revert b19250eec6f92308f237a09a43e8e3e2355345b9. * guix/store.scm (build-accumulator): Turn into a procedure. Call CONTINUE when the store is not eq? to the initial store. (map/accumulate-builds): Adjust accordingly. * tests/store.scm ("map/accumulate-builds and different store"): New test.
* store: 'map/accumulate-builds' processes the whole list in case of cutoff.Ludovic Courtès2021-09-15
| | | | | | | | | | | | | | | | | Fixes <https://issues.guix.gnu.org/50264>. Reported by Lars-Dominik Braun <lars@6xq.net>. This fixes a regression introduced in fa81971cbae85b39183ccf8f51e8d96ac88fb4ac whereby 'map/accumulate-builds' would return REST (the tail of LST) without applying PROC on it. The effect would be that 'lower-inputs' in (guix gexp) would dismiss those elements, leading to derivations with correct builders but only a subset of the inputs they should have had. * guix/store.scm (map/accumulate-builds): Add #:cutoff parameter and remove 'accumulation-cutoff' variable. Call PROC on the elements of REST. * tests/store.scm ("map/accumulate-builds cutoff"): New test.
* store: Add 'map/accumulate-builds' cutoff to address pathological cases.Ludovic Courtès2021-08-10
| | | | | | | | | | | | | | | | | | | | | Fixes <https://bugs.gnu.org/49439>. Reported by Ricardo Wurmus <rekado@elephly.net>. Previously, a command such as: guix environment pigx-scrnaseq could lead to unbounded memory growth and could even fail to complete when some items are missing from the store. This was because 'map/accumulate-builds' callees would keep making .drv build requests that were turned into <unresolved> nodes; in this case, there are often many identical build requests. Stopping accumulation earlier allows us to unlock the situation by proceeding with the first few build requests instead of spinning until we've accumulated all the build requests. * guix/store.scm (map/accumulate-builds): Define 'accumulation-cutoff'. Use a loop when iterating over LST and maintain a counter of unresolved nodes met so far; return when the counter exceeds ACCUMULATION-CUTOFF.
* Start enabling substitutes from bordeaux.guix.gnu.org.Christopher Baines2021-06-18
| | | | | | | | | | | | In addition to substitutes from ci.guix.gnu.org. There are more changes that can be made in the future, but these changes seem like a good start. * config-daemon.ac (guix_substitute_urls): Add https://bordeaux.guix.gnu.org. * guix/scripts/substitute.scm (%default-substitute-urls): Add http://bordeaux.guix.gnu.org. * guix/store.scm (%default-substitute-urls): Add bordeaux.guix.gnu.org. * doc/guix.texi: Adjust accordingly. * doc/contributing.texi: Adjust accordingly.
* store: 'references/cached' now uses a per-session cache.Ludovic Courtès2021-06-08
| | | | | | * guix/store.scm (%reference-cache): Remove. (%reference-cache-id): New variable. (references/cached): Rewrite in terms of it.
* store: Remove 'references/substitutes'.Ludovic Courtès2021-06-08
| | | | | | | | | This procedure lost its only user in commit 710854304b1ab29332edcb76f3de532e0724c197. * guix/store.scm (references/substitutes): Remove. * tests/store.scm ("references/substitutes missing reference info") ("references/substitutes with substitute info"): Remove.
* store: Generalize cache lookup recording.Ludovic Courtès2021-06-08
| | | | | * guix/store.scm (cache-lookup-recorder): New procedure. (record-cache-lookup!): Define in terms of it.
* store: Support dynamic allocation of per-connection caches.Ludovic Courtès2021-06-08
| | | | | | | | | | | | | | | | | * guix/store.scm (<store-connection>)[object-cache]: Remove. [caches]: New field. (open-connection, port->connection): Adjust '%make-store-connection' calls accordingly. (%store-connection-caches, %object-cache-id): New variables. (allocate-store-connection-cache, vector-set) (store-connection-cache, set-store-connection-cache) (set-store-connection-caches!, set-store-connection-cache!): New procedures. (cache-object-mapping): Add #:cache parameter. (set-store-connection-object-cache!): Remove. (lookup-cached-object): Use 'store-connection-cache'. (run-with-store): Use 'store-connection-caches' and 'set-store-connection-caches!'.
* maint: Require Guile 3.0.Ludovic Courtès2021-06-01
| | | | | | | | | | | | * configure.ac: Require Guile 3.0. * doc/guix.texi (Requirements): Adjust accordingly. * gnu/packages/package-management.scm (guile2.2-guix): Remove. * guix/lint.scm (exception-with-kind-and-args?): Remove 'cond-expand'. * guix/scripts/deploy.scm (deploy-machine*): Likewise. * guix/store.scm (call-with-store): Likewise. * guix/swh.scm (http-get*, http-post*): Likewise. * guix/ui.scm (without-compiler-optimizations, guard*) (call-with-error-handling): Likewise.
* store: Export 'connect-to-daemon'.Ludovic Courtès2021-05-08
| | | | * guix/store.scm (connect-to-daemon): Make public. Improve docstring.
* store: 'open-connection' never returns #f.Ludovic Courtès2021-05-08
| | | | | * guix/store.scm (open-connection)[handshake-error]: New procedure. Call it in code paths that would previously return #f.
* store: Micro-optimize object cache lookup.Ludovic Courtès2021-02-23
| | | | | | | | This avoids a closure allocation when 'lookup-cached-object' is called. * guix/store.scm (lookup-cached-object): Avoid optional/keyword arguments and inline. (%mcached): Adjust accordingly.
* store: Object cache profiling shows the number of entries.Ludovic Courtès2021-02-23
| | | | | * guix/store.scm (record-cache-lookup!): Add 'size' variable; keep it up-to-date and display it.
* store: 'store-path-hash-part' really returns false for invalid file names.Ludovic Courtès2021-02-04
| | | | | | | The "store-path-hash-part #f", due to a SRFI-64 bug, was marked as successful even though 'store-path-hash-part' was throwing an exception. * guix/store.scm (store-path-hash-part): Really return #f.
* store: Add 'find-roots' RPC.Ludovic Courtès2021-01-22
| | | | | | | * guix/serialization.scm (read-string-pairs): New procedure. * guix/store.scm (read-arg): Add support for 'string-pairs'. (find-roots): New procedure. * tests/store.scm ("add-indirect-root and find-roots"): New test.
* store: Fix ensure-path docstring.Mathieu Othacehe2020-11-23
| | | | | | Suggested by Ludovic Courtès. * guix/store.scm (ensure-path): Fix the docstring.
* guix: store: Add ensure-path.Mathieu Othacehe2020-11-22
| | | | * guix/store.scm (ensure-path): New procedure.
* store: 'set-build-options' sends LC_MESSAGES, not LC_ALL.Ludovic Courtès2020-09-02
| | | | | * guix/store.scm (set-build-options): Change #:locale default value to (setlocale LC_MESSAGES).
* ssh: 'send-files' displays a progress bar.Ludovic Courtès2020-08-31
| | | | | | | | * guix/store.scm (export-paths): Add #:start, #:progress, and #:finish parameters and honor them. * guix/ssh.scm (prepare-to-send, notify-transfer-progress) (notify-transfer-completion): New procedures. (send-files): Pass #:start, #:progress, and #:finish to 'export-paths'.
* store: 'with-store' returns as many values as its body.Ludovic Courtès2020-08-28
| | | | | | | | | Fixes <https://bugs.gnu.org/42912>. Reported by Ricardo Wurmus <rekado@elephly.net>. * guix/store.scm (call-with-store)[thunk]: Wrap call to PROC in 'call-with-values'. * tests/store.scm ("with-store, multiple values"): New test.
* store: Use '=' instead of 'eqv?'.Ludovic Courtès2020-06-26
| | | | | | | | | Fixes <https://bugs.gnu.org/42047>. Reported by Jan Nieuwenhuizen <janneke@gnu.org>. * guix/store.scm (open-connection): Use '=' instead of 'eqv?'. This works around <https://bugs.gnu.org/42060> while also being more accurate since the arguments are known to be integers.
* store: Use buffered I/O for all protocol writesLars-Dominik Braun2020-06-09
| | | | | | | | | * guix/store.scm (run-gc) Use buffered output port. (export-path) Same. (add-file-tree-to-store) Same. (set-build-options): Same. Add explicit flush. Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* store: 'mapm/accumulate-builds' preserves '%current-target-system'.Ludovic Courtès2020-05-14
| | | | | | | | | | Fixes <https://bugs.gnu.org/41182>. * guix/store.scm (mapm/accumulate-builds): Pass #:system and #:target to 'run-with-store'. * tests/store.scm ("mapm/accumulate-builds, %current-target-system"): New test. * tests/guix-pack.sh: Add 'guix pack -d --target' test.
* Merge branch 'master' into core-updatesMarius Bakke2020-04-08
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: etc/news.scm gnu/local.mk gnu/packages/check.scm gnu/packages/cross-base.scm gnu/packages/gimp.scm gnu/packages/java.scm gnu/packages/mail.scm gnu/packages/sdl.scm gnu/packages/texinfo.scm gnu/packages/tls.scm gnu/packages/version-control.scm
| * store: 'with-store' uses 'with-exception-handler'.Ludovic Courtès2020-04-05
| | | | | | | | | | | | | | | | | | | | This ensures the stack is not unwound before the exception is re-thrown, as was the case since 8ed597f4a261fe188de82cd1f5daed83dba948eb, leading to '&store-protocol-error' being uncaught by 'with-error-handling' in (guix scripts build) & co. * guix/store.scm (call-with-store): Define 'thunk'. Add 'cond-expand' to use 'with-exception-handler' on 'guile-3' and 'catch' otherwise.
| * store: 'with-store' doesn't close the store upon abort.Ludovic Courtès2020-04-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes <https://bugs.gnu.org/40428>. Reported by Marius Bakke <mbakke@fastmail.com> and 白い熊. Regression introduced with the first uses of 'with-build-handler' in commit 62195b9a8fd6846117c5d7698842748300d13e31 and subsequent. * guix/store.scm (call-with-store): Use 'catch #t' instead of 'dynamic-wind'. This ensures STORE remains open when a non-local exit other than an exception occurs, such as an abort to the build handler prompt. * tests/store.scm ("with-build-handler + with-store"): New test.
* | Merge branch 'master' into core-updatesMarius Bakke2020-03-30
|\| | | | | | | | | | | | | | | | | | | | | Conflicts: gnu/packages/admin.scm gnu/packages/commencement.scm gnu/packages/guile.scm gnu/packages/linux.scm gnu/packages/package-management.scm gnu/packages/pulseaudio.scm gnu/packages/web.scm
| * store: Add 'references/cached'.Ludovic Courtès2020-03-29
| | | | | | | | * guix/store.scm (references/cached): New procedure.
| * store: Add 'map/accumulate-builds'.Ludovic Courtès2020-03-29
| | | | | | | | | | | | | | | | * guix/store.scm (<unresolved>): New record type. (build-accumulator, map/accumulate-builds, mapm/accumulate-builds): New procedures. * tests/store.scm ("map/accumulate-builds", "mapm/accumulate-builds"): New tests.
* | Merge branch 'master' into core-updatesMarius Bakke2020-03-27
|\| | | | | | | | | | | | | | | | | | | | | | | Conflicts: gnu/packages/icu4c.scm gnu/packages/man.scm gnu/packages/python-xyz.scm guix/scripts/environment.scm guix/scripts/pack.scm guix/scripts/package.scm guix/scripts/pull.scm guix/store.scm
| * store: Add 'with-build-handler'.Ludovic Courtès2020-03-22
| | | | | | | | | | | | | | * guix/store.scm (current-build-prompt): New variable. (call-with-build-handler, invoke-build-handler): New procedures. (with-build-handler): New macro. * tests/store.scm ("with-build-handler"): New test.
* | Merge branch 'master' into core-updatesMarius Bakke2020-03-18
|\|
| * store: Fix many guix commands failing on some locales.Florian Pelz2020-03-14
| | | | | | | | | | | | | | | | | | | | | | | | Partly fixes <https://bugs.gnu.org/39970>. At least 'guix environment', 'guix install' and 'guix pull' on 'az_AZ.utf8' and 'tr_TR.utf8' were affected. * guix/store.scm (store-path-hash-part): Move base path detection to ... (store-path-base): ... this new exported procedure. (store-path-package-name): Use it instead of locale-dependent regexps. (store-regexp*): Remove.
generated by cgit v1.2.3 (git 2.25.4) at 2024-05-16 05:02:33 +0000