aboutsummaryrefslogtreecommitdiff
path: root/tests/nar.scm
Commit message (Collapse)AuthorAge
* daemon: Do not deduplicate files smaller than 8 KiB.Ludovic Courtès2021-11-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Files smaller than 8 KiB typically represent ~70% of the entries in /gnu/store/.links but only contribute to ~4% of the space savings afforded by deduplication. Not considering these files for deduplication speeds up file insertion in the store and, more importantly, leaves 'removeUnusedLinks' with fewer entries to traverse, thereby speeding it up proportionally. Partly fixes <https://issues.guix.gnu.org/24937>. * config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK definition. * guix/store/deduplication.scm (%deduplication-minimum-size): New variable. (deduplicate)[loop]: Do not recurse when FILE's size is below %DEDUPLICATION-MINIMUM-SIZE. (dump-port): New procedure. (dump-file/deduplicate)[hash]: Turn into... [dump-and-compute-hash]: ... this thunk. Call 'deduplicate' only when SIZE is greater than %DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'. * nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where st.st_size < deduplicationMinSize. * nix/libstore/local-store.hh (deduplicationMinSize): New declaration. * nix/libstore/optimise-store.cc (deduplicationMinSize): New variable. (LocalStore::optimisePath_): Return when PATH is a symlink or smaller than 'deduplicationMinSize'. * tests/derivations.scm ("identical files are deduplicated"): Produce files bigger than %DEDUPLICATION-MINIMUM-SIZE. * tests/nar.scm ("restore-file-set with directories (signed, valid)"): Likewise. * tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"): New test. ("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than %DEDUPLICATION-MINIMUM-SIZE. * tests/store.scm ("substitute, deduplication"): Likewise.
* nar: Deduplicate files right as they are restored.Ludovic Courtès2020-12-15
| | | | | | | | | | | | | This avoids having to traverse and re-read the files that we have just restored, thereby reducing I/O. * guix/serialization.scm (dump-file): New procedure. (restore-file): Add #:dump-file parameter and honor it. * guix/store/deduplication.scm (tee, dump-file/deduplicate): New procedures. * guix/nar.scm (restore-one-item): Pass #:dump-file to 'restore-file'. (finalize-store-file): Pass #:deduplicate? #f to 'register-items'. * tests/nar.scm <top level>: Call 'setenv' to set "NIX_STORE".
* serialization: 'restore-file' sets canonical timestamp and permissions.Ludovic Courtès2020-12-15
| | | | | | | | | | | * guix/serialization.scm (restore-file): Set the permissions and mtime of FILE. * guix/nar.scm (finalize-store-file): Pass #:reset-timestamps? #f to 'register-items'. * tests/nar.scm (rm-rf): Add 'chmod' calls to ensure files are writable. ("write-file + restore-file with symlinks"): Ensure every file in OUTPUT passes 'canonical-file?'. * tests/guix-archive.sh: Run "chmod -R +w" before "rm -rf".
* serialization: 'fold-archive' notifies about directory processing completion.Ludovic Courtès2020-12-15
| | | | | | | | | * guix/serialization.scm (fold-archive): Call PROC with a 'directory-complete tag when done with a directory. (restore-file): Handle it. * guix/scripts/archive.scm (list-contents): Likewise. * guix/scripts/challenge.scm (archive-contents): Likewise. * tests/nar.scm ("write-file-tree + fold-archive"): Adjust accordingly.
* serialization: Add 'fold-archive'.Ludovic Courtès2019-12-12
| | | | | | | | * guix/serialization.scm (read-contents): Remove. (read-file-type, fold-archive): New procedures. (restore-file): Rewrite in terms of 'fold-archive'. * tests/nar.scm ("write-file-tree + fold-archive") ("write-file-tree + fold-archive, flat file"): New tests.
* serialization: 'restore-file' errors out upon non-convertible file names.Ludovic Courtès2019-01-18
| | | | | | | | | Fixes <https://bugs.gnu.org/33603>. Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com>. * guix/serialization.scm (port-conversion-strategy): New variable. (restore-file): Parameterize it. * tests/nar.scm ("restore-file with non-UTF8 locale"): New test.
* deduplication: Restore directory mtime and permissions after deduplication.Ludovic Courtès2018-11-13
| | | | | | | | | Fixes <https://bugs.gnu.org/33361>. * guix/store/deduplication.scm (replace-with-link): Call 'set-file-time' and 'chmod' after 'rename-file'. * tests/nar.scm ("restore-file-set with directories (signed, valid)"): New test.
* tests: Check file canonicalization for 'restore-file-set'.Ludovic Courtès2018-11-13
| | | | | | * guix/tests.scm (canonical-file?): New procedure. * tests/nar.scm ("restore-file-set (signed, valid)"): Check that every item of FILES matches 'canonical-file?'.
* tests: Remove check for a feature that appeared in Guile 2.0.10.Ludovic Courtès2018-11-13
| | | | * tests/nar.scm: Remove test for 'open-sha256-input-port'.
* Switch to Guile-Gcrypt.Ludovic Courtès2018-09-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This removes (guix hash) and (guix pk-crypto), which now live as part of Guile-Gcrypt (version 0.1.0.) * guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm, tests/hash.scm, tests/pk-crypto.scm: Remove. * configure.ac: Test for Guile-Gcrypt. Remove LIBGCRYPT and LIBGCRYPT_LIBDIR assignments. * m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove. * README: Add Guile-Gcrypt to the dependencies; move libgcrypt as "required unless --disable-daemon". * doc/guix.texi (Requirements): Likewise. * gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm, guix/git.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm, guix/import/gnu.scm, guix/import/hackage.scm, guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm, guix/pki.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/download.scm, guix/scripts/hash.scm, guix/scripts/pack.scm, guix/scripts/publish.scm, guix/scripts/refresh.scm, guix/scripts/substitute.scm, guix/store.scm, guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm, tests/builders.scm, tests/challenge.scm, tests/cpan.scm, tests/crate.scm, tests/derivations.scm, tests/gem.scm, tests/nar.scm, tests/opam.scm, tests/pki.scm, tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm, tests/store.scm, tests/substitute.scm: Adjust imports. * gnu/system/vm.scm: Likewise. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (expression->derivation-in-linux-vm)[config]: Remove. (iso9660-image)[config]: Remove. (qemu-image)[config]: Remove. (system-docker-image)[config]: Remove. * guix/scripts/pack.scm: Adjust imports. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (self-contained-tarball)[build]: Call 'make-config.scm' without #:libgcrypt argument. (squashfs-image)[libgcrypt]: Remove. [build]: Call 'make-config.scm' without #:libgcrypt. (docker-image)[config, json]: Remove. [build]: Add GUILE-GCRYPT to the extensions Remove (guix config) from the imported modules. * guix/self.scm (specification->package): Remove "libgcrypt", add "guile-gcrypt". (compiled-guix): Remove #:libgcrypt. [guile-gcrypt]: New variable. [dependencies]: Add it. [*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call. Add #:extensions. [*config*]: Remove #:libgcrypt from 'make-config.scm' call. (%dependency-variables): Remove %libgcrypt. (make-config.scm): Remove #:libgcrypt. * build-aux/build-self.scm (guile-gcrypt): New variable. (make-config.scm): Remove #:libgcrypt. (build-program)[fake-gcrypt-hash]: New variable. Add (gcrypt hash) to the imported modules. Adjust load path assignments. * gnu/packages/package-management.scm (guix)[propagated-inputs]: Add GUILE-GCRYPT. [arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search path.
* serialization: Add 'write-file-tree'.Ludovic Courtès2018-07-19
| | | | | | | | * guix/serialization.scm (write-contents-from-port): New procedure. (write-contents): Write in terms of 'write-contents-from-port'. (filter/sort-directory-entries, write-file-tree): New procedures. (write-file): Rewrite in terms of 'write-file-tree'. * tests/nar.scm ("write-file-tree + restore-file"): New test.
* build: Require Guile >= 2.0.9.Ludovic Courtès2017-03-18
| | | | | | | | | | | | | | | | * configure.ac: Bump requirement to 2.0.9. * doc/guix.texi (Requirements): Adjust accordingly. * README (Requirements): Likewise. * build-aux/download.scm: Remove workaround for <http://bugs.gnu.org/13095>. * guix/build/download.scm: Likewise. (http-fetch)[post-2.0.7?]: Remove. Remove conditional code for not POST-2.0.7?. * guix/http-client.scm: Remove workaround for <http://bugs.gnu.org/13095>. (http-fetch)[post-2.0.7?]: Remove. Remove conditional code for not POST-2.0.7?. * guix/serialization.scm (read-latin1-string): Remove mention of 2.0.9. * tests/nar.scm: Use (ice-9 control). (let/ec): Remove.
* hash: Close or flush sha256 output ports before calling their 'get' procedure.Ludovic Courtès2017-03-08
| | | | | | | | | | Slightly different buffering of custom binary output ports in Guile 2.1.7 would lead tests missing 'close-port' or 'force-output' to fail. * tests/hash.scm ("open-sha256-port, hello"): Add call to 'force-output' before call to 'get'. * tests/nar.scm ("write-file puts file in C locale collation order"): Call 'close-port' before calling 'get-hash'.
* serialization: Add #:select? parameter to 'write-file'.Ludovic Courtès2016-06-12
| | | | | * guix/serialization.scm (write-file): Add #:select? parameter and honor it. * tests/nar.scm ("write-file #:select? + restore-file"): New test.
* build: Add a Guile custom test driver using SRFI-64.Mathieu Lirzin2016-04-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before that '.log' files for scheme tests were fragmented and not included in test-suite.log. This unifies the semantics of SRFI-64 API with Automake test suite. * build-aux/test-driver.scm: New file. * Makefile.am (SCM_LOG_DRIVER, AM_SCM_LOG_DRIVER_FLAGS): New variables. (SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables. (AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0. * test-env.in: Silence guix-daemon. * doc/guix.texi (Running the Test Suite): Describe how to display the detailed results. Bug reports require only 'test-suite.log' file. * tests/base32.scm, tests/build-utils.scm, tests/builders.scm, tests/challenge.scm, tests/cpan.scm, tests/cpio.scm, tests/cran.scm, tests/cve.scm, tests/derivations.scm, tests/elpa.scm, tests/file-systems.scm, tests/gem.scm, tests/gexp.scm, tests/gnu-maintenance.scm, tests/grafts.scm, tests/graph.scm, tests/gremlin.scm, tests/hackage.scm, tests/hash.scm, tests/import-utils.scm, tests/lint.scm, tests/monads.scm, tests/nar.scm, tests/packages.scm, tests/pk-crypto.scm, tests/pki.scm, tests/profiles.scm, tests/publish.scm, tests/pypi.scm, tests/records.scm, tests/scripts-build.scm, tests/scripts.scm, tests/services.scm, tests/sets.scm, tests/size.scm, tests/snix.scm, tests/store.scm, tests/substitute.scm, tests/syscalls.scm, tests/system.scm, tests/ui.scm, tests/union.scm, tests/upstream.scm, tests/utils.scm: Don't exit at the end of test groups. * tests/containers.scm: Likewise. Use 'test-skip' instead of exiting with error code 77.
* tests: Work around Btrfs handling of nlink for empty directories.Ludovic Courtès2015-08-26
| | | | | | | | | Fixes <http://bugs.gnu.org/21280>. Reported by Leo Famulari <leo@famulari.name>. * tests/nar.scm (file-tree-equal?): Use #t as the initial result. Btrfs would set nlink to 1 instead of 2 for an empty directory, thereby leading 'file-tree-equal?' to return #f.
* tests: Move 'file=?' to (guix tests).Ludovic Courtès2015-06-24
| | | | | * tests/nar.scm (file-tree-equal?)[file=?]: Move to... * guix/tests.scm (file=?): ... here. New procedure.
* serialization: Check for EOF and incomplete input conditions.Ludovic Courtès2015-02-07
| | | | | | | | | | | | Fixes <http://bugs.gnu.org/19756>. Reported by <sleep_walker@suse.cz>. * guix/serialization.scm (currently-restored-file): New variable. (get-bytevector-n*): New procedure. (read-int, read-long-long, read-string, read-latin1-string, read-contents): Use it instead of 'get-bytevector-n'. (restore-file): Parameterize 'currently-restored-file' and set it. * tests/nar.scm ("restore-file with incomplete input"): New test.
* tests: When a store item is to be deleted, make it statistically unique.Ludovic Courtès2014-11-04
| | | | | | | | | | | | | | Fixes <http://bugs.gnu.org/18935>. Reported by Mark H Weaver <mhw@netris.org>. Previously, if tests/nar.scm ran concurrently with tests/gexp.scm, the store item "foo" containing the string "Hello, world!" could be deleted (via 'delete-paths') from the tests/nar.scm daemon while the tests/gexp.scm daemon would still consider it live. * tests/nar.scm ("restore-file-set (missing signature)"): Use (random-text) rather than "Hello, world!", to avoid concurrent deletion of store item "foo" used in tests/gexp.scm.
* Break module cycle involving (guix store) and (guix ui).Ludovic Courtès2014-10-09
| | | | | | | | | | | | | | | | | | | | Before, there was a cycle along the lines of: (guix store) -> (guix nar) -> (guix ui) -> (guix store) This caused problems, as discussed at: http://lists.gnu.org/archive/html/guix-devel/2014-10/msg00109.html This patch removes cycles in the (guix ...) modules. * guix/nar.scm (&nar-error, &nar-read-error, dump, write-contents, read-contents, %archive-version-1, write-file, restore-file): Move to... * guix/serialization.scm: ... here. * guix/store.scm: Remove dependency on (guix nar). * guix/scripts/hash.scm, guix/scripts/offload.scm, guix/scripts/substitute-binary.scm, tests/nar.scm, tests/store.scm, tests/substitute-binary.scm: Adjust accordingly.
* Factorize test suite support in (guix tests).Ludovic Courtès2014-08-23
| | | | | | | | | | | | | | | | | | | | * guix/tests.scm: New file. * Makefile.am (noinst_DATA): New variable. (GOBJECTS): Add guix/tests.go. * tests/builders.scm (%store): Use 'open-connection-for-tests' from (guix tests). * tests/derivations.scm: Likewise. * tests/monads.scm: Likewise. * tests/packages.scm: Likewise. * tests/profiles.scm: Likewise. * tests/union.scm: Likewise. * tests/gexp.scm: Likewise. (guile-for-build): Remove. Use (%guile-for-build) instead. * tests/nar.scm (make-random-bytevector, %seed, random-text): Remove. (populate-file): Change 'make-random-bytevector' to 'random-bytevector'. Use (guix tests). * tests/store.scm (%seed, random-text): Remove. Use (guix tests).
* nar: Produce archives with files sorted in C collation order.Ludovic Courtès2014-02-21
| | | | | | | * guix/nar.scm (write-file) <directory>: Pass 'string<?' as the second argument to 'scandir'. * tests/nar.scm ("write-file puts file in C locale collation order"): New test.
* nar: 'write-file' can write to non-file ports.Ludovic Courtès2014-02-21
| | | | | | * guix/nar.scm (write-contents): Use 'sendfile' only when P is a file port. * tests/nar.scm ("write-file supports non-file output ports"): New test.
* nar: Add 'restore-file-set', for use by build hooks.Ludovic Courtès2014-01-24
| | | | | | | | | | | | | | | * guix/nar.scm (&nar-invalid-hash-error, &nar-signature-error): New condition types. (&nar-error): Add 'file' and 'port' fields. (&nar-read-error): Remove 'port' and 'file' fields. (lock-store-file, unlock-store-file, finalize-store-file, temporary-store-directory, restore-file-set): New procedures. * tests/nar.scm (%seed): New variable. (random-text): New procedure. ("restore-file-set (signed, valid)", "restore-file-set (missing signature)", "restore-file-set (corrupt)"): New tests. * po/Makevars (XGETTEXT_OPTIONS): Add '--keyword=message'.nar fixes * po/POTFILES.in: Add guix/nar.scm.
* tests: Fix out-of-source builds.Ludovic Courtès2013-05-11
| | | | | * tests/nar.scm (%test-dir): Compute relative to `pre-inst-env', which is in $top_builddir.
* tests: Remove temporary directory created by nar.scm.Ludovic Courtès2013-04-12
| | | | | * tests/nar.scm ("write-file + restore-file with symlinks"): Add (rm-rf output).
* nar: Add support for symlinks.Ludovic Courtès2013-04-12
| | | | | | | | | | | * guix/nar.scm (write-file): Add case for type `symlink'. (restore-file): Likewise. * tests/nar.scm (random-file-size, make-file-tree, delete-file-tree, with-file-tree, file-tree-equal?, make-random-bytevector, populate-file): New procedures. (%test-dir): New variable. ("write-file + restore-file"): Use `%test-dir' and `file-tree-equal?'. ("write-file + restore-file with symlinks"): New test.
* nar: Implement restoration from Nar.Ludovic Courtès2013-04-08
* guix/nar.scm (&nar-error, &nar-read-error): New condition types. (dump): New procedure. (write-contents)[dump]: Remove. Use the one above instead. (read-contents, write-file, restore-file): New procedures. (%archive-version-1): New variable.
generated by cgit v1.2.3 (git 2.45.0) at 2024-11-16 06:19:33 +0000