summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorLudovic Courtès <ludo@gnu.org>2017-03-20 11:48:23 +0100
committerLudovic Courtès <ludo@gnu.org>2017-03-20 11:48:23 +0100
commitececade69fcc9c4fdfec8d0f980f32f39acbe888 (patch)
tree2827652008b39e3a69d47fb3891269c41805f885
parente5854abdd050ed93c0c8d429d14ab8954a5be602 (diff)
downloadguix-artwork-ececade69fcc9c4fdfec8d0f980f32f39acbe888.tar
guix-artwork-ececade69fcc9c4fdfec8d0f980f32f39acbe888.tar.gz
website: Add post about 'guix pack'.
* website/posts/creating-bundles-with-guix-pack.md: New file.
-rw-r--r--website/posts/creating-bundles-with-guix-pack.md250
1 files changed, 250 insertions, 0 deletions
diff --git a/website/posts/creating-bundles-with-guix-pack.md b/website/posts/creating-bundles-with-guix-pack.md
new file mode 100644
index 0000000..d24ef86
--- /dev/null
+++ b/website/posts/creating-bundles-with-guix-pack.md
@@ -0,0 +1,250 @@
+title: Creating bundles with guix pack
+date: 2017-03-20 14:00
+author: Ludovic Courtès
+tags: pack bundles
+---
+Guix just got a new command,
+[dubbed `guix pack`](https://lists.gnu.org/archive/html/guix-devel/2017-03/msg00322.html),
+which we think many developers will find useful.
+
+Last week we were celebrating the
+[release of GNU Guile 2.2.0](https://www.gnu.org/software/guile/news/gnu-guile-220-released.html),
+the Scheme implementation that powers Guix. This is a major milestone
+and Guile developers naturally wanted to make it easy for users to
+discover all the goodies of 2.2.0 as soon as possible. One of the major
+roadblocks to that, as for any non-trivial piece of software, is
+deployment: because your distro is unlikely to have Guile 2.2.0 packaged
+on Day 1, you have to build it by yourself, which means getting the
+right dependencies installed and then building Guile itself. That’s not
+difficult for a developer, but it’s certainly cumbersome.
+
+Andy Wingo, the driving force behind Guile, thought that it would be
+nice to propose a binary tarball of Guile 2.2.0 on the day of its
+release. Guix had already been providing
+[binary tarballs](https://www.gnu.org/software/guix/manual/html_node/Binary-Installation.html)
+for a couple of years, so why not do the same for Guile? Essentially,
+the new `guix pack` command is a generalization of what Guix was already
+using.
+
+#### Making packs
+
+So how does it work? The basic idea is simple: you type
+
+```
+guix pack guile
+```
+
+and you the command returns in `/gnu/store` a good old tarball that
+contains binaries for Guile and all its dependencies. If you run, say,
+
+```
+guix pack guile emacs geiser
+```
+
+then you get a complete “Guile SDK” containing Guile, Emacs,
+[Geiser](http://nongnu.org/geiser), and all their dependencies.
+
+When you extract the tarball, you get a `/gnu/store` directory with a
+bunch of sub-directories with
+[these long hashes](https://www.gnu.org/software/guix/manual/html_node/Features.html),
+one of which is the “profile” containing Guile, Emacs, and Geiser.
+
+You wouldn’t want to ask users to type
+`/gnu/store/war325pv1iixj13k6y8yplzagpknfn0c-profile/bin/guile` to
+launch Guile, though. So `guix pack` has a command-line option to
+create symlinks in the image.
+
+```
+guix pack -S /opt/gnu/bin=bin guile emacs geiser
+```
+
+The command above creates a `/opt/gnu/bin` symlink to the `bin`
+directory of the profile in the tarball, such that users can simply type
+`/opt/gnu/bin/guile` to run Guile.
+
+Recipients of a binary tarball are expected to either extract it in
+their root file system (yes!) where it will create `/gnu` and `/opt/gnu`
+in this case:
+
+```
+# cd /
+# tar xf /path/to/pack.tar.gz
+# /opt/gnu/bin/guile --version
+guile (GNU Guile) 2.2.0
+```
+
+… or they can chroot into it, possibly relying on
+[user namespaces](http://man7.org/linux/man-pages/man7/user_namespaces.7.html)
+and thereby avoiding root privileges:
+
+```
+$ mkdir /tmp/pack
+$ cd /tmp/pack
+$ tar xf /path/to/pack.tar.gz
+$ unshare -mrf chroot . /opt/gnu/bin/guile --version
+guile (GNU Guile) 2.2.0
+```
+
+The good thing with this is that, because Guix captures the _complete
+dependency graph_ of packages, the tarball contains everything that’s
+needed to run Guile and is going to work in exactly the same way on any
+system that runs the kernel Linux!
+
+#### Bells and whistles
+
+Of course a popular approach to run such “application bundles” is
+[Docker](https://www.docker.com). Since the image format for Docker is
+[documented](https://github.com/docker/docker/blob/master/image/spec/v1.2.md)
+and fairly easy to produce, we added an option to produce images in this
+format (Ricardo Wurmus
+[initially contributed Docker support](https://lists.gnu.org/archive/html/guix-devel/2017-01/msg00188.html)
+for the low-level
+[`guix archive`](https://www.gnu.org/software/guix/manual/html_node/Invoking-guix-archive.html)
+tool but we found that it made more sense to have it in `guix pack`):
+
+```
+guix pack -f docker -S /opt/gnu=/ guile emacs geiser
+```
+
+The resulting tarball can be passed to
+[`docker load`](https://docs.docker.com/engine/reference/commandline/load/),
+and people can then use `docker run` to actually run the application.
+
+One of the goodies that comes for free is cross-compilation: Guix
+supports cross-compilation, so you can create a pack consisting of
+software cross-compiled for a given platform, specified by the
+[usual GNU triplet](https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Specifying-Target-Triplets.html).
+For example, the following command creates a pack with binaries for
+GNU/Linux on ARMv7:
+
+```
+guix pack --target=arm-linux-gnueabihf guile
+```
+
+… while the command below creates a pack with binaries for MinGW:
+
+```
+guix pack --target=i686-w64-mingw32 guile
+```
+
+All the
+[package transformation options](https://www.gnu.org/software/guix/manual/html_node/Package-Transformation-Options.html)
+that Guix supports are available to `guix pack`. Let’s say you’re a
+developer of a large piece of software such as a web browser like
+[IceCat](https://gnu.org/s/gnuzilla) and you’d like your users to test
+whether the current master branch actually fixes the bug you attempted
+to fix. In this case, you can build a pack of IceCat, but replace the
+source that’s specified in the distribution with the snapshot of master
+you’re interested in:
+
+```
+guix pack icecat --with-source=./icecat-48.8.0.master.tar.gz
+```
+
+Of course the resulting pack is going to be pretty big in this case, but
+I’m sure the general pattern can be useful.
+
+#### Wait, didn’t you say that “app bundles get it wrong”?
+
+It turns out that we Guix developers
+[have](https://archive.fosdem.org/2016/schedule/event/deployments_with_gnu_guix/)
+[been](https://arxiv.org/abs/1506.02822)
+[saying](https://fosdem.org/2017/schedule/event/hpc_deployment_guix/)
+that binary “application bundles” à la Docker are problematic for a
+number of reasons:
+
+ 1. Composability: each bundle comes with a complete operating system,
+ minus the kernel, and there is little or no sharing happening among
+ bundles, notably in terms of disk space and memory usage.
+ 2. Security updates: since an “app bundle” is essentially a complete
+ operating system, one has to be careful and apply security updates
+ to all the software in each bundle. Unfortunately, that doesn’t
+ always happen as has been
+ [famously](http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html)
+ [reported](https://www.infoq.com/news/2015/05/Docker-Image-Vulnerabilities)
+ on several occasions.
+ 3. Reproducibility: Docker images, for instance, are often hardly
+ “reproducible” in the sense of a
+ [reproducible build process](https://reproducible-builds.org/docs/definition/).
+ First, `Dockerfile`s start out with a “base layer” that is
+ typically a huge binary blob of some major distro. On top of that,
+ they run a number of commands such as `apt-get install` whose
+ result likely depends on the time at which they are run. Docker’s
+ [best practices document](https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/)
+ suggests ways to mitigate the problem, such as “version pinning”,
+ but the whole approach remains rather brittle.
+ 4. Experimentation: Once you have this big binary blob, sure you can
+ run the application you wanted, but you can do little more than
+ that—you may or may not be able to find the corresponding source
+ code, and you’d have a hard time fiddling with one of the
+ components of the software stack.
+
+We pride ourselves with having a tool set that caters to some of the use
+cases that “app bundles” and “containerization” try to address while
+having none of these drawbacks. So how do Guix packs fit into that
+picture?
+
+First of all, the intended use case is different: we view `guix pack` as
+a tool that makes it easy to try out a piece of software on a non-Guix
+machine. But it is clear that for production, our recommendation is to
+use Guix directly, to get security updates and generally address all the
+above issues. :-)
+
+That said, let’s see how these issues affect Guix packs. First,
+composability of Guix packs turns out to be pretty good. If you receive
+two different Guix packs for different pieces of software, you can
+unpack both in your root directory (or union-mount them in the same
+place): packages that differ have a different `/gnu/store` file name
+with a different hash, so they won’t collide; packages that are
+identical (say the C library or GTK+) will have the same `/gnu/store`
+file name so they’ll actually be shared.
+
+That means that for security updates, you could always fetch a new pack
+of your application with the security updates and extract it in place.
+However, that requires you as a user to manually pay attention to
+vulnerabilities in all the software that comes with the pack, so clearly
+you’re better off using Guix instead and regularly upgrading. No
+wonders.
+
+Packs themselves are reproducible bit-by-bit. If you know the Guix
+commit that was used to build a given pack, you can thus run the same
+`guix pack` command on another machine and verify that you get the exact
+same tarball. Currently not 100% of the packages Guix provide are
+reproducible bit-by-bit; we’re getting closer to that goal though, in
+part due to the fact that Guix builds are isolated by default, and also
+thanks to the efforts of everyone in the
+[Reproducible Builds](https://reproducible-builds.org) project to
+address sources of non-determinism in free software.
+
+Because Guix packs are reproducible, you can not only reproduce the
+exact same pack but also create packs with variants of the software—for
+instance, changing the version of one of the packages in the stack. Of
+course this part requires you to have Guix installed somewhere, but at
+least you can easily fiddle with the software stack and “compile” your
+own variant of the software stack down to a new pack.
+
+We hope you’ll enjoy packs and Guix, and would welcome your feedback on
+[the `guix-devel` mailing list](https://www.gnu.org/software/guix/about/#contact)
+and on `#guix` on Freenode!
+
+
+#### About GNU Guix
+
+[GNU Guix](https://www.gnu.org/software/guix) is a transactional package
+manager for the GNU system. The Guix System Distribution or GuixSD is
+an advanced distribution of the GNU system that relies on GNU Guix and
+[respects the user's
+freedom](https://www.gnu.org/distros/free-system-distribution-guidelines.html).
+
+In addition to standard package management features, Guix supports
+transactional upgrades and roll-backs, unprivileged package management,
+per-user profiles, and garbage collection. Guix uses low-level
+mechanisms from the Nix package manager, except that packages are
+defined as native [Guile](https://www.gnu.org/software/guile) modules,
+using extensions to the [Scheme](http://schemers.org) language. GuixSD
+offers a declarative approach to operating system configuration
+management, and is highly customizable and hackable.
+
+GuixSD can be used on an i686 or x86_64 machine. It is also possible to
+use Guix on top of an already installed GNU/Linux system, including on
+mips64el and armv7.