GNU Guix
Package manager,
Hello, my name is Chris, and I'm talking about package
management with GNU Guix.
I first found out about Guix sometime in late 2015, early
2016.
I remember installing it the night before FOSDEM back in
2016 and I started using and contributing to Guix soon
after.
Dependable
Flexible
General Purpose
Some of the concepts involved in Guix take a bit of
getting used to, so I've tried to break down the benefits
and properties in to 3 areas.
Dependability, flexibility, and Guix's position as a
general purpose package manager.
Dependable
The immutable store is a rock solid base to build on
Package behaviour can be fixed at build time
You can rollback changes
Starting with dependability, I consider Guix to be a very
dependable package manager.
The main building block for dependability is a immutable
store, a directory where data can be added, but not
changed. With Guix, the location for this is somewhat
fixed, at /gnu/store.
When you build packages with Guix, you can build them in
such a way that the behaviour will be fixed at build
time. I'll give some concrete examples of this later on.
The other part of dependability is handling problems that
inevitably arise. Sometimes things go wrong with software,
and the ideal first step is to just back out the change
you just made. The architecture that Guix uses supports
this elegantly.
Flexible
Use prebuilt packages, or build from source
Tweak package definitions, and add your own
No conflicts between different packages
Having spent a couple of years now doing a few odd things
with Guix, I can say for sure that it's really flexible.
Guix supports downloading binaries from elsewhere to
install, but also building those same packages from
source, and that can come in really useful if you start
tweaking package definitions or adding your own.
Guix also elegantly supports handling multiple versions of
the same bit of software at once, or even multiple
different packages of the same version of the same bit of
software, built with different arguments say.
General purpose
Advancement through diversity
No barriers of programming language or field of use
Leverage Guix wherever you need it
Works on multiple distributions and architectures
Finally, Guix is a general purpose package
manager. Personally, this is something I have a big
preference for. I think the problems of package management
are best tackled without fracturing the effort along lines
like programming languages, or fields of use.
While Guix is a general purpose package manager, it also
has specific tooling, for example if you want to generate
Docker images, you can pack up some Guix packages in to a
Docker image with ease.
Guix as a package manager works on many systems, including
GuixSD, a system based around Guix. This means that it
sits across that historical divide between package
managers for operating systems, and other package
managers.
$ guix package --install hello
$ hello
Hello, world!
On to the practical bit now, lets take a look at a few
examples of using Guix packages.
You might be able to guess what running this command does.
It will look at the available packages, find one called
hello, and then install it to the current users profile.
In case you haven't encountered hello, it's purpose is
pretty simple. It provides a binary called hello that
prints out hello world when you run it.
Modified, © Ricardo Wurmus
The really key bit with this method of using Guix packages
is the profiles.
Let's take another example. Here you can see the default
layout for a users default profile. In this case they have
two packages installed, samtools and bowtie.
You'd usually expect to:
- have a .guix-profile symlink in your home directory
- that points to another symlink somewhere in /var/guix
- which points to another symlink which is named with a
number
This number is the generation of the profile. The
generation symlink then points to a directory in the
store.
It is that directory in the store where the profile is
stored, and while it will contain files, these files will
be symlinks to other items in the store.
All of this complexity is normally hidden, as a user, you
just source the /etc/profile file within the profile, and
then all the environment variables you need will be
set. In this case, the PATH environment variable will be
set so that you can run samtools and bowtie2.
guix package -r bowtie
Modified, © Ricardo Wurmus
Things get interesting when you make a change. Say you
remove the bowtie package.
A new generation of your profile will be generated,
pointing to a new profile in the store, now not containing
bowtie, and just containing samtools.
The guix-profile symlink in /var/guix has been updated to
point at this new generation.
Going back to dependability, something important has
happened here, or rather not happened.
The bowtie package it still in the store, and the 42nd
generation of the profile is as well. Nothing in the store
has been deleted. This means that you can roll back to the
previous generation by changing one symlink.
guix package --roll-back
Modified, © Ricardo Wurmus
The roll back can be done by running the guix package
command with the --roll-back flag.
Let's just flip through that so you can see the changes.
To recap, the immutable nature of the store, and the
ability to roll back profiles makes Guix very dependable.
Development environment for hello
$ guix environment hello
$ type -p gcc
/gnu/store/32812c4rxsprnyifhbv21pd6d796r1w8-profile/bin/gcc
On to some flexibility now.
The guix package command deals with profiles, but the guix
environment command does as well.
The purpose of this command is to create development
environments, without having to install the packages
through the guix package command. It's pretty flexible
itself though!
Just running this command will drop you in to a subshell
using a new profile created to contain the packages you'd
need to build the hello package, for example, things like
gcc will be on the path.
guix environment --ad-hoc hello guile
/gnu/store/5w8jk6p57l5an1d39lngw560r2bw2dx1-profile/bin
├── guild -> /gnu/store/yih…6fk-guile-2.2.2/bin/guild
├── guile -> /gnu/store/yih…6fk-guile-2.2.2/bin/guile
├── guile-config -> /gnu/store/yih…6fk-guile-2.2.2/bin/guile-config
├── guile-snarf -> /gnu/store/yih…6fk-guile-2.2.2/bin/guile-snarf
├── guile-tools -> /gnu/store/yih…6fk-guile-2.2.2/bin/guile-tools
└── hello -> /gnu/store/wf6…2sx-hello-2.10/bin/hello
Looking past the purpose of the guix environment command,
in general it offers a really flexible way of generating
arbitrary profiles.
Here is how you generate a profile containing the hello
and guile packages.
(Next)
If you were to look at the profile in the store, you'd see
something like this. The bin directory of the profile
contains symlinks to the bin directories of the hello and
guile packages.
This is how profiles compose multiple packages together.
Recap
Install to your users profile:
$ guix package -i ...
Enter a development environment:
$ guix environment ...
Enter a environment with specific packages:
$ guix environment --ad-hoc ...
Let's recap quickly.
We've looked at two commands, guix package and guix
environment.
Both work with packages and profiles, but in slightly
different ways.
Let's now take a look at some more interesting ways of
using Guix packages.
$ guix environment --container --ad-hoc hello
$ echo /gnu/store/*
/gnu/store/…-ncurses-6.0 /gnu/store/…-profile
/gnu/store/…-manual-database /gnu/store/…-gcc-5.4.0-lib
/gnu/store/…-info-dir /gnu/store/…-bash-4.4.12
/gnu/store/…-readline-7.0 /gnu/store/…-glibc-2.25
/gnu/store/…-hello-2.10 /gnu/store/…-bash-static-4.4.12
Usually, Guix packages explicitly rely on their
dependencies by pointing at them with absolute paths to
those items in the store.
This is really useful for reliability, as the store is
immutable, this means that dependencies used in this way
won't change.
It also provides for some flexibility, as you know what's
needed, you can remove everything else.
This is what this command does. It's similar to the last
guix environment command, but with --container.
Now the bash shell is started with an isolated view of the
system, for example it's in a different process namespace,
and different mount namespace with only access to the
parts of the store that are required, the current
directory, and some key parts of the root file system.
(Next)
If you run this command, and then look in the store,
you'll see only what is needed to run bash and hello.
Note that here I'm talking about containers just in terms
of the sense of Linux namespaces. Not really anything
else. Somehow the term "containers" has got dragged in to
issues around managing state and software packages, but in
this case, the architecture of Guix allows for some clear
separation between the two concepts.
guix graph --type=references hello
Now this is only possible as Guix can find all the
necessary store items for all the things you want in the
container. Sometimes this is called a closure.
This is a graph generated with Guix for the references for
the hello package.
This was generated with the guix graph command, and this
command and others like guix size can be really useful
when digging in to the packages and understanding the
graph.
$ ldd /gnu/store/wf65hjwqwpz4wllasn63zysi5irql2sx-hello-2.10/bin/hello
linux-vdso.so.1 (0x00007ffece166000)
libgcc_s.so.1 => /gnu/store/3x53yv4v144c9xp02rs64z7j597kkqax-gcc-5.4.0-lib/lib/libgcc_s.so.1 (0x00007fee83076000)
libc.so.6 => /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/libc.so.6 (0x00007fee82cd7000)
/gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/ld-linux-x86-64.so.2 (0x00007fee8328d000)
Back to some more dependability now.
What does it mean that the hello package references the
glibc package?
Well in this case, one of the references is in the
libraries used by the hello binary.
hello is linking against the libc shared library with an
absolute path.
This gives you shared libraries, rather than static
linking, which can help with space issues, but as the
store is immutable you get similar dependability to static
linking.
This is a concrete example of fixing the behavior of a
package at build time. If you were linking against a libc
shared library which could change, then maybe this could
affect the hello package.
guix package -i hello --profile=test-profile
$ ls -l test-profile*
... test-profile -> test-profile-1-link
... test-profile-1-link ->
/gnu/store/7pyxcs7yh7xbl3lbjdf5g6z0b4kl7pzf-profile
Back to some flexibility, maybe having one profile per
user isn't enough.
This isn't a problem, as the guix package command can use
and create different profiles.
Here the profile is called test-profile, and what that
means is Guix creates some symlinks to manage that profile
in the current directory.
Guix + Direnv
use guix --ad-hoc gcc-toolchain python python-lxml
$ cd /tmp/foo
direnv: loading .envrc
direnv: using guix --ad-hoc gcc-toolchain python python-lxml
direnv: export +PAGER +PYTHONPATH ~CPLUS_INCLUDE_PATH
~C_INCLUDE_PATH ~LIBRARY_PATH ~PATH
This is a neat way that I use guix environment, and hook
it up with another program called direnv.
direnv is an environment switcher for the shell, and guix
environment can specify environment variables to load a
profile. Therefore, you can combine the two so that when
you enter a directory, direnv will invoke guix to load the
environment.
Direnv is configured through using a file named .envrc in
the directory you want it to be active in.
The example I'm showing here is to create an environment
containing gcc-toolchain, python and python-lxml when you
enter the /tmp/foo directory.
guix pack
The guix pack command creates a shrink-wrapped pack or
software bundle.
The guix pack command can be really practical. Offering a
way to get some of the benefits of using Guix, but also
some of the benefits of binary bundles of software.
$ guix pack guile emacs geiser
...
...
/gnu/store/v5dq15c61k0gb5w8w9psgszd56vnca0v-tarball-pack.tar.gz
Here, packing up guile emacs and geiser will give you a
tarball containing those store items, but also all of
their direct and indirect dependencies.
$ guix pack -f docker guile emacs geiser
...
...
/gnu/store/mym3ipc9v0ki9y9vijjmyqrsajc1y3sz-docker-pack.tar.gz
Generating a tarball is the default, but the other format
currently available is docker.
Writing a package definition
Importers available for: gnu, nix, pypi, cpan, hackage,
stackage, elpa, gem, cran, crate, texlive and json.
Guix has definitions for around six and a half thousand
packages, but you might want to use something that is
missing. One thing you can do is look at packaging it!
Guix contains several importers, which can use existing
metadata from some source, to construct a package
definition.
Depending on the importer, and how complete the
information it had available was, you might be able to
build this package straight away, or it might need some
tweaking.
Hello package definition
(define-public hello
(package
(name "hello")
(version "2.10")
(source (origin
(method url-fetch)
(uri (string-append "mirror://gnu/hello/hello-" version
".tar.gz"))
(sha256
(base32
"0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
(build-system gnu-build-system)
(synopsis "Hello, GNU world: An example GNU package")
(description
"GNU Hello prints the message \"Hello, world!\" and then exits. It
serves as an example of standard GNU coding practices. As such, it supports
command-line arguments, multiple languages, and so on.")
(home-page "https://www.gnu.org/software/hello/")
(license gpl3+)))
Here is an example package definition. This one is for the
hello package we looked at earlier.
This package definition is pretty simple, as the build
system takes care of the entire process. Guix is capable
of packaging complicated and difficult software though.
It's also pretty amazing to think about what that short
definition just shown represents.
One part of why Guix packages are so dependable is that
the definitions specify exactly what that package is,
including all its dependencies, referred to as inputs.
Often these inputs are packages themselves, plus maybe
some data, the source tarball in the case of hello. The
inputs for all the dependent packages are also exactly
specified.
Derivation graph for the hello package
This means you can build up a graph like this, showing the
parts involved in the entire process of creating this
package.
Now I don't expect you to read this, but I have coloured
some of the nodes differently. The black ones are the
fixed bits of data, where the hash is known in advance.
The purple ones are the derivations, that maybe take some
data, and some outputs from other derivations and generate
some outputs themselves.
The hello package is just one of these purple coloured
nodes.
Now you might have noticed earlier that the store items
contain a random looking string of characters.
For the black nodes in this graph, this hash roughly
corresponds to the data itself.
For the purple nodes in the graph, the hash represents the
process that is represented by that node.
As hello is at the top of this directed acyclic graph,
that means if any of the data represented by the black
nodes changes, or any of the derivations represented by
the purple nodes changes, you'll get a different hello
package, a different item in the store.
Talk to others on
#guix on freenode
For the manual, mailing lists, papers, blog posts and
talks, go to
https://gnu.org/s/guix/help
Thank you for listening, I hope you've enjoyed this very
quick look at package management with Guix.
If you'd like to talk to others, there is an active IRC
channel, #guix on Freenode.
From the website, you can find a online version of the
manual, and details on the mailing lists.
Tomorrow, there are also a couple of talks involving Guix
in the high performance computing track. Including a talk
by Ludovic Courtès, the creator of the Guix project.
Modified, © Ricardo Wurmus
Modified, © Ricardo Wurmus
Modified, © Ricardo Wurmus
Derivation graph for the hello package