Merge branch 'master' into core-updates

1 files changed, 322 insertions, 59 deletions

diff --git a/doc/guix.texi b/doc/guix.texi
index f8d71fdace..c10479ff12 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -11,7 +11,7 @@
 
 @copying
 Copyright @copyright{} 2012, 2013, 2014 Ludovic Courtès@*
-Copyright @copyright{} 2013 Andreas Enge@*
+Copyright @copyright{} 2013, 2014 Andreas Enge@*
 Copyright @copyright{} 2013 Nikita Karetnikov
 
 Permission is granted to copy, distribute and/or modify this document
@@ -828,6 +828,17 @@ name: libgc
 version: 7.2alpha6
 @end example
 
+Similarly, to show the name of all the packages available under the
+terms of the GNU@tie{}LGPL version 3:
+
+@example
+$ guix package -s "" | recsel -p name -e 'license ~ "LGPL 3"'
+name: elfutils
+
+name: gmp
+@dots{}
+@end example
+
 @item --list-installed[=@var{regexp}]
 @itemx -I [@var{regexp}]
 List the currently installed packages in the specified profile, with the
@@ -1305,6 +1316,7 @@ package definitions.
 * The Store::                   Manipulating the package store.
 * Derivations::                 Low-level interface to package derivations.
 * The Store Monad::             Purely functional interface to the store.
+* G-Expressions::               Manipulating build expressions.
 @end menu
 
 @node Defining Packages
@@ -1762,13 +1774,21 @@ to a Bash executable in the store:
                            "echo hello world > $out\n" '())))
   (derivation store "foo"
               bash `("-e" ,builder)
+              #:inputs `((,bash) (,builder))
               #:env-vars '(("HOME" . "/homeless"))))
 @result{} #<derivation /gnu/store/@dots{}-foo.drv => /gnu/store/@dots{}-foo>
 @end lisp
 
-As can be guessed, this primitive is cumbersome to use directly.  An
-improved variant is @code{build-expression->derivation}, which allows
-the caller to directly pass a Guile expression as the build script:
+As can be guessed, this primitive is cumbersome to use directly.  A
+better approach is to write build scripts in Scheme, of course!  The
+best course of action for that is to write the build code as a
+``G-expression'', and to pass it to @code{gexp->derivation}.  For more
+information, @ref{G-Expressions}.
+
+Once upon a time, @code{gexp->derivation} did not exist and constructing
+derivations with build code written in Scheme was achieved with
+@code{build-expression->derivation}, documented below.  This procedure
+is now deprecated in favor of the much nicer @code{gexp->derivation}.
 
 @deffn {Scheme Procedure} build-expression->derivation @var{store} @
        @var{name} @var{exp} @
@@ -1816,20 +1836,6 @@ containing one file:
 @result{} #<derivation /gnu/store/@dots{}-goo.drv => @dots{}>
 @end lisp
 
-@cindex strata of code
-Remember that the build expression passed to
-@code{build-expression->derivation} is run by a separate Guile process
-than the one that calls @code{build-expression->derivation}: it is run
-by a Guile process launched by the daemon, typically in a chroot.  So,
-while there is a single language for both the @dfn{host} and the build
-side, there are really two @dfn{strata} of code: the host-side, and the
-build-side code@footnote{The term @dfn{stratum} in this context was
-coined by Manuel Serrano et al. in the context of their work on Hop.}.
-This distinction is important to keep in mind, notably when using
-higher-level constructs such as @var{gnu-build-system} (@pxref{Defining
-Packages}).  For this reason, Guix modules that are meant to be used in
-the build stratum are kept in the @code{(guix build @dots{})} name
-space.
 
 @node The Store Monad
 @section The Store Monad
@@ -1873,11 +1879,12 @@ Consider this ``normal'' procedure:
 
 Using @code{(guix monads)}, it may be rewritten as a monadic function:
 
+@c FIXME: Find a better example, one that uses 'mlet'.
 @example
 (define (sh-symlink)
   ;; Same, but return a monadic value.
-  (mlet %store-monad ((sh (package-file bash "bin")))
-    (derivation-expression "sh" `(symlink ,sh %output))))
+  (gexp->derivation "sh"
+                    #~(symlink (string-append #$bash "/bin/bash") #$output)))
 @end example
 
 There are two things to note in the second version: the @code{store}
@@ -1978,21 +1985,206 @@ directory of @var{package}.  When @var{file} is omitted, return the name
 of the @var{output} directory of @var{package}.
 @end deffn
 
-@deffn {Monadic Procedure} derivation-expression @var{name} @var{exp} @
-       [#:system (%current-system)] [#:inputs '()] @
-       [#:outputs '("out")] [#:hash #f] @
-       [#:hash-algo #f] [#:env-vars '()] [#:modules '()] @
-       [#:references-graphs #f] [#:guile-for-build #f]
-Monadic version of @code{build-expression->derivation}
-(@pxref{Derivations}).
-@end deffn
-
 @deffn {Monadic Procedure} package->derivation @var{package} [@var{system}]
 Monadic version of @code{package-derivation} (@pxref{Defining
 Packages}).
 @end deffn
 
 
+@node G-Expressions
+@section G-Expressions
+
+@cindex G-expression
+@cindex build code quoting
+So we have ``derivations'', which represent a sequence of build actions
+to be performed to produce an item in the store (@pxref{Derivations}).
+Those build actions are performed when asking the daemon to actually
+build the derivations; they are run by the daemon in a container
+(@pxref{Invoking guix-daemon}).
+
+@cindex strata of code
+It should come as no surprise that we like to write those build actions
+in Scheme.  When we do that, we end up with two @dfn{strata} of Scheme
+code@footnote{The term @dfn{stratum} in this context was coined by
+Manuel Serrano et al.@: in the context of their work on Hop.  Oleg
+Kiselyov, who has written insightful
+@url{http://okmij.org/ftp/meta-programming/#meta-scheme, essays and code
+on this topic}, refers to this kind of code generation as
+@dfn{staging}.}: the ``host code''---code that defines packages, talks
+to the daemon, etc.---and the ``build code''---code that actually
+performs build actions, such as making directories, invoking
+@command{make}, etc.
+
+To describe a derivation and its build actions, one typically needs to
+embed build code inside host code.  It boils down to manipulating build
+code as data, and Scheme's homoiconicity---code has a direct
+representation as data---comes in handy for that.  But we need more than
+Scheme's normal @code{quasiquote} mechanism to construct build
+expressions.
+
+The @code{(guix gexp)} module implements @dfn{G-expressions}, a form of
+S-expressions adapted to build expressions.  G-expressions, or
+@dfn{gexps}, consist essentially in three syntactic forms: @code{gexp},
+@code{ungexp}, and @code{ungexp-splicing} (or simply: @code{#~},
+@code{#$}, and @code{#$@@}), which are comparable respectively to
+@code{quasiquote}, @code{unquote}, and @code{unquote-splicing}
+(@pxref{Expression Syntax, @code{quasiquote},, guile, GNU Guile
+Reference Manual}).  However, there are major differences:
+
+@itemize
+@item
+Gexps are meant to be written to a file and run or manipulated by other
+processes.
+
+@item
+When a package or derivation is unquoted inside a gexp, the result is as
+if its output file name had been introduced.
+
+@item
+Gexps carry information about the packages or derivations they refer to,
+and these dependencies are automatically added as inputs to the build
+processes that use them.
+@end itemize
+
+To illustrate the idea, here is an example of a gexp:
+
+@example
+(define build-exp
+  #~(begin
+      (mkdir #$output)
+      (chdir #$output)
+      (symlink (string-append #$coreutils "/bin/ls") 
+               "list-files")))
+@end example
+
+This gexp can be passed to @code{gexp->derivation}; we obtain a
+derivation that builds a directory containing exactly one symlink to
+@file{/gnu/store/@dots{}-coreutils-8.22/bin/ls}:
+
+@example
+(gexp->derivation "the-thing" build-exp)
+@end example
+
+As one would expect, the @code{"/gnu/store/@dots{}-coreutils-8.22"} string is
+substituted to the reference to the @var{coreutils} package in the
+actual build code, and @var{coreutils} is automatically made an input to
+the derivation.  Likewise, @code{#$output} (equivalent to @code{(ungexp
+output)}) is replaced by a string containing the derivation's output
+directory name.  The syntactic form to construct gexps is summarized
+below.
+
+@deffn {Scheme Syntax} #~@var{exp}
+@deffnx {Scheme Syntax} (gexp @var{exp})
+Return a G-expression containing @var{exp}.  @var{exp} may contain one
+or more of the following forms:
+
+@table @code
+@item #$@var{obj}
+@itemx (ungexp @var{obj})
+Introduce a reference to @var{obj}.  @var{obj} may be a package or a
+derivation, in which case the @code{ungexp} form is replaced by its
+output file name---e.g., @code{"/gnu/store/@dots{}-coreutils-8.22}.
+
+If @var{obj} is a list, it is traversed and any package or derivation
+references are substituted similarly.
+
+If @var{obj} is another gexp, its contents are inserted and its
+dependencies are added to those of the containing gexp.
+
+If @var{obj} is another kind of object, it is inserted as is.
+
+@item #$@var{package-or-derivation}:@var{output}
+@itemx (ungexp @var{package-or-derivation} @var{output})
+This is like the form above, but referring explicitly to the
+@var{output} of @var{package-or-derivation}---this is useful when
+@var{package-or-derivation} produces multiple outputs (@pxref{Packages
+with Multiple Outputs}).
+
+@item #$output[:@var{output}]
+@itemx (ungexp output [@var{output}])
+Insert a reference to derivation output @var{output}, or to the main
+output when @var{output} is omitted.
+
+This only makes sense for gexps passed to @code{gexp->derivation}.
+
+@item #$@@@var{lst}
+@itemx (ungexp-splicing @var{lst})
+Like the above, but splices the contents of @var{lst} inside the
+containing list.
+
+@end table
+
+G-expressions created by @code{gexp} or @code{#~} are run-time objects
+of the @code{gexp?} type (see below.)
+@end deffn
+
+@deffn {Scheme Procedure} gexp? @var{obj}
+Return @code{#t} if @var{obj} is a G-expression.
+@end deffn
+
+G-expressions are meant to be written to disk, either as code building
+some derivation, or as plain files in the store.  The monadic procedures
+below allow you to do that (@pxref{The Store Monad}, for more
+information about monads.)
+
+@deffn {Monadic Procedure} gexp->derivation @var{name} @var{exp} @
+       [#:system (%current-system)] [#:inputs '()] @
+       [#:hash #f] [#:hash-algo #f] @
+       [#:recursive? #f] [#:env-vars '()] [#:modules '()] @
+       [#:references-graphs #f] [#:local-build? #f] @
+       [#:guile-for-build #f]
+Return a derivation @var{name} that runs @var{exp} (a gexp) with
+@var{guile-for-build} (a derivation) on @var{system}.
+
+Make @var{modules} available in the evaluation context of @var{EXP};
+@var{MODULES} is a list of names of Guile modules from the current
+search path to be copied in the store, compiled, and made available in
+the load path during the execution of @var{exp}---e.g., @code{((guix
+build utils) (guix build gnu-build-system))}.
+
+The other arguments are as for @code{derivation} (@pxref{Derivations}).
+@end deffn
+
+@deffn {Monadic Procedure} gexp->script @var{name} @var{exp}
+Return an executable script @var{name} that runs @var{exp} using
+@var{guile} with @var{modules} in its search path.
+
+The example below builds a script that simply invokes the @command{ls}
+command:
+
+@example
+(use-modules (guix gexp) (gnu packages base))
+
+(gexp->script "list-files"
+              #~(execl (string-append #$coreutils "/bin/ls")
+                       "ls"))
+@end example
+
+When ``running'' it through the store (@pxref{The Store Monad,
+@code{run-with-store}}), we obtain a derivation that produces an
+executable file @file{/gnu/store/@dots{}-list-files} along these lines:
+
+@example
+#!/gnu/store/@dots{}-guile-2.0.11/bin/guile -ds
+!#
+(execl (string-append "/gnu/store/@dots{}-coreutils-8.22"/bin/ls")
+       "ls")
+@end example
+@end deffn
+
+@deffn {Monadic Procedure} gexp->file @var{name} @var{exp}
+Return a derivation that builds a file @var{name} containing @var{exp}.
+
+The resulting file holds references to all the dependencies of @var{exp}
+or a subset thereof.
+@end deffn
+
+Of course, in addition to gexps embedded in ``host'' code, there are
+also modules containing build tools.  To make it clear that they are
+meant to be used in the build stratum, these modules are kept in the
+@code{(guix build @dots{})} name space.
+
+
 @c *********************************************************************
 @node Utilities
 @chapter Utilities
@@ -2412,6 +2604,7 @@ to join!  @ref{Contributing}, for information about how you can help.
 @node Installing Debugging Files
 @section Installing Debugging Files
 
+@cindex debugging files
 Program binaries, as produced by the GCC compilers for instance, are
 typically written in the ELF format, with a section containing
 @dfn{debugging information}.  Debugging information is what allows the
@@ -2442,7 +2635,7 @@ installs the debugging information for the GNU C Library and for GNU
 Guile:
 
 @example
-guix package -i glibc:debug -i guile:debug
+guix package -i glibc:debug guile:debug
 @end example
 
 GDB must then be told to look for debug files in the user's profile, by
@@ -2457,9 +2650,16 @@ GDB}):
 From there on, GDB will pick up debugging information from the
 @code{.debug} files under @file{~/.guix-profile/lib/debug}.
 
+In addition, you will most likely want GDB to be able to show the source
+code being debugged.  To do that, you will have to unpack the source
+code of the package of interest (obtained with @code{guix build
+--source}, @pxref{Invoking guix build}), and to point GDB to that source
+directory using the @code{directory} command (@pxref{Source Path,
+@code{directory},, gdb, Debugging with GDB}).
+
 @c XXX: keep me up-to-date
 The @code{debug} output mechanism in Guix is implemented by the
-@code{gnu-build-system} (@pxref{Defining Packages}).  Currently, it is
+@code{gnu-build-system} (@pxref{Build Systems}).  Currently, it is
 opt-in---debugging information is available only for those packages
 whose definition explicitly declares a @code{debug} output.  This may be
 changed to opt-out in the future, if our build farm servers can handle
@@ -2570,6 +2770,7 @@ needed is to review and apply the patch.
 * Package Naming::       What's in a name?
 * Version Numbers::      When the name is not enough.
 * Python Modules::       Taming the snake.
+* Perl Modules::         Little pearls.
 @end menu
 
 @node Software Freedom
@@ -2611,12 +2812,15 @@ the string in the @code{name} field of a package definition.  This name
 is used by package management commands such as
 @command{guix package} and @command{guix build}.
 
-Both are usually the same and correspond to the lowercase conversion of the
-project name chosen upstream.  For instance, the GNUnet project is packaged
-as @code{gnunet}.  We do not add @code{lib} prefixes for library packages,
-unless these are already part of the official project name.  But see
-@ref{Python Modules} for special rules concerning modules for
-the Python language.
+Both are usually the same and correspond to the lowercase conversion of
+the project name chosen upstream, with underscores replaced with
+hyphens.  For instance, GNUnet is available as @code{gnunet}, and
+SDL_net as @code{sdl-net}.
+
+We do not add @code{lib} prefixes for library packages, unless these are
+already part of the official project name.  But see @pxref{Python
+Modules} and @ref{Perl Modules} for special rules concerning modules for
+the Python and Perl languages.
 
 
 @node Version Numbers
@@ -2678,6 +2882,19 @@ for instance, the module python-dateutil is packaged under the names
 @code{python-dateutil} and @code{python2-dateutil}.
 
 
+@node Perl Modules
+@subsection Perl Modules
+
+Perl programs standing for themselves are named as any other package,
+using the lowercase upstream name.
+For Perl packages containing a single class, we use the lowercase class name,
+replace all occurrences of @code{::} by dashes and prepend the prefix
+@code{perl-}.
+So the class @code{XML::Parser} becomes @code{perl-xml-parser}.
+Modules containing several classes keep their lowercase upstream name and
+are also prepended by @code{perl-}.  Such modules tend to have the word
+@code{perl} somewhere in their name, which gets dropped in favor of the
+prefix.  For instance, @code{libwww-perl} becomes @code{perl-libwww}.
 
 
 
@@ -2895,9 +3112,8 @@ Linux-Libre kernel, initial RAM disk, and boot loader looks like this:
 
 @findex operating-system
 @lisp
-(use-modules (gnu services base)   ; for '%base-services'
+(use-modules (gnu)   ; for 'user-account', '%base-services', etc.
              (gnu services ssh)    ; for 'lsh-service'
-             (gnu system shadow)   ; for 'user-account'
              (gnu packages base)   ; Coreutils, grep, etc.
              (gnu packages bash)   ; Bash
              (gnu packages admin)  ; dmd, Inetutils
@@ -2911,6 +3127,12 @@ Linux-Libre kernel, initial RAM disk, and boot loader looks like this:
    (host-name "komputilo")
    (timezone "Europe/Paris")
    (locale "fr_FR.UTF-8")
+   (bootloader (grub-configuration
+                 (device "/dev/sda")))
+   (file-systems (list (file-system
+                         (device "/dev/disk/by-label/root")
+                         (mount-point "/")
+                         (type "ext3"))))
    (users (list (user-account
                  (name "alice")
                  (password "")
@@ -2986,6 +3208,29 @@ operating system is instantiate.  Currently the following values are
 supported:
 
 @table @code
+@item build
+Build the operating system's derivation, which includes all the
+configuration files and programs needed to boot and run the system.
+This action does not actually install anything.
+
+@item init
+Populate the given directory with all the files necessary to run the
+operating system specified in @var{file}.  This is useful for first-time
+installations of the GNU system.  For instance:
+
+@example
+guix system init my-os-config.scm /mnt
+@end example
+
+copies to @file{/mnt} all the store items required by the configuration
+specified in @file{my-os-config.scm}.  This includes configuration
+files, packages, and so on.  It also creates other essential files
+needed for the system to operate correctly---e.g., the @file{/etc},
+@file{/var}, and @file{/run} directories, and the @file{/bin/sh} file.
+
+This command also installs GRUB on the device specified in
+@file{my-os-config}, unless the @option{--no-grub} option was passed.
+
 @item vm
 @cindex virtual machine
 Build a virtual machine that contain the operating system declared in
@@ -2994,9 +3239,23 @@ Build a virtual machine that contain the operating system declared in
 The VM shares its store with the host system.
 
 @item vm-image
-Return a virtual machine image of the operating system declared in
-@var{file} that stands alone.  Use the @option{--image-size} option to
-specify the size of the image.
+@itemx disk-image
+Return a virtual machine or disk image of the operating system declared
+in @var{file} that stands alone.  Use the @option{--image-size} option
+to specify the size of the image.
+
+When using @code{vm-image}, the returned image is in qcow2 format, which
+the QEMU emulator can efficiently use.
+
+When using @code{disk-image}, a raw disk image is produced; it can be
+copied as is to a USB stick, for instance.  Assuming @code{/dev/sdc} is
+the device corresponding to a USB stick, one can copy the image on it
+using the following command:
+
+@example
+# dd if=$(guix system disk-image my-os.scm) of=/dev/sdc
+@end example
+
 @end table
 
 @var{options} can contain any of the common build options provided by
@@ -3039,29 +3298,33 @@ like:
 
 @lisp
 (define (nscd-service)
-  (mlet %store-monad ((nscd (package-file glibc "sbin/nscd")))
+  (with-monad %store-monad
     (return (service
              (documentation "Run libc's name service cache daemon.")
              (provision '(nscd))
-             (start `(make-forkexec-constructor ,nscd "-f" "/dev/null"
-                                                "--foreground"))
-             (stop  `(make-kill-destructor))
-
-             (respawn? #f)
-             (inputs `(("glibc" ,glibc)))))))
+             (activate #~(begin
+                           (use-modules (guix build utils))
+                           (mkdir-p "/var/run/nscd")))
+             (start #~(make-forkexec-constructor
+                       (string-append #$glibc "/sbin/nscd")
+                       "-f" "/dev/null" "--foreground"))
+             (stop #~(make-kill-destructor))
+             (respawn? #f)))))
 @end lisp
 
 @noindent
-The @code{inputs} field specifies that this service depends on the
-@var{glibc} package---the package that contains the @command{nscd}
-program.  The @code{start} and @code{stop} fields are expressions that
-make use of dmd's facilities to start and stop processes (@pxref{Service
-De- and Constructors,,, dmd, GNU dmd Manual}).  The @code{provision}
-field specifies the name under which this service is known to dmd, and
-@code{documentation} specifies on-line documentation.  Thus, the
-commands @command{deco start ncsd}, @command{deco stop nscd}, and
-@command{deco doc nscd} will do what you would expect (@pxref{Invoking
-deco,,, dmd, GNU dmd Manual}).
+The @code{activate}, @code{start}, and @code{stop} fields are G-expressions
+(@pxref{G-Expressions}).  The @code{activate} field contains a script to
+run at ``activation'' time; it makes sure that the @file{/var/run/nscd}
+directory exists before @command{nscd} is started.
+
+The @code{start} and @code{stop} fields refer to dmd's facilities to
+start and stop processes (@pxref{Service De- and Constructors,,, dmd,
+GNU dmd Manual}).  The @code{provision} field specifies the name under
+which this service is known to dmd, and @code{documentation} specifies
+on-line documentation.  Thus, the commands @command{deco start ncsd},
+@command{deco stop nscd}, and @command{deco doc nscd} will do what you
+would expect (@pxref{Invoking deco,,, dmd, GNU dmd Manual}).
 
 
 @c *********************************************************************