From 21531add3205e400707c8fbfd841845f9a71863a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= Date: Sun, 2 Mar 2014 22:39:48 +0100 Subject: offload: Use Guile-SSH instead of GNU lsh. * guix/scripts/offload.scm ()[ssh-options]: Remove. [host-key, host-key-type]: New fields. (%lsh-command, %lshg-command, user-lsh-private-key): Remove. (user-openssh-private-key, private-key-from-file*): New procedures. (host-key->type+key, open-ssh-session): New procedures. (remote-pipe): Remove 'mode' parameter. Rewrite in terms of 'open-ssh-session' etc. Update users. (send-files)[missing-files]: Rewrite using the bidirectional channel port. Remove call to 'call-with-compressed-output-port'. (retrieve-files): Remove call to 'call-with-decompressed-port'. (machine-load): Remove exit status logic. * doc/guix.texi (Requirements): Mention Guile-SSH. (Daemon Offload Setup): Document 'host-key' and 'private-key'. Show the default value on each @item line. * m4/guix.m4 (GUIX_CHECK_GUILE_SSH): New macro. * config-daemon.ac: Use 'GUIX_CHECK_GUILE_SSH'. Set 'HAVE_DAEMON_OFFLOAD_HOOK' as a function of that. --- doc/guix.texi | 69 ++++++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 49 insertions(+), 20 deletions(-) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index ebb138e15d..f1cb007aa9 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -566,6 +566,12 @@ allow you to use the @command{guix import pypi} command (@pxref{Invoking guix import}). It is of interest primarily for developers and not for casual users. +@item +@c Note: We need at least 0.10.2 for 'channel-send-eof'. +Support for build offloading (@pxref{Daemon Offload Setup}) depends on +@uref{https://github.com/artyom-poptsov/guile-ssh, Guile-SSH}, +version 0.10.2 or later. + @item When @url{http://zlib.net, zlib} is available, @command{guix publish} can compress build byproducts (@pxref{Invoking guix publish}). @@ -814,9 +820,11 @@ available on the system---making it much harder to view them as @cindex offloading @cindex build hook -When desired, the build daemon can @dfn{offload} -derivation builds to other machines -running Guix, using the @code{offload} @dfn{build hook}. When that +When desired, the build daemon can @dfn{offload} derivation builds to +other machines running Guix, using the @code{offload} @dfn{build +hook}@footnote{This feature is available only when +@uref{https://github.com/artyom-poptsov/guile-ssh, Guile-SSH} is +present.}. When that feature is enabled, a list of user-specified build machines is read from @file{/etc/guix/machines.scm}; every time a build is requested, for instance via @code{guix build}, the daemon attempts to offload it to one @@ -832,16 +840,18 @@ The @file{/etc/guix/machines.scm} file typically looks like this: (list (build-machine (name "eightysix.example.org") (system "x86_64-linux") + (host-key "ssh-ed25519 AAAAC3Nza@dots{}") (user "bob") - (speed 2.)) ; incredibly fast! + (speed 2.)) ;incredibly fast! (build-machine (name "meeps.example.org") (system "mips64el-linux") + (host-key "ssh-rsa AAAAB3Nza@dots{}") (user "alice") (private-key (string-append (getenv "HOME") - "/.lsh/identity-for-guix")))) + "/.ssh/identity-for-guix")))) @end example @noindent @@ -875,31 +885,50 @@ The user account to use when connecting to the remote machine over SSH. Note that the SSH key pair must @emph{not} be passphrase-protected, to allow non-interactive logins. +@item host-key +This must be the machine's SSH @dfn{public host key} in OpenSSH format. +This is used to authenticate the machine when we connect to it. It is a +long string that looks like this: + +@example +ssh-ed25519 AAAAC3NzaC@dots{}mde+UhL hint@@example.org +@end example + +If the machine is running the OpenSSH daemon, @command{sshd}, the host +key can be found in a file such as +@file{/etc/ssh/ssh_host_ed25519_key.pub}. + +If the machine is running the SSH daemon of GNU@tie{}lsh, +@command{lshd}, the host key is in @file{/etc/lsh/host-key.pub} or a +similar file. It can be converted to the OpenSSH format using +@command{lsh-export-key} (@pxref{Converting keys,,, lsh, LSH Manual}): + +@example +$ lsh-export-key --openssh < /etc/lsh/host-key.pub +ssh-rsa AAAAB3NzaC1yc2EAAAAEOp8FoQAAAQEAs1eB46LV@dots{} +@end example + @end table A number of optional fields may be specified: -@table @code - -@item port -Port number of SSH server on the machine (default: 22). +@table @asis -@item private-key -The SSH private key file to use when connecting to the machine. +@item @code{port} (default: @code{22}) +Port number of SSH server on the machine. -Currently offloading uses GNU@tie{}lsh as its SSH client -(@pxref{Invoking lsh,,, GNU lsh Manual}). Thus, the key file here must -be an lsh key file. This may change in the future, though. +@item @code{private-key} (default: @file{~/.ssh/id_rsa}) +The SSH private key file to use when connecting to the machine, in +OpenSSH format. -@item parallel-builds -The number of builds that may run in parallel on the machine (1 by -default.) +@item @code{parallel-builds} (default: @code{1}) +The number of builds that may run in parallel on the machine. -@item speed +@item @code{speed} (default: @code{1.0}) A ``relative speed factor''. The offload scheduler will tend to prefer machines with a higher speed factor. -@item features +@item @code{features} (default: @code{'()}) A list of strings denoting specific features supported by the machine. An example is @code{"kvm"} for machines that have the KVM Linux modules and corresponding hardware support. Derivations can request features by @@ -915,7 +944,7 @@ machines, since offloading works by invoking the @code{guix archive} and this is the case by running: @example -lsh build-machine guile -c "'(use-modules (guix config))'" +ssh build-machine guile -c "'(use-modules (guix config))'" @end example There is one last thing to do once @file{machines.scm} is in place. As -- cgit v1.2.3 From cf283dd92eb5ef2dee4b761bb23f6dca2525cd55 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= Date: Sat, 5 Nov 2016 00:47:34 +0100 Subject: offload: Rewrite to make direct RPCs to the remote daemon. * guix/scripts/offload.scm ()[daemon-socket]: New field. (connect-to-remote-daemon): New procedure. (%gc-root-file, register-gc-root, remove-gc-roots, offload): Remove. (transfer-and-offload): Rewrite using 'connect-to-remote-daemon' and RPCs over SSH. (store-import-channel, store-export-channel): New procedures. (send-files, retrieve-files): Rewrite using these. --- doc/guix.texi | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index f1cb007aa9..b8e37055e6 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -921,6 +921,10 @@ Port number of SSH server on the machine. The SSH private key file to use when connecting to the machine, in OpenSSH format. +@item @code{daemon-socket} (default: @code{"/var/guix/daemon-socket/socket"}) +File name of the Unix-domain socket @command{guix-daemon} is listening +to on that machine. + @item @code{parallel-builds} (default: @code{1}) The number of builds that may run in parallel on the machine. -- cgit v1.2.3 From f88371e86602a9b3d86f2030709f719778613552 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= Date: Mon, 21 Nov 2016 20:41:17 +0800 Subject: services: Add opensmtpd service. * gnu/services/mail.scm (): New record type. (%default-opensmtpd-config-file, %opensmtpd-accounts): New variables. (opensmtpd-shepherd-service, opensmtpd-activation): New procedures. (opensmtpd-service-type): New variable. * doc/guix.texi (Mail Services): Document it. --- doc/guix.texi | 42 +++++++++++++++++++++++++++++++++--------- 1 file changed, 33 insertions(+), 9 deletions(-) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index b8e37055e6..137fec8d7a 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -10075,16 +10075,11 @@ For MariaDB, the root password is empty. @cindex mail @cindex email The @code{(gnu services mail)} module provides Guix service definitions -for mail services. Currently the only implemented service is Dovecot, -an IMAP, POP3, and LMTP server. +for email services: IMAP, POP3, and LMTP servers, as well as mail +transport agents (MTAs). Lots of acronyms! These services are detailed +in the subsections below. -Guix does not yet have a mail transfer agent (MTA), although for some -lightweight purposes the @code{esmtp} relay-only MTA may suffice. Help -is needed to properly integrate a full MTA, such as Postfix. Patches -welcome! - -To add an IMAP/POP3 server to a GuixSD system, add a -@code{dovecot-service} to the operating system definition: +@subsubheading Dovecot Service @deffn {Scheme Procedure} dovecot-service [#:config (dovecot-configuration)] Return a service that runs the Dovecot IMAP/POP3/LMTP mail server. @@ -11440,6 +11435,35 @@ could instantiate a dovecot service like this: (string ""))) @end example +@subsubheading OpenSMTPD Service + +@deffn {Scheme Variable} opensmtpd-service-type +This is the type of the @uref{https://www.opensmtpd.org, OpenSMTPD} +service, whose value should be an @code{opensmtpd-configuration} object +as in this example: + +@example +(service opensmtpd-service-type + (opensmtpd-configuration + (config-file (local-file "./my-smtpd.conf")))) +@end example +@end deffn + +@deftp {Data Type} opensmtpd-configuration +Data type regresenting the configuration of opensmtpd. + +@table @asis +@item @code{package} (default: @var{opensmtpd}) +Package object of the OpenSMTPD SMTP server. + +@item @code{config-file} (default: @var{%default-opensmtpd-file}) +File-like object of the OpenSMTPD configuration file to use. By default +it listens on the loopback network interface, and allows for mail from +users and daemons on the local machine, as well as permitting email to +remote servers. Run @command{man smtpd.conf} for more information. + +@end table +@end deftp @node Kerberos Services @subsubsection Kerberos Services -- cgit v1.2.3 From c940b8e682ef375f46c69ede4218d0a4d75317b5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= Date: Sat, 26 Nov 2016 12:47:14 +0800 Subject: doc: mysql-configuration: Fix typo. * doc/guix.texi (Database Services): Fix typo of 'mysql-configuration'. --- doc/guix.texi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index 137fec8d7a..5747484b20 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -10053,7 +10053,7 @@ Return a service that runs @command{mysqld}, the MySQL or MariaDB database server. The optional @var{config} argument specifies the configuration for -@command{mysqld}, which should be a @code{} object. +@command{mysqld}, which should be a @code{} object. @end deffn @deftp {Data Type} mysql-configuration -- cgit v1.2.3 From 9a5187b687e659da86000b32c951e2f55e1f74bb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= Date: Mon, 28 Nov 2016 16:19:04 +0100 Subject: doc: Document NSS incompatibility issues on foreign distros. * doc/guix.texi (Application Setup)[Name Service Switch]: New subsection. --- doc/guix.texi | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index 5747484b20..ce1e5d075a 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -1242,6 +1242,56 @@ data in the right format. This is important because the locale data format used by different libc versions may be incompatible. +@subsection Name Service Switch + +@cindex name service switch, glibc +@cindex NSS (name service switch), glibc +@cindex nscd (name service caching daemon) +@cindex name service caching daemon (nscd) +When using Guix on a foreign distro, we @emph{strongly recommend} that +the system run the GNU C library's @dfn{name service cache daemon}, +@command{nscd}, which should be listening on the +@file{/var/run/nscd/socket} socket. Failing to do that, applications +installed with Guix may fail to look up host names or user accounts, or +may even crash. The next paragraphs explain why. + +@cindex @file{nsswitch.conf} +The GNU C library implements a @dfn{name service switch} (NSS), which is +an extensible mechanism for ``name lookups'' in general: host name +resolution, user accounts, and more (@pxref{Name Service Switch,,, libc, +The GNU C Library Reference Manual}). + +@cindex Network information service (NIS) +@cindex NIS (Network information service) +Being extensible, the NSS supports @dfn{plugins}, which provide new name +lookup implementations: for example, the @code{nss-mdns} plugin allow +resolution of @code{.local} host names, the @code{nis} plugin allows +user account lookup using the Network information service (NIS), and so +on. These extra ``lookup services'' are configured system-wide in +@file{/etc/nsswitch.conf}, and all the programs running on the system +honor those settings (@pxref{NSS Configuration File,,, libc, The GNU C +Reference Manual}). + +When they perform a name lookup---for instance by calling the +@code{getaddrinfo} function in C---applications first try to connect to +the nscd; on success, nscd performs name lookups on their behalf. If +the nscd is not running, then they perform the name lookup by +themselves, by loading the name lookup services into their own address +space and running it. These name lookup services---the +@file{libnss_*.so} files---are @code{dlopen}'d, but they may come from +the host system's C library, rather than from the C library the +application is linked against (the C library coming from Guix). + +And this is where the problem is: if your application is linked against +Guix's C library (say, glibc 2.24) and tries to load NSS plugins from +another C library (say, @code{libnss_mdns.so} for glibc 2.22), it will +likely crash or have its name lookups fail unexpectedly. + +Running @command{nscd} on the system, among other advantages, eliminates +this binary incompatibility problem because those @code{libnss_*.so} +files are loaded in the @command{nscd} process, not in applications +themselves. + @subsection X11 Fonts @cindex fonts -- cgit v1.2.3 From e9c72306fdfd6a60158918850cb25d0ff3837d16 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= Date: Tue, 29 Nov 2016 15:07:07 +0100 Subject: refresh: Warn about packages that lack an updater. * guix/upstream.scm (package-update-path): Rename to... (package-latest-release): ... this. Remove 'version>?' check. (package-latest-release*): New procedure. (package-update): Use it. * guix/scripts/refresh.scm (lookup-updater): Rename to... (lookup-updater-by-name): ... this. (warn-no-updater): New procedure. (update-package): Add #:warn? parameter and honor it. (check-for-package-update): New procedure. (guix-refresh)[warn?]: New variable. Replace inline code when UPDATE? is false with a call to 'check-for-package-update'. Pass WARN? to 'check-for-package-update' and 'update-package'. * doc/guix.texi (Invoking guix refresh): Document it. Fix a couple of typos. --- doc/guix.texi | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index ce1e5d075a..4677e5cf79 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5250,10 +5250,19 @@ gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18. gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0 @end example -It does so by browsing the FTP directory of each package and determining -the highest version number of the source tarballs therein. The command +Alternately, one can specify packages to consider, in which case a +warning is emitted for packages that lack an updater: + +@example +$ guix refresh coreutils guile guile-ssh +gnu/packages/ssh.scm:205:2: warning: no updater for guile-ssh +gnu/packages/guile.scm:136:12: guile would be upgraded from 2.0.12 to 2.0.13 +@end example + +@command{guix refresh} browses the upstream repository of each package and determines +the highest version number of the releases therein. The command knows how to update specific types of packages: GNU packages, ELPA -packages, etc.---see the documentation for @option{--type} below. The +packages, etc.---see the documentation for @option{--type} below. There are many packages, though, for which it lacks a method to determine whether a new upstream release is available. However, the mechanism is extensible, so feel free to get in touch with us to add a new method! @@ -5293,7 +5302,7 @@ usually run from a checkout of the Guix source tree (@pxref{Running Guix Before It Is Installed}): @example -$ ./pre-inst-env guix refresh -s non-core +$ ./pre-inst-env guix refresh -s non-core -u @end example @xref{Defining Packages}, for more information on package definitions. @@ -5359,7 +5368,7 @@ In addition, @command{guix refresh} can be passed one or more package names, as in this example: @example -$ ./pre-inst-env guix refresh -u emacs idutils gcc-4.8.4 +$ ./pre-inst-env guix refresh -u emacs idutils gcc@@4.8 @end example @noindent -- cgit v1.2.3