path: root/doc/cuirass.texi

\input texinfo
@setfilename cuirass.info
@documentencoding UTF-8
@include version.texi
@settitle Cuirass Reference Manual
@setchapternewpage odd

@copying

This manual is for Cuirass version @value{VERSION}, a build automation
server.

Copyright @copyright{} 2016, 2017 Mathieu Lirzin@*
Copyright @copyright{} 2017 Mathieu Othacehe

@quotation
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.  A
copy of the license is included in the section entitled ``GNU Free
Documentation License''.
@end quotation
@end copying

@dircategory Software development
@direntry
* Cuirass: (cuirass).       Build automation server.
@end direntry

@titlepage
@title Cuirass Reference Manual
@subtitle Build automation server
@subtitle for version @value{VERSION}, @value{UPDATED}
@author by Mathieu Lirzin

@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@contents

@ifnottex
@node Top
@top Cuirass
@insertcopying
@end ifnottex

@c *********************************************************************
@menu
* Introduction::                What is Cuirass about?

Tutorial sections:
* Overview::                    A quick tour of Cuirass

Reference sections:
* Invocation::                  How to run Cuirass.
* Database::                    About the database schema.

* Contributing::                Your help needed!
* GNU Free Documentation License::  The license of this manual.
* Concept Index::               Concepts.

@end menu

@c *********************************************************************
@node Introduction
@unnumbered Introduction
@cindex introduction

@dfn{Cuirass} is a general-purpose build automation server that checks
out source files from @acronym{VCS, Version Control System}
repositories, executes a @dfn{build job}, and notifies the results of
that job.  A build job consists of a combination of tasks such as
compiling source code to binary code and running automated tests.
Notification is achieved by using a database that stores the build
results associated with an HTTP server that provides a convenient way to
access them.

Cuirass is inspired by the @url{https://nixos.org/hydra/, Hydra}
continuous build system. Unlike Hydra, it is built on top of the
@url{https://www.gnu.org/software/guix/, GNU Guix} functional package
manager.

The goal of Cuirass is to provide both on-demand, scheduled, and
triggered builds.  A Build server is an important tool in the software
development process, because it allows modifying the source code while
ensuring the portability and robustness of those changes.  It is the
basis of the @dfn{Continuous integration} practice.

@menu
* Continuous Integration::      A Software development practice
@end menu

@c *********************************************************************
@node Continuous Integration
@unnumberedsec Continuous Integration

@c *********************************************************************
@node Overview
@chapter Overview

@command{cuirass} acts as a daemon polling @acronym{VCS, version control
system} repositories for changes, and evaluating a derivation when
something has changed (@pxref{Derivations, Derivations,, guix, Guix}).
As a final step the derivation is realized and the result of that build
allows you to know if the job succeeded or not.

What is actually done by @command{cuirass} is specified in a @dfn{job
specification} which is represented as an association list which is a
basic and traditional Scheme data structure.  Here is an example of what
a specification might look like:

@lisp
 `((#:name . "hello")
   (#:url . "git://git.savannah.gnu.org/guix.git")
   (#:branch . "master")
   (#:no-compile? . #t)
   (#:load-path . ".")
   (#:proc . cuirass-jobs)
   (#:file . "/tmp/drv-file.scm")
   (#:arguments (subset . "hello")))
@end lisp

In this specification the keys are Scheme keywords which have the nice
property of being self evaluating.  This means that they can't refer to
another value like symbols do.

Currently the only way to add those specifications to cuirass is to put
a list of them in a file and set the @code{--specifications} command
line option argument with the file name when launching the daemon
(@pxref{Invocation}).  The specifications are persistent (they are kept
in a SQLite database) so the next time @command{cuirass} is run the
previously added specifications will remain active even if you don't
keep the @code{--specifications} option.

@c *********************************************************************
@node Invocation
@chapter Invoking cuirass
@cindex invoking cuirass
@cindex cuirass invocation
@cindex options for invoking cuirass

The usual way to invoke @code{cuirass} is as follows:

@example
cuirass --specifications @var{specs}
@end example

Additionally the following options can be used.

@table @code
@item --one-shot
Instead of executing @code{cuirass} as a daemon looping over the jobs.
Only evaluate and build the specifications once.

@item --cache-directory=@var{directory}
@var{directory} is the place where the VCS repositories used by the jobs
are stored.

@item --specifications=@var{specifications-file}
@itemx -S @var{specifications-file}
Add the specifications defined in @var{specifications-file} in the job
database before launching the evaluation and build processes.

@item --database=@var{database}
@itemx -D @var{database}
Use @var{database} as the database containing the jobs and the past
build results. Since @code{cuirass} uses SQLite as a database engine,
@var{database} must be a file name.  If the file doesn't exist, it will
be created.

@item --port=@var{num}
@itemx -p @var{num}
Make the HTTP interface listen on port @var{num}.  Use port 8080 by
default.

@item --interval=@var{n}
@itemx -I @var{n}
Wait @var{n} seconds between each poll.

@item --use-substitutes
This can be useful when you are not interested in building the
dependencies of a particular job.

@item --version
@itemx -V
Display the actual version of @code{cuirass}.

@item --help
@itemx -h
Display an help message that summarize all the options provided.
@end table

@c *********************************************************************
@node Database
@chapter Database schema
@cindex cuirass database
@cindex sqlite database
@cindex persistent configuration

Cuirass uses a SQLite database to store information about jobs and past
build results, but also to coordinate the execution of jobs.

The database contains the following tables: @code{Specifications},
@code{Stamps}, @code{Evaluations}, @code{Derivations}, and
@code{Builds}.  The purpose of each of these tables is explained below.

@section Specifications
@cindex specifications, database

This table stores specifications describing the repository from whence
Cuirass fetches code and the environment in which it will be processed.
Entries in this table must have values for the following text fields:

@table @code
@item repo_name
This field holds the name of the repository.  This field is also the
primary key of this table.

@item url
The URL of the repository.

@item load_path
This text field holds the name of the subdirectory in the checked out
repository that is passed to the @code{evaluate} tool as the Guile load
path.  This directory is interpreted relative to the repository in the
Cuirass cache directory.  This will usually be the current directory
@code{"."}.

@item file
The absolute name of the Scheme file containing PROC.

@item proc
This text field holds the name of the procedure in the Scheme file FILE
that produces a list of jobs.

@item arguments
A list of arguments to be passed to PROC.  This can be used to produce a
different set of jobs using the same PROC.
@end table

The following columns are optional:

@table @code
@item branch
This text field determines which branch of the repository Cuirass should
check out.

@item tag
This text field is an alternative to using BRANCH or REVISION.  It tells
Cuirass to check out the repository at the specified tag.

@item revision
This text field is an alternative to using BRANCH or TAG.  It tells
Cuirass to check out the repository at a particular revision.  In the
case of a git repository this would be a commit hash.

@item no_compile_p
When this integer field holds the value @code{1} Cuirass will skip
compilation for the specified repository.
@end table

@section Stamps
@cindex stamps, database

When a specification is processed, the repository must be downloaded at
a certain revision as specified.  The @code{Stamps} table stores the
current revision for every specification when it is being processed.

The table only has two text columns: @code{specification}, which
references a specification from the @code{Specifications} table via the
field @code{repo_name}, and @code{stamp}, which holds the revision
(e.g. a commit hash).

@section Evaluations
@cindex evaluations, database

An evaluation relates a specification with the revision of the
repository specified therein.  Derivations and builds (see below) each
belong to a specific evaluation.

The @code{Evaluations} table has the following columns:

@table @code
@item id
This is an automatically incrementing numeric identifier.

@item specification
This field holds the @code{repo_name} of a specification from the
@code{Specifications} table.

@item revision
This text field holds the revision string (e.g. a git commit) of the
repository specified in the related specification.
@end table

@section Derivations
@cindex derivations, database

This table associates a tuple of the absolute derivation file name and
evaluation identifier with a job name.

@table @code
@item derivation
This column holds the absolute file name of the Guix derivation that is
supposed to be evaluated for this job.

@item evaluation
This field holds the @code{id} of an evaluation from the
@code{Evaluations} table.

@item job_name
This text field holds the name of the job.

@item system
This text field holds the system name of the derivation.

@item nix_name
This text field holds the name of the derivation ---e.g.,
@code{coreutils-8.24}.

@end table

@section Builds
@cindex builds, database

This table holds records of completed or failed package builds.  Note
that builds are not in a one to one relationship with derivations in
order to keep track of non-deterministic compilations.

@table @code
@item id
This is an automatically incrementing numeric identifier.

@item derivation
This text field holds the absolute name of the derivation file that
resulted in this build.

@item evaluation
This integer field references the evaluation identifier from the
@code{Evaluations} table, indicating to which evaluation this build
belongs.

@item log
This text field holds the absolute file name of the build log file.

@item status
This integer field holds the build status of the derivation.

@item timestamp
This integer field holds a timestamp taken at build creation time.

@item starttime
This integer field holds a timestamp taken at build start time.
Currently, it has the same value as the @code{timestamp} above.

@item stoptime
This integer field holds a timestamp taken at build stop time.
Currently, it has the same value as the @code{timestamp} above.

@end table

@section Outputs
@cindex outputs, database

This table keep tracks for every eventual build outputs. Each build
stored in @code{Builds} table may have zero (if it has failed), one or
multiple outputs.

@table @code
@item build
This field holds the @code{id} of a build from the
@code{Builds} table.

@item name
This text field holds the name of the output.

@item path
This text field holds the path of the output.

@end table
@end table


@c *********************************************************************
@node Contributing
@chapter Contributing

Everyone is welcome to contribute to Cuirass.  You can report bugs, send
patches and share your ideas with others by sending emails the
@email{guix-devel@@gnu.org, mailing list}.

Development is done using the Git distributed version control system.
Thus, access to the repository is not strictly necessary.  We welcome
contributions in the form of patches as produced by @code{git
format-patch}.  Please write commit logs in the ChangeLog format
(@pxref{Change Logs,,, standards, GNU Coding Standards}); you can check
the commit history for examples.

When posting a patch to the mailing list, use @samp{[PATCH] @dots{}} as
a subject.  You may use your email client or the @command{git
send-email} command.  We prefer to get patches in plain text messages,
either inline or as MIME attachments.  You are advised to pay attention
if your email client changes anything like line breaks or indentation
which could potentially break the patches.

@c *********************************************************************
@node GNU Free Documentation License
@appendix GNU Free Documentation License
@cindex license, GNU Free Documentation License
@include fdl-1.3.texi

@c *********************************************************************
@node Concept Index
@unnumbered Concept Index
@printindex cp

@bye