path: root/Documentation/pbuilder-doc.xml

<?xml version="1.0" encoding="euc-jp"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" "docbookx.dtd"> 
<!-- the original of this documentation is in pbuilder source tarball -->
<book>
  <title>pbuilder User's Manual</title>
  <chapter>
    <title>Introducing pbuilder</title>
    <sect1>
      <title>Aims of pbuilder</title>
      <para>
	<command>pbuilder</command> stands for 
	Personal Builder, and it is a automatic Debian Package Building system
	for personal environments.
	<command>pbuilder</command> aims to be an 
	easy-to-setup system
	for auto-building Debian packages inside a clean-room
	environment, so that it is possible to verify that
	a package can be built on most Debian installations.
	Clean-room environment is achieved through use of a chroot image,
	so that only minimal packages will be installed inside the
	chroot.
      </para>
      <para>
	Debian distribution consists of free software
	accompanied with source.
	The source code within Debian "main" section
	must build within Debian "main",
	with only the explicitly specified build-dependencies
	installed.
      </para>
      <para>
	The aim of pbuilder is very different from other 
	autobuilding systems in that its aim is not in trying to build
	as many packages. It does not try to guess
	what a package needs, and in most cases it tries the
	worst choice of all if there was a choice to be made.
      </para>
      <para>
	In this way, <command>pbuilder</command> tries to ensure 
	that packages 
	tested against pbuilder will build in 
	most Debian environments, hopefully resulting
	in a good overall Debian source-buildability.
      </para>
      <para>
	The goal of making Debian buildable from source is 
	somewhat achieved, and has progressed well.
	It is known that Debian 3.0 is not quite 
	buildable from source, but the next version should 
	be better.
      </para>
    </sect1>
  </chapter>
  <chapter>
    <title>Using pbuilder</title>
    <sect1>
      <title>Creating base chroot image</title>
      <para>
	<command>pbuilder create</command>
	will create base chroot image.
	Distribution code-name needs to be specified with 
	<command><option>--distribution</option></command>
	command-line option.
	Usually, "sid" is the proper distribution.
      </para>
      <para>
	<command>debootstrap</command> is used to create 
	the bare minimum Debian installation, 
	and then build-essential packages are installed on top 
	of the minimum installation using <command>apt-get</command> 
	inside the chroot.
      </para>
      <para>
	For fuller documentation of command-line options, see
	pbuilder.8 manual page
      </para>
    </sect1>
    <sect1>
      <title>Updating base chroot image</title>
      <para><command>pbuilder update</command>
	will update the chroot image.
	It will extract the chroot, invoke <command>apt-get update</command>
	and <command>apt-get dist-upgrade</command> inside the 
	chroot, and then recreate the base tarball.
      </para>
      <para>
	It is possible to switch the distribution which the chroot 
	tarball is targetted at at this point.
	Specify <command><option>--distribution <parameter>sid</parameter></option> <option>--override-config</option></command> to change the distribution
	to sid.
	<footnote>
	  <para>However, only upgrading is really supported.</para>
	</footnote>
      </para>
      <para>
	For fuller documentation of command-line options, see
	pbuilder.8 manual page
      </para>
    </sect1>
    <sect1>
      <title>Building a package using chroot image</title>
      <para>
	To build a package inside the chroot, invoke
	<command>pbuilder build <option>whatever.dsc</option></command>.
	<command>pbuilder</command> will extract 
	chroot image to a temporal working directory,
	and satisfy the build-dependency inside the chroot,
	and build the package.
	The built packages will be moved to a 
	directory specified with 
	<command><option>--buildresult</option></command>
	command-line option.
      </para>
      <para>
	<command><option>--basetgz</option></command> option can be 
	used to specify which chroot image to use.
      </para>
      <para>
	<command>pbuilder</command> will extract a fresh chroot image
	created with <command>pbuilder build</command>
	and updated with <command>pbuilder update</command>,
	and populate the chroot with build-dependency by parsing 
	debian/control and invoking <command>apt-get</command>.
      </para>
      <para>
	For fuller documentation of command-line options, see
	pbuilder.8 manual page
      </para>
    </sect1>
    <sect1>
      <title>Configuration Files</title>
      <para>
	It is possible to specify all settings by command-line
	option. However, for convenience it is possible to 
	use a configuration file.
      </para>
      <para>
	<filename>/etc/pbuilderrc</filename> and 
	<filename>${HOME}/.pbuilderrc</filename>
	are read in when pbuilder is invoked.
	The possible options are documented in 
	pbuilderrc.5 documentation.
      </para>
    </sect1>
    <sect1>
      <title>Building packages as non-root</title>
      <para>
	<command>pbuilder</command> requires full root privilage 
	when it is satisfying the build-dependency but most packages do not 
	need root privilage, or even do not build when they are root.
	<command>pbuilder </command> can create a user only used 
	inside <command>pbuilder </command> and use that user id when
	building, and use <command>fakeroot</command> command
	when root privilage is required.
      </para>
      <para>
	BUILDUSERID needs should be set to a value for a user id that
	does not exist on the system, so that it is harder for 
	packages that are being built with 
	<command>pbuilder</command> to do harm to the main system.
	BUILDUSERNAME needs to be set to some value, and 
	pbuilder will use that user id and use fakeroot for building.
      </para>
      <para>
	Using the fakerooting method, pbuilder will run with 
	root privilage when it is required, when installing
	packages to the chroot.
      </para>
    </sect1>
    <sect1>
      <title>Using pbuilder for backporting</title>
      <para>
	pbuilder can be used for backporting software from 
	the latest Debian distribution to 
	older stable distribution, by using a chroot that contains
	image of older distribution, and building packages inside the
	chroot.
	There are several points to consider, and due to the following reasons,
	automatic backporting is usually not possible, and 
	manual interaction is required:
      </para>
      <itemizedlist>
	<listitem>
	  <para>Build-Dependency in stable may not be enough to build a package in unstable distribution, so package may need more than what exists in stable</para>
	</listitem>
	<listitem>
	  <para>Stable distribution may have bugs that have been fixed in unstable that needs to be worked around.</para></listitem>
	<listitem>
	  <para>Package in unstable distribution may have problem building even for unstable.</para>
	</listitem>
      </itemizedlist>
    </sect1>
    <sect1>
      <title>Mass-building packages</title>
      <para>
	pbuilder can be automated, because its operation is 
	noninteractive.
	It is possible to run pbuilder through multiple packages 
	noninteractively.
	There are several such scripts known to exist.
	Junichi Uekawa has been running such script since 2001,
	and has been filing bugs on packages that fail the 
	test of pbuilder. There were several problems with autobuilding:
      </para>
      <itemizedlist>
	<listitem>
	  <para>Build-Dependency needs to install noninteractively, but 
	    some packages are so broken that they cannot install 
	    without interaction (like postgresql)</para>
	</listitem>
	<listitem>
	  <para>When a library package breaks, or gcc/gcj/g++ breaks, 
	    or even bison, a large number of build failure are reported.
	    (gcj-3.0 which had no "javac", bison which got more strict, etc.)
	  </para>
	</listitem>
	<listitem>
	  <para>Some people were quite hostile against build failure reports.</para>
	</listitem>
      </itemizedlist>
      <para>
	But most of these problems are now getting solved.
	Only about 10% of Debian now fail to build from source (29 Dec 2002).
      </para>
      <para>
	A script that was used by Junichi Uekawa is now included in 
	pbuilder distribution, as <command>pbuildd.sh</command>.
	It is available in <filename>/usr/share/doc/pbuilder/examples/pbuildd/</filename>
	and its configuration is in <filename>/etc/pbuilder/pbuildd-config.sh</filename>.
	It should be easy enough to set up for people who are used to 
	pbuilder. It has been running for quite a while, and it should be 
	possible to set the application up on your system also.
	However, it is a new introduction, and please file bugs
	to the Debian BTS if you know of possible problems,
	or improved on the script considerably.
      </para>
    </sect1>
    <sect1>
      <title>Auto-backporting scripts</title>
      <para>
	There are some people who use pbuilder to automatically backport
	a subset of packages to the stable distribution.
	Any URL?
      </para>
    </sect1>
    <sect1>
      <title>Using pbuilder for automated testing of packages</title>
      <para>
	pbuilder can be used for automated testing of pacakges.
	It has the feature of allowing hooks to be placed,
	and these hooks can try to install packages inside
	the chroot, or run them, or whatever else that 
	can be done. Some known tests and ideas:
      </para>
      <itemizedlist>
	<listitem>
	  <para>Automatic install-remove-upgrade-remove-install-purge-upgrade-purge testsuite (distributed as an example)</para>
	</listitem>
	<listitem>
	  <para>Automatic lintian running (distributed as an example)</para>
	</listitem>
	<listitem>
	  <para>Automatic debian-test of the package?</para>
	</listitem>
      </itemizedlist>
    </sect1>
  </chapter>
  <chapter>
    <title>Enhanced and experimental features of pbuilder</title>
    <para>
      There are some advanced features, above that of the 
      basic feature of pbuilder, for some specific purposes.
    </para>
    <sect1>
      <title>Using User-mode-linux</title>
      <para>
	<command>pbuilder-uml</command> exists.
	Invoking that command instead of <command>pbuilder</command>
	it is possible to use user-mode-linux.
	The advantage of using user-mode-linux is that 
	it does not require root privilege to run,
	and it does Copy-on-write, which is probably  much faster than
	conventional pbuilder method.
      </para>
      <para>
	The problem is that this relies on User-mode-linux
	which is a relatively new project, and has not quite
	matured, as opposed to conventional pbuilder which rely 
	on <command>chroot</command> and <command>tar</command> 
	and <command>gzip</command>, which are known to work
	on most Unix systems.
      </para>
      <para>
	Currently there are problems with rootstrap that 
	stops <command>pbuilder-uml</command> from starting,
	and help, or success reports would be appreciated.
      </para>
    </sect1>
    <sect1>
      <title>Using LVM</title>
      <para>
	LVM has snapshot function that features Copy-on-write images.
	That could be used for pbuilder just it can be used for 
	user-mode-linux pbuilder port.
	It may prove to be faster, but it is not implemented yet,
	and so no measurement has been made, yet.
      </para>
    </sect1>
  </chapter>
  <chapter>
    <title>Minor details</title>
    <sect1>
      <title>Documentation history </title>
      <para>
	This document is started on 28 Dec 2002 by
	Junichi Uekawa, trying to document what is known
	about pbuilder.
      </para>
      <para>
	This documentation is available from pbuilder source tarball,
	and from CVS repository of pbuilder (which might not be 
	public yet now).
	A copy of this documentation can be found in
	<ulink url="http://www.netfort.gr.jp/~dancer/software/pbuilder-doc/pbuilder-doc.html">Netfort page for pbuilder</ulink>.
	The homepage for pbuilder is 
	<ulink url="http://www.netfort.gr.jp/~dancer/software/pbuilder.html">
	  http://www.netfort.gr.jp/~dancer/software/pbuilder.html
	</ulink>
      </para>
    </sect1>
    <sect1>
      <title>Inaccurate Background of pbuilder</title>
      <para>
	The following is most possibly inaccurate account of how 
	pbuilder happened to come, and other attempts to 
	make something like pbuilder to happen.
      </para>
      <sect2>
	<title>The Time Before PBuilder</title>
	<para>
	  There were dbuild, which was a shell script to build
	  Debian packages from source. Lars Wirzenius wrote that 
	  script, and it was good, short, and simple (probably).
	  There was nothing like build-depends then (I think), and it was simple.
	  It could have been improved, I don't have the source off-hand.
	</para>
	<para>
	  debbuild was probably written by James Troup. I don't know it 
	  because I have never seen the actual code, I could only find some
	  references to it on the net, and mailing list logs.
	</para>
	<para>
	  sbuild is a perl script to build Debian package from source.
	  It parses Build-Dependency, and performs other misc checks,
	  and has a lot of hacks to actually get things building,
	  including a table of what package to use when virtual packages are
	  specified (does it do that still?).
	  It supports use of local database for packages which do not 
	  have build-dependency. It was written by Ronan Hodek, 
	  and I think it was patched and fixed and extended by
	  several people. It is part of wanna-build, and used extensively
	  in Debian buildd system. I think it was maintained
	  mostly by Ryan Murray.
	</para>
      </sect2>
      <sect2>
	<title>Birth of PBuilder</title>
	<para>
	  wanna-build (sbuild) was quite difficult to set up, and it was 
	  never a Debian package. dbuild was something that predated
	  Build-Depends.
	</para>
	<para>
	  Building package from source using Build-Depends 
	  information within a chroot sounded trivial; and 
	  pbuilder was born. It was initially a shell script
	  with only a few lines, which called debootstrap
	  and chroot and dpkg-buildpackage in the same run,
	  but soon, it was decided that's too slow.
	</para>
	<para>
	  Yes, and it took almost an year to get things somewhat 
	  right, and in the middle of the process, Debian 3.0
	  was released. Yay.
	  Debian 3.0 wasn't completely buildable with pbuilder,
	  but the amount of packages which are not buildable
	  are steadily decreasing. (I hope)
	</para>
      </sect2>
      <sect2>
	<title>And the second year of its life</title>
	<para>
	  And someone wanted pbuilder to run as not root,
	  and User-mode-linux has become more useful as time passed,
	  I've started experimenting with pbuilder-uml.
	  pbuilder-uml has not been able to run as often as it should,
	  and bootstrapping user-mode-linux environment has been
	  pretty hard.
	</para>
      </sect2>


    </sect1>
  </chapter>
</book>