From f1a7ed88a0fd70c2bae72c4fd4f45021f43d53ec Mon Sep 17 00:00:00 2001 From: Junichi Uekawa Date: Mon, 20 Aug 2007 08:00:22 +0900 Subject: * update lvmpbuilder, thanks to Kapil Hari Paranjape (closes: #389884) --- examples/lvmpbuilder/README | 58 +++++--- examples/lvmpbuilder/STRATEGY | 29 ++++ examples/lvmpbuilder/lib/lvmbuilder-checkparams | 146 +++++++++++++++++++ examples/lvmpbuilder/lib/lvmbuilder-modules | 138 ++++++++++++++++++ examples/lvmpbuilder/lib/lvmbuilder-unimplemented | 51 +++++++ examples/lvmpbuilder/lvmbuilder | 167 +++++++++++----------- examples/lvmpbuilder/pbuilderrc | 71 --------- examples/lvmpbuilder/sample_device_creator | 25 ---- examples/lvmpbuilder/sample_setup_base | 34 ----- 9 files changed, 484 insertions(+), 235 deletions(-) create mode 100644 examples/lvmpbuilder/STRATEGY create mode 100755 examples/lvmpbuilder/lib/lvmbuilder-checkparams create mode 100755 examples/lvmpbuilder/lib/lvmbuilder-modules create mode 100755 examples/lvmpbuilder/lib/lvmbuilder-unimplemented delete mode 100644 examples/lvmpbuilder/pbuilderrc delete mode 100755 examples/lvmpbuilder/sample_device_creator delete mode 100755 examples/lvmpbuilder/sample_setup_base (limited to 'examples') diff --git a/examples/lvmpbuilder/README b/examples/lvmpbuilder/README index d2a5cb0..15647d9 100644 --- a/examples/lvmpbuilder/README +++ b/examples/lvmpbuilder/README @@ -1,36 +1,50 @@ -The files in this directory (except pbuilderrc) were written by -(C) 2006 Kapil Hari Paranjape . -They are placed in the public domain. You can do with them exactly as you wish. +The files under this directory were written by Kapil Hari Paranjape + and are based on pbuilder by Junichi Uekawa. -These are scripts to help pbuilder run with the device mapper or LVM. -(You do need "dmsetup" to be installed on your system.) +They are placed under the GPL (see the header of each file for +details). + +These files are scripts to help pbuilder run with the linux device mapper. I hope the files are reasonably well commented. If not please ask me. -The rough order of things is as follows. +The order of things is roughly as follows. + +1. Create/assign two block devices for build work. The BASEDEV device + should have enough space to hold the basic debian buildd root + (an unpacked base.tgz from pbuilder). The COWDEV device should have + enough space to actually perform the build. + + WARNING: All data on these two devices will be wiped out + in the process of using lvmbuilder. You have been warned. -1. Use a script like "sample_device_creator" to create two block -devices. The BASE device should have enough space to hold the basic -buildd environment (an unpacked base.tgz from pbuilder). The COW -device should have enough space to perform the build. +1a.You could use a script like "sample_device_creator" to create + two block devices. This creates "loop"back block devices out + of files. + + WARNING: This is a sample script which I no longer use. + Instead I have two real block devices reserved for build work. 2. You should edit your ~/.pbuilderrc and add two lines - BASEDEV= - COWDEV= + BASEDEV= + COWDEV= + you can also give these on the command line with + --basedev + and + --cowdev + +3. Edit the lvmbuilder script to give the location of the lib + subdirectory. -3. Use a script like "sample_setup_base" to setup the BASE device. -Essentially this ensures that BASE device has a working (ext2) file -system which has the entire chroot (e.g. just unpack base.tgz to this -device). In addition, the BASE device *must* be "e2fsck -f"'ed in -order that the resize works in the next step. Optionally you can set -the BASE device to be a read-only device after this step. +4. After this you can use lvmbuilder in place of pbuilder for + the "create", "update", "build", "login" and "execute" operations. + A number of command line options of pbuilder do not make + sense in the lvm context --- these should generate error + messages. See the file STRATEGY for details. -4. Now you can run "lvmbuilder " to build your package. - (a) It ought to be faster than unpacking base.tgz - (b) It will make no changes to the BASE device. Hope this helps! -- Kapil Hari Paranjape -Thu, 28 Sep 2006 12:40:12 +0530 +Wed, 15 Aug 2007 09:55:17 +0530 diff --git a/examples/lvmpbuilder/STRATEGY b/examples/lvmpbuilder/STRATEGY new file mode 100644 index 0000000..3c9c9e9 --- /dev/null +++ b/examples/lvmpbuilder/STRATEGY @@ -0,0 +1,29 @@ +1. We require that the user has defined (either + in the environment or in one of the pbuilderrc + files), two devices: + BASEDEV + This is the device where the persistent + ext2 filesystem containing the basic build + file system lies (what is usually kept + in base.tgz). This file system should be + e2fsck -f clean. create/update/clean go here. + COWDEV + This is the device to which the writes + will actually go during the build/login/execute. + +2. For create/update/clean, we just mount BASEDEV in the appropriate + place and run pbuilder create/update/clean --no-tgz. We run an + e2fsck -f at the end. + +3. For all other operations: + a. we create a new device TOTALDEV consisting of BASEDEV + extended by zeroes by the size of COWDEV. + d. we create a snapshot version WORKDEV of TOTALDEV with + the writes going to COWDEV. + e. we resize2fs the filesystem WORKDEV. This + step could be *slow*. + f. we mount WORKDEV in the appropriate place and run + pbuilder with the appropriate command. + g. on exit or error we must unmount WORKDEV and + remove WORKDEV and TOTALDEV. + diff --git a/examples/lvmpbuilder/lib/lvmbuilder-checkparams b/examples/lvmpbuilder/lib/lvmbuilder-checkparams new file mode 100755 index 0000000..e43cb11 --- /dev/null +++ b/examples/lvmpbuilder/lib/lvmbuilder-checkparams @@ -0,0 +1,146 @@ +#! /bin/bash +# this is sourced from lvmbuilder to process the optional parameters. +# lvmbuilder -- Debian package builder using LVM +# Copyright (C) 2007 Kapil Hari Paranjape +# based on: +# pbuilder -- personal Debian package builder +# Copyright (C) 2001-2006 Junichi Uekawa +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + + +# We check all the parameters that are relevant to us +# and add the rest to the pbuilder command line to +# be used to call pbuilder +. /usr/lib/pbuilder/pbuilder-loadconfig +. $LVMBLIB/lvmbuilder-unimplemented + + +# Storage for the command line +declare -a PBCMDLINE + +# The devices +TOTALNAME=lvmbuilder-total +TOTALDEV=/dev/mapper/$TOTALNAME +WORKNAME=lvmbuilder-work +WORKDEV=/dev/mapper/$WORKNAME + +while [ -n "$1" ]; do + case "$1" in +# First the meaningful options + --basedev) + BASEDEV="$2"; + shift; shift; + ;; + --cowdev) + COWDEV="$2"; + shift; shift; + ;; + --buildplace) + BUILDPLACE="$2"; + shift; shift; + ;; + --debug) + PBCMDLINE+=("--debug"); + set -x; + shift; + ;; + --configfile) + if [ ! -f "$2" ]; then + echo "E: Config file $2 does not exist" >&2 + exit 1 + fi + . "$2"; + PBCMDLINE+=("--configfile" "$2"); + shift; shift; + ;; +# We also grab those options that are likely to be meaningless with +# lvmbuilder + --basetgz) + BASETGZ="$2"; + warn_option $1; + shift; shift; + ;; + --hookdir) + HOOKDIR="$2"; + warn_option $1; + shift; shift; + ;; + --preserve-buildplace) + PRESERVE_BUILDPLACE="yes" + warn_option $1; + shift; + ;; + --bindmounts) + BINDMOUNTS="${BINDMOUNTS} $2" + warn_option $1; + shift; shift; + ;; + --save-after-login|--save-after-exec) + SAVE_AFTER_LOGIN=yes; + warn_option $1; + shift; + ;; + --internal-chrootexec) + CHROOTEXEC="$2" + warn_option $1; + shift; shift; + ;; + --internal-build-uml) + INTERNAL_BUILD_UML="yes" + IGNORE_UMOUNT="no" + warn_option $1; + shift; + ;; + --|*.dsc) + break; + ;; + *) + PBCMDLINE+=("$1"); + shift; + ;; + esac +done + +# Add a no-targz option at the end of the cmdline to override +# any conflicting options set by the user +PBCMDLINE+=("--no-targz"); + +# These variables must be defined properly in order for us to proceed! +BUILDPLACE=${BUILDPLACE:?"E: Build root directory is not defined"} +if [ ! -d "$BUILDPLACE" ] ; then + echo "E: Directory $2 does not exist" >&2 + exit 1 +else + BUILDPLACE=$(readlink -f "$BUILDPLACE") + PBCMDLINE+=("--buildplace" "$BUILDPLACE") +fi +BASEDEV=${BASEDEV:?"E: Base device is not defined"} +if [ ! -b "$BASEDEV" ] ; then + echo "E: Base device is not a block device" >&2 + exit 1 +else + BASEDEV=$(readlink -f "$BASEDEV") +fi +COWDEV=${COWDEV:?"E: Cow device is not defined"} +if [ ! -b "$COWDEV" ]; then + echo "E: Cow device is not a block device" >&2 + exit 1 +else + COWDEV=$(readlink -f "$COWDEV") +fi + +PBCMDLINE+=("$@") + diff --git a/examples/lvmpbuilder/lib/lvmbuilder-modules b/examples/lvmpbuilder/lib/lvmbuilder-modules new file mode 100755 index 0000000..8857f09 --- /dev/null +++ b/examples/lvmpbuilder/lib/lvmbuilder-modules @@ -0,0 +1,138 @@ +#! /bin/bash +# The functions used to mount and unmount the filesystems for lvmbuilder +# lvmbuilder -- Debian package builder using LVM +# Copyright (C) 2007 Kapil Hari Paranjape +# based on: +# pbuilder -- personal Debian package builder +# Copyright (C) 2001-2006 Junichi Uekawa +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +# lvmbuilder by Kapil Hari Paranjape 2007 Aug 15 +# pbuilder by Junichi Uekawa 2001 Aug 25 + +set -e + +function clean_basedev(){ + echo -n "This will wipe out $BASEDEV. Proceed?[y/N]" + read ans + if [ "$ans" != "y" -a "$ans" != "Y" ] + then + exit 0 + fi + echo "Making new file system at $BASEDEV." + mke2fs $BASEDEV +} + +function unmount_basedev(){ + umount $BASEDEV + e2fsck -f -v $BASEDEV +} + +function setup_workplace(){ + # Size computations + BASESIZE=$(blockdev --getsize $BASEDEV) + COWSIZE=$(blockdev --getsize $COWDEV) + TOTAL=$[ $COWSIZE + $BASESIZE ] + + # Load the necessary modules + modprobe dm_zero dm_mirror dm_snapshot dm_mod + + ## Create the total base device + # It consists of the base device extended by zeroes. + # The length of the zeroes is the size of the cow. + ( echo 0 $BASESIZE linear $BASEDEV 0 ; \ + echo $BASESIZE $COWSIZE zero ) | \ + dmsetup create $TOTALNAME + + ## Create the work device + # It is the snapshot device with the all writes + # going to the cow device. The file-system is resized + # to the maximum extent possible. + echo 0 $TOTAL snapshot $TOTALDEV $COWDEV n 16 | \ + dmsetup create $WORKNAME + + [ -b $WORKDEV ] && resize2fs $WORKDEV + mount $WORKDEV $BUILDPLACE +} + +function takedown_workplace(){ + umount /dev/mapper/lvmbuilder-work + dmsetup remove lvmbuilder-work + dmsetup remove lvmbuilder-total +} + +function showhelp(){ + cat <&2 +} +# This portion is not yet used but lists the unimplemented or +# undefined variables. +LVMBADVARS=("APTCACHEHARDLINK" \ + "HOOKDIR" \ + "BASETGZ" \ + "PRESERVE_BUILDPLACE" \ + "BINDMOUNTS" \ + "CHROOTEXEC" \ + "INTERNAL_BUILD_UML" \ + "IGNORE_UMOUNT") + +function unimplemented(){ + for var in "${LVMBADVARS[@]}" + do + if [ -n "${!var}" ] + then + echo "W: Cannot use variable $var with lvmbuilder." >&2 + fi + done +} + diff --git a/examples/lvmpbuilder/lvmbuilder b/examples/lvmpbuilder/lvmbuilder index 788908e..eb373f3 100755 --- a/examples/lvmpbuilder/lvmbuilder +++ b/examples/lvmpbuilder/lvmbuilder @@ -1,88 +1,89 @@ -#!/bin/sh -#The files in this directory (except pbuilderrc) were written by -#(C) 2006 Kapil Hari Paranjape . -#They are placed in the public domain. You can do with them exactly as you wish. - -# This assumes: -# A. -# You have added the following lines to you ~/.pbuilderrc -# BASEDEV= -# COWDEV= +#! /bin/bash +# lvmbuilder -- Debian package builder using LVM +# Copyright (C) 2007 Kapil Hari Paranjape +# based on: +# pbuilder -- personal Debian package builder +# Copyright (C) 2001-2006 Junichi Uekawa # -# B. -# That the BASE and COW device have been properly set up. -# Specifically, you must run: -# e2fsck -f $BASEDEV -# It is too painful to run this everytime! +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. # -# You can also run -# blockdev --setro $BASEDEV -# This tries to ensure that the $BASDEV remains pristine. +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. # -# C. -# It also assumes that /dev/mapper/{total,work} -# do not exist already. - - -if [ $# -lt 1 ] -then - echo You need to supply the name of the dsc file - exit 1 -fi - -PKGDSC=$1 - -# Source the pbuilderrc file -. /usr/lib/pbuilder/pbuilder-loadconfig - - -if [ -z "$BASEDEV" -o -z "$COWDEV" ] -then - echo Please define BASEDEV and COWDEV in your ~/.pbuilderrc file +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +# lvmbuilder by Kapil Hari Paranjape 2007 Aug 15 +# pbuilder by Junichi Uekawa 2001 Aug 25 + +set -e + +# Base directory for LVM stuff +LVMBLIB=./lib 'LVMBLIB needs to be defined' + +# For safe file creation +umask 0022 + +# command line operation +OP="$1" +shift + +case "$OP" in + --create|create) + . ${LVMBLIB}/lvmbuilder-checkparams + . ${LVMBLIB}/lvmbuilder-modules + clean_basedev + mount $BASEDEV $BUILDPLACE + trap unmount_basedev exit + pbuilder "$OP" "${PBCMDLINE[@]}" + ;; + --update|update) + . ${LVMBLIB}/lvmbuilder-checkparams + . ${LVMBLIB}/lvmbuilder-modules + mount $BASEDEV $BUILDPLACE + trap unmount_basedev exit + pbuilder "$OP" "${PBCMDLINE[@]}" + ;; + --clean|clean) + . ${LVMBLIB}/lvmbuilder-checkparams + . ${LVMBLIB}/lvmbuilder-modules + clean_basedev + . /usr/lib/pbuilder/pbuilder-checkparams + if [ -n "$APTCACHE" ]; then + echo "Cleaning [$APTCACHE]" + clean_subdirectories "$APTCACHE" || true + fi + ;; + --build|build|--login|login|--execute|execute) + . ${LVMBLIB}/lvmbuilder-checkparams + . ${LVMBLIB}/lvmbuilder-modules + setup_workplace + trap takedown_workplace exit + pbuilder "$OP" "${PBCMDLINE[@]}" + ;; + --dumpconfig|dumpconfig) + . ${LVMBLIB}/lvmbuilder-checkparams + . ${LVMBLIB}/lvmbuilder-modules + echo " -> start dump config" + echo " -> set" + set; + echo " -> env" + env; + echo " -> end dump config" + ;; + --debuild|debuild) + echo "E: The debuild mode of pbuilder is currently not supported by lvmbuilder." >&2 exit 1 -fi - -# Size computations - -BASESIZE=$(blockdev --getsize $BASEDEV) -COWSIZE=$(blockdev --getsize $COWDEV) -TOTAL=$[ $COWSIZE + $BASESIZE ] - -## Create the total base device -# It consists of the base device extended by zeroes. -# The length of the zeroes is the size of the cow. -dd if=/dev/zero of=$COWDEV bs=4K count=16 - -( echo 0 $BASESIZE linear $BASEDEV 0 ; \ - echo $BASESIZE $COWSIZE zero ) | \ - dmsetup create total - -## Create the work device -# It is the snapshot device with the all writes -# going to the cow device. The file-system is resized -# to the maximum extent possible. -echo 0 $TOTAL snapshot /dev/mapper/total $COWDEV p 16 | \ - dmsetup create work - -resize2fs /dev/mapper/work - -mount /dev/mapper/work $BUILDPLACE - -# run build -pbuilder build --no-targz $PKGDSC - -## Check the status -# If pbuilder returned with -# an error then we must stop -# and be able to examine the work space. -[ $? != 0 ] && \ -( echo Error during build. $BUILDPLACE contains the build place. ; \ - exit 1 ) - -## Clean Up -umount $BUILDPLACE -dmsetup remove work -dmsetup remove total - -echo Done. + ;; + *) + . ${LVMBLIB}/lvmbuilder-modules + showhelp + ;; +esac diff --git a/examples/lvmpbuilder/pbuilderrc b/examples/lvmpbuilder/pbuilderrc deleted file mode 100644 index d8b50e0..0000000 --- a/examples/lvmpbuilder/pbuilderrc +++ /dev/null @@ -1,71 +0,0 @@ -# this is your configuration file for pbuilder. -# the file in /usr/share/pbuilder/pbuilderrc is the default template. -# /etc/pbuilderrc is the one meant for editing. -# -# read pbuilderrc.5 document for notes on specific options. - -BASETGZ=/var/cache/pbuilder/base.tgz -#EXTRAPACKAGES=gcc3.0-athlon-builder -#export DEBIAN_BUILDARCH=athlon -BUILDPLACE=/var/cache/pbuilder/build/ -#NONUSMIRRORSITE="http://ftp.jp.debian.org/debian-non-US" -#OTHERMIRROR="deb http://oak:9999/security/ sarge/updates/" -#OTHERMIRROR="deb file:/home/mirrors/debian/ testing main" -#MIRRORSITE=http://oak:9999/debian/ -MIRRORSITE=http://ftp.iitm.ac.in/debian/ -#export http_proxy=http://your-proxy:8080/ -USEPROC=yes -USEDEVPTS=yes -USEDEVFS=no -BUILDRESULT=/var/cache/pbuilder/result/ - -# specifying the distribution forces the distribution on "pbuilder update" -#DISTRIBUTION=etch -#specify the cache for APT -APTCACHE="/var/cache/pbuilder/aptcache/" -APTCACHEHARDLINK="no" -REMOVEPACKAGES="lilo" -#HOOKDIR="/usr/lib/pbuilder/hooks" -HOOKDIR="" - -# make debconf not interact with user -export DEBIAN_FRONTEND="noninteractive" - -DEBEMAIL="Kapil Hari Paranjape " - -#for pbuilder debuild -BUILDSOURCEROOTCMD="fakeroot" -PBUILDERROOTCMD="sudo" - -#default is to build everything. Passed on to dpkg-buildpackage -#DEBBUILDOPTS="-b" -DEBBUILDOPTS="-k5416E5B8" - -#APT configuration files directory -APTCONFDIR="" - -# the username and ID used by pbuilder, inside chroot. Needs fakeroot, really -BUILDUSERID=1234 -BUILDUSERNAME=pbuilder - -# BINDMOUNTS is a space separated list of things to mount -# inside the chroot. -#BINDMOUNTS="/home " - -# Set the debootstrap variant to 'buildd' type. -DEBOOTSTRAPOPTS[0]='--variant=buildd' -# or work around bug in debootstrap 3.0.0 (314858) -# unset DEBOOTSTRAPOPTS - -# Set the PATH I am going to use inside pbuilder: default is "/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin" -export PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin" - -# SHELL variable is used inside pbuilder by commands like 'su'; and they need sane values -export SHELL=/bin/bash - -# The name of debootstrap command. -DEBOOTSTRAP="debootstrap" - -# Extra variables for LVM build -BASEDEV=/dev/store/pbuilder -COWDEV=/dev/store/cow diff --git a/examples/lvmpbuilder/sample_device_creator b/examples/lvmpbuilder/sample_device_creator deleted file mode 100755 index 4c38810..0000000 --- a/examples/lvmpbuilder/sample_device_creator +++ /dev/null @@ -1,25 +0,0 @@ -#!/bin/sh -#The files in this directory (except pbuilderrc) were written by -#(C) 2006 Kapil Hari Paranjape . -#They are placed in the public domain. You can do with them exactly as you wish. - -## Uncomment these lines after filling the -## full path of a device with at least 2G -# DISK=/dev/hda5 -## ...and... -## Comment these lines to use the actual device -## You entered above -LOOPTMP=$(mktemp /tmp/loop.XXXXXX) -dd if=/dev/zero of=$LOOPTMP bs=10M count=200 -DISK=$(losetup -f) -losetup $DISK $LOOPTMP - -# Possible steps to create base and cow devices -pvcreate $DISK -vgcreate pbuilder $DISK -lvcreate -L 500M -n base pbuilder -lvcreate -L 1.5G -n cow pbuilder - -echo BASEDEV=$(ls /dev/pbuilder/base) && \ -echo COWDEV=$(ls /dev/pbuilder/cow) - diff --git a/examples/lvmpbuilder/sample_setup_base b/examples/lvmpbuilder/sample_setup_base deleted file mode 100755 index 5ce5ce5..0000000 --- a/examples/lvmpbuilder/sample_setup_base +++ /dev/null @@ -1,34 +0,0 @@ -#!/bin/sh -#The files in this directory (except pbuilderrc) were written by -#(C) 2006 Kapil Hari Paranjape . -#They are placed in the public domain. You can do with them exactly as you wish. - -# Source the pbuilderrc file -. /usr/lib/pbuilder/pbuilder-loadconfig - -echo $HOME : $BASEDEV : $COWDEV - -if [ -z "$BASEDEV" -o -z "$COWDEV" ] -then - echo Please define BASEDEV and COWDEV in the pbuilderrc file - exit 1 -fi - -# setup for creation -mke2fs -O dir_index $BASEDEV -mount $BASEDEV $BUILDPLACE - -# Use debootstrap to create -DISTRIBUTION=${DISTRIBUTION:-"sid"} -$DEBOOTSTRAP ${DEBOOTSTRAPOPTS[@]} $DISTRIBUTION $BUILDPLACE $MIRRORSITE - -# run update -pbuilder update --no-targz - -# finalise base -umount $BUILDPLACE -e2fsck -f $BASEDEV -blockdev --setro $BASEDEV - -echo Done. - -- cgit v1.2.3