Create debian VM images

Bob Mottram 7c62cee36d Merge branch 'master' of git://git.liw.fi/vmdebootstrap 2 years ago
bin 2750fe0888 Enable systemd-resolved for jessie 2 years ago
common 29c91afaec Extend prepare_apt_source to use a specified local mirror. 2 years ago
doc 80759a16ac add note on custom packages 2 years ago
examples 4bf2d516fb Advise use of log-mode for readable logs 2 years ago
man b73dd28dd4 add --dry-run to the docs 3 years ago
vmdebootstrap ee3c08db33 allow for differences between jessie and stretch 2 years ago
yarns 891282b6d4 be explicit about the distro 2 years ago
.gitignore e4e6de36f0 add a note on pandoc for make -C yarns 3 years ago
COPYING e383d0cdf4 Add missing copy of GPLv3 6 years ago
MANIFEST.in 94d7aff02b Add a squashfs script to use a standard tarball. 3 years ago
NEWS 0f9b13fcb9 prepare for merge 4 years ago
README 87a9b1d1b6 Add pre-commit hook and document usage 3 years ago
README.md 8561747421 Initial commit 2 years ago
TODO acafe6a9cc remove note on qcow2, now completed 3 years ago
pre-commit.sh 87a9b1d1b6 Add pre-commit hook and document usage 3 years ago
setup.py 369740cc28 prepare for 1.6 release 2 years ago
vmextract.py 57f49749e6 initial 2to3 compat changes 3 years ago
vmsquashtar.py 513a847a24 Expand to allow operation on a rootfs directory. 3 years ago

README

README for vmdebootstrap
========================

`debootstrap` installs a basic Debian system into a directory, for use with
`chroot`(8). `vmdebootstrap` is a wrapper around it to install Debian into
a disk image, which can be used with a virtual machine (such as KVM).

See the manual page and `vmdebootstrap --help` for details on how to
use the program. The manual page has an example.

Limitations
-----------

`vmdebootstrap` is aimed principally at creating virtual machines, not
installers or prebuilt installation images. It is possible to create
prebuilt installation images for some devices but this depends on the
specific device. (A 'prebuilt installation image' is a single image file
which can be written to physical media in a single operation and which
allows the device to boot directly into a fully installed system - in a
similar way to how a virtual machine would behave.)

* `vmdebootstrap` assumes that all operations take place on a local image
file, not a physical block device / removable media.
* `vmdebootstrap` is intended to be used with tools like `qemu` on the
command line to launch a new virtual machine. Not all devices have
virtualisation support in hardware.

This has implications for `u-boot` support in some cases. If the device
can support reading the bootloader from a known partition, like the
Beaglebone-black, then `vmdebootstrap` can provide space for the bootloader
and the image will work as a prebuilt installation image. If the device
expects that the bootloader exists at a specific offset and therefore
requires that the bootloader is written as an image not as a binary which
can be copied into an existing partition, `vmdebootstrap` is unable to
include that bootloader image into the virtual machine image.

It is possible to wrap `vmdebootstrap` in such a way as to prepare a
*physical block device* with a bootloader image and then deploy the
bootstrap on top. However, this does require physical media to be
inserted and removed each time the wrapper is executed. Once you have
working media, an image can be created using ``dd`` to read back from
the media to an image file, allowing other media to be written with a
single image file. To do this, use the `--tarball` option to `vmdebootstrap`
instead of the `--image`` option. Then setup the physical media and
bootloader image as required for the device, redefine the partitions to
make space for the rootfs, create a filesystem on the physical media and
unpack the `vmdebootstrap` tarball onto that filesystem.

What you need
-------------

In order to use vmdebootstrap, you'll need a few things:

* debootstrap
* extlinux
* qemu-img (in the qemu-utils package in Debian)
* parted
* mbr
* kpartx
* python-cliapp (see http://liw.fi/cliapp/)
* python-distro-info

Testing vmdebootstrap from git
------------------------------

There is a strongly recommended git pre-commit hook available
for vmdebootstrap development - it requires the ``cmdtest``
package::

ln -s ../../pre-commit.sh .git/hooks/pre-commit

Running vmdebootstrap from git
------------------------------

$ sudo PYTHONPATH=. ./bin/vmdebootstrap

This has changed slightly with version 1.0 with the need for
PYTHONPATH to reference the module approach for support handlers.

vmdebootstrap modules
---------------------

The single vmdebootstrap script has been refactored to be the top
level settings parser and validator and the point where the other
modules (handlers) get to be called in a collaborative sequence.

The new modules are an attempt to work with a DRY process as well
as keeping the source code itself maintainable. Handler functions
need to check settings at the start so that calls to the handlers
can be retained in a simple flow. Where a function needs code from
multiple handlers, that function needs to be in the vmdebootstrap
script but these should, ideally, be single calls into dedicated
calls from the relevant handlers which can return True|False or
raise cliapp.AppException to affect subsequent flow. Handlers must
NOT hook into other handlers, except Base or constants, only the
vmdebootstrap script has the full set, so use function arguments to
pass variables populated by different handlers. Wherever possible,
large sections of new functionality need to be added as new handlers.

pylint
------

When using pylint, the following option is advised:

$ pylint --ignore-imports=y vmdebootstrap

(Despite the name of the option, this only ignores imports when
computing similarities and various handlers will end up needing
similar imports, it makes no sense to complain about that.)

Apart from that, vmdebootstrap uses pylint and contains comments to
disable certain pylint checks in certain areas. pylint compatibility
will make it easier to accept patches, just follow the existing pattern
of pylint usage. pylint is far from perfect but can be helpful.

Testing UEFI support
--------------------

There is EFI firmware available to use with QEMU when testing images
built using the UEFI support, but this software is in Debian non-free
due to patent concerns. If you choose to use it to test UEFI builds,
a secondary change is also needed to symlink the provided OVMF.fd to
the file required by QEMU: bios-256k.bin and then tell QEMU about the
location of this file with the -L option:

$ qemu-system-x86_64 -L /usr/share/ovmf/ -machine accel=kvm \
-m 4096 -smp 2 -drive file=amd64.img,format=raw

Note the use of -drive file=,format=raw which is needed for newer
versions of QEMU.

The vmextract helper
--------------------

Once the image is built, various files can be generated or modified
during the install operations and some of these files can be useful
when testing the image. One example is the initrd built by the process
of installing a Debian kernel. Rather than having to mount the image
and copy the files manually, the vmextract helper can do it for you,
without needing root privileges.

$ /usr/share/vmdebootstrap/vmextract.py --verbose \
--image bbb/bbb-debian-armmp.img --boot \
--path /boot/initrd.img-3.14-2-armmp \
--path /lib/arm-linux-gnueabihf/libresolv.so.2

This uses python-guestfs (a Recommended package for vmdebootstrap) to
prepare a read-only version of the image - in this case with the /boot
partition also mounted - and copies files out into the current working
directory.

The integration test suite
--------------------------

To run the vmdebootstrap integration test suite, you need the yarn
tool, which is part of cmdtest. See and the
cmdtest package in Debian. The command to run the tests is:

sudo yarns/run-tests

You can skip the slow tests that actually build images, by setting the
`TESTS` variable:

sudo yarns/run-tests --env TESTS=fast

To format the test suite document:
(needs pandoc installed)

make -C yarns

Legalese
--------

Copyright 2011-2013 Lars Wirzenius
Copyright 2012 Codethink Limited

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see .