Cloud images and vmbuilder

Introduction

With Ubuntu being on of the most used operating systems on most of the cloud platforms, the availability of stable and secure cloud images has become very important. Starting with 12.04 the utilization of cloud images outside of a cloud infrastructure has been improved. It is now possible to use those images to create a virtual machine without the need of a complete installation.

Creating virtual machines using cloud images

Cloud images for the supported versions of Ubuntu are available at the following URL :

  • http://cloud-images.ubuntu.com/

When used in conjunction with a tool called cloud-localds which is part of the cloud-utils package starting with Ubuntu 12.10 those images can be used to create a ready to use virtual machine. The following instructions should give you access to a working virtual machine.

Required packages

The following packages will be required in order to use cloud images as virtual machines :

  • kvm

  • cloud-utils

  • genisoimage

Get the Ubuntu Cloud Image

The Ubuntu Cloud Image can be downloaded from the Internet by various means. This example shows how to easily download the 12.04 Precise image using wget :

wget -O my_new_vm.img.dist http://cloud-images.ubuntu.com/server/releases\
/12.04/release/ubuntu-12.04-server-cloudimg-amd64-disk1.img

Create the user-data file

The user-data file contains configuration elements that will be provided to the cloud image and applied at the first boot of the virtual machine using cloud-init. The first three elements, password, chpasswd and ssh_pwauth are mandatory. You should add an ssh key that you have created beforehand using ssh-keygen otherwise you will not be able to connect remotely to your virtual machine.

Use the following command to create the my-user-data file that will contain your user specific data :

$ cat > my-user-data <<EOF
#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: True
ssh_authorized_keys:
 - ssh-rsa {insert your own ssh public key here}
EOF

Convert the cloud-image to Qemu format

The two qemu-img command is not strictly necessary :

  • The 'convert' converts the compressed qcow2 disk image as downloaded to an uncompressed version. If you don't do this the image will still boot, but reads will undergo decompression. This will improve the performance of your virtual machines.

Use the following command to prepare your file to be used as a virtual machine disk:

$ qemu-img convert -O qcow2 my_new_vm.img.dist my_new_vm.img

create the disk with NoCloud data on it

This action will create a second disk image that will be provided to the virtual machine as a second disk. The cloud-init initialization process will fetch this data and configure the virtual machine appropriately

$ cloud-localds my-seed.img my-user-data

Create the XML domain definition file

You will need to tailor the following XML domain definition file to your need in order to create the libvirt domain. If the files that you have generated are in /home/ubuntu, the template can be used as is.

Use the following command to create the template file :

$ cat > my_new_vm.xml <<EOF
<domain type='kvm'>
  <name>my_new_vm</name>
  <memory unit='MiB'>1024</memory>
  <currentMemory unit='MiB'>1024</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <os>
    <type arch='x86_64' machine='pc-1.2'>hvm</type>
    <boot dev='hd'/>
    <bootmenu enable='no'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/ubuntu/my_new_vm.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/home/ubuntu/my-seed.img'/>
      <target dev='hda' bus='ide'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <interface type='network'>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <graphics type='vnc' port='-1' autoport='yes'/>
  </devices>
</domain>
EOF

Create the VM using libvirt

The last few commands remaining are standard libvirt commands used to define and start your virtual machine :

$ virsh define my_new_vm.xml
$ virsh start my_new_vm
$ virsh console my_new_vm

If everything goes as planned, you should see the boot sequence appear in your console session. After the normal boot sequence you will see something similar to the following :

cloud-init start-local running: Wed, 10 Apr 2013 12:30:25 +0000. up 1.67 seconds
no instance data found in start-local
ci-info: lo    : 1 127.0.0.1       255.0.0.0       .
ci-info: eth0  : 1 192.168.122.113 255.255.255.0   52:54:00:c2:fd:e1
ci-info: route-0: 0.0.0.0         192.168.122.1   0.0.0.0         eth0   UG
ci-info: route-1: 192.168.122.0   0.0.0.0         255.255.255.0   eth0   U
cloud-init start running: Wed, 10 Apr 2013 12:30:30 +0000. up 6.30 seconds
found data source: DataSourceNoCloud [seed=/dev/sda]

This section can be particularly useful to identify the IP address of the virtual machine that you have just started. The cloud-init sequence will continue, creating the SSH information. It should indicates proper completion by the following line :

cloud-init boot finished at Wed, 10 Apr 2013 12:30:35 +0000. Up 10.93 seconds

Your new virtual machine is now available. You can exit out of the virsh console command using <Ctrl>]

You can now connect to your virtual machine using the ssh key that you have created previously :

$ ssh -i $HOME/.ssh/id_rsa [email protected]

Vmbuilder

Vmbuilder is now maintained by the community as it is no longer used to generate the cloud images. It can still be used as described and it should let you create functioning virtual machines

What is vmbuilder

. Vmbuilder will fetch the various package and build a virtual machine tailored for your needs in about a minute. vmbuilder is a script that automates the process of creating a ready to use Linux based VM. The currently supported hypervisors are KVM and Xen.

You can pass command line options to add extra packages, remove packages, choose which version of Ubuntu, which mirror etc. On recent hardware with plenty of RAM, tmpdir in /dev/shm or using a tmpfs, and a local mirror, you can bootstrap a VM in less than a minute.

First introduced as a shell script in Ubuntu 8.04 LTS, ubuntu-vm-builder started with little emphasis as a hack to help developers test their new code in a virtual machine without having to restart from scratch each time. As a few Ubuntu administrators started to notice this script, a few of them went on improving it and adapting it for so many use case that Soren Hansen (the author of the script and Ubuntu virtualization specialist, not the golf player) decided to rewrite it from scratch for Intrepid as a python script with a few new design goals:

  • Develop it so that it can be reused by other distributions.

  • Use a plugin mechanisms for all virtualization interactions so that others can easily add logic for other virtualization environments.

  • Provide an easy to maintain web interface as an option to the command line interface.

But the general principles and commands remain the same.

Initial Setup

It is assumed that you have installed and configured libvirt and KVM locally on the machine you are using. For details on how to perform this, please refer to:

We also assume that you know how to use a text based text editor such as nano or vi. If you have not used any of them before, you can get an overview of the various text editors available by reading the PowerUsersTextEditors page. This tutorial has been done on KVM, but the general principle should remain on other virtualization technologies.

Install vmbuilder

The name of the package that we need to install is python-vm-builder. In a terminal prompt enter:

sudo apt-get install python-vm-builder

If you are running Hardy, you can still perform most of this using the older version of the package named ubuntu-vm-builder, there are only a few changes to the syntax of the tool.

Defining Your Virtual Machine

Defining a virtual machine with Ubuntu's vmbuilder is quite simple, but here are a few thing to consider:

  • If you plan on shipping a virtual appliance, do not assume that the end-user will know how to extend disk size to fit their need, so either plan for a large virtual disk to allow for your appliance to grow, or explain fairly well in your documentation how to allocate more space. It might actually be a good idea to store data on some separate external storage.

  • Given that RAM is much easier to allocate in a VM, RAM size should be set to whatever you think is a safe minimum for your appliance.

The vmbuilder command has 2 main parameters: the virtualization technology (hypervisor) and the targeted distribution. Optional parameters are quite numerous and can be found using the following command:

vmbuilder kvm ubuntu --help

Base Parameters

As this example is based on KVM and Ubuntu 13.04 (Raring Ringtail), and we are likely to rebuild the same virtual machine multiple time, we'll invoke vmbuilder with the following first parameters:

sudo vmbuilder kvm ubuntu --suite raring --flavour virtual --arch i386  \
-o --libvirt qemu:///system

The --suite defines the Ubuntu release, the --flavour specifies that we want to use the virtual kernel (that's the one used to build a JeOS image), the --arch tells that we want to use a 32 bit machine, the -o tells vmbuilder to overwrite the previous version of the VM and the --libvirt tells to inform the local virtualization environment to add the resulting VM to the list of available machines.

Notes:

  • Because of the nature of operations performed by vmbuilder, it needs to have root privilege, hence the use of sudo.

  • If your virtual machine needs to use more than 3Gb of ram, you should build a 64 bit machine (--arch amd64).

  • Until Ubuntu 8.10, the virtual kernel was only built for 32 bit architecture, so if you want to define an amd64 machine on Hardy, you should use --flavour server instead.

Installation Parameters

Assigning a fixed IP address

As a virtual appliance that may be deployed on various very different networks, it is very difficult to know what the actual network will look like. In order to simplify configuration, it is a good idea to take an approach similar to what network hardware vendors usually do, namely assigning an initial fixed IP address to the appliance in a private class network that you will provide in your documentation. An address in the range 192.168.0.0/255 is usually a good choice.

To do this we'll use the following parameters:

  • --ip ADDRESS: IP address in dotted form (defaults to dhcp if not specified)

  • --hostname NAME: Set NAME as the hostname of the guest.

  • --mask VALUE: IP mask in dotted form (default: 255.255.255.0)

  • --net VALUE: IP net address (default: X.X.X.0)

  • --bcast VALUE: IP broadcast (default: X.X.X.255)

  • --gw ADDRESS: Gateway address (default: X.X.X.1)

  • --dns ADDRESS: Name server address (default: X.X.X.1)

We assume for now that default values are good enough, so the resulting invocation becomes:

sudo vmbuilder kvm ubuntu --suite raring --flavour virtual --arch i386 \
-o --libvirt qemu:///system --ip 192.168.0.100 --hostname myvm 
Bridging

Because our appliance will be likely to need to be accessed by remote hosts, we need to configure libvirt so that the appliance uses bridge networking. To do this add the --bridge option to the command:

sudo vmbuilder kvm ubuntu --suite raring --flavour virtual --arch i386 \
-o --libvirt qemu:///system --ip 192.168.0.100 --hostname myvm --bridge br0

You will need to have previously setup a bridge interface, see Bridging for more information. Also, if the interface name is different change br0 to the actual bridge interface.

Partitioning

Partitioning of the virtual appliance will have to take into consideration what you are planning to do with is. Because most appliances want to have a separate storage for data, having a separate /var would make sense.

In order to do this vmbuilder provides us with --part:

--part PATH
  Allows you to specify a partition table in a partition file, located at PATH. Each
  line of the partition file should specify (root first):
      mountpoint size
  where  size  is  in megabytes. You can have up to 4 virtual disks, a new disk starts
  on a line with '---'.  ie :
      root 1000
      /opt 1000
      swap 256
      ---
      /var 2000
      /log 1500

In our case we will define a text file name vmbuilder.partition which will contain the following:

root 8000
swap 4000
---
/var 20000

Note that as we are using virtual disk images, the actual sizes that we put here are maximum sizes for these volumes.

Our command line now looks like:

sudo vmbuilder kvm ubuntu --suite raring --flavour virtual --arch i386 \
-o --libvirt qemu:///system --ip 192.168.0.100 --hostname myvm --part vmbuilder.partition

Using a "\" in a command will allow long command strings to wrap to the next line.

User and Password

Again setting up a virtual appliance, you will need to provide a default user and password that is generic so that you can include it in your documentation. We will see later on in this tutorial how we will provide some security by defining a script that will be run the first time a user actually logs in the appliance, that will, among other things, ask him to change his password. In this example I will use 'user' as my user name, and 'default' as the password.

To do this we use the following optional parameters:

  • --user USERNAME: Sets the name of the user to be added. Default: ubuntu.

  • --name FULLNAME: Sets the full name of the user to be added. Default: Ubuntu.

  • --pass PASSWORD: Sets the password for the user. Default: ubuntu.

Our resulting command line becomes:

sudo vmbuilder kvm ubuntu --suite raring --flavour virtual --arch i386 \
-o --libvirt qemu:///system --ip 192.168.0.100 --hostname myvm --part \
vmbuilder.partition --user user --name user --pass default

Installing Required Packages

In this example we will be installing a package (Limesurvey) that accesses a MySQL database and has a web interface. We will therefore require our OS to provide us with:

  • Apache

  • PHP

  • MySQL

  • OpenSSH Server

  • Limesurvey (as an example application that we have packaged)

This is done using vmbuilder by specifying the --addpkg option multiple times:

--addpkg PKG
  Install PKG into the guest (can be specfied multiple times)

However, due to the way vmbuilder operates, packages that have to ask questions to the user during the post install phase are not supported and should instead be installed while interactivity can occur. This is the case of Limesurvey, which we will have to install later, once the user logs in.

Other packages that ask simple debconf question, such as mysql-server asking to set a password, the package can be installed immediately, but we will have to reconfigure it the first time the user logs in.

If some packages that we need to install are not in main, we need to enable the additional repositories using --comp and --ppa:

--components COMP1,COMP2,...,COMPN
           A comma separated list of distro components to include (e.g. main,universe).
           This defaults to "main"
--ppa=PPA  Add ppa belonging to PPA to the vm's sources.list.

Limesurvey not being part of the archive at the moment, we'll specify it's PPA (personal package archive) address so that it is added to the VM /etc/apt/source.list, so we add the following options to the command line:

--addpkg apache2 --addpkg apache2-mpm-prefork --addpkg apache2-utils \
         --addpkg apache2.2-common --addpkg dbconfig-common --addpkg libapache2-mod-php5 \
         --addpkg mysql-client --addpkg php5-cli --addpkg php5-gd --addpkg php5-ldap \
         --addpkg php5-mysql --addpkg wwwconfig-common --addpkg mysql-server --ppa nijaba

Speed Considerations

Package Caching

When vmbuilder creates builds your system, it has to go fetch each one of the packages that composes it over the network to one of the official repositories, which, depending on your internet connection speed and the load of the mirror, can have a big impact on the actual build time. In order to reduce this, it is recommended to either have a local repository (which can be created using apt-mirror) or using a caching proxy such as apt-proxy. The later option being much simpler to implement and requiring less disk space, it is the one we will pick in this tutorial. To install it, simply type:

sudo apt-get install apt-proxy

Once this is complete, your (empty) proxy is ready for use on http://mirroraddress:9999 and will find ubuntu repository under /ubuntu. For vmbuilder to use it, we'll have to use the --mirror option:

--mirror=URL  Use Ubuntu mirror at URL instead of the default, which
              is http://archive.ubuntu.com/ubuntu for official
              arches and http://ports.ubuntu.com/ubuntu-ports
              otherwise

So we add to the command line:

--mirror http://mirroraddress:9999/ubuntu

The mirror address specified here will also be used in the /etc/apt/sources.list of the newly created guest, so it is useful to specify here an address that can be resolved by the guest or to plan on reseting this address later on.

Install a Local Mirror

If we are in a larger environment, it may make sense to setup a local mirror of the Ubuntu repositories. The package apt-mirror provides you with a script that will handle the mirroring for you. You should plan on having about 20 gigabyte of free space per supported release and architecture.

By default, apt-mirror uses the configuration file in /etc/apt/mirror.list. As it is set up, it will replicate only the architecture of the local machine. If you would like to support other architectures on your mirror, simply duplicate the lines starting with "deb", replacing the deb keyword by /deb-{arch} where arch can be i386, amd64, etc... For example, on an amd64 machine, to have the i386 archives as well, you will have (some lines have been split to fit the format of this document):

deb  http://archive.ubuntu.com/ubuntu raring main restricted universe multiverse 
/deb-i386  http://archive.ubuntu.com/ubuntu raring main restricted universe multiverse

deb  http://archive.ubuntu.com/ubuntu raring-updates main restricted universe multiverse 
/deb-i386  http://archive.ubuntu.com/ubuntu raring-updates main
 restricted universe multiverse 

deb http://archive.ubuntu.com/ubuntu/ raring-backports main restricted universe multiverse 
/deb-i386  http://archive.ubuntu.com/ubuntu raring-backports main
 restricted universe multiverse 

deb http://security.ubuntu.com/ubuntu raring-security main restricted universe multiverse 
/deb-i386  http://security.ubuntu.com/ubuntu raring-security main
 restricted universe multiverse 

deb http://archive.ubuntu.com/ubuntu raring main/debian-installer
 restricted/debian-installer universe/debian-installer multiverse/debian-installer 
/deb-i386 http://archive.ubuntu.com/ubuntu raring main/debian-installer
 restricted/debian-installer universe/debian-installer multiverse/debian-installer 

Notice that the source packages are not mirrored as they are seldom used compared to the binaries and they do take a lot more space, but they can be easily added to the list.

Once the mirror has finished replicating (and this can be quite long), you need to configure Apache so that your mirror files (in /var/spool/apt-mirror if you did not change the default), are published by your Apache server. For more information on Apache see HTTPD - Apache2 Web Server.

Package the Application

Two option are available to us:

  • The recommended method to do so is to make a Debian package. Since this is outside of the scope of this tutorial, we will not perform this here and invite the reader to read the documentation on how to do this in the Ubuntu Packaging Guide. In this case it is also a good idea to setup a repository for your package so that updates can be conveniently pulled from it. See the Debian Administration article for a tutorial on this.

  • Manually install the application under /opt as recommended by the FHS guidelines.

In our case we'll use Limesurvey as example web application for which we wish to provide a virtual appliance. As noted before, we've made a version of the package available in a PPA (Personal Package Archive).

Useful Additions

Configuring Automatic Updates

To have your system be configured to update itself on a regular basis, we will just install unattended-upgrades, so we add the following option to our command line:

--addpkg unattended-upgrades

As we have put our application package in a PPA, the process will update not only the system, but also the application each time we update the version in the PPA.

ACPI Event Handling

For your virtual machine to be able to handle restart and shutdown events it is being sent, it is a good idea to install the acpid package as well. To do this we just add the following option:

--addpkg acpid

Final Command

Here is the command with all the options discussed above:

sudo vmbuilder kvm ubuntu --suite raring --flavour virtual --arch i386 -o \ 
         --libvirt qemu:///system --ip 192.168.0.100 --hostname myvm \
         --part vmbuilder.partition --user user --name user --pass default \
         --addpkg apache2 --addpkg apache2-mpm-prefork --addpkg apache2-utils \
         --addpkg apache2.2-common --addpkg dbconfig-common \ 
         --addpkg libapache2-mod-php5 --addpkg mysql-client --addpkg php5-cli \ 
         --addpkg php5-gd --addpkg php5-ldap --addpkg php5-mysql \
         --addpkg wwwconfig-common --addpkg mysql-server \
         --addpkg unattended-upgrades --addpkg acpid --ppa nijaba \
         --mirror http://mirroraddress:9999/ubuntu 
         

Resources

If you are interested in learning more, have questions or suggestions, please contact the Ubuntu Server Team at: