Post-Installation Configuration

The configuration file nova.conf is installed in /etc/nova by default. A default set of options are already configured in nova.conf when you install manually.

Create a nova group, so you can set permissions on the configuration file:

$ sudo addgroup nova

The nova.conf file should have its owner set to root:nova, and mode set to 0640, since the file could contain your MySQL server’s username and password. You also want to ensure that the nova user belongs to the nova group.

$ sudo usermod -g nova nova
$ chown -R username:nova /etc/nova
$ chmod 640 /etc/nova/nova.conf

Most configuration information is available in the nova.conf configuration option file, which is in the /etc/nova directory.

You can use a particular configuration option file by using the option (nova.conf) parameter when running one of the nova-* services. This inserts configuration option definitions from the given configuration file name, which may be useful for debugging or performance tuning.

If you want to maintain the state of all the services, you can use the state_path configuration option to indicate a top-level directory for storing data related to the state of Compute including images if you are using the Compute object store.

You can place comments in the nova.conf file by entering a new line with a # sign at the beginning of the line. To see a listing of all possible configuration options, refer to the tables in this guide. Here are some general purpose configuration options that you can use to learn more about the configuration option file and the node.

 Example nova.conf Configuration Files

The following sections describe many of the configuration option settings that can go into the nova.conf files. Copies of each nova.conf file need to be copied to each compute node. Here are some sample nova.conf files that offer examples of specific configurations.

 Small, private cloud

Here is a simple example nova.conf file for a small private cloud, with all the cloud controller services, database server, and messaging server on the same server. In this case, CONTROLLER_IP represents the IP address of a central server, BRIDGE_INTERFACE represents the bridge such as br100, the NETWORK_INTERFACE represents an interface to your VLAN setup, and passwords are represented as DB_PASSWORD_COMPUTE for your Compute (nova) database password, and RABBIT PASSWORD represents the password to your message queue installation.

[DEFAULT]

# LOGS/STATE
verbose=True
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
rootwrap_config=/etc/nova/rootwrap.conf

# SCHEDULER
compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler

# VOLUMES
# configured in cinder.conf

# COMPUTE
libvirt_type=qemu
compute_driver=libvirt.LibvirtDriver
instance_name_template=instance-%08x
api_paste_config=/etc/nova/api-paste.ini

# COMPUTE/APIS: if you have separate configs for separate services
# this flag is required for both nova-api and nova-compute
allow_resize_to_same_host=True

# APIS
osapi_compute_extension=nova.api.openstack.compute.contrib.standard_extensions
ec2_dmz_host=192.168.206.130
s3_host=192.168.206.130

# RABBITMQ
rabbit_host=192.168.206.130

# GLANCE
image_service=nova.image.glance.GlanceImageService
glance_api_servers=192.168.206.130:9292

# NETWORK
network_manager=nova.network.manager.FlatDHCPManager
force_dhcp_release=True
dhcpbridge_flagfile=/etc/nova/nova.conf
firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
# Change my_ip to match each host
my_ip=192.168.206.130
public_interface=eth0
vlan_interface=eth0
flat_network_bridge=br100
flat_interface=eth0

# NOVNC CONSOLE
novncproxy_base_url=http://192.168.206.130:6080/vnc_auto.html
# Change vncserver_proxyclient_address and vncserver_listen to match each compute host
vncserver_proxyclient_address=192.168.206.130
vncserver_listen=192.168.206.130

# AUTHENTICATION
auth_strategy=keystone
[keystone_authtoken]
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = nova
admin_password = nova
signing_dirname = /tmp/keystone-signing-nova

# DATABASE
[database]
connection=mysql://nova:[email protected]/nova

 KVM, Flat, MySQL, and Glance, OpenStack or EC2 API

This example nova.conf file is from an internal Rackspace test system used for demonstrations.

[DEFAULT]

# LOGS/STATE
verbose=True
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
rootwrap_config=/etc/nova/rootwrap.conf

# SCHEDULER
compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler

# VOLUMES
# configured in cinder.conf

# COMPUTE
libvirt_type=qemu
compute_driver=libvirt.LibvirtDriver
instance_name_template=instance-%08x
api_paste_config=/etc/nova/api-paste.ini

# COMPUTE/APIS: if you have separate configs for separate services
# this flag is required for both nova-api and nova-compute
allow_resize_to_same_host=True

# APIS
osapi_compute_extension=nova.api.openstack.compute.contrib.standard_extensions
ec2_dmz_host=192.168.206.130
s3_host=192.168.206.130

# RABBITMQ
rabbit_host=192.168.206.130

# GLANCE
image_service=nova.image.glance.GlanceImageService
glance_api_servers=192.168.206.130:9292

# NETWORK
network_manager=nova.network.manager.FlatDHCPManager
force_dhcp_release=True
dhcpbridge_flagfile=/etc/nova/nova.conf
firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
# Change my_ip to match each host
my_ip=192.168.206.130
public_interface=eth0
vlan_interface=eth0
flat_network_bridge=br100
flat_interface=eth0

# NOVNC CONSOLE
novncproxy_base_url=http://192.168.206.130:6080/vnc_auto.html
# Change vncserver_proxyclient_address and vncserver_listen to match each compute host
vncserver_proxyclient_address=192.168.206.130
vncserver_listen=192.168.206.130

# AUTHENTICATION
auth_strategy=keystone
[keystone_authtoken]
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = nova
admin_password = nova
signing_dirname = /tmp/keystone-signing-nova

# DATABASE
[database]
connection=mysql://nova:[email protected]/nova

 

Figure 2.1. KVM, Flat, MySQL, and Glance, OpenStack or EC2 API

 XenServer, Flat networking, MySQL, and Glance, OpenStack API

This example nova.conf file is from an internal Rackspace test system.

verbose
nodaemon
network_manager=nova.network.manager.FlatManager
image_service=nova.image.glance.GlanceImageService
flat_network_bridge=xenbr0
compute_driver=xenapi.XenAPIDriver
xenapi_connection_url=https://<XenServer IP>
xenapi_connection_username=root
xenapi_connection_password=supersecret
xenapi_image_upload_handler=nova.virt.xenapi.image.glance.GlanceStore
rescue_timeout=86400
use_ipv6=true

# To enable flat_injected, currently only works on Debian-based systems
flat_injected=true
ipv6_backend=account_identifier
ca_path=./nova/CA

# Add the following to your conf file if you're running on Ubuntu Maverick
xenapi_remap_vbd_dev=true
[database]
connection=mysql://root:<password>@127.0.0.1/nova

 

Figure 2.2. KVM, Flat, MySQL, and Glance, OpenStack or EC2 API

You can use nova.conf file to configure where Compute logs events, the level of logging, and log formats.

To customize log formats for OpenStack Compute, use these configuration option settings.

See the section called “Hypervisors” for details.

There are different methods of authentication for the OpenStack Compute project, including no authentication. The preferred system is the OpenStack Identity Service, code-named Keystone.

To customize authorization settings for Compute, see these configuration settings in nova.conf.

To customize certificate authority settings for Compute, see these configuration settings in nova.conf.

To customize Compute and the Identity service to use LDAP as a backend, refer to these configuration settings in nova.conf.

You can configure Compute to use both IPv4 and IPv6 addresses for communication by putting it into a IPv4/IPv6 dual stack mode. In IPv4/IPv6 dual stack mode, instances can acquire their IPv6 global unicast address by stateless address autoconfiguration mechanism [RFC 4862/2462]. IPv4/IPv6 dual stack mode works with VlanManager and FlatDHCPManager networking modes. In VlanManager, different 64bit global routing prefix is used for each project. In FlatDHCPManager, one 64bit global routing prefix is used for all instances.

This configuration has been tested with VM images that have IPv6 stateless address autoconfiguration capability (must use EUI-64 address for stateless address autoconfiguration), a requirement for any VM you want to run with an IPv6 address. Each node that executes a nova-* service must have python-netaddr and radvd installed.

On all nova-nodes, install python-netaddr:

$ sudo apt-get install python-netaddr

On all nova-network nodes install radvd and configure IPv6 networking:

$ sudo apt-get install radvd
$ sudo bash -c "echo 1 > /proc/sys/net/ipv6/conf/all/forwarding"
$ sudo bash -c "echo 0 > /proc/sys/net/ipv6/conf/all/accept_ra"

Edit the nova.conf file on all nodes to set the use_ipv6 configuration option to True. Restart all nova- services.

When using the command nova network-create you can add a fixed range for IPv6 addresses. You must specify public or private after the create parameter.

$ nova network-create public --fixed-range-v4 fixed_range_v4 --vlan vlan_id --vpn vpn_start --fixed-range-v6 fixed_range_v6

You can set IPv6 global routing prefix by using the --fixed_range_v6 parameter. The default is: fd00::/48. When you use FlatDHCPManager, the command uses the original value of --fixed_range_v6. When you use VlanManager, the command creates prefixes of subnet by incrementing subnet id. Guest VMs uses this prefix for generating their IPv6 global unicast address.

Here is a usage example for VlanManager:

$ nova network-create public --fixed-range-v4 10.0.1.0/24 --vlan 100 --vpn 1000 --fixed-range-v6 fd00:1::/48

Here is a usage example for FlatDHCPManager:

$ nova network-create public  --fixed-range-v4 10.0.2.0/24 --fixed-range-v6 fd00:1::/48

	Note
	Only cloud administrators can perform live migrations. If your cloud is configured to use cells, you can perform live migration within but not between cells.

Migration enables an administrator to move a virtual machine instance from one compute host to another. This feature is useful when a compute host requires maintenance. Migration can also be useful to redistribute the load when many VM instances are running on a specific physical machine.

The migration types are:

The types of live migration are:

The following sections describe how to configure your hosts and compute nodes for migrations by using the KVM and XenServer hypervisors.

 KVM-Libvirt

Enable true live migration

Prerequisites

• Hypervisor: KVM with libvirt

• Shared storage: NOVA-INST-DIR/instances/ (for example, /var/lib/nova/instances) has to be mounted by shared storage. This guide uses NFS but other options, including the OpenStack Gluster Connector are available.

• Instances: Instance can be migrated with iSCSI based volumes

Notes

Because the Compute service does not use the libvirt live migration functionality by default, guests are suspended before migration and might experience several minutes of downtime. For details, see the section called “Enable true live migration”. This guide assumes the default value for instances_path in your nova.conf file (NOVA-INST-DIR/instances). If you have changed the state_path or instances_path variables, modify accordingly. You must specify vncserver_listen=0.0.0.0 or live migration does not work correctly.

Example Compute installation environment

• Prepare at least three servers; for example, HostA, HostB, and HostC.

• HostA is the Cloud Controller, and should run these services: nova-api, nova-scheduler, nova-network, cinder-volume, and nova-objectstore.

• HostB and HostC are the compute nodes that run nova-compute.

• Ensure that NOVA-INST-DIR (set with state_path in the nova.conf file) is the same on all hosts.

• In this example, HostA is the NFSv4 server that exports NOVA-INST-DIR/instances, and HostB and HostC mount it.

 

Procedure 2.1. To configure your system

1. Configure your DNS or /etc/hosts and ensure it is consistent across all hosts. Make sure that the three hosts can perform name resolution with each other. As a test, use the ping command to ping each host from one another.

   $ ping HostA
   $ ping HostB
   $ ping HostC

2. Ensure that the UID and GID of your nova and libvirt users are identical between each of your servers. This ensures that the permissions on the NFS mount works correctly.

3. Follow the instructions at the Ubuntu NFS HowTo to setup an NFS server on HostA, and NFS Clients on HostB and HostC.

   The aim is to export NOVA-INST-DIR/instances from HostA, and have it readable and writable by the nova user on HostB and HostC.

4. Using your knowledge from the Ubuntu documentation, configure the NFS server at HostA by adding this line to the /etc/exports file:

   NOVA-INST-DIR/instances HostA/255.255.0.0(rw,sync,fsid=0,no_root_squash)

   Change the subnet mask (255.255.0.0) to the appropriate value to include the IP addresses of HostB and HostC. Then restart the NFS server:

   $ /etc/init.d/nfs-kernel-server restart
   $ /etc/init.d/idmapd restart

5. Set the 'execute/search' bit on your shared directory.

   On both compute nodes, make sure to enable the 'execute/search' bit to allow qemu to be able to use the images within the directories. On all hosts, run the following command:

   $ chmod o+x NOVA-INST-DIR/instances 

6. Configure NFS at HostB and HostC by adding this line to the /etc/fstab file:

   HostA:/ /NOVA-INST-DIR/instances nfs4 defaults 0 0

   Make sure that you can mount the exported directory can be mounted:

   $ mount -a -v

   Check that HostA can see the "NOVA-INST-DIR/instances/" directory:

   $ ls -ld NOVA-INST-DIR/instances/

   drwxr-xr-x 2 nova nova 4096 2012-05-19 14:34 nova-install-dir/instances/

   Perform the same check at HostB and HostC, paying special attention to the permissions (nova should be able to write):

   $ ls -ld NOVA-INST-DIR/instances/

   drwxr-xr-x 2 nova nova 4096 2012-05-07 14:34 nova-install-dir/instances/

   $ df -k

   Filesystem           1K-blocks      Used Available Use% Mounted on
   /dev/sda1            921514972   4180880 870523828   1% /
   none                  16498340      1228  16497112   1% /dev
   none                  16502856         0  16502856   0% /dev/shm
   none                  16502856       368  16502488   1% /var/run
   none                  16502856         0  16502856   0% /var/lock
   none                  16502856         0  16502856   0% /lib/init/rw
   HostA:        921515008 101921792 772783104  12% /var/lib/nova/instances  ( <--- this line is important.)

7. Update the libvirt configurations so that the calls can be made securely. These methods enable remote access over TCP and are not documented here, please consult your network administrator for assistance in deciding how to configure access.

   • SSH tunnel to libvirtd's UNIX socket

   • libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption

   • libvirtd TCP socket, with TLS for encryption and x509 client certs for authentication

   • libvirtd TCP socket, with TLS for encryption and Kerberos for authentication

   Restart libvirt. After you run the command, ensure that libvirt is successfully restarted:

   $ stop libvirt-bin && start libvirt-bin
   $ ps -ef | grep libvirt

   root 1145 1 0 Nov27 ? 00:00:03 /usr/sbin/libvirtd -d -l

8. Configure your firewall to allow libvirt to communicate between nodes.

   For information about ports that are used with libvirt, see the libvirt documentation By default, libvirt listens on TCP port 16509 and an ephemeral TCP range from 49152 to 49261 is used for the KVM communications. As this guide has disabled libvirt auth, you should take good care that these ports are only open to hosts within your installation.

9. You can now configure options for live migration. In most cases, you do not need to configure any options. The following chart is for advanced usage only.

Table 2.7. Description of configuration options for livemigration

Configuration option=Default value	Description
live_migration_bandwidth=0	(IntOpt) Maximum bandwidth to be used during migration, in Mbps
live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE, VIR_MIGRATE_PEER2PEER	(StrOpt) Migration flags to be set for live migration
live_migration_retry_count=30	(IntOpt) Number of 1 second retries needed in live_migration
live_migration_uri=qemu+tcp://%s/system	(StrOpt) Migration target URI (any included "%s" is replaced with the migration target hostname)

 Enable true live migration

By default, the Compute service does not use the libvirt live migration functionality. To enable this functionality, add the following line to the nova.conf file:

live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE

The Compute service does not use libvirt's live migration by default because there is a risk that the migration process never ends. This can happen if the guest operating system dirties blocks on the disk faster than they can migrated.

 XenServer

Shared storage

Block migration

 Shared storage

Prerequisites

• Compatible XenServer hypervisors. For more information, see the Requirements for Creating Resource Pools section of the XenServer Administrator's Guide.

• Shared storage. An NFS export, visible to all XenServer hosts.

  	Note
  	For the supported NFS versions, see the NFS VHD section of the XenServer Administrator's Guide.

To use shared storage live migration with XenServer hypervisors, the hosts must be joined to a XenServer pool. To create that pool, a host aggregate must be created with special metadata. This metadata is used by the XAPI plug-ins to establish the pool.

 

Procedure 2.2. To use shared storage live migration with XenServer hypervisors

1. Add an NFS VHD storage to your master XenServer, and set it as default SR. For more information, please refer to the NFS VHD section in the XenServer Administrator's Guide.

2. Configure all the compute nodes to use the default sr for pool operations. Add this line to your nova.conf configuration files across your compute nodes:

   sr_matching_filter=default-sr:true

3. Create a host aggregate:

   $ nova aggregate-create <name-for-pool> <availability-zone>

   The command displays a table that contains the ID of the newly created aggregate.

   Now add special metadata to the aggregate, to mark it as a hypervisor pool:

   $ nova aggregate-set-metadata <aggregate-id> hypervisor_pool=true

   $ nova aggregate-set-metadata <aggregate-id> operational_state=created

   Make the first compute node part of that aggregate:

   $ nova aggregate-add-host <aggregate-id> <name-of-master-compute>

   At this point, the host is part of a XenServer pool.

4. Add additional hosts to the pool:

   $ nova aggregate-add-host <aggregate-id> <compute-host-name>

   	Note
   	At this point, the added compute node and the host are shut down, to join the host to the XenServer pool. The operation fails, if any server other than the compute node is running/suspended on your host.

 Block migration

Prerequisites

• Compatible XenServer hypervisors. The hypervisors must support the Storage XenMotion feature. See your XenServer manual to make sure your edition has this feature.

	Notes
	To use block migration, you must use the --block-migrate parameter with the live migration command. Block migration works only with EXT local storage SRs, and the server must not have any volumes attached.

Resize (or Server resize) is the ability to change the flavor of a server, thus allowing it to upscale or downscale according to user needs. For this feature to work properly, some underlying virt layers may need further configuration; this section describes the required configuration steps for each hypervisor layer provided by OpenStack.