Current Series Release Notes¶

New Features¶

• A new boolean config option serial_console is added to the section compute-feature-enabled. If enabled, tests, which validate the behavior of Nova’s serial console feature (an alternative to VNC, RDP, SPICE) can be executed.

• Add a new client to handle the domain configuration feature from the identity v3 API.

• A new config option in the network-feature-enabled section, floating_ips, to specify whether floating ips are available in the cloud under test. By default this is set to True.

• Add force detach volume feature API to v2 volumes_client library. This feature enables the possibility to force a volume to detach, and roll back an unsuccessful detach operation after you disconnect the volume.

• Defines the identity v3 OS-EP-FILTER extension API client. This client manages associations between endpoints, projects along with groups.

• The delete_volume method of the VolumesClient class now has an additional **params argument that enables passing additional information in the query string of the HTTP request.

• Add list volume transfers with details API to v2 transfers_client library. This feature enables the possibility to list volume transfers with details.

• A new config option ‘manage_snapshot_ref’ is added in the volume section, to specify snapshot ref parameter for different storage backend drivers when managing an existing snapshot. By default it is set to fit the LVM driver.

• Add show host API to the volume v2 hosts_client library. This feature enables the possibility to show details for a host.

• Define v3 volumes_client for the volume service as a library interface, allowing other projects to use this module as a stable library without maintenance changes. Add show volume summary API to v3 volumes_client library, min_microversion of this API is 3.12.
  • volumes_client(v3)

• Define v2 quota_classes_client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.
  • quota_classes_client(v2)

• A new configuration flag api_v2_admin is introduced in the identity feature flag group to allow for enabling/disabling all identity v2 admin tests. The new flag only applies when the existing api_v2 flag is set to True

• A new attr decorator has been added in the tempest.lib.decorators module. For example, use it to tag specific tests, which could be leveraged by test runners to run only a subset of Tempest tests.

• A new related_bug decorator has been added to tempest.lib.decorators. Use it to decorate and tag a test that was added in relation to a launchpad bug report.

• Define v2.0 tags_client for the network service as a library interface, allowing other projects to use this module as a stable library without maintenance changes.
  • tags_client(v2.0)

Upgrade Notes¶

• Switch to use Glance v2 APIs in volume tests, by adding the Glance v2 client images_client.

• The Heat API tests have been removed from tempest, they were unmaintained. The future direction of api test for heat is their in-tree Gabbi tests

• verify_tempest_config command starts using extension_client of cinder v2 API only, because cinder v3 API is current and v2 and v1 are deprecated and v3 extension API is the same as v2. Then we can reuse the v2 client for v3 API also.

Deprecation Notes¶

• Image APIs in compute are deprecated, Image native APIs are recommended. And Glance v1 APIs are deprecated and v2 APIs are current. Image client compute_images_client and Glance v1 APIs are removed in volume tests.

• The config option forbid_global_implied_dsr from the IdentityFeature group is now deprecated. This feature flag was introduced to support testing of old OpenStack versions which are not supported anymore.

• Deprecate default value for configuration parameter v3_endpoint_type of identity section in OpenStack Pike and modify the default value to publicURL in OpenStack Q release.

Bug Fixes¶

• When receiving nullable list as a response body, tempest.lib rest_client module raised an exception without valid json deserialization. A new release fixes this bug.

New Features¶

• Add a new client to handle the OAUTH consumers feature from the identity API.

• Add missing API call, list all role inference rules, to the roles_client library. This feature enables the possibility of listing all role inference rules in the system.

• Add cascade parameter to volumes_client. This option provides the ability to delete a volume and have Cinder handle deletion of snapshots associated with that volume by passing an additional argument to volume delete, “cascade=True”.

• Define the compute server evacuate client method in the servers_client library.

• Add the list auth projects API to the identity client library. This feature enables the possibility to list projects that are available to be scoped to based on the X-Auth-Token provided in the request.

• Add versions_client module for image service. This new module provides list_versions() method which shows API versions from Image service.

• Add the list security groups by server API to the servers_client library. This feature enables the possibility to list security groups for a server instance.

• Add versions_client module for identity service. This new module provides list_versions() method which shows API versions from Identity service.

• Add versions_client module for volume service. This new module provides list_versions() method which shows API versions from Volume service.

• Interface show_quota_set of compute quotas_client has been extended to include the argument “detail”, which allows for detailed quota set information for a project to be retrieved, if set to True.

• Add remote_client under tempest.lib. This remote_client under tempest.lib is defined as stable interface, and now this module provides the following essential methods.
  • exec_command
  • validate_authentication
  • ping_host

• Adds a new cli option to tempest run, –combine, which is used to indicate you want the subunit stream output combined with the previous run’s in the testr repository

• Add update encryption type API to the v2 encryption_types_client library. This feature enables the possibility to update an encryption type for an existing volume type.

• Add the unmanage volume API service method in v2 volumes_client library. Define v2 volume_manage_client client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.
  • volume_manage_client(v2)

• Add server tags APIs to the servers_client library. This feature enables the possibility of upating, deleting and checking existence of a tag on a server, as well as updating and deleting all tags on a server.

• Define volume transfers service clients as libraries. The following volume transfers service clients are defined as library interface.
  • transfers_client(v2)

Upgrade Notes¶

• The JSON_ENC and TXT_ENC option in the _error_checker section have been added with additional content-type which are defined in RFC7231 but missing in the currnt rest_client.py file. The lack of these additional content-type will cause defcore test to fail for OpenStack public cloud which uses tomcat module in the api gateway. The additions are application/json;charset=utf-8, text/html;charset=utf-8,“text/plain;charset=utf-8“

• The default value of rand_name()’s prefix argument is changed to ‘tempest’ from None to identify resources are created by Tempest.

• The volume config option ‘api_v1’ default is changed to False because the volume v1 API has been deprecated since Juno release.

• The call_until_true of test module is removed because it was marked as deprecated and Tempest provides it from test_utils as a stable interface instead. Please switch to use test_utils.call_until_true if necessary.

• Remove Cinder v1 API tests. Cinder v1 API has been deprecated since Juno release, and Juno is not supported by current Tempest. Then Cinder v1 API tests are removed from Tempest.

• The deprecated config option ‘allow_port_security_disabled’ from compute_feature_enabled group has been removed.

• Below deprecated config options from compute group have been removed. Corresponding config options already been available in validation group.
  • compute.image_ssh_user (available as validation.image_ssh_user)
  • compute.ssh_user (available as validation.image_ssh_user)
  • scenario.ssh_user (available as validation.image_ssh_user)
  • compute.network_for_ssh (available as validation.network_for_ssh)
  • compute.ping_timeout `` (available as ``validation.ping_timeout)

• The deprecated config option ‘dvr_extra_resources’ from network group has been removed. This option was for extra resources which were provisioned to bind a router to Neutron L3 agent. The extra resources need to be provisioned in Liberty release or older, and are not required since Mitaka release. Current Tempest doesn’t support Liberty, so this option has been removed from Tempest.

• The deprecated config option ‘reseller’ from identity_feature_enabled group has been removed.

• The deprecated volume_services option in the volume_feature_enabled section has now been removed.

• Tempest now defaults to using Keystone v3 API for the authentication, because Keystone v3 API is CURRENT and the v2 API is deprecated.

Deprecation Notes¶

• The deactivate_image configuration switch from the config module is deprecated. It was added to support the older-than-kilo releases which we don’t support anymore.

• The dvr_extra_resources configuration switch from the config module is deprecated. It was added to support the Liberty Release which we don’t support anymore.

• Glance v1 APIs are deprecated and v2 are current. Tempest should tests only v2 APIs. Below API version selection config options for glance have been deprecated and will be removed in future.
  • CONF.image_feature_enabled.api_v2
  • CONF.image_feature_enabled.api_v1

• The resources_prefix is marked as deprecated because it is enough to set ‘tempest’ as the prefix on rand_name() to ideintify resources which are created by Tempest and no projects set this option on OpenStack dev community.

• The skip_unless_attr decorator in lib/decorators.py has been deprecated, please use the standard testtools.skipUnless and testtools.skipIf decorators.

• The skip_unless_config and skip_if_config decorators in the config module have been deprecated and will be removed in the Queens dev cycle. Use the testtools.skipUnless (or a variation of) instead.

• Volume v1 API is deprecated and the v3 are CURRENT. Tempest doesn’t need to test the v1 API as the default. The volume config option ‘api_v1’ has been marked as deprecated.

• Deprecate the client_parameters argument in tempest.lib.services.clients.ServiceClients. The parameter is actually not honoured already - see https://bugs.launchpad.net/tempest/+bug/1680915

• Deprecate volume v2 transfers resource methods from volumes_client(v2) libraries. Same methods are available in new transfers service client: transfers_client(v2) The following methods of volume v2 volumes_clients have been deprecated:
  • create_volume_transfer (v2.volumes_client)
  • show_volume_transfer (v2.volumes_client)
  • list_volume_transfers (v2.volumes_client)
  • delete_volume_transfer (v2.volumes_client)
  • accept_volume_transfer (v2.volumes_client)

Bug Fixes¶

• Fix below volume v2 service clients to make v2 API call: Bug#1667354
  • SchedulerStatsClient
  • CapabilitiesClient

• The ‘sahara’ config option in the ‘service-available’ group has been moved to the sahara plugin (openstack/sahara-tests) along with tests and service client during the Ocata timeframe. A ‘sahara’ config option was left over on Tempest side, and it’s removed now. As long as the sahara plugin is installed, this change as no impact on users of sahara tests.

Other Notes¶

• OpenStack Releases Supported after this release are Newton and Ocata

  The release under current development as of this tag is Pike, meaning that every Tempest commit is also tested against master branch during the Pike cycle. However, this does not necessarily mean that using Tempest as of this tag will work against Pike (or future releases) cloud.

New Features¶

• As in the [doc]: http://developer.openstack.org/api-ref/image/v2/metadefs-index.html, there are some apis are not included, add them.

  • namespace_objects_client(v2)
  • namespace_tags_client(v2)

• Add the implied roles feature API to the roles_client library. This feature enables the possibility to create inferences rules between roles (a role being implied by another role).

• Define v2 snapshot_manage_client client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.
  • snapshot_manage_client(v2)

• Added customized JSON schema format checker for ‘date-time’ format. Compute response schema will be validated against customized format checker.

Upgrade Notes¶

• The default value for the allow_port_security_disabled option in the compute-feature-enabled section has been changed from False to True.

• The default value for the reseller option in the identity-feature-enabled section has been changed from False to True.

• The default value for the volume_services option in the volume_feature_enabled section has been changed from False to True.

• The deprecated compute microversion config options from ‘compute-feature-enabled’ group have been removed. Those config options are available under ‘compute’ group to configure the min and max microversion for compute service. * CONF.compute.min_microversion * CONF.compute.max_microversion

• Below deprecated config options from compute group have been removed. Corresponding config options already been available in validation group.
  • compute.use_floatingip_for_ssh (available as validation.connect_method)
  • compute.ssh_auth_method (available as validation.auth_method)
  • compute.image_ssh_password (available as validation.image_ssh_password)
  • compute.ssh_shell_prologue (available as validation.ssh_shell_prologue)
  • compute.ping_size `` (available as ``validation.ping_size)
  • compute.ping_count `` (available as ``validation.ping_count)
  • compute.floating_ip_range `` (available as ``validation.floating_ip_range)

• The deprecated input-scenario config options and group have been removed. The input scenarios functionality already being removed from tempest and from this release, their corresponding config options too.

• Below deprecated network config options have been removed. Those config options already been renamed to below meaningful names.
  • tenant_network_cidr (removed) -> project_network_cidr
  • tenant_network_mask_bits (removed) -> project_network_mask_bits
  • tenant_network_v6_cidr (removed) -> project_network_v6_cidr
  • tenant_network_v6_mask_bits (removed) -> project_network_v6_mask_bits
  • tenant_networks_reachable (removed) -> project_networks_reachable

Deprecation Notes¶

• The allow_port_security_disabled option in the compute-feature-enabled section is now deprecated.

• The reseller option in the identity-feature-enabled section is now deprecated.

• The volume_services option in the volume_feature_enabled section is now deprecated.

Other Notes¶

• OpenStack releases supported at this time are Mitaka, Newton, and Ocata.

  The release under current development as of this tag is Pike, meaning that every Tempest commit is also tested against master during the Pike cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Pike (or future releases) cloud.

New Features¶

• The cred_provider abstract class which serves as the basis for both of tempest’s cred providers, pre-provisioned credentials and dynamic credentials, is now a library interface. This provides the common signature required for building a credential provider.

• The cred_client module was added to tempest.lib. This module provides a wrapper to the keystone services client which provides a uniform interface that abstracts out the differences between keystone api versions.

• As in the [doc]: http://developer.openstack.org/api-ref/image/v2/metadefs-index.html, there are some apis are not included, add them.

  • namespace_properties_client(v2)

• A Neutron Service Providers client is added to deal with resources of the ‘/service-providers’ route.

• A new optional parameter port for ssh client (tempest.lib.common.ssh.Client) to specify destination port for a host. The default value is 22.

• Define the Volume v3 service clients as library interfaces, allowing other projects to use these modules as stable libraries without maintenance changes.
  • messages_client(v3)

• The volume_limits client was added to tempest.lib.

• Define volume service clients as libraries. The following volume service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.
  • volumes_client(v1)
  • volumes_client(v2)
  • capabilities_client(v2)
  • scheduler_stats_client(v2)

• Define the identity v3 service client domains_client as a library. Add domains_client to the library interface so the other projects can use this module as a stable library without any maintenance changes.

Upgrade Notes¶

• Add an error translation to list_versions() of versions_client of both compute and network. This can affect users who are expecting that these clients return error status code instead of the exception. It is needed to change the code for handling the exception like the other clients code.

• The Stress tests framework and all the stress tests have been removed.

• All tests for the Ironic project have been removed from Tempest. Those exist as a Tempest plugin in the Ironic project.

• The Negative Tests Generator has been removed (it was not used by any Tempest tests).

• All tests for the Sahara project have been removed from Tempest. They now live as a Tempest plugin in the openstack/sahara-tests repository.

Deprecation Notes¶

• The api_extensions config option in the compute-feature-enabled group is now deprecated. This option will be removed from tempest when all the OpenStack releases supported by tempest no longer support the API extensions mechanism. This was removed from Nova during the Newton cycle, so this will be removed at the Mitaka EOL.

• The bootable config option in the volume_feature_enabled group is removed because the corresponding feature os-set_bootable has been implemented 2.5 years ago and all OpenStack versions which are supported by Tempest should support the feature.

Other Notes¶

• OpenStack releases supported at this time are Mitaka and Newton.

  The release under current development as of this tag is Ocata, meaning that every Tempest commit is also tested against master during the Ocata cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ocata (or future releases) cloud.

New Features¶

• Define identity service clients as libraries. Add new service clients to the library interface so the other projects can use these modules as stable libraries without any maintenance changes.

  • identity_client(v2)
  • groups_client(v3)
  • trusts_client(v3)
  • users_client(v3)
  • identity_client(v3)
  • roles_client(v3)
  • inherited_roles_client(v3)
  • credentials_client(v3)

• Define volume service clients as libraries. The following volume service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.
  • backups_client
  • encryption_types_client (v1)
  • encryption_types_client (v2)
  • qos_clients (v1)
  • qos_clients (v2)
  • snapshots_client (v1)
  • snapshots_client (v2)

• Define the v1 and v2 types_client clients for the volume service as library interfaces, allowing other projects to use these modules as stable libraries without maintenance changes.

Upgrade Notes¶

• the already deprecated tempest-cleanup standalone command has been removed. The corresponding functionalities can be accessed through the unified tempest command (tempest cleanup).

Deprecation Notes¶

• Oslo.utils provides same method get_ipv6_addr_by_EUI64, so deprecate it in Newton and remove it in Ocata.

• The call_until_true function is moved from the tempest.test module to the tempest.lib.common.utils.test_utils module. Backward compatibility is preserved until Ocata.

Other Notes¶

• OpenStack releases supported at this time are Liberty, Mitaka, and Newton.

  The release under current development as of this tag is Ocata, meaning that every Tempest commit is also tested against master during the Ocata cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ocata (or future releases) cloud.

New Features¶

• RestClient now supports setting timeout in urllib3.poolmanager. Clients will use CONF.service_clients.http_timeout for timeout value to wait for http request to response.

• KeystoneAuthProvider will accept http_timeout and will use it in get_credentials.

• Define identity service clients as libraries. The following identity service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • endpoints_client(v3)
  • policies_client (v3)
  • regions_client(v3)
  • services_client(v3)
  • projects_client(v3)

• The Tempest plugin interface contains a new optional method, which allows plugins to declare and automatically register any service client defined in the plugin.

• tempest.lib exposes a new stable interface, the clients module and ServiceClients class, which provides a convenient way for plugin tests to access service clients defined in Tempest as well as service clients defined in all loaded plugins. The new ServiceClients class only exposes for now the service clients which are in tempest.lib, i.e. compute, network and image. The remaining service clients (identity, volume and object-storage) will be added in future updates.

• A new optional interface TempestPlugin.get_service_clients is available to plugins. It allows them to declare any service client they implement. For now this is used by tempest only, for auto-registration of service clients in the new class ServiceClients.

• A new singleton class clients.ClientsRegistry is available. It holds the service clients registration data from all plugins. It is used by ServiceClients for auto-registration of the service clients implemented in plugins.

• A new helper method service_client_config has been added to the stable module config.py that returns extracts from configuration into a dictionary the configuration settings relevant for the initisialisation of a service client.

• Define volume service clients as libraries The following volume service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • availability_zone_client(v1)
  • availability_zone_client(v2)
  • extensions_client(v1)
  • extensions_client(v2)
  • hosts_client(v1)
  • hosts_client(v2)
  • quotas_client(v1)
  • quotas_client(v2)
  • services_client(v1)
  • services_client(v2)

Upgrade Notes¶

• The nova_cert option default is changed to False. The nova certification management APIs were a hold over from ec2, and are not used by any other parts of nova. They are deprecated for removal in nova after the newton release. This makes false a more sensible default going forward.

• The previously deprecated Javelin utility has been removed from Tempest. As an alternative Ansible can be used to construct similar yaml workflows to what Javelin used to provide.

Deprecation Notes¶

• The new clients module provides a stable alternative to tempest classes manager.Manager and clients.Manager. manager.Manager only exists now to smoothen the transition of plugins to the new interface, but it will be removed shortly without further notice.

New Features¶

• Adds a network version client for querying Neutron’s API version discovery URL (“GET /”).

• Tempest library auth interface now supports scope. Scope allows to control the scope of tokens requested via the identity API. Identity V2 supports unscoped and project scoped tokens, but only the latter are implemented. Identity V3 supports unscoped, project and domain scoped token, all three are available.

• Adds the tempest run command to the unified tempest CLI. This new command is used for running tempest tests.

• Adds tempest workspaces command and WorkspaceManager. This is used to have a centralized repository for managing different tempest configurations.

• Adds subunit-describe-calls. A parser for subunit streams to determine what REST API calls are made inside of a test and in what order they are called.

  • Input can be piped in or a file can be specified
  • Output is shortened for stdout, the output file has more information

• Tempest library auth interface now supports filtering with catalog name. Note that filtering by name is only successful if a known service type is provided.

• Define identity service clients as libraries The following identity service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • endpoints_client(v2)
  • roles_client(v2)
  • services_client(v2)
  • tenants_client(v2)
  • users_client(v2)

• Define image service clients as libraries The following image service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • image_members_client(v1)
  • images_client(v1)
  • image_members_client(v2)
  • images_client(v2)
  • namespaces_client(v2)
  • resource_types_client(v2)
  • schemas_client(v2)

• A new test_utils module has been added to tempest.lib.common.utils. It should hold any common utility functions that help writing Tempest tests.

• A new utility function called call_and_ignore_notfound_exc has been added to the test_utils module. That function call another function passed as parameter and ignore the NotFound exception if it raised.

• Define routers_client as stable library interface. The routers_client module is defined as library interface, so the other projects can use the module as stable library without any maintenance changes.

• The RestClient (in tempest.lib.common.rest_client) now supports POSTing and PUTing data with chunked transfer encoding. Just pass an iterable object as the body argument and set the chunked argument to True.

• A new generator called chunkify is added in tempest.lib.common.utils.data_utils that yields fixed-size chunks (slices) from a Python sequence.

Upgrade Notes¶

• The input scenarios functionality no longer exists in tempest. This caused a large number of issues for limited benefit and was only used by a single test, test_server_basic_ops. If you were using this functionality you’ll now have to do it manually with a script and/or tempest workspaces

• The integrated dashboard scenario test has been removed and is now in a separate tempest plugin tempest-horizon. The removed test coverage can be used by installing tempest-horizon on the server where you run tempest.

• The deprecated legacy credential provider has been removed. The only way to configure credentials in tempest now is to use the dynamic or preprovisioned credential providers

• All tests for the Trove project have been removed from tempest. They now live as a tempest plugin in the trove project.

• The location on disk that the tempest init command looks for has changed. Previously it would attempt to use python packaging’s data files to guess where setuptools/distutils were installing data files, which was incredibly unreliable and depended on how you installed tempest and which versions of setuptools, distutils, and python you had installed. Instead, now it will use either /etc/tempest, $XDG_CONFIG_PATH/.config/tempest, or ~/.tempest/etc (attempted in that order). If none of these exist it will create an empty ~/.tempest/etc directory. If you were relying on the previous behavior and none of these directories were being used you will need to move the files to live in one of these directories.

Deprecation Notes¶

• tempest.lib.common.utils.misc.find_test_caller has been moved into the tempest.lib.common.utils.test_utils module. Calling the find_test_caller function with its old location is deprecated.

• All the options in the input-scenario group are now deprecated. These were only used in tree by the now removed input scenarios functionality in test_server_basic_ops. They were only deprecated because there could be external consumers via plugins. They will be removed during the Ocata cycle.

Other Notes¶

• OpenStack Releases Supported after this release are Liberty and Mitaka The release under current development as of this tag is Newton, meaning that every Tempest commit is also tested against master during the Newton cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Newton (or future releases) cloud.

New Features¶

• Tempest library interface addition(API Microversion testing interfaces).

• Compute Microversion testing support in Service Clients.

Other Notes¶

• OpenStack Releases Supported at this time are Kilo, Liberty, Mitaka The release under current development as of this tag is Newton, meaning that every Tempest commit is also tested against master during the Newton cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Newton (or future releases) cloud.

New Features¶

• Tempest library interface

Other Notes¶

• OpenStack Releases Supported at this time are the same as in the previous release 9, Kilo and Liberty.

  The release under current development as of this tag is Mitaka, meaning that every Tempest commit is also tested against master during the Mitaka cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Mitaka (or future releases) cloud.

• Start using reno for managing release notes.