OpenStack Integration Test Suite Guide

Scenario tests simulate a typical end user action workflow to test the integration points between services. The testing framework conducts the set up, tests the integration between services, and then it is torn down. You should tag the tests with which services they relate to, to make it clear which client libraries the test uses.

A scenario is based on a use case, for example:

API tests validate the OpenStack API. Tests use the OpenStack Integration Test Suite implementation of the Openstack API. Both valid and invalid JSON can be used to make sure error responses are valid. Tests can be run independently, and do not rely on the state left by the previous test.

Edit the undercloud.conf file, located in the stack user’s home directory. By default, enable_tempest is set to false. Change this to true.

enable_tempest = true

You are now ready to install the tempest packages and plug-ins, described in Section 3.3, “Installing the OpenStack Integration Test Suite Packages”.

To run the OpenStack Integration Test Suite, you need to first install the necessary packages and create a configuration file that will tell the Integration Test Suite where to find the various OpenStack services and other testing behaviour switches.

On the controller node, as a root user, create a virtual machine called tempest. This machine must run Red Hat Enterprise Linux 7.3 or greater. It also needs to be able to reach the cloud, but it does not have to be part of the cloud. For more information, see Creating Guests with Virt-Manager.

Also, before installing the OpenStack Integration Test Suite, the following networks are required within your Red Hat OpenStack Platform environment:

These networks must be connected through a router.

Create the private network:

$ openstack network create <network_name> --share
$ openstack subnet create <subnet_name> --subnet-range <address/prefix>
$ openstack router create <router_name>
$ openstack router add subnet <router_name> <subnet_name>

Create the public network:

$ openstack network create <network_name> --external \
  --provider-network-type flat
$ openstack subnet create <subnet_name> --subnet-range <address/prefix> \
  --gateway <default_gateway> --no-dhcp --network <network_name>
$ openstack router set <router_name> --external_gateway <public_network_name>

You are now ready to install and configure the OpenStack Integration Test Suite within the tempest virtual machine. For more information, see Section 3.3, “Installing the OpenStack Integration Test Suite Packages”.

Verify your current tempest configuration:

# tempest verify-config -o <output>

output is the output file where your updated configuration is written. This is different from your original configuration file.

The default location for log files is the logs directory within your tempest workspace.

To change this directory, in tempest.conf, under the [DEFAULT] section, set log_dir to the desired directory:

[DEFAULT]
log_dir = <directory>

If you have your own logging configuration file, in tempest.conf, under the [DEFAULT] section, set log_config_append to your file:

[DEFAULT]
log_config_append = <file>

If this is set, all other logging configuration in tempest.conf will be ignored, including log_dir.

The OpenStack Integration Test Suite provides stable interfaces to test the API microversions. This section describes how to implement microversion tests using these interfaces.

You first need to configure options in the tempest.conf configuration file to specify the target microversions. This is to make sure that the supported microversions match the microversions used in the OpenStack cloud. You can run multiple microversion tests in a single Integration Test Suite operation by specifying a range of target microversions.

For example, to limit the range of microversions for the compute service, in the [compute] section of your configuration file, assign values to the min_microversion and max_microversion parameters:

[compute]
min_microversion = 2.14
max_microversion = latest

Smoke testing is a type of preliminary testing which only covers the most important functionality. While they are not comprehensive, running smoke tests can save time if they do identify a problem.

To run the smoke tests:

# ostestr '.*smoke'

A whitelist file is a file which contains regular expressions to select tests to include. Regular expressions are separated by a newline.

To use a whitelist file:

# ostestr --whitelist-file <whitelist_file>

Alternatively:

# ostestr -w <whitelist_file>

A blacklist file is a file which contains regular expressions to select tests to exclude. Regular expressions are separated by a newline.

To use a blacklist file:

# ostestr --blacklist-file <blacklist_file>

Alternatively:

# ostestr -b <blacklist_file>

Run the tests serially:

# ostestr --serial

Run the tests in parallel (this is the default):

# ostestr --parallel

Specify the number of workers to use when running tests in parallel:

# ostestr --concurrency <workers>

Alternatively:

# ostestr -c <workers>

By default, this is set to the number of CPUs.

First you must initialize the saved state. This creates the file saved_state.json, which prevents the cleanup from deleting objects that need to be kept. Typically you would run cleanup with --init-saved-state prior to a tempest run. If this is not the case, saved_state.json must be edited to remove objects you want cleanup to delete.

# tempest cleanup --init-saved-state

Run the cleanup:

# tempest cleanup

A dry run lists the files that would be deleted by a cleanup, but does not delete any files. The files are listed in the dry_run.json file.

# tempest cleanup --dry-run

Delete users and tenants created by tempest:

# tempest cleanup --delete-tempest-conf-objects