Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 6. Running the certification test suite

Complete the following steps on the test server to run certification tests on the OpenStack deployment under test or test client. Run the tests on OpenStack plugins and drivers for the following services - Block Storage and Shared File System Storage.

Prerequisites

  • You have prepared the test host system as per the instructions provided in Setting up the test environment.
  • You have a working directory on the test host system with write privileges.

Procedure

  1. To initialize the certification test environment, enter the OpenShift namespace for the RHOSO control plane (default is openstack), and then select a certification test suite. 

    # rhoso-cert-init

    The current release supports a cinder test suite and four manila suites. For more information on debugging test failures, see debug suite.

  2. The certification test environment is automatically configured with most of the tempest settings required by each test suite. Running the test suites requires additional settings that you can specify in a tempest-config-overrides file.

    Create the tempest-conf-overrides file and enter the necessary configuration settings.

    Note

    In the tempest-conf-overrides file, enter the necessary configuration settings as separate lines and do not enter comments.

    For example, you can configure the following options by providing an appropriate value in the angular braces (<>).

    Sample configuration settings

    share.share_network_id <network_uuid>
share.alt_share_network_id <network_uuid>
share.admin_share_network_id <network_uuid>
share.security_service <security_service_mapping>
share.backend_replication_type <replication_style>
share.username_for_user_rules <Username>
share.override_ip_for_nfs_access <IP/CIDR>
volume.build_interval 10
volume.build_timeout 300
volume.storage_protocol <iSCSI, FC, NVMe-TCP, NFS, etc.>
volume.vendor_name <Driver's vendor name>

  3. Optional: Create tempest accounts.

    It is beneficial to use a dedicated set of tempest test accounts for some test suites, for example, the manila suites with DHSS=True. Using dedicated test accounts prevents a tempest test from deleting a dynamically created resource, while another test is using the resource. For more information about tempest test accounts, see Using tempest test accounts.

  4. Run the certification test suite. 

    # rhoso-cert-run

    Each test suite will run a series of subtests, and when it completes it reports a Pass or Fail status.

    Note

    In the current RHOSO version, all the tests are executed, including both mandatory and optional tests, such as consistency groups and multi-attach volumes. The optional tests may fail if your driver does not support those features. This is acceptable and does not impact the overall certification process.

    You can view the detailed test results, including the debug data after the certification logs are generated.

  5. Extract the log files. 

    # rhoso-cert-logs

    The system prompts you to include a must-gather report. To speed up the extraction process of the tempest test results, skip the must-gather report by typing no. However, if you are submitting the test results for certification, you must include the must-gather report.

    The logs and debug data are generated in a subdirectory and the name of the subdirectory is displayed at the end of the command.

    Sample output

    Here is an example of the sample output received while extracting the logs for the cinder test suite without a must-gather report. In this example the extracted logs are stored in the directory rhoso-cert-cinder-2024-Aug-01_11-48-08. 

    # rhoso-cert-logs
Spawning a pod to access the logs in PVC rhoso-cert-cinder-6f855...
pod/rhoso-cert-cinder-logs created
Waiting for the rhoso-cert-cinder-logs pod to be ready...
Saving tempest logs...
Saving logs from the individual Tempest pods:
Saving rhoso-cert-cinder-volumes-workflow-step-0-q9btc.log...
Saving rhoso-cert-cinder-backups-workflow-step-1-5p9xf.log...
Saving rhoso-cert-cinder-multi-attach-volume-workflow-step-2-zpwh2.log...
Saving rhoso-cert-cinder-consistency-groups-workflow-step-3-tmb24.log...
Collect a must-gather report? [y/N] : n

Done.
Logs are stored in rhoso-cert-cinder-2024-Aug-01_11-48-08

  6. Save the generated log files. 

    # rhoso-cert-save

    A filtered subset of the generated test result files are stored in the default location /var/log/redhat-certification/. You can upload the generated test result files for Red Hat’s review.

  7. After uploading the test result files, clean up the host system. 

    # rhoso-cert-cleanup

    You can execute this command after completing the certification process, or before selecting a different test suite.

6.1. Optional: Running the debug test suite

The debug test suite executes a single group of tests specified in the rhoso-cert-debug.yaml file. The debug test suite helps you to perform the following tasks:

  • Debug the tempest test failures that you encounter when running the entire cinder, manila or neutron test suite.
  • Rerun specific set of tests that failed but are required for result submission.

Procedure

  1. Copy the YAML file related to the debug test suite to your working directory. 

    # cp /usr/share/redhat-certification-rhoso/rhoso-cert-debug.yaml <current test directory>
  2. Open the local rhoso-cert-debug.yaml file in an editor, and update the list of tests. The list can include just a single test, or a group of tests that you want to execute.
  3. Run the command rhoso-cert-init.
  4. Select the debug test suite.
  5. Rerun the commands rhoso-cert-run and rhoso-cert-logs to execute the selected list of tempest tests.
  6. Collect the logs for Red Hat’s analysis and verification.

6.2. Optional: Using tempest test accounts

Tempest provides a mechanism for specifying a set of test resources by using an accounts.yaml file. It is optional to use test accounts, but it is helpful if the tempest tests fail when a test resource gets prematurely deleted.

You can pre-create the test accounts by providing an accounts.yaml file in the certification working directory. Refer to tempest’s sample accounts.yaml for knowing the structure of this file. You must have three times the number of test accounts as there are concurrent test threads. Since the default test concurrency is 4, the accounts.yaml file used must list 12 test accounts. These accounts must have 'administrator' users in addition to 'member' users and must also include pre-created networks.

Alternatively, the RHOSO certification tools can create the test accounts and the accounts.yaml file when you perform the following steps:

Note

You must manually clean up the test accounts and their resources after completing the certification process.

Procedure

  1. Run the debug suite and ignore any failures. 

    # rhoso-cert-init
# rhoso-cert-run

  2. Generate the tempest test accounts and accounts.yaml file. 

    # rhoso-cert-test-accounts

  3. Clean up after running the debug suite. 

    # rhoso-cert-cleanup

  4. Re-initialize the certification test environment and select the desired test suite. 

    # rhoso-cert-init
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.