Chapter 4. Certification tests
The Red Hat OpenStack application policy includes multiple tests each with a series of subtests and checks. Different certifications will require different tests. Following are the certification tests:
4.1. Overview of system report
The System Report test, also known as openstack/sosreport, captures the basic sosreport. Red Hat uses a tool called sos to collect the configuration and diagnostics information from a Red Hat Enterprise Linux(RHEL) system, and to assist the customers in troubleshooting their system by following the recommended practices.
The system report subtest ensures that the sos tool functions as expected on the image or system and captures a basic sosreport.
Success Criteria
- A basic sosreport can be captured from the system under test
- The test status will be PASS if a valid rpm version captures and collects the openstack data.
Additional resources
- For more information about sosreport, see What is an sosreport and how to create one in Red Hat Enterprise Linux?
4.2. Overview of supportability test
The Supportability test, also known as openstack/supportable, ensures that the test environment is compliant with Red Hat’s support policy. The test confirms that the test node (an OpenStack deployment-under-test) consists only of components RHOSP and RHEL that are supported by Red Hat or the Partner.
An OpenStack deployment-under-test refers to the node where the plugin/application-under-test is installed and also the Undercloud Director node.
4.2.1. Kernel subtest
The kernel subtest checks the kernel module running on the test environment. The version of the kernel can be either the original General Availability (GA) version or any subsequent kernel update released for the RHEL major and minor releases.
The kernel subtest also ensures that the kernel is not tainted when running in the environment.
Success criteria
- The running kernel is a Red Hat kernel.
- The running kernel is released by Red Hat for use with the RHEL version.
- The running kernel is not tainted.
- The running kernel has not been modified.
4.2.2. Kernel modules subtest
The kernel modules subtest verifies that loaded kernel modules are released by Red Hat, either as part of the kernel’s package or added through a Red Hat Driver Update. The kernel module subtest also ensures that kernel modules do not identify as Technology Preview.
Success criteria
- The kernel modules are released by Red Hat and supported.
Additional resources
4.2.3. Hardware Health subtest
The Hardware Health subtest checks the system’s health by testing if the hardware is supported, meets the requirements, and has any known hardware vulnerabilities. The subtest does the following:
Checks that the Red Hat Enterprise Linux (RHEL) kernel does not identify hardware as unsupported. When the kernel identifies unsupported hardware, it will display an unsupported hardware message in the system logs and/or trigger an unsupported kernel taint. This subtest prevents customers from possible production risks which may arise from running Red Hat products on unsupported configurations and environments.
In hypervisor, partitioning, cloud instances, and other virtual machine situations, the kernel may trigger an unsupported hardware message or taint based on the hardware data presented to RHEL by the virtual machine (VM).
Checks that the system under test (SUT) meets the minimum hardware requirements.
- RHEL 8 and 9: Minimum system RAM should be 1.5GB, per CPU logical core count.
- Checks if the kernel has reported any known hardware vulnerabilities, if those vulnerabilities have mitigations and if those mitigations have resolved the vulnerability. Many mitigations are automatic to ensure that customers do not need to take active steps to resolve vulnerabilities. In some cases this is not possible; where most of these remaining cases require changes to the configuration of the system BIOS/firmware which may not be modifiable by customers in all situations.
- Confirms the system does not have any offline CPUs.
- Confirms if Simultaneous Multithreading (SMT) is available, enabled, and active in the system.
Failing any of these tests will result in a WARN from the test suite and should be verified by the partner to have correct and intended behavior.
Success criteria
- The kernel does not have the UNSUPPORTEDHARDWARE taint bit set.
- The kernel does not report an unsupported hardware system message.
- The kernel should not report any vulnerabilities with mitigations as vulnerable.
- The kernel does not report the logic core to installed memory ratio as out of range.
- The kernel does not report CPUs in an offline state.
Additional resources
4.2.4. Installed RPMs subtest
The installed RPMs subtest verifies that RPM packages installed on the system are released by Red Hat and not modified. Modified packages may introduce risks and impact the supportability of the customer’s environment. You might install non-Red Hat packages if necessary, but you must add them to your product’s documentation, and they must not modify or conflict with any Red Hat packages.
Red Hat will review the output of this test if you install non-Red Hat packages.
Success criteria
- The installed Red Hat RPMs are not modified.
- The installed non-Red Hat RPMs are necessary and documented.
- The installed non-Red Hat RPMs do not conflict with Red Hat RPMs or software.
Additional resources
4.2.5. SELinux subtest
Security-Enhanced Linux (SELinux) adds Mandatory Access Control (MAC) to the Linux kernel, and is enabled by default in RHEL. The SELinux subtest confirms that SELinux is running in enforcing mode on the OpenStack deployment-under test.
SELinux policy is administratively-defined, enforced system-wide, and is not set at user discretion reducing vulnerability to privilege escalation attacks helping limit the damage made by configuration mistakes.
Success criteria
SELinux is configured and running in enforcing mode on the OpenStack deployment-under-test.
Additional resources
- For more information on SELinux in RHEL, see SELinux Users and Administrators Guide
4.3. Director test
The Director test also known as openstack/director ensures that the deployment-under-test is originally installed using Red Hat OpenStack Platform Director. This test is required for all OpenStack software certifications.
Red Hat OpenStack Platform Director is the supported toolset for installing and managing a Red Hat OpenStack Platform environment in production. It helps in easy installation of a lean and robust OpenStack cloud and is targeted specifically for enterprise cloud environments where updates, upgrades and infrastructure control are critical for underlying OpenStack operations.
Success criteria
The deployment under test is originally installed using Red Hat OpenStack Platform Director.
Additional resources
- For more information about installing Red Hat OpenStack Platform Director, see Director Installation and Usage Guide.
4.4. VNF testing configuration report test
The VNF testing configuration report test is applicable only for VNF certification. In this test, a Partner selects the operating system on which the VNF is based; provides the link to a report, or uploads a VNF configuration testing report file. The VNF SME, reviews the report that describes the installation, configuration, and testing details that the Partner conducts.
The format of the required report is pre-defined and is at the Partner’s discretion, however it will need to include the following information:
Hardware configuration
- Server make and model
- CPU: make, model, speed, cores, HT
- NIC make/model
- Networking HW make/model
- Storage make/model
- Traffic Generator make/model
Firmware configuration
- Server firmware version
- BMC firmware version
- NIC firmware version
System software configuration
- Version and architecture of Red Hat Enterprise Linux used on the host
- NUMA, Cores, Huge Pages config.
- OpenStack Platform version
- Third party OSP plug-ins used, with versions
- Storage software used
- Architectural / Topology Diagram
VNF configuration
- VNF version
- vCPUs, Memory, storage used in testing
- Dataplane acceleration (ovs-dpdk, sr-iov, etc.)
- Cores allocation - DPDK vs. application
- Bandwidth, IOPS, latency, etc. requirements
Test cases performed:
- Instantiation, termination, scale out/in, healing, HA, and others
