Chapter 13. Hardening infrastructure and virtualization

You must perform the following checks when you add any hardware to your cloud environment:

Keep Red Hat OpenStack Platform (RHOSP) updated for security, performance and supportability.

You must update your SSH keys if they are less than 2048 bits in the following scenarios:

Run the ssh_key_rotation.yaml Ansible Playbook to safely automate the process of rotating your SSH keys.

On the overcloud, backup keys are stored in the following directory:

/home/{{ ansible_user_id }}/backup_keys/{{ ansible_date_time.epoch }}/authorized_keys"

Enable only the hardware and software features that you use, so that less code is exposed to the possibility of attack. There are some features that you should only enable in trusted environments.

You must enable PCI passthrough for specific use cases, such as Network Functions Virtualization (NFV). Do not enable PCI passthrough unless it is necessary for your deployment.

Security-Enhanced Linux (SELinux) is an implementation of mandatory access control (MAC). MAC limits the impact of an attack by restricting what a process or application is permitted to do on a system. For more information on SELinux, see What is SELinux?.

SELinux policies have been pre-configured for Red Hat OpenStack Platform (RHOSP) services. On RHOSP, SELinux is configured to run each QEMU process under a separate security context. In RHOSP, SELinux policies help protect hypervisor hosts and instances against the following threats:

In RHOSP, instance image files on disk are labeled with SELinux data type svirt_image_t. When the instance is powered on, SELinux appends a random numerical identifier to the image. A random numerical identifier can prevent a compromised OpenStack instance from gaining unauthorized access to other containers. SELinux is capable of assigning up to 524,288 numeric identifiers on each hypervisor node.

The OpenStack services that come with Red Hat OpenStack Platform run within containers. Containerization allows for the development and upgrade of services without dependency related conflicts. When a service runs within a container, potential vulnerabilities to that service are also contained.

You can get information about each service that is running in your environment by using the following steps:

You can make changes to containerized services that persist when the container is restarted, but that do not affect the permanent configuration of your Red Hat OpenStack Platform (RHOSP) cluster. This is useful for testing configuration changes, or enabling debug-level logs when troubleshooting. You can revert changes manually. Alternatively, running a redeploy on your RHOSP cluster resets all parameters to their permanent configurations.

Use configuration files that are located in /var/lib/config-data/puppet-generated/[service] to make temporary changes to a service. The following example enables debugging on the OpenStack Compute service (nova):

You can make permanent changes to containerized services in Red Hat OpenStack Platform (RHOSP) services with heat. Use an existing template that you used when you first deployed RHOSP, or create a new template to add to your deployment script. In the following example, the private key size for libvirt is increased to 4096.

Physical servers use complex firmware to enable and operate server hardware and lights-out management cards, which can have their own security vulnerabilities, potentially allowing system access and interruption. Hardware vendors issue firmware updates to address security vulnerabilities. Firmware updates are installed separately from operating system updates.

You must create an operational security process that retrieves, tests, and implements firmware updates on a regular schedule, noting that firmware updates often require a reboot of the physical hosts to become effective.

You can set a banner that displays a console message to all users that connect over SSH. You can add banner text to /etc/issue using the following parameters in an environment file. Consider customizing this sample text to suit your requirements.

resource_registry:
  OS::TripleO::Services::Sshd:
    /usr/share/openstack-tripleo-heat-templates/deployment/sshd/sshd-baremetal-puppet.yaml

parameter_defaults:
  BannerText: |
   ******************************************************************
   * This system is for the use of authorized users only. Usage of  *
   * this system may be monitored and recorded by system personnel. *
   * Anyone using this system expressly consents to such monitoring *
   * and is advised that if such monitoring reveals possible        *
   * evidence of criminal activity, system personnel may provide    *
   * the evidence from such monitoring to law enforcement officials.*
   ******************************************************************

To apply this change to your deployment, save the settings as a file called ssh_banner.yaml, and then pass it to the overcloud deploy command as follows. The <full environment> indicates that you must still include all of your original deployment parameters. For example:

    openstack overcloud deploy --templates \
      -e <full environment> -e  ssh_banner.yaml

Maintaining a record of all audit events helps you establish a system baseline, perform troubleshooting, or analyze the sequence of events that led to a certain outcome. The audit system is capable of logging many types of events, such as changes to the system time, changes to Mandatory or Discretionary Access Control, and the creation or deletion of users or groups.

Rules can be created using an environment file, which are then injected by director into /etc/audit/audit.rules. For example:

    resource_registry:
      OS::TripleO::Services::AuditD: /usr/share/openstack-tripleo-heat-templates/deployment/auditd/auditd-baremetal-puppet.yaml
    parameter_defaults:
      AuditdRules:
        'Record Events that Modify User/Group Information':
          content: '-w /etc/group -p wa -k audit_rules_usergroup_modification'
          order  : 1
        'Collects System Administrator Actions':
          content: '-w /etc/sudoers -p wa -k actions'
          order  : 2
        'Record Events that Modify the Systems Mandatory Access Controls':
          content: '-w /etc/selinux/ -p wa -k MAC-policy'
          order  : 3

Firewall rules are automatically applied on overcloud nodes during deployment, and are intended to only expose the ports required to get OpenStack working. You can specify additional firewall rules as needed. For example, to add a rule for a Zabbix monitoring system:

parameter_defaults:
  ControllerExtraConfig:
    ExtraFirewallRules:
      '301 allow zabbix':
      dport: 10050
      proto: tcp
      source: 10.0.0.8

You can add rules that restrict access. The number used for the rule definition determines the precedence of this rule. To apply a rule to restrict access you must specify a lower value. For example, the rule number for RabbitMQ is 109 by default. Any rule that you add to restrain it, must have a lower value. In this example, 098 and 099 are arbitrarily chosen lower numbers:

parameter_defaults:
  ControllerParameters
    ExtraFirewallRules:
      '098 allow rabbit from internalapi network':
        dport: [4369,5672,25672]
        proto: tcp
        source: 10.0.0.0/24
      '099 drop other rabbit access:
        dport: [4369,5672,25672]
        proto: tcp
        action: drop

To determine the number of a rule, you can inspect the iptables rule on the appropriate node. For example, this is the RabbitMQ rule:

iptables-save
[...]
-A INPUT -p tcp -m multiport --dports 4369,5672,25672 -m comment --comment "109 rabbitmq" -m state --state NEW -j ACCEPT

Alternatively, you can obtain the rule number from the puppet definition. For example, the RabbitMQ rules are stored in puppet/services/rabbitmq.yaml:

    ExtraFirewallRules:
      '109 rabbitmq':
        dport:
          - 4369
          - 5672
          - 25672

You can set the following parameters for a rule:

AIDE (Advanced Intrusion Detection Environment) is a file and directory integrity checker. It is used to detect incidents of unauthorized file tampering or changes. For example, AIDE can alert you if system password files are changed.

AIDE analyzes the system files and compiles an integrity database of file hashes. This database is used to verify the integrity of the files and directories and detect changes. For more information, see Reporting options supported by the AIDE service and How the AIDE service affects system upgrades.

    MyAlias = p+i+n+u+g+s+b+m+c+sha512

SecureTTY allows you to disable root access for any console device (tty). This behavior is managed by entries in the /etc/securetty file. For example:

  resource_registry:
    OS::TripleO::Services::Securetty: ../puppet/services/securetty.yaml

  parameter_defaults:
    TtyValues:
      - console
      - tty1
      - tty2
      - tty3
      - tty4
      - tty5
      - tty6

A thorough auditing process can help you review the ongoing security posture of your OpenStack deployment. This is especially important for keystone, due to its role in the security model.

Red Hat OpenStack Platform has adopted Cloud Auditing Data Federation (CADF) as the data format for audit events, with the keystone service generating CADF events for Identity and Token operations. You can enable CADF auditing for keystone using KeystoneNotificationFormat:

  parameter_defaults:
    KeystoneNotificationFormat: cadf

To enforce password requirements for new system users (non-keystone), director can add entries to /etc/login.defs by following these example parameters:

  resource_registry:
    OS::TripleO::Services::LoginDefs: ../puppet/services/login-defs.yaml

  parameter_defaults:
    PasswordMaxDays: 60
    PasswordMinDays: 1
    PasswordMinLen: 5
    PasswordWarnAge: 7
    FailDelay: 4