Chapter 2. Active-Active Disaster Recovery
This chapter provides instructions to configure Red Hat Virtualization for disaster recovery using the active-active disaster recovery solution.
2.1. Active-Active Overview
Red Hat Virtualization supports an active-active disaster recovery failover configuration that can span two sites. Both sites are active, and if the primary site becomes unavailable, the Red Hat Virtualization environment will continue to operate in the secondary site to ensure business continuity.
The active-active failover is achieved by configuring a stretch cluster where hosts capable of running the virtual machines are located in the primary and secondary site. All the hosts belong to the same Red Hat Virtualization cluster.
You require replicated storage that is writeable on both sites to allow virtual machines to migrate between sites and continue running on the site’s storage.
Figure 2.1. Stretch Cluster Configuration
Virtual machines will migrate to the secondary site if the primary site becomes unavailable. The virtual machines will automatically failback to the primary site when the site becomes available and the storage is replicated in both sites.
Figure 2.2. Failed Over Stretch Cluster
To ensure virtual machine failover and failback works:
- Virtual machines must be configured to be highly available, and each virtual machine must have a lease on a target storage domain to ensure the virtual machine can start even without power management.
- Soft enforced virtual machine to host affinity must be configured to ensure the virtual machines only start on the selected hosts.
For more information see Improving Uptime with Virtual Machine High Availability and Affinity Groups in the Virtual Machine Management Guide.
The stretched cluster configuration can be implemented using a self-hosted engine environment, or a standalone Manager environment. For more information about the different types of deployments see Red Hat Virtualization Architecture in the Product Guide.
2.1.1. Network Considerations
All hosts in the cluster must be on the same broadcast domain over an L2 network. This means that connectivity between the two sites needs to be L2.
The maximum latency requirements between the sites across the L2 network differs for the two setups. The standalone Manager environment requires a maximum latency of 100ms, while the self-hosted engine environment requires a maximum latency of 7ms.
2.1.2. Storage Considerations
The storage domain for Red Hat Virtualization can be made of either block devices (SAN - iSCSI or FCP) or a file system (NAS - NFS, GlusterFS, or other POSIX compliant file systems). For more information about Red Hat Virtualization storage see Storage in the Administration Guide.
The sites require synchronously replicated storage that is writeable on both sites with shared layer 2 (L2) network connectivity. The replicated storage is required to allow virtual machines to migrate between sites and continue running on the site’s storage. All storage replication options supported by Red Hat Enterprise Linux 7 and later can be used in the stretch cluster.
If you have a custom multipath configuration that is recommended by the storage vendor, see the instructions and important limitations in Customizing Multipath Configurations for SAN Vendors.
2.2. Configure a Self-hosted Engine Stretch Cluster Environment
This procedure provides instructions to configure a stretch cluster using a self-hosted engine deployment.
Prerequisites:
- A writable storage server in both sites with L2 network connectivity.
- Real-time storage replication service to duplicate the storage.
Limitations:
- Maximum 7ms latency between sites.
Configuring the Self-hosted Engine Stretch Cluster
- Deploy the self-hosted engine. See Installing Red Hat Virtualization as a self-hosted engine using the Cockpit web interface.
- Install additional self-hosted engine nodes in each site and add them to your cluster. See Adding Self-hosted Engine Nodes to the Red Hat Virtualization Manager in Installing Red Hat Virtualization as a self-hosted engine using the Cockpit web interface.
- Optionally, install additional standard hosts. See Adding Standard Hosts to the Red Hat Virtualization Manager in Installing Red Hat Virtualization as a self-hosted engine using the Cockpit web interface.
- Configure the SPM priority to be higher on all hosts in the primary site to ensure SPM failover to the secondary site occurs only when all hosts in the primary site are unavailable. See SPM Priority in the Administration Guide.
- Configure all virtual machines that need to failover as highly available, and ensure that the virtual machine has a lease on the target storage domain. See Configuring a Highly Available Virtual Machine in the Virtual Machine Management Guide.
- Configure virtual machine to host soft affinity and define the behavior you expect from the affinity group. See Affinity Groups in the Virtual Machine Management Guide and Scheduling Policies in the Administration Guide.
The active-active failover can be manually performed by placing the main site’s hosts into maintenance mode.
2.3. Configure a Standalone Manager Stretch Cluster Environment
This procedure provides instructions to configure a stretch cluster using a standalone Manager deployment.
Prerequisites:
- A writable storage server in both sites with L2 network connectivity.
- Real-time storage replication service to duplicate the storage.
Limitations:
- Maximum 100ms latency between sites.
The Manager must be highly available for virtual machines to failover and failback between sites. If the Manager goes down with the site, the virtual machines will not failover.
The standalone Manager is only highly available when managed externally. For example:
- Using Red Hat’s High Availability Add-On.
- As a highly available virtual machine in a separate virtualization environment.
- Using Red Hat Enterprise Linux Cluster Suite.
- In a public cloud.
Configuring the Standalone Manager Stretch Cluster
- Install and configure the Red Hat Virtualization Manager. See Installing Red Hat Virtualization as a standalone Manager with local databases.
- Install the hosts in each site and add them to the cluster. See Installing Hosts for Red Hat Virtualization in Installing Red Hat Virtualization as a standalone Manager with local databases.
- Configure the SPM priority to be higher on all hosts in the primary site to ensure SPM failover to the secondary site occurs only when all hosts in the primary site are unavailable. See SPM Priority in the Administration Guide.
- Configure all virtual machines that need to failover as highly available, and ensure that the virtual machine has a lease on the target storage domain. See Configuring a Highly Available Virtual Machine in the Virtual Machine Management Guide.
- Configure virtual machine to host soft affinity and define the behavior you expect from the affinity group. See Affinity Groups in the Virtual Machine Management Guide and Scheduling Policies in the Administration Guide.
The active-active failover can be manually performed by placing the main site’s hosts into maintenance mode.
