Appendix B. Testing the Active-Passive Configuration
You must test your disaster recovery solution after configuring it. This section provides multiple options to test the active-passive disaster recovery configuration.
- Test failover while the primary site remains active and without interfering with virtual machines on the primary site’s storage domains. See Discreet Failover Test.
- Test failover and failback using specific storage domains attached to the the primary site, therefore allowing the primary site to remain active. See Discreet Failover and Failback Test.
- Test failover and failback for an impending disaster where you have a grace period to failover to the secondary site, or an unplanned shutdown of the primary site. See Full Failover and Failback test.
Ensure that you completed all the steps to configure your active-passive configuration before running any of these tests.
B.1. Discreet Failover Test
This test simulates a failover while the primary site and all its storage domains remain active, allowing users to continue working in the primary site. To enable this scenario, you must disable replication between the primary storage domains and the replicated (secondary) storage domains. During this test the primary site is unaware of the failover activities on the secondary site.
This test will not allow you to test the failback functionality.
Ensure that no production tasks are performed after the failover. For example, ensure that email systems are blocked from sending emails to real users, or redirect emails elsewhere. If systems are used to directly manage other systems, prohibit access to the systems or ensure that they access parallel systems in the secondary site.
Performing the discreet failover test:
- Disable storage replication between the primary and replicated storage domains, and ensure that all replicated storage domains are in read/write mode.
Run the command to fail over to the secondary site:
# ansible-playbook playbook --tags "fail_over"For more information, see Executing a Failback.
- Verify that all relevant storage domains, virtual machines, and templates are registered and running successfully.
Restoring the environment to its active-passive state:
- Detach the storage domains from the secondary site.
- Enable storage replication between the primary and secondary storage domains.
B.2. Discreet Failover and Failback Test
For this test you must define testable storage domains that will be used specifically for testing the failover and failback. These storage domains must be replicated so that the replicated storage can be attached to the secondary site. This allows you to test the failover while users continue to work in the primary site.
You should define the testable storage domains on a separate storage server that can be offline without affecting the primary storage domains used for production in the primary site.
For more information about failing over the environment, cleaning the environment, and performing the failback, see Executing a Failover, Cleaning the Primary Site, and Executing a Failback.
Procedure: Discreet failover test:
- Stop the test storage domains in the primary site. You can do this by, for example, shutting down the server host or blocking it with a firewall rule.
- Disable the storage replication between the testable storage domains and ensure that all replicated storage domains used for the test are in read/write mode.
- Place the test primary storage domains into read-only mode.
Run the command to fail over to the secondary site:
# ansible-playbook playbook --tags "fail_over"- Verify that all relevant storage domains, virtual machines, and templates are registered and running successfully.
Procedure: Discreet failback test*
Run the command to clean the primary site and remove all inactive storage domains and related virtual machines and templates:
# ansible-playbook playbook --tags "clean_engine"Run the failback command:
# ansible-playbook playbook --tags "fail_back"- Enable replication from the primary storage domains to the secondary storage domains.
- Verify that all relevant storage domains, virtual machines, and templates are registered and running successfully.
B.3. Full Failover and Failback test
This test performs a full failover and failback between the primary and secondary site. You can simulate the disaster by shutting down the primary site’s hosts or by adding firewall rules to block writing to the storage domains.
For more information about failing over the environment, cleaning the environment, and performing the failback, see Executing a Failover, Cleaning the Primary Site, and Executing a Failback.
Procedure: Failover test
- Disable storage replication between the primary and replicated storage domains and ensure that all replicated storage domains are in read/write mode.
Run the command to fail over to the secondary site:
# ansible-playbook playbook --tags "fail_over"- Verify that all relevant storage domains, virtual machines, and templates are registered and running successfully.
Procedure: Failback test
- Synchronize replication between the secondary site’s storage domains and the primary site’s storage domains. The secondary site’s storage domains must be in read/write mode and the primary site’s storage domains must be in read-only mode.
Run the command to clean the primary site and remove all inactive storage domains and related virtual machines and templates:
# ansible-playbook playbook --tags "clean_engine"Run the failback command:
# ansible-playbook playbook --tags "fail_back"- Enable replication from the primary storage domains to the secondary storage domains.
- Verify that all relevant storage domains, virtual machines, and templates are registered and running successfully.
