Self-Hosted Engine Guide
Installing and Maintaining the Red Hat Virtualization Self-Hosted Engine
Abstract
Chapter 1. Introduction Copy linkLink copied to clipboard!
System Type
|
Supported Versions
|
---|---|
Red Hat Enterprise Linux host
|
7.2 or later
|
Red Hat Virtualization Host
|
7.2 or later
|
HostedEngine-VM (Manager)
|
7
|
Important
Chapter 2. Deploying Self-Hosted Engine Copy linkLink copied to clipboard!
2.1. Deploying Self-Hosted Engine on Red Hat Enterprise Linux Hosts Copy linkLink copied to clipboard!
2.1.1. Installing the Self-Hosted Engine Packages Copy linkLink copied to clipboard!
Procedure 2.1. Installing the Self-Hosted Engine
- Install the self-hosted engine packages:
yum install ovirt-hosted-engine-setup
# yum install ovirt-hosted-engine-setup
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Install the RHV-M Virtual Appliance package for the Manager virtual machine installation:
yum install rhevm-appliance
# yum install rhevm-appliance
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.1.2. Configuring a RHEL-Based Self-Hosted Engine Copy linkLink copied to clipboard!
hosted-engine
script is provided to assist with configuring the host, and the Manager virtual machine. The script asks you a series of questions, and configures your environment based on your answers.
Note
hosted-engine --check-deployed
command to check whether a self-hosted engine has already been deployed. An error will only be displayed if no self-hosted engine has been deployed. If a self-hosted engine has already been deployed, subsequent deployments will fail. See Chapter 3, Troubleshooting a Self-Hosted Engine Deployment if you need to troubleshoot an existing deployement, or clean up a failed deployment in order to redeploy the self-hosted engine.
Prerequisites
- You must have a freshly installed Red Hat Enterprise Linux 7 system with the ovirt-hosted-engine-setup package installed.
- You must have prepared storage for your self-hosted engine environment. At least two storage domains are required:For more information on preparing storage for your deployment, see the Storage chapter of the Administration Guide.
- A shared storage domain dedicated to the Manager virtual machine. This domain is created during the self-hosted engine deployment, and must be at least 60 GB.
- A data storage domain for regular virtual machine data. This domain must be added to the self-hosted engine environment after completing the deployment.
Warning
Red Hat strongly recommends that you have additional active data storage domains available in the same data center as the self-hosted engine environment.If you deploy the self-hosted engine in a data center with only one active data storage domain, and if that data storage domain is corrupted, you will be unable to add new data storage domains or to remove the corrupted data storage domain. You will have to redeploy the self-hosted engine.If you are using an ISO storage domain, Red Hat recommends that the ISO domain not be within the hosted engine virtual machine.
Important
If you are using iSCSI storage, do not use the same iSCSI target for the shared storage domain and data storage domain. - You must have a fully qualified domain name prepared for your Manager and the host. Forward and reverse lookup records must both be set in the DNS.
- You must have the RHV-M Virtual Appliance for the Manager installation. If you have not manually installed the rhevm-appliance package, it will be downloaded and installed automatically.
- To use the RHV-M Virtual Appliance for the Manager installation, one directory must be at least 5 GB. The
hosted-engine
script first checks if/var/tmp
has enough space to extract the appliance files. If not, you can specify a different directory or mount external storage. The VDSM user and KVM group must have read, write, and execute permissions on the directory.
Procedure 2.2. Configuring a RHEL-based Self-Hosted Engine
Initiating Hosted Engine Deployment
Run thehosted-engine
script. To escape the script at any time, use the CTRL+D keyboard combination to abort deployment. It is recommended to use thescreen
window manager to run the script to avoid losing the session in case of network or terminal disruption. If not already installed, install the screen package, which is available in the standard Red Hat Enterprise Linux repository.yum install screen
# yum install screen
Copy to Clipboard Copied! Toggle word wrap Toggle overflow screen
# screen
Copy to Clipboard Copied! Toggle word wrap Toggle overflow hosted-engine --deploy
# hosted-engine --deploy
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Note
In the event of session timeout or connection disruption, runscreen -d -r
to recover thehosted-engine
deployment session.Configuring Storage
Select the type of storage to use.During customization use CTRL-D to abort. Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs3, nfs4)[nfs3]:
During customization use CTRL-D to abort. Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs3, nfs4)[nfs3]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For NFS storage types, specify the full address, using either the FQDN or IP address, and path name of the shared storage domain.
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/nfs
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/nfs
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For iSCSI, specify the iSCSI portal IP address, port, user name and password, and select a target name from the auto-detected list. You can only select one iSCSI target during the deployment.
Please specify the iSCSI portal IP address: Please specify the iSCSI portal port [3260]: Please specify the iSCSI portal user: Please specify the iSCSI portal password: Please specify the target name (auto-detected values) [default]:
Please specify the iSCSI portal IP address: Please specify the iSCSI portal port [3260]: Please specify the iSCSI portal user: Please specify the iSCSI portal password: Please specify the target name (auto-detected values) [default]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For Gluster storage, specify the full address, using either the FQDN or IP address, and path name of the shared storage domain.
Important
Only replica 3 Gluster storage is supported. Ensure the following configuration has been made:- In the
/etc/glusterfs/glusterd.vol
file on all three Gluster servers, setrpc-auth-allow-insecure
toon
.option rpc-auth-allow-insecure on
option rpc-auth-allow-insecure on
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure the volume as follows:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/gluster_volume
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/gluster_volume
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For Fibre Channel, the host bus adapters must be configured and connected, and the
hosted-engine
script will auto-detect the LUNs available. The LUNs must not contain any existing data.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Configuring the Network
The script detects possible network interface controllers (NICs) to use as a management bridge for the environment. It then checks your firewall configuration and offers to modify it for console (SPICE or VNC) access.Please indicate a nic to set rhvm bridge on: (eth1, eth0) [eth1]: iptables was detected on your computer, do you wish setup to configure it? (Yes, No)[Yes]: Yes Please indicate a pingable gateway IP address [X.X.X.X]:
Please indicate a nic to set rhvm bridge on: (eth1, eth0) [eth1]: iptables was detected on your computer, do you wish setup to configure it? (Yes, No)[Yes]: Yes Please indicate a pingable gateway IP address [X.X.X.X]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring the Virtual Machine
Selectfor the boot device type, and the script will automatically detect the RHV-M Virtual Appliances available. Specify the memory size.Copy to Clipboard Copied! Toggle word wrap Toggle overflow SpecifyYes
if you want cloud-init to take care of the initial configuration of the Manager virtual machine. Specify for cloud-init to take care of tasks like setting the root password, configuring networking, configuring the host name, injecting an answers file forengine-setup
to use, and runningengine-setup
on boot. Optionally, select if you have an existing cloud-init script to take care of more sophisticated functions of cloud-init. Specify the FQDN for the Manager virtual machine. Specify a MAC address for the HostedEngine-VM, or accept a randomly generated one. The MAC address can be used to update your DHCP and DNS server prior to installing the operating system on the virtual machine.Note
For more information on cloud-init, see https://cloudinit.readthedocs.org/en/latest/.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring the Hosted Engine
Specify a name for the host to be identified in the Administration Portal, and the password for theadmin@internal
user to access the Administration Portal. Provide the name and TCP port number of the SMTP server, the email address used to send email notifications, and a comma-separated list of email addresses to receive these notifications.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuration Preview
Before proceeding, thehosted-engine
script displays the configuration values you have entered, and prompts for confirmation to proceed with these values.Creating the Manager Virtual Machine
The script creates the Manager virtual machine, starts theovirt-engine
and high availability services, and connects the host and shared storage domain to the Manager virtual machine.Copy to Clipboard Copied! Toggle word wrap Toggle overflow When thehosted-engine
deployment script completes successfully, the Red Hat Virtualization Manager is configured and running on your host. The Manager has already configured the data center, cluster, host, the Manager virtual machine, and a shared storage domain dedicated to the Manager virtual machine.Important
Log in as the admin@internal user to continue configuring the Manager and add further resources. You must create another data domain for the data center to be initialized to host regular virtual machine data, and for the Manager virtual machine to be visible. See Storage in the Administration Guide for different storage options and on how to add a data storage domain.Link your Red Hat Virtualization Manager to a directory server so you can add additional users to the environment. Red Hat Virtualization supports many directory server types; for example, Red Hat Directory Server (RHDS), Red Hat Identity Management (IdM), Active Directory, and many other types. Add a directory server to your environment using theovirt-engine-extension-aaa-ldap-setup
interactive setup script. For more information, see Configuring an External LDAP Provider in the Administration Guide.Theovirt-hosted-engine-setup
script also saves the answers you gave during configuration to a file, to help with disaster recovery. If a destination is not specified using the--generate-answer=<file>
argument, the answer file is generated at/etc/ovirt-hosted-engine/answers.conf
.Note
SSH password authentication is not enabled by default on the RHV-M Virtual Appliance. You can enable SSH password authentication by accessing the Red Hat Virtualization Manager virtual machine through the SPICE or VNC console. Verify that thesshd
service is running. Edit/etc/ssh/sshd_config
and change the following two options toyes
:PasswordAuthentication
PermitRootLogin
Restart thesshd
service for the changes to take effect.Subscribing to the Required Entitlements
Subscribe the Manager virtual machine to the required entitlements. See Subscribing to the Required Entitlements in the Installation Guide for more information.
2.2. Deploying Self-Hosted Engine on Red Hat Virtualization Host Copy linkLink copied to clipboard!
hosted-engine
script assists with configuring the host and Manager virtual machine. The script asks you a series of questions, and configures your environment based on your answers.
Prerequisites
- You must have a freshly installed Red Hat Virtualization Host system. The Performance Profile in the System sub-tab of the Cockpit user interface must be set to
virtual-host
. - You must have prepared storage for your self-hosted engine environment. At least two storage domains are required:For more information on preparing storage for your deployment, see the Storage chapter of the Administration Guide.
- A shared storage domain dedicated to the Manager virtual machine. This domain is created during the self-hosted engine deployment, and must be at least 60 GB.
- A data storage domain for regular virtual machine data. This domain must be added to the self-hosted engine environment after completing the deployment.
Warning
Red Hat strongly recommends that you have additional active data storage domains available in the same data center as the self-hosted engine environment.If you deploy the self-hosted engine in a data center with only one active data storage domain, and if that data storage domain is corrupted, you will be unable to add new data storage domains or to remove the corrupted data storage domain. You will have to redeploy the self-hosted engine.If you are using an ISO storage domain, Red Hat recommends that the ISO domain not be within the hosted engine virtual machine.
Important
If you are using iSCSI storage, do not use the same iSCSI target for the shared storage domain and data storage domain. - You must have a fully qualified domain name prepared for your Manager and the host. Forward and reverse lookup records must both be set in the DNS.
- To use the RHV-M Virtual Appliance for the Manager installation, one directory must be at least 5 GB. The
hosted-engine
script first checks if/var/tmp
has enough space to extract the appliance files. If not, you can specify a different directory or mount external storage. The VDSM user and KVM group must have read, write, and execute permissions on the directory.
Procedure 2.3. Configuring a RHVH-based Self-Hosted Engine
Obtaining the RHV-M Virtual Appliance
Download the RHV-M Virtual Appliance from the Customer Portal:- Log in to the Customer Portal at https://access.redhat.com.
- Click Downloads in the menu bar.
- Click Red Hat Virtualization > to access the product download page.
- Choose the appliance for the correct Red Hat Virtualization version and click.
Secure copy the OVA file to the Red Hat Virtualization Host:scp rhvm-appliance.ova root@host.example.com:/usr/share
scp rhvm-appliance.ova root@host.example.com:/usr/share
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Initiating Self-Hosted Engine Deployment
Log in to the Cockpit user interface at https://HostIPorFQDN:9090 and navigate to Virtualization > Hosted Engine. Click .The text fields in the deployment script are pre-populated with a default answer if one is available; change or enter your answers as necessary.Note
In this procedure, the deployment questions are presented in text form. In the UI, clickwhen prompted.During customization use CTRL-D to abort. Continuing will configure this host for serving as hypervisor and create a VM where you have to install the engine afterwards. Are you sure you want to continue? (Yes, No)[Yes]:
During customization use CTRL-D to abort. Continuing will configure this host for serving as hypervisor and create a VM where you have to install the engine afterwards. Are you sure you want to continue? (Yes, No)[Yes]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring Storage
Select the type of storage to use.Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs3, nfs4)[nfs3]:
Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs3, nfs4)[nfs3]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For NFS storage types, specify the full address, using either the FQDN or IP address, and path name of the shared storage domain.
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/nfs
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/nfs
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For iSCSI, specify the iSCSI portal IP address, port, user name and password, and select a target name from the auto-detected list. You can only select one iSCSI target during the deployment.
Please specify the iSCSI portal IP address:
Please specify the iSCSI portal IP address:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please specify the iSCSI portal port [3260]:
Please specify the iSCSI portal port [3260]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please specify the iSCSI portal user:
Please specify the iSCSI portal user:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please specify the iSCSI portal password:
Please specify the iSCSI portal password:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please specify the target name (auto-detected values) [default]:
Please specify the target name (auto-detected values) [default]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For Gluster storage, specify the full address, using either the FQDN or IP address, and path name of the shared storage domain.
Important
Only replica 3 Gluster storage is supported. Ensure the following configuration has been made:- In the
/etc/glusterfs/glusterd.vol
file on all three Gluster servers, setrpc-auth-allow-insecure
toon
.option rpc-auth-allow-insecure on
option rpc-auth-allow-insecure on
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure the volume as follows:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/gluster_volume
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/gluster_volume
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For Fibre Channel, the host bus adapters must be configured and connected, and the
hosted-engine
script will auto-detect the LUNs available. The LUNs must not contain any existing data.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Configuring the Network
The script checks your firewall configuration and offers to modify it for console (SPICE or VNC) access. It then detects possible network interface controllers (NICs) to use as a management bridge for the environment.iptables was detected on your computer, do you wish setup to configure it? (Yes, No)[Yes]: Yes
iptables was detected on your computer, do you wish setup to configure it? (Yes, No)[Yes]: Yes
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please indicate a pingable gateway IP address [X.X.X.X]:
Please indicate a pingable gateway IP address [X.X.X.X]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please indicate a nic to set ovirtmgmt bridge on: (eth1, eth0) [eth1]:
Please indicate a nic to set ovirtmgmt bridge on: (eth1, eth0) [eth1]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring the Virtual Machine
Selectfor the boot device type, and then specify the path to the RHV-M Virtual Appliance. If the/var/tmp
directory does not have enough space, specify a different directory. The VDSM user and KVM group must have read, write, and execute permissions on the directory.Please specify the device to boot the VM from (choose disk for the oVirt engine appliance) (cdrom, disk, pxe) [disk]: disk
Please specify the device to boot the VM from (choose disk for the oVirt engine appliance) (cdrom, disk, pxe) [disk]: disk
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please specify the console type you would like to use to connect to the VM (vnc, spice) [vnc]: vnc
Please specify the console type you would like to use to connect to the VM (vnc, spice) [vnc]: vnc
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Using an oVirt engine appliance could greatly speed-up ovirt hosted-engine deploy. You could get oVirt engine appliance installing ovirt-engine-appliance rpm. Please specify path to OVF archive you would like to use [None]: /path/to/rhvm-appliance.ova
Using an oVirt engine appliance could greatly speed-up ovirt hosted-engine deploy. You could get oVirt engine appliance installing ovirt-engine-appliance rpm. Please specify path to OVF archive you would like to use [None]: /path/to/rhvm-appliance.ova
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Specifyif you want cloud-init to take care of the initial configuration of the Manager virtual machine. Specify for cloud-init to take care of tasks like setting the root password, configuring networking, configuring the host name, injecting an answers file forengine-setup
to use, and runningengine-setup
on boot. Optionally, select if you have an existing cloud-init script to take care of more sophisticated functions of cloud-init.Note
For more information on cloud-init, see https://cloudinit.readthedocs.org/en/latest/.Would you like to use cloud-init to customize the appliance on the first boot (Yes, No)[Yes]? Yes
Would you like to use cloud-init to customize the appliance on the first boot (Yes, No)[Yes]? Yes
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Would you like to generate on-fly a cloud-init ISO image (of no-cloud type) or do you have an existing one (Generate, Existing)[Generate]? Generate
Would you like to generate on-fly a cloud-init ISO image (of no-cloud type) or do you have an existing one (Generate, Existing)[Generate]? Generate
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please provide the FQDN you would like to use for the engine appliance. Note: This will be the FQDN of the engine VM you are now going to launch. It should not point to the base host or to any other existing machine. Engine VM FQDN: (leave it empty to skip): manager.example.com
Please provide the FQDN you would like to use for the engine appliance. Note: This will be the FQDN of the engine VM you are now going to launch. It should not point to the base host or to any other existing machine. Engine VM FQDN: (leave it empty to skip): manager.example.com
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Automatically execute engine-setup on the engine appliance on first boot (Yes, No)[Yes]? Yes
Automatically execute engine-setup on the engine appliance on first boot (Yes, No)[Yes]? Yes
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Automatically restart the engine VM as a monitored service after engine-setup (Yes, No)[Yes]? Yes
Automatically restart the engine VM as a monitored service after engine-setup (Yes, No)[Yes]? Yes
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please provide the domain name you would like to use for the engine appliance. Engine VM domain: [localdomain] example.com
Please provide the domain name you would like to use for the engine appliance. Engine VM domain: [localdomain] example.com
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Enter root password that will be used for the engine appliance (leave it empty to skip): p@ssw0rd
Enter root password that will be used for the engine appliance (leave it empty to skip): p@ssw0rd
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Confirm appliance root password: p@ssw0rd
Confirm appliance root password: p@ssw0rd
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please specify the number of virtual CPUs for the VM [Defaults to appliance OVF value: [2]:
Please specify the number of virtual CPUs for the VM [Defaults to appliance OVF value: [2]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You may specify a unicast MAC address for the VM or accept a randomly generated default [00:16:3e:77:b2:a4]:
You may specify a unicast MAC address for the VM or accept a randomly generated default [00:16:3e:77:b2:a4]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please specify the memory size of the VM in MB (Defaults to maximum available): [12722]:
Please specify the memory size of the VM in MB (Defaults to maximum available): [12722]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow How should the engine VM network be configured (DHCP, Static)[DHCP]? Static
How should the engine VM network be configured (DHCP, Static)[DHCP]? Static
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please enter the IP address to be used for the engine VM: 192.168.x.x
Please enter the IP address to be used for the engine VM: 192.168.x.x
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please provide a comma-separated list (max 3) of IP addresses of domain name servers for the engine VM Engine VM DNS (leave it empty to skip):
Please provide a comma-separated list (max 3) of IP addresses of domain name servers for the engine VM Engine VM DNS (leave it empty to skip):
Copy to Clipboard Copied! Toggle word wrap Toggle overflow SpecifyYes
to copy the/etc/hosts
file from the host to the Manager virtual machine for host name resolution.Add lines for the appliance itself and for this host to /etc/hosts on the engine VM? Note: ensuring that this host could resolve the engine VM hostname is still up to you (Yes, No)[No] Yes
Add lines for the appliance itself and for this host to /etc/hosts on the engine VM? Note: ensuring that this host could resolve the engine VM hostname is still up to you (Yes, No)[No] Yes
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring the Self-Hosted Engine
Specify a name for the host to be identified in the Administration Portal, and the password for theadmin@internal
user to access the Administration Portal. Provide the name and TCP port number of the SMTP server, the email address used to send email notifications, and a comma-separated list of email addresses to receive these notifications.Enter engine admin password: p@ssw0rd
Enter engine admin password: p@ssw0rd
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Confirm engine admin password: p@ssw0rd
Confirm engine admin password: p@ssw0rd
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Enter the name which will be used to identify this host inside the Administrator Portal [hosted_engine_1]:
Enter the name which will be used to identify this host inside the Administrator Portal [hosted_engine_1]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please provide the name of the SMTP server through which we will send notifications [localhost]:
Please provide the name of the SMTP server through which we will send notifications [localhost]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please provide the TCP port number of the SMTP server [25]:
Please provide the TCP port number of the SMTP server [25]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please provide the email address from which notifications will be sent [root@localhost]:
Please provide the email address from which notifications will be sent [root@localhost]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Please provide a comma-separated list of email addresses which will get notifications [root@localhost]:
Please provide a comma-separated list of email addresses which will get notifications [root@localhost]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuration Preview
Before proceeding, thehosted-engine
script displays the configuration values you have entered, and prompts for confirmation to proceed with these values.Please confirm installation settings (Yes, No)[Yes]: Yes
Please confirm installation settings (Yes, No)[Yes]: Yes
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The script creates the Manager virtual machine, starts theovirt-engine
and high availability services, and connects the host and shared storage domain to the Manager virtual machine.When thehosted-engine
deployment script completes successfully, the Red Hat Virtualization Manager is configured and running on your host. The Manager has already configured the data center, cluster, host, the Manager virtual machine, and a shared storage domain dedicated to the Manager virtual machine.Important
Log in to the Administration Portal as the admin@internal user to continue configuring the Manager and add further resources. You must create another data domain for the data center to be initialized to host regular virtual machine data, and for the Manager virtual machine to be visible. See Storage in the Administration Guide for different storage options and on how to add a data storage domain.Link your Red Hat Virtualization Manager to a directory server so you can add additional users to the environment. Red Hat Virtualization supports many directory server types; for example, Red Hat Directory Server (RHDS), Red Hat Identity Management (IdM), Active Directory, and many other types. Add a directory server to your environment using theovirt-engine-extension-aaa-ldap-setup
interactive setup script. For more information, see Configuring an External LDAP Provider in the Administration Guide.The script also saves the answers you gave during configuration to a file, to help with disaster recovery. If a destination is not specified using the--generate-answer=<file>
argument, the answer file is generated at/etc/ovirt-hosted-engine/answers.conf
.Note
SSH password authentication is not enabled by default on the RHV-M Virtual Appliance. You can enable SSH password authentication by accessing the Red Hat Virtualization Manager virtual machine through the SPICE or VNC console. Verify that thesshd
service is running. Edit/etc/ssh/sshd_config
and change the following two options toyes
:PasswordAuthentication
PermitRootLogin
Restart thesshd
service for the changes to take effect.Subscribing to the Required Entitlements
Subscribe the Manager virtual machine to the required entitlements. See Subscribing to the Required Entitlements in the Installation Guide for more information.
2.3. Administering the Manager Virtual Machine Copy linkLink copied to clipboard!
hosted-engine
utility is provided to assist with administering the Manager virtual machine. It can be run on any self-hosted engine nodes in the environment. For all the options, run hosted-engine --help
. For additional information on a specific command, run hosted-engine --command --help
.
Chapter 3. Troubleshooting a Self-Hosted Engine Deployment Copy linkLink copied to clipboard!
hosted-engine --check-deployed
. An error will only be displayed if the self-hosted engine has not been deployed.
3.1. Troubleshooting the Manager Virtual Machine Copy linkLink copied to clipboard!
Procedure 3.1. Troubleshooting the Manager Virtual Machine
- Check the status of the Manager virtual machine by running
hosted-engine --vm-status
.Note
Any changes made to the Manager virtual machine will take about 20 seconds before they are reflected in the status command output.If the Manager virtual machine is up and running as normal, you will see the following output:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - If theis bad or the is down, enable the global maintenance mode so that the hosts are no longer managed by the HA services.
- In the Administration Portal, right-click the Manager virtual machine, and select Enable Global HA Maintenance.
- You can also set the maintenance mode from the command line:
hosted-engine --set-maintenance --mode=global
# hosted-engine --set-maintenance --mode=global
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- If the Manager virtual machine is down, start the Manager virtual machine. If the virtual machine is up, skip this step.
hosted-engine ---vm-start
# hosted-engine ---vm-start
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Set the console password:
hosted-engine --add-console-password
# hosted-engine --add-console-password
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Connect to the console. When prompted, enter the password set in the previous step. For more console options, see https://access.redhat.com/solutions/2221461.
hosted-engine --console
# hosted-engine --console
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Determine why the Manager virtual machine is down or in a bad health state. Check
/var/log/messages
and/var/log/ovirt-engine/engine.log
. After fixing the issue, reboot the Manager virtual machine. - Log in to the Manager virtual machine as root and verfiy that the
ovirt-engine
service is up and running:systemctl status ovirt-engine.service
# systemctl status ovirt-engine.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - After ensuring the Manager virtual machine is up and running, close the console session and disable the maintenance mode to enable the HA services again:
hosted-engine --set-maintenance --mode=none
# hosted-engine --set-maintenance --mode=none
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Additional Troubleshooting Commands:
Important
hosted-engine --reinitialize-lockspace
: This command is used when the sanlock lockspace is broken. Ensure that the global maintenance mode is enabled and that the Manager virtual machine is stopped before reinitializing the sanlock lockspaces.hosted-engine --clean-metadata
: Remove the metadata for a host's agent from the global status database. This makes all other hosts forget about this host. Ensure that the target host is down and that the global maintenance mode is enabled.hosted-engine --check-liveliness
: This command checks the liveliness page of the ovirt-engine service. You can also check by connecting tohttps://engine-fqdn/ovirt-engine/services/health/
in a web browser.hosted-engine --connect-storage
: This command instructs VDSM to prepare all storage connections needed for the host and and the Manager virtual machine. This is normally run in the back-end during the self-hosted engine deployment. Ensure that the global maintenance mode is enabled if you need to run this command to troubleshoot storage issues.
3.2. Cleaning Up a Failed Self-hosted Engine Deployment Copy linkLink copied to clipboard!
Failed to connect to broker, the number of errors has exceeded the limit.
Failed to connect to broker, the number of errors has exceeded the limit.
Chapter 4. Migrating from Bare Metal to a RHEL-Based Self-Hosted Environment Copy linkLink copied to clipboard!
4.1. Migrating to a Self-Hosted Environment Copy linkLink copied to clipboard!
hosted-engine
script to assist with the task. The script asks you a series of questions, and configures your environment based on your answers. The Manager from the standard Red Hat Virtualization environment is referred to as the BareMetal-Manager in the following procedure.
engine-setup
in a standard installation, in the migration process engine-setup
must be run manually so that you can restore the BareMetal-Manager backup file on the new Manager virtual machine beforehand.
- Run the
hosted-engine
script to configure the host to be used as a self-hosted engine node and to create a new Red Hat Virtualization virtual machine. - Back up the the engine database and configuration files using the
engine-backup
tool, copy the backup to the new Manager virtual machine, and restore the backup using the--mode=restore
parameter ofengine-backup
. Runengine-setup
to complete the Manager virtual machine configuration. - Follow the
hosted-engine
script to complete the setup.
Prerequisites
- Prepare a new host with the ovirt-hosted-engine-setup package installed. See Section 2.1, “Deploying Self-Hosted Engine on Red Hat Enterprise Linux Hosts” for more information on subscriptions and package installation. The host must be a supported version of the current Red Hat Virtualization environment.
Note
If you intend to use an existing host, place the host in maintenance and remove it from the existing environment. See Removing a Host in the Administration Guide for more information. - Prepare storage for your self-hosted engine environment. The self-hosted engine requires a shared storage domain dedicated to the Manager virtual machine. This domain is created during deployment, and must be at least 60 GB. For more information on preparing storage for your deployment, see the Storage chapter of the Administration Guide.
Important
If you are using iSCSI storage, do not use the same iSCSI target for the shared storage domain and data storage domain. - Obtain the RHV-M Virtual Appliance by installing the rhevm-appliance package. The RHV-M Virtual Appliance is always based on the latest supported Manager version. Ensure the Manager version in your current environment is updated to the latest supported Y-stream version as the Manager version needs to be the same for the migration.
- To use the RHV-M Virtual Appliance for the Manager installation, ensure one directory is at least 5 GB. The
hosted-engine
script first checks if/var/tmp
has enough space to extract the appliance files. If not, you can specify a different directory or mount external storage. The VDSM user and KVM group must have read, write, and execute permissions on the directory. - The fully qualified domain name of the new Manager must be the same fully qualified domain name as that of the BareMetal-Manager. Forward and reverse lookup records must both be set in DNS.
- You must have access and can make changes to the BareMetal-Manager.
- The virtual machine to which the BareMetal-Manager is being migrated must have the same amount of RAM as the physical machine from which the BareMetal-Manager is being migrated. If you must migrate to a virtual machine that has less RAM than the physical machine from which the BareMetal-Manager is migrated, see the following Red Hat Knowledgebase article: https://access.redhat.com/articles/2705841.
Procedure 4.1. Migrating to a Self-Hosted Environment
Initiating a Self-Hosted Engine Deployment
Note
If you are updating from version 3.5 or earlier, you must run the commandhosted-engine --deploy --config-append=/etc/ovirt-hosted-engine/answers.conf
. The fileanswers.conf
must contain the parameterOVEHOSTED_NETWORK/bridgeName=str:rhevm
. Upgrading from version 3.5 to version 3.6 or later causes the default management network to be non-operational unless this parameter is set.Run thehosted-engine
script. To escape the script at any time, use the CTRL+D keyboard combination to abort deployment. It is recommended to use thescreen
window manager to run the script to avoid losing the session in case of network or terminal disruption. If not already installed, install the screen package, which is available in the standard Red Hat Enterprise Linux repository.yum install screen
# yum install screen
Copy to Clipboard Copied! Toggle word wrap Toggle overflow screen
# screen
Copy to Clipboard Copied! Toggle word wrap Toggle overflow hosted-engine --deploy
# hosted-engine --deploy
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Note
In the event of session timeout or connection disruption, runscreen -d -r
to recover thehosted-engine
deployment session.Configuring Storage
Select the type of storage to use.During customization use CTRL-D to abort. Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs3, nfs4)[nfs3]:
During customization use CTRL-D to abort. Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs3, nfs4)[nfs3]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For NFS storage types, specify the full address, using either the FQDN or IP address, and path name of the shared storage domain.
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/nfs
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/nfs
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For iSCSI, specify the iSCSI portal IP address, port, user name and password, and select a target name from the auto-detected list. You can only select one iSCSI target during the deployment.
Please specify the iSCSI portal IP address: Please specify the iSCSI portal port [3260]: Please specify the iSCSI portal user: Please specify the iSCSI portal password: Please specify the target name (auto-detected values) [default]:
Please specify the iSCSI portal IP address: Please specify the iSCSI portal port [3260]: Please specify the iSCSI portal user: Please specify the iSCSI portal password: Please specify the target name (auto-detected values) [default]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For Gluster storage, specify the full address, using either the FQDN or IP address, and path name of the shared storage domain.
Important
Only replica 3 Gluster storage is supported. Ensure the following configuration has been made:- In the
/etc/glusterfs/glusterd.vol
file on all three Gluster servers, setrpc-auth-allow-insecure
toon
.option rpc-auth-allow-insecure on
option rpc-auth-allow-insecure on
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure the volume as follows:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/gluster_volume
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/gluster_volume
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For Fibre Channel, the host bus adapters must be configured and connected, and the
hosted-engine
script will auto-detect the LUNs available. The LUNs must not contain any existing data.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Configuring the Network
The script detects possible network interface controllers (NICs) to use as a management bridge for the environment. It then checks your firewall configuration and offers to modify it for console (SPICE or VNC) access HostedEngine-VM. Provide a pingable gateway IP address, to be used by theovirt-ha-agent
to help determine a host's suitability for running HostedEngine-VM.Please indicate a nic to set rhvm bridge on: (eth1, eth0) [eth1]: iptables was detected on your computer, do you wish setup to configure it? (Yes, No)[Yes]: Please indicate a pingable gateway IP address [X.X.X.X]:
Please indicate a nic to set rhvm bridge on: (eth1, eth0) [eth1]: iptables was detected on your computer, do you wish setup to configure it? (Yes, No)[Yes]: Please indicate a pingable gateway IP address [X.X.X.X]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring the Virtual Machine
The script creates a virtual machine to be configured as the Red Hat Virtualization Manager, referred to in this procedure as HostedEngine-VM. Selectfor the boot device type, and the script will automatically detect the RHV-M Appliances available. Select an appliance.Copy to Clipboard Copied! Toggle word wrap Toggle overflow SpecifyYes
if you want cloud-init to take care of the initial configuration of the Manager virtual machine. Specify for cloud-init to take care of tasks like setting the root password, configuring networking, and configuring the host name. Optionally, select if you have an existing cloud-init script to take care of more sophisticated functions of cloud-init. Specify the FQDN for the Manager virtual machine. This must be the same FQDN provided for the BareMetal-Manager.Note
For more information on cloud-init, see https://cloudinit.readthedocs.org/en/latest/.Copy to Clipboard Copied! Toggle word wrap Toggle overflow You must answerNo
to the following question so that you can restore the BareMetal-Manager backup file on HostedEngine-VM before runningengine-setup
.Automatically execute engine-setup on the engine appliance on first boot (Yes, No)[Yes]? No
Automatically execute engine-setup on the engine appliance on first boot (Yes, No)[Yes]? No
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configure the Manager domain name, root password, networking, hardware, and console access details.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring the Self-Hosted Engine
Specify the name for Host-HE1 to be identified in the Red Hat Virtualization environment, and the password for theadmin@internal
user to access the Administration Portal. Finally, provide the name and TCP port number of the SMTP server, the email address used to send email notifications, and a comma-separated list of email addresses to receive these notifications.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuration Preview
Before proceeding, thehosted-engine
script displays the configuration values you have entered, and prompts for confirmation to proceed with these values.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Creating HostedEngine-VM
The script creates the virtual machine to be configured as HostedEngine-VM and provides connection details. You must manually runengine-setup
after restoring the backup file on HostedEngine-VM before thehosted-engine
script can proceed on Host-HE1.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Connect to the virtual machine using the VNC protocol with the following command. Replace FQDN with the fully qualified domain name or the IP address of the self-hosted engine node./bin/remote-viewer vnc://FQDN:5900
# /bin/remote-viewer vnc://FQDN:5900
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Enabling SSH on HostedEngine-VM
SSH password authentication is not enabled by default on the RHV-M Virtual Appliance. Connect to HostedEngine-VM via VNC and enable SSH password authentication so that you can access the virtual machine via SSH later to restore the BareMetal-Manager backup file and configure the new Manager. Verify that thesshd
service is running. Edit/etc/ssh/sshd_config
and change the following two options toyes
:[...] PermitRootLogin yes [...] PasswordAuthentication yes
[...] PermitRootLogin yes [...] PasswordAuthentication yes
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Restart thesshd
service for the changes to take effect.systemctl restart sshd.service
# systemctl restart sshd.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Disabling BareMetal-Manager
Connect to BareMetal-Manager, the Manager of your established Red Hat Virtualization environment, and stop theovirt-engine
service and prevent it from running.systemctl stop ovirt-engine.service systemctl disable ovirt-engine.service
# systemctl stop ovirt-engine.service # systemctl disable ovirt-engine.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Note
Though stopping BareMetal-Manager from running is not obligatory, it is recommended as it ensures no changes are made to the environment after the backup is created. Additionally, it prevents BareMetal-Manager and HostedEngine-VM from simultaneously managing existing resources.Updating DNS
Update your DNS so that the FQDN of the Red Hat Virtualization environment correlates to the IP address of HostedEngine-VM and the FQDN previously provided when configuring thehosted-engine
deployment script on Host-HE1. In this procedure, FQDN was set as manager.example.com because in a migrated hosted-engine setup, the FQDN provided for the engine must be identical to that given in the engine setup of the original engine.Creating a Backup of BareMetal-Manager
Connect to BareMetal-Manager and run theengine-backup
command with the--mode=backup
,--file=FILE
, and--log=LogFILE
parameters to specify the backup mode, the name of the backup file created and used for the backup, and the name of the log file to be created to store the backup log.engine-backup --mode=backup --file=FILE --log=LogFILE
# engine-backup --mode=backup --file=FILE --log=LogFILE
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copying the Backup File to HostedEngine-VM
On BareMetal-Manager, secure copy the backup file to HostedEngine-VM. In the following example, manager.example.com is the FQDN for HostedEngine-VM, and /backup/ is any designated folder or path. If the designated folder or path does not exist, you must connect to HostedEngine-VM and create it before secure copying the backup from BareMetal-Manager.scp -p FILE LogFILE manager.example.com:/backup/
# scp -p FILE LogFILE manager.example.com:/backup/
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Registering HostedEngine-VM
Register HostedEngine-VM with Red Hat Subscription Management and enable the required repositories. See Subscribing to the Required Entitlements in the Installation Guide.Restoring the Backup File on HostedEngine-VM
Use theengine-backup
tool to restore a complete backup. If you configured the BareMetal-Manager database(s) manually duringengine-setup
, follow the instructions at Section 6.2.3, “Restoring the Self-Hosted Engine Manager Manually” to restore the backup environment manually.- If you are only restoring the Manager, run:
engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --restore-permissions
# engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --restore-permissions
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - If you are restoring the Manager and Data Warehouse, run:
engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --provision-dwh-db --restore-permissions
# engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --provision-dwh-db --restore-permissions
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
If successful, the following output displays:You should now run engine-setup. Done.
You should now run engine-setup. Done.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring HostedEngine-VM
Configure the restored Manager virtual machine. This process identifies the existing configuration settings and database content. Confirm the settings. Upon completion, the setup provides an SSH fingerprint and an internal Certificate Authority hash.engine-setup
# engine-setup
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy to Clipboard Copied! Toggle word wrap Toggle overflow Synchronizing the Host and the Manager
Return to Host-HE1 and continue thehosted-engine
deployment script by selecting option 1:(1) Continue setup - oVirt-Engine installation is ready and ovirt-engine service is up
(1) Continue setup - oVirt-Engine installation is ready and ovirt-engine service is up
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The script displays the internal Certificate Authority hash, and prompts you to select the cluster to which to add Host-HE1.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Shutting Down HostedEngine-VM
Shut down HostedEngine-VM.shutdown -h now
# shutdown -h now
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Setup Confirmation
Return to Host-HE1 to confirm it has detected that HostedEngine-VM is down.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 5. Maintenance and Upgrading Resources Copy linkLink copied to clipboard!
5.1. Maintaining the Self-Hosted Engine Copy linkLink copied to clipboard!
global
- All high-availability agents in the cluster are disabled from monitoring the state of the Manager virtual machine. The global maintenance mode must be applied for any setup or upgrade operations that require theovirt-engine
service to be stopped, such as upgrading to a later version of Red Hat Virtualization.local
- The high-availability agent on the node issuing the command is disabled from monitoring the state of the Manager virtual machine. The node is exempt from hosting the Manager virtual machine while in local maintenance mode; if hosting the Manager virtual machine when placed into this mode, the Manager will migrate to another node, provided there is one available. The local maintenance mode is recommended when applying system changes or updates to a self-hosted engine node.none
- Disables maintenance mode, ensuring that the high-availability agents are operating.
Procedure 5.1. Maintaining a RHEL-Based Self-Hosted Engine (Local Maintenance)
- Place a self-hosted engine node into the local maintenance mode:
- In the Administration Portal, select a self-hosted engine node and click Maintenance. The local maintenance mode is automatically triggered for that node.
- You can also set the maintenance mode from the command line:
hosted-engine --set-maintenance --mode=local
# hosted-engine --set-maintenance --mode=local
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- After you have completed any maintenance tasks, disable the maintenance mode:
hosted-engine --set-maintenance --mode=none
# hosted-engine --set-maintenance --mode=none
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Procedure 5.2. Maintaining a RHEL-Based Self-Hosted Engine (Global Maintenance)
- Place a self-hosted engine node into the global maintenance mode:
- In the Administration Portal, right-click the Manager virtual machine, and select Enable Global HA Maintenance.
- You can also set the maintenance mode from the command line:
hosted-engine --set-maintenance --mode=global
# hosted-engine --set-maintenance --mode=global
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- After you have completed any maintenance tasks, disable the maintenance mode:
hosted-engine --set-maintenance --mode=none
# hosted-engine --set-maintenance --mode=none
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
5.2. Removing a Host from a Self-Hosted Engine Environment Copy linkLink copied to clipboard!
Procedure 5.3. Removing a Host from a Self-Hosted Engine Environment
- In the Administration Portal, click the Hosts tab. Select the self-hosted engine node and click Maintenance to set it to the local maintenance mode.
- Click Edit.
- Click the Undeploy radio button. This action stops thesub-tab and select the
ovirt-ha-agent
andovirt-ha-broker
services and removes the self-hosted engine configuration file.Note
The host will be visible, but shown as unavailable when runninghosted-engine --vm-status
. This is because the metadata is not removed when a host is undeployed. The removed host will disappear in approximately a week when the metadata is not refreshed. To force the removal of the host's metadata follow the procedure in How to clean the metadata of redeployed Hosts in a Hosted Engine setup. - Optionally, click Remove to open the Remove Host(s) confirmation window, and click .
5.3. Upgrading a RHEL-Based Self-Hosted Engine Environment Copy linkLink copied to clipboard!
Important
Note
Important
Note
- Place the high-availability agents that manage the Manager virtual machine into the global maintenance mode.
- Enable the required repositories on the host and update the ovirt-hosted-engine-setup and rhevm-appliance packages.
- Run
hosted-engine --upgrade-appliance
to upgrade the Manager virtual machine. During the upgrade you will be asked to create a backup of the 3.6 Manager and copy it to the host machine where the upgrade is being performed. - Update the hosts.
- After the Manager virtual machine and all hosts in the cluster have been updated, change the cluster compatibility version to 4.0.
hosted-engine-backup-*
.
Prerequisites
- The
/var/tmp
directory must have at least 5 GB of free space to extract the appliance files. If it does not, you can specify a different directory or mount alternate storage that does have the required space. The VDSM user and KVM group must have read, write, and execute permissions on the directory. - The self-hosted engine storage domain must have additional free space for the new appliance being deployed (50 GB by default). To increase the storage on iSCSI or Fibre Channel storage, you must manually extend the LUN size on the storage and then extend the storage domain using the Manager. See Increasing iSCSI or FCP Storage in the Administration Guide for more information about resizing a LUN.
Procedure 5.4. Upgrading the Self-Hosted Engine
- Disable the high-availability agents on all the self-hosted engine nodes. To do this run the following command on any host in the cluster.
hosted-engine --set-maintenance --mode=global
# hosted-engine --set-maintenance --mode=global
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Note
Runhosted-engine --vm-status
to confirm that the environment is in maintenance mode. - On the host that is currently set as SPM and contains the Manager virtual machine, enable the required repository.
subscription-manager repos --enable=rhel-7-server-rhv-4-mgmt-agent-rpms
# subscription-manager repos --enable=rhel-7-server-rhv-4-mgmt-agent-rpms
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Migrate all virtual machines except the Manager virtual machine to alternate hosts.
- On the host, update the Manager virtual machine packages.
yum update ovirt-hosted-engine-setup rhevm-appliance
# yum update ovirt-hosted-engine-setup rhevm-appliance
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the rhevm-appliance package is missing, install it manually before updating ovirt-hosted-engine-setup.yum install rhevm-appliance
# yum install rhevm-appliance
Copy to Clipboard Copied! Toggle word wrap Toggle overflow yum update ovirt-hosted-engine-setup
# yum update ovirt-hosted-engine-setup
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Run the upgrade utility to upgrade the Manager virtual machine. If not already installed, install the screen package, which is available in the standard Red Hat Enterprise Linux repository.
yum install screen
# yum install screen
Copy to Clipboard Copied! Toggle word wrap Toggle overflow screen
# screen
Copy to Clipboard Copied! Toggle word wrap Toggle overflow hosted-engine --upgrade-appliance
# hosted-engine --upgrade-appliance
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Note
You will be prompted to select the appliance if more than one is detected, and to create a backup of the Manager database and provide its full location. - After the upgrade is complete, disable global maintenance:
hosted-engine --set-maintenance --mode=none
# hosted-engine --set-maintenance --mode=none
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
hosted-engine --vm-poweroff
command, then rollback the upgrade by running hosted-engine --rollback-upgrade
.
5.4. Upgrading a RHEV-H-Based Self-Hosted Engine Environment Copy linkLink copied to clipboard!
Important
Important
- Place the high-availability agents that manage the Manager virtual machine into the global maintenance mode.
- Add a new RHVH 4.0 host to your environment.
- Migrate the Manager virtual machine to the new host and set as SPM.
- Run the upgrade utility to upgrade the Manager. During the upgrade procedure you will be asked to create a backup of the 3.6 Manager and copy it to the host machine where the upgrade is being performed.
- Update the hosts.
- After the Manager virtual machine and all hosts in the cluster have been updated, change the cluster compatibility version to 4.0.
hosted-engine-backup-*
.
Prerequisites
- The
/var/tmp
directory must have at least 5 GB of free space to extract the appliance files. If it does not, you can specify a different directory or mount alternate storage that has the required space. The VDSM user and KVM group must have read, write, and execute permissions on the directory. - The self-hosted engine storage domain must have additional free space for the new appliance being deployed (50 GB by default). To increase the storage on iSCSI or Fibre Channel storage, you must manually extend the LUN size on the storage and then extend the storage domain using the Manager. See Increasing iSCSI or FCP Storage in the Administration Guide for more information about resizing a LUN.
Procedure 5.5. Upgrading the Self-Hosted Engine
- Install a new RHVH 4.0 host. See the Installation Guide for instructions to install RHVH.
- Add the new host to your environment. Run the deployment script on the new host:
hosted-engine --deploy
# hosted-engine --deploy
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Specify the existing shared storage domain when prompted; the script then detects that this is an additional host setup. See Installing Additional Hosts to a Self-Hosted Environment in the Red Hat Enterprise Virtualization 3.6 Self-Hosted Engine Guide for more details about the questions asked by the script during an additional host deployment. - Disable the high-availability agents on all the self-hosted engine nodes. To do this run the following command on any host in the cluster.
hosted-engine --set-maintenance --mode=global
# hosted-engine --set-maintenance --mode=global
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Note
Runhosted-engine --vm-status
to confirm that the environment is in maintenance mode. - Download the RHV-M Virtual Appliance from the Customer Portal and copy it to the new host:
- Log in to the Customer Portal at https://access.redhat.com.
- Click Downloads in the menu bar.
- Click Red Hat Virtualization > to access the product download page.
- Choose the appliance for Red Hat Virtualization 4.0 and click.
Secure copy the OVA file to the Red Hat Virtualization Host:scp rhvm-appliance.ova root@host.example.com:/usr/share
scp rhvm-appliance.ova root@host.example.com:/usr/share
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Migrate the Manager virtual machine to the RHVH 4.0 host and set the host as the Storage Pool Manager (SPM).
- Run the upgrade script to upgrade the Manager virtual machine. If not already installed, install the screen package.
yum install screen
# yum install screen
Copy to Clipboard Copied! Toggle word wrap Toggle overflow screen
# screen
Copy to Clipboard Copied! Toggle word wrap Toggle overflow hosted-engine --upgrade-appliance
# hosted-engine --upgrade-appliance
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Note
The script will ask for the location of the RHV-M Virtual Appliance you copied to the host. It will also prompt you to create a backup of the Manager database and provide its full location. - After the upgrade is complete, disable global maintenance:
hosted-engine --set-maintenance --mode=none
# hosted-engine --set-maintenance --mode=none
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
hosted-engine --vm-poweroff
command, then rollback the upgrade by running hosted-engine --rollback-upgrade
.
5.5. Upgrading Hosts in a Self-Hosted Engine Environment Copy linkLink copied to clipboard!
subscription-manager repos --disable=rhel-7-server-rhev-mgmt-agent-rpms
# subscription-manager repos --disable=rhel-7-server-rhev-mgmt-agent-rpms
subscription-manager repos --enable=rhel-7-server-rhv-4-mgmt-agent-rpms
# subscription-manager repos --enable=rhel-7-server-rhv-4-mgmt-agent-rpms
5.6. Updating the Self-Hosted Engine Manager Between Minor Releases Copy linkLink copied to clipboard!
- Ensure you have subscribed the Manager virtual machine to the required entitlements. See Subscribing to the Required Entitlements in the Installation Guide for more information.
- Place the system in global maintenance mode. See Section 5.1, “Maintaining the Self-Hosted Engine” for details.
- Follow the procedures for updating between minor versions using
engine-setup
. See Updates Between Minor Releases in the Installation Guide for details. - Disable global maintenance mode. See Section 5.1, “Maintaining the Self-Hosted Engine” for details.
Chapter 6. Backing up and Restoring a RHEL-Based Self-Hosted Environment Copy linkLink copied to clipboard!
Note
Procedure 6.1. Workflow for Backing Up the Self-Hosted Engine Environment
- The Manager virtual machine is running on
Host 2
and the six regular virtual machines in the environment are balanced across the three hosts.PlaceHost 1
into maintenance mode. This will migrate the virtual machines onHost 1
to the other hosts, freeing it of any virtual load and enabling it to be used as a failover host for the backup. Host 1
is in maintenance mode. The two virtual machines it previously hosted have been migrated to Host 3.Useengine-backup
to create backups of the environment. After the backup has been taken,Host 1
can be activated again to host virtual machines, including the Manager virtual machine.
Procedure 6.2. Workflow for Restoring the Self-Hosted Engine Environment
Host 1
has been used to deploy a new self-hosted engine and has restored the backup taken in the previous example procedure. Deploying the restored environment involves additional steps to that of a regular self-hosted engine deployment:- After Red Hat Virtualization Manager has been installed on the Manager virtual machine, but before
engine-setup
is first run, restore the backup using theengine-backup
tool. - After
engine-setup
has configured and restored the Manager, log in to the Administration Portal and removeHost 1
, which will be present from the backup. If oldHost 1
is not removed, and is still present in the Manager when finalizing deployment on newHost 1
, the Manager virtual machine will not be able to synchronize with newHost 1
and the deployment will fail.
AfterHost 1
and the Manager virtual machine have synchronized and the deployment has been finalized, the environment can be considered operational on a basic level. With only one self-hosted engine node, the Manager virtual machine is not highly available. However, if necessary, high-priority virtual machines can be started onHost 1
.Any standard RHEL-based hosts - hosts that are present in the environment but are not self-hosted engine nodes - that are operational will become active, and the virtual machines that were active at the time of backup will now be running on these hosts and available in the Manager.Host 2
andHost 3
are not recoverable in their current state. These hosts need to be removed from the environment, and then added again to the environment using thehosted-engine
deployment script. For more information on these actions, see Section 6.2.4, “Removing Non-Operational Hosts from a Restored Self-Hosted Engine Environment” and Chapter 7, Installing Additional Hosts to a Self-Hosted Environment.Host 2
andHost 3
have been re-deployed into the restored environment. The environment is now as it was in the first image, before the backup was taken, with the exception that the Manager virtual machine is hosted onHost 1
.
6.1. Backing up the Self-Hosted Engine Manager Virtual Machine Copy linkLink copied to clipboard!
engine-backup
tool and can be performed without interrupting the ovirt-engine
service. The engine-backup
tool only allows you to back up the Red Hat Virtualization Manager virtual machine, but not the self-hosted engine node that runs the Manager virtual machine or other virtual machines hosted in the environment.
Procedure 6.3. Backing up the Original Red Hat Virtualization Manager
Preparing the Failover Host
A failover host, one of the self-hosted engine nodes in the environment, must be placed into maintenance mode so that it has no virtual load at the time of the backup. This host can then later be used to deploy the restored self-hosted engine environment. Any of the self-hosted engine nodes can be used as the failover host for this backup scenario, however the restore process is more straightforward ifHost 1
is used. The default name for theHost 1
host ishosted_engine_1
; this was set when thehosted-engine
deployment script was initially run.- Log in to one of the self-hosted engine nodes.
- Confirm that the
hosted_engine_1
host isHost 1
:hosted-engine --vm-status
# hosted-engine --vm-status
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Log in to the Administration Portal.
- Click the Hosts tab.
- Select the
hosted_engine_1
host in the results list, and click . - Click.
Depending on the virtual load of the host, it may take some time for all the virtual machines to be migrated. Proceed to the next step after the host status has changed toMaintenance
.Creating a Backup of the Manager
On the Manager virtual machine, back up the configuration settings and database content, replacing [EngineBackupFile] with the file name for the backup file, and [LogFILE] with the file name for the backup log.engine-backup --mode=backup --file=[EngineBackupFile] --log=[LogFILE]
# engine-backup --mode=backup --file=[EngineBackupFile] --log=[LogFILE]
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Backing up the Files to an External Server
Back up the files to an external server. In the following example, [Storage.example.com] is the fully qualified domain name of a network storage server that will store the backup until it is needed, and /backup/ is any designated folder or path. The backup files must be accessible to restore the configuration settings and database content.scp -p [EngineBackupFiles] [Storage.example.com:/backup/EngineBackupFiles]
# scp -p [EngineBackupFiles] [Storage.example.com:/backup/EngineBackupFiles]
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Activating the Failover Host
Bring thehosted_engine_1
host out of maintenance mode.- Log in to the Administration Portal.
- Click the Hosts tab.
- Select
hosted_engine_1
from the results list. - Click.
6.2. Restoring the Self-Hosted Engine Environment Copy linkLink copied to clipboard!
engine-backup
tool.
- Create a newly installed Red Hat Enterprise Linux host and run the
hosted-engine
deployment script. - Restore the Red Hat Virtualization Manager configuration settings and database content in the new Manager virtual machine.
- Remove self-hosted engine nodes in a Non Operational state and re-install them into the restored self-hosted engine environment.
Prerequisites
- To restore a self-hosted engine environment, you must prepare a newly installed Red Hat Enterprise Linux system on a physical host.
- The operating system version of the new host and Manager must be the same as that of the original host and Manager.
- You must have Red Hat Subscription Manager entitlements for your new environment. For a list of the required repositories, see Subscribing to the Required Entitlements in the Installation Guide.
- The fully qualified domain name of the new Manager must be the same fully qualified domain name as that of the original Manager. Forward and reverse lookup records must both be set in DNS.
- You must prepare storage for the new self-hosted engine environment to use as the Manager virtual machine's shared storage domain. This domain must be at least 60 GB. For more information on preparing storage for your deployment, see the Storage chapter of the Administration Guide.
6.2.1. Creating a New Self-Hosted Engine Environment to be Used as the Restored Environment Copy linkLink copied to clipboard!
Host 1
, used in Section 6.1, “Backing up the Self-Hosted Engine Manager Virtual Machine” uses the default hostname of hosted_engine_1
which is also used in this procedure. Due to the nature of the restore process for the self-hosted engine, before the final synchronization of the restored engine can take place, this failover host will need to be removed, and this can only be achieved if the host had no virtual load when the backup was taken. You can also restore the backup on a separate hardware which was not used in the backed up environment and this is not a concern.
Important
Procedure 6.4. Creating a New Self-Hosted Environment to be Used as the Restored Environment
Updating DNS
Update your DNS so that the fully qualified domain name of the Red Hat Virtualization environment correlates to the IP address of the new Manager. In this procedure, fully qualified domain name was set as Manager.example.com. The fully qualified domain name provided for the engine must be identical to that given in the engine setup of the original engine that was backed up.Initiating Hosted Engine Deployment
On the newly installed Red Hat Enterprise Linux host, run thehosted-engine
deployment script. To escape the script at any time, use the CTRL+D keyboard combination to abort deployment. If running thehosted-engine
deployment script over a network, it is recommended to use thescreen
window manager to avoid losing the session in case of network or terminal disruption. Install the screen package first if not installed.screen
# screen
Copy to Clipboard Copied! Toggle word wrap Toggle overflow hosted-engine --deploy
# hosted-engine --deploy
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Preparing for Initialization
The script begins by requesting confirmation to use the host as a hypervisor for use in a self-hosted engine environment.Continuing will configure this host for serving as hypervisor and create a VM where you have to install oVirt Engine afterwards. Are you sure you want to continue? (Yes, No)[Yes]:
Continuing will configure this host for serving as hypervisor and create a VM where you have to install oVirt Engine afterwards. Are you sure you want to continue? (Yes, No)[Yes]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring Storage
Select the type of storage to use.During customization use CTRL-D to abort. Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs3, nfs4)[nfs3]:
During customization use CTRL-D to abort. Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs3, nfs4)[nfs3]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For NFS storage types, specify the full address, using either the FQDN or IP address, and path name of the shared storage domain.
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/nfs
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/nfs
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For iSCSI, specify the iSCSI portal IP address, port, user name and password, and select a target name from the auto-detected list. You can only select one iSCSI target during the deployment.
Please specify the iSCSI portal IP address: Please specify the iSCSI portal port [3260]: Please specify the iSCSI portal user: Please specify the iSCSI portal password: Please specify the target name (auto-detected values) [default]:
Please specify the iSCSI portal IP address: Please specify the iSCSI portal port [3260]: Please specify the iSCSI portal user: Please specify the iSCSI portal password: Please specify the target name (auto-detected values) [default]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For Gluster storage, specify the full address, using either the FQDN or IP address, and path name of the shared storage domain.
Important
Only replica 3 Gluster storage is supported. Ensure the following configuration has been made:- In the
/etc/glusterfs/glusterd.vol
file on all three Gluster servers, setrpc-auth-allow-insecure
toon
.option rpc-auth-allow-insecure on
option rpc-auth-allow-insecure on
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure the volume as follows:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/gluster_volume
Please specify the full shared storage connection path to use (example: host:/path): storage.example.com:/hosted_engine/gluster_volume
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For Fibre Channel, the host bus adapters must be configured and connected, and the
hosted-engine
script will auto-detect the LUNs available. The LUNs must not contain any existing data.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Configuring the Network
The script detects possible network interface controllers (NICs) to use as a management bridge for the environment. It then checks your firewall configuration and offers to modify it for console (SPICE or VNC) access the Manager virtual machine. Provide a pingable gateway IP address, to be used by theovirt-ha-agent
, to help determine a host's suitability for running a Manager virtual machine.Please indicate a nic to set ovirtmgmt bridge on: (eth1, eth0) [eth1]: iptables was detected on your computer, do you wish setup to configure it? (Yes, No)[Yes]: Please indicate a pingable gateway IP address [X.X.X.X]:
Please indicate a nic to set ovirtmgmt bridge on: (eth1, eth0) [eth1]: iptables was detected on your computer, do you wish setup to configure it? (Yes, No)[Yes]: Please indicate a pingable gateway IP address [X.X.X.X]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring the New Manager Virtual Machine
The script creates a virtual machine to be configured as the new Manager virtual machine. Specify the boot device and, if applicable, the path name of the installation media, the image alias, the CPU type, the number of virtual CPUs, and the disk size. Specify a MAC address for the Manager virtual machine, or accept a randomly generated one. The MAC address can be used to update your DHCP server prior to installing the operating system on the Manager virtual machine. Specify memory size and console connection type for the creation of Manager virtual machine.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Identifying the Name of the Host
Specify the password for theadmin@internal
user to access the Administration Portal.A unique name must be provided for the name of the host, to ensure that it does not conflict with other resources that will be present when the engine has been restored from the backup. The namehosted_engine_1
can be used in this procedure because this host was placed into maintenance mode before the environment was backed up, enabling removal of this host between the restoring of the engine and the final synchronization of the host and the engine.Enter engine admin password: Confirm engine admin password: Enter the name which will be used to identify this host inside the Administration Portal [hosted_engine_1]:
Enter engine admin password: Confirm engine admin password: Enter the name which will be used to identify this host inside the Administration Portal [hosted_engine_1]:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuring the Hosted Engine
Provide the fully qualified domain name for the new Manager virtual machine. This procedure uses the fully qualified domain name Manager.example.com. Provide the name and TCP port number of the SMTP server, the email address used to send email notifications, and a comma-separated list of email addresses to receive these notifications.Important
The fully qualified domain name provided for the engine (Manager.example.com) must be the same fully qualified domain name provided when original Manager was initially set up.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configuration Preview
Before proceeding, thehosted-engine
deployment script displays the configuration values you have entered, and prompts for confirmation to proceed with these values.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Creating the New Manager Virtual Machine
The script creates the virtual machine to be configured as the Manager virtual machine and provides connection details. You must install an operating system on it before thehosted-engine
deployment script can proceed on Hosted Engine configuration.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Using the naming convention of this procedure, connect to the virtual machine using VNC with the following command:/usr/bin/remote-viewer vnc://hosted_engine_1.example.com:5900
/usr/bin/remote-viewer vnc://hosted_engine_1.example.com:5900
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Installing the Virtual Machine Operating System
Connect to Manager virtual machine and install a Red Hat Enterprise Linux 7 operating system.Synchronizing the Host and the Manager
Return to the host and continue thehosted-engine
deployment script by selecting option 1:(1) Continue setup - VM installation is complete
(1) Continue setup - VM installation is complete
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy to Clipboard Copied! Toggle word wrap Toggle overflow Installing the Manager
Connect to the new Manager virtual machine, register it with Red Hat Subscription Management, and enable the required repositories. See Subscribing to the Required Entitlements in the Installation Guide.Ensure the latest versions of all installed packages are in use, and install the rhevm packages.yum update
# yum update
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Note
Reboot the machine if any kernel related packages have been updated.yum install rhevm
# yum install rhevm
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
6.2.2. Restoring the Self-Hosted Engine Manager Copy linkLink copied to clipboard!
engine-backup
tool to automate the restore of the configuration settings and database content for a backed-up self-hosted engine Manager virtual machine and Data Warehouse. The procedure only applies to components that were configured automatically during the initial engine-setup
. If you configured the database(s) manually during engine-setup
, follow the instructions at Section 6.2.3, “Restoring the Self-Hosted Engine Manager Manually” to restore the back-up environment manually.
Procedure 6.5. Restoring the Self-Hosted Engine Manager
- Secure copy the backup files to the new Manager virtual machine. This example copies the files from a network storage server to which the files were copied in Section 6.1, “Backing up the Self-Hosted Engine Manager Virtual Machine”. In this example, Storage.example.com is the fully qualified domain name of the storage server, /backup/EngineBackupFiles is the designated file path for the backup files on the storage server, and /backup/ is the path to which the files will be copied on the new Manager.
scp -p Storage.example.com:/backup/EngineBackupFiles /backup/
# scp -p Storage.example.com:/backup/EngineBackupFiles /backup/
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Use the
engine-backup
tool to restore a complete backup.- If you are only restoring the Manager, run:
engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --restore-permissions
# engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --restore-permissions
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - If you are restoring the Manager and Data Warehouse, run:
engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --provision-dwh-db --restore-permissions
# engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --provision-dwh-db --restore-permissions
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
If successful, the following output displays:You should now run engine-setup. Done.
You should now run engine-setup. Done.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure the restored Manager virtual machine. This process identifies the existing configuration settings and database content. Confirm the settings. Upon completion, the setup provides an SSH fingerprint and an internal Certificate Authority hash.
engine-setup
# engine-setup
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy to Clipboard Copied! Toggle word wrap Toggle overflow Removing the Host from the Restored Environment
If the deployment of the restored self-hosted engine is on new hardware that has a unique name not present in the backed-up engine, skip this step. This step is only applicable to deployments occurring on the failover host,hosted_engine_1
. Because this host was present in the environment at time the backup was created, it maintains a presence in the restored engine and must first be removed from the environment before final synchronization can take place.- Log in to the Administration Portal.
- Click the Hosts tab. The failover host,
hosted_engine_1
, will be in maintenance mode and without a virtual load, as this was how it was prepared for the backup. - Click.
- Click.
Note
If the host you are trying to remove becomes non-operational, see Section 6.2.4, “Removing Non-Operational Hosts from a Restored Self-Hosted Engine Environment” for instructions on how to force the removal of a host.Synchronizing the Host and the Manager
Return to the host and continue thehosted-engine
deployment script by selecting option 1:(1) Continue setup - engine installation is complete
(1) Continue setup - engine installation is complete
Copy to Clipboard Copied! Toggle word wrap Toggle overflow [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational...
[ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational...
Copy to Clipboard Copied! Toggle word wrap Toggle overflow At this point,hosted_engine_1
will become visible in the Administration Portal with Installing and Initializing states before entering a Non Operational state. The host will continue to wait for the VDSM host to become operational until it eventually times out. This happens because another host in the environment maintains the Storage Pool Manager (SPM) role andhosted_engine_1
cannot interact with the storage domain because the SPM host is in a Non Responsive state. When this process times out, you are prompted to shut down the virtual machine to complete the deployment. When deployment is complete, the host can be manually placed into maintenance mode and activated through the Administration Portal.[ INFO ] Still waiting for VDSM host to become operational... [ ERROR ] Timed out while waiting for host to start. Please check the logs. [ ERROR ] Unable to add hosted_engine_2 to the manager Please shutdown the VM allowing the system to launch it as a monitored service. The system will wait until the VM is down.
[ INFO ] Still waiting for VDSM host to become operational... [ ERROR ] Timed out while waiting for host to start. Please check the logs. [ ERROR ] Unable to add hosted_engine_2 to the manager Please shutdown the VM allowing the system to launch it as a monitored service. The system will wait until the VM is down.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Shut down the new Manager virtual machine.
shutdown -h now
# shutdown -h now
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Return to the host to confirm it has detected that the Manager virtual machine is down.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Activate the host.
- Log in to the Administration Portal.
- Click the Hosts tab.
- Select
hosted_engine_1
and click the Maintenance button. The host may take several minutes before it enters maintenance mode. - Click thebutton.
Once active,hosted_engine_1
immediately contends for SPM, and the storage domain and data center become active. - Migrate virtual machines to the active host by manually fencing the Non Responsive hosts. In the Administration Portal, right-click the hosts and select Confirm 'Host has been Rebooted'.Any virtual machines that were running on that host at the time of the backup will now be removed from that host, and move from an Unknown state to a Down state. These virtual machines can now be run on
hosted_engine_1
. The host that was fenced can now be forcefully removed using the REST API.
hosted_engine_1
is active and is able to run virtual machines in the restored environment. The remaining self-hosted engine nodes in Non Operational state can now be removed by following the steps in Section 6.2.4, “Removing Non-Operational Hosts from a Restored Self-Hosted Engine Environment” and then re-installed into the environment by following the steps in Chapter 7, Installing Additional Hosts to a Self-Hosted Environment.
Note
6.2.3. Restoring the Self-Hosted Engine Manager Manually Copy linkLink copied to clipboard!
Procedure 6.6. Restoring the Self-Hosted Engine Manager
- Manually create an empty database to which the database content in the backup can be restored. The following steps must be performed on the machine where the database is to be hosted.
- If the database is to be hosted on a machine other than the Manager virtual machine, install the postgresql-server package. This step is not required if the database is to be hosted on the Manager virtual machine because this package is included with the rhevm package.
yum install postgresql-server
# yum install postgresql-server
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Initialize the
postgresql
database, start thepostgresql
service, and ensure this service starts on boot:postgresql-setup initdb systemctl start postgresql.service systemctl enable postgresql.service
# postgresql-setup initdb # systemctl start postgresql.service # systemctl enable postgresql.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Enter the postgresql command line:
su postgres psql
# su postgres $ psql
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Create the
engine
user:postgres=# create role engine with login encrypted password 'password';
postgres=# create role engine with login encrypted password 'password';
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If you are also restoring Data Warehouse, create theovirt_engine_history
user on the relevant host:postgres=# create role ovirt_engine_history with login encrypted password 'password';
postgres=# create role ovirt_engine_history with login encrypted password 'password';
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Create the new database:
postgres=# create database database_name owner engine template template0 encoding 'UTF8' lc_collate 'en_US.UTF-8' lc_ctype 'en_US.UTF-8';
postgres=# create database database_name owner engine template template0 encoding 'UTF8' lc_collate 'en_US.UTF-8' lc_ctype 'en_US.UTF-8';
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If you are also restoring the Data Warehouse, create the database on the relevant host:postgres=# create database database_name owner ovirt_engine_history template template0 encoding 'UTF8' lc_collate 'en_US.UTF-8' lc_ctype 'en_US.UTF-8';
postgres=# create database database_name owner ovirt_engine_history template template0 encoding 'UTF8' lc_collate 'en_US.UTF-8' lc_ctype 'en_US.UTF-8';
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Exit the postgresql command line and log out of the postgres user:
postgres=# \q $ exit
postgres=# \q $ exit
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Edit the
/var/lib/pgsql/data/pg_hba.conf
file as follows:- For each local database, replace the existing directives in the section starting with
local
at the bottom of the file with the following directives:host database_name user_name 0.0.0.0/0 md5 host database_name user_name ::0/0 md5
host database_name user_name 0.0.0.0/0 md5 host database_name user_name ::0/0 md5
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - For each remote database:
- Add the following line immediately underneath the line starting with
Local
at the bottom of the file, replacing X.X.X.X with the IP address of the Manager:host database_name user_name X.X.X.X/32 md5
host database_name user_name X.X.X.X/32 md5
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Allow TCP/IP connections to the database. Edit the
/var/lib/pgsql/data/postgresql.conf
file and add the following line:listen_addresses='*'
listen_addresses='*'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This example configures thepostgresql
service to listen for connections on all interfaces. You can specify an interface by giving its IP address. - Open the default port used for PostgreSQL database connections, and save the updated firewall rules:
iptables -I INPUT 5 -p tcp -s Manager_IP_Address --dport 5432 -j ACCEPT service iptables save
# iptables -I INPUT 5 -p tcp -s Manager_IP_Address --dport 5432 -j ACCEPT # service iptables save
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- Restart the
postgresql
service:systemctl restart postgresql.service
# systemctl restart postgresql.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- Secure copy the backup files to the new Manager virtual machine. This example copies the files from a network storage server to which the files were copied in Section 6.1, “Backing up the Self-Hosted Engine Manager Virtual Machine”. In this example, Storage.example.com is the fully qualified domain name of the storage server, /backup/EngineBackupFiles is the designated file path for the backup files on the storage server, and /backup/ is the path to which the files will be copied on the new Manager.
scp -p Storage.example.com:/backup/EngineBackupFiles /backup/
# scp -p Storage.example.com:/backup/EngineBackupFiles /backup/
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Restore a complete backup or a database-only backup with the
--change-db-credentials
parameter to pass the credentials of the new database. The database_location for a database local to the Manager islocalhost
.Note
The following examples use a--*password
option for each database without specifying a password, which will prompt for a password for each database. Passwords can be supplied for these options in the command itself, however this is not recommended as the password will then be stored in the shell history. Alternatively,--*passfile=
password_file options can be used for each database to securely pass the passwords to theengine-backup
tool without the need for interactive prompts.- Restore a complete backup:
engine-backup --mode=restore --file=file_name --log=log_file_name --change-db-credentials --db-host=database_location --db-name=database_name --db-user=engine --db-password
# engine-backup --mode=restore --file=file_name --log=log_file_name --change-db-credentials --db-host=database_location --db-name=database_name --db-user=engine --db-password
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If Data Warehouse is also being restored as part of the complete backup, include the revised credentials for the additional database:engine-backup --mode=restore --file=file_name --log=log_file_name --change-db-credentials --db-host=database_location --db-name=database_name --db-user=engine --db-password --change-dwh-db-credentials --dwh-db-host=database_location --dwh-db-name=database_name --dwh-db-user=ovirt_engine_history --dwh-db-password
engine-backup --mode=restore --file=file_name --log=log_file_name --change-db-credentials --db-host=database_location --db-name=database_name --db-user=engine --db-password --change-dwh-db-credentials --dwh-db-host=database_location --dwh-db-name=database_name --dwh-db-user=ovirt_engine_history --dwh-db-password
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Restore a database-only backup restoring the configuration files and the database backup:
engine-backup --mode=restore --scope=files --scope=db --file=file_name --log=file_name --change-db-credentials --db-host=database_location --db-name=database_name --db-user=engine --db-password
# engine-backup --mode=restore --scope=files --scope=db --file=file_name --log=file_name --change-db-credentials --db-host=database_location --db-name=database_name --db-user=engine --db-password
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The example above restores a backup of the Manager database.engine-backup --mode=restore --scope=files --scope=dwhdb --file=file_name --log=file_name --change-dwh-db-credentials --dwh-db-host=database_location --dwh-db-name=database_name --dwh-db-user=ovirt_engine_history --dwh-db-password
# engine-backup --mode=restore --scope=files --scope=dwhdb --file=file_name --log=file_name --change-dwh-db-credentials --dwh-db-host=database_location --dwh-db-name=database_name --dwh-db-user=ovirt_engine_history --dwh-db-password
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The example above restores a backup of the Data Warehouse database.
If successful, the following output displays:You should now run engine-setup. Done.
You should now run engine-setup. Done.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure the restored Manager virtual machine. This process identifies the existing configuration settings and database content. Confirm the settings. Upon completion, the setup provides an SSH fingerprint and an internal Certificate Authority hash.
engine-setup
# engine-setup
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy to Clipboard Copied! Toggle word wrap Toggle overflow Removing the Host from the Restored Environment
If the deployment of the restored self-hosted engine is on new hardware that has a unique name not present in the backed-up engine, skip this step. This step is only applicable to deployments occurring on the failover host,hosted_engine_1
. Because this host was present in the environment at time the backup was created, it maintains a presence in the restored engine and must first be removed from the environment before final synchronization can take place.- Log in to the Administration Portal.
- Click the Hosts tab. The failover host,
hosted_engine_1
, will be in maintenance mode and without a virtual load, as this was how it was prepared for the backup. - Click.
- Click.
Synchronizing the Host and the Manager
Return to the host and continue thehosted-engine
deployment script by selecting option 1:(1) Continue setup - engine installation is complete
(1) Continue setup - engine installation is complete
Copy to Clipboard Copied! Toggle word wrap Toggle overflow [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational...
[ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational...
Copy to Clipboard Copied! Toggle word wrap Toggle overflow At this point,hosted_engine_1
will become visible in the Administration Portal with Installing and Initializing states before entering a Non Operational state. The host will continue to wait for the VDSM host to become operational until it eventually times out. This happens because another host in the environment maintains the Storage Pool Manager (SPM) role andhosted_engine_1
cannot interact with the storage domain because the SPM host is in a Non Responsive state. When this process times out, you are prompted to shut down the virtual machine to complete the deployment. When deployment is complete, the host can be manually placed into maintenance mode and activated through the Administration Portal.[ INFO ] Still waiting for VDSM host to become operational... [ ERROR ] Timed out while waiting for host to start. Please check the logs. [ ERROR ] Unable to add hosted_engine_2 to the manager Please shutdown the VM allowing the system to launch it as a monitored service. The system will wait until the VM is down.
[ INFO ] Still waiting for VDSM host to become operational... [ ERROR ] Timed out while waiting for host to start. Please check the logs. [ ERROR ] Unable to add hosted_engine_2 to the manager Please shutdown the VM allowing the system to launch it as a monitored service. The system will wait until the VM is down.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Shut down the new Manager virtual machine.
shutdown -h now
# shutdown -h now
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Return to the host to confirm it has detected that the Manager virtual machine is down.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Activate the host.
- Log in to the Administration Portal.
- Click the Hosts tab.
- Select
hosted_engine_1
and click the Maintenance button. The host may take several minutes before it enters maintenance mode. - Click thebutton.
Once active,hosted_engine_1
immediately contends for SPM, and the storage domain and data center become active. - Migrate virtual machines to the active host by manually fencing the Non Responsive hosts. In the Administration Portal, right-click the hosts and select Confirm 'Host has been Rebooted'.Any virtual machines that were running on that host at the time of the backup will now be removed from that host, and move from an Unknown state to a Down state. These virtual machines can now be run on
hosted_engine_1
. The host that was fenced can now be forcefully removed using the REST API.
hosted_engine_1
is active and is able to run virtual machines in the restored environment. The remaining self-hosted engine nodes in Non Operational state can now be removed by following the steps in Section 6.2.4, “Removing Non-Operational Hosts from a Restored Self-Hosted Engine Environment” and then re-installed into the environment by following the steps in Chapter 7, Installing Additional Hosts to a Self-Hosted Environment.
Note
6.2.4. Removing Non-Operational Hosts from a Restored Self-Hosted Engine Environment Copy linkLink copied to clipboard!
Fencing the Non-Operational Host
In the Administration Portal, right-click the hosts and select Confirm 'Host has been Rebooted'.Any virtual machines that were running on that host at the time of the backup will now be removed from that host, and move from an Unknown state to a Down state. The host that was fenced can now be forcefully removed using the REST API.Retrieving the Manager Certificate Authority
Connect to the Manager virtual machine and use the command line to perform the following requests with cURL.Use aGET
request to retrieve the Manager Certificate Authority (CA) certificate for use in all future API requests. In the following example, the--output
option is used to designate the file hosted-engine.ca as the output for the Manager CA certificate. The--insecure
option means that this initial request will be without a certificate.curl --output hosted-engine.ca --insecure https://[Manager.example.com]/ca.crt
# curl --output hosted-engine.ca --insecure https://[Manager.example.com]/ca.crt
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Retrieving the GUID of the Host to be Removed
Use aGET
request on the hosts collection to retrieve the Global Unique Identifier (GUID) for the host to be removed. The following example includes the Manager CA certificate file, and uses theadmin@internal
user for authentication, the password for which will be prompted once the command is executed.curl --request GET --cacert hosted-engine.ca --user admin@internal https://[Manager.example.com]/api/hosts
# curl --request GET --cacert hosted-engine.ca --user admin@internal https://[Manager.example.com]/api/hosts
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This request returns the details of all of the hosts in the environment. The host GUID is a hexadecimal string associated with the host name. For more information on the Red Hat Virtualization REST API, see the Red Hat Virtualization REST API Guide.Removing the Fenced Host
Use aDELETE
request using the GUID of the fenced host to remove the host from the environment. In addition to the previously used options this example specifies headers to specify that the request is to be sent and returned using eXtensible Markup Language (XML), and the body in XML that sets theforce
action to betrue
.curl --request DELETE --cacert hosted-engine.ca --user admin@internal --header "Content-Type: application/xml" --header "Accept: application/xml" --data "<action><force>true</force></action>" https://[Manager.example.com]/api/hosts/ecde42b0-de2f-48fe-aa23-1ebd5196b4a5
curl --request DELETE --cacert hosted-engine.ca --user admin@internal --header "Content-Type: application/xml" --header "Accept: application/xml" --data "<action><force>true</force></action>" https://[Manager.example.com]/api/hosts/ecde42b0-de2f-48fe-aa23-1ebd5196b4a5
Copy to Clipboard Copied! Toggle word wrap Toggle overflow ThisDELETE
request can be used to remove every fenced host in the self-hosted engine environment, as long as the appropriate GUID is specified.Removing the Self-Hosted Engine Configuration from the Host
Remove the host's self-hosted engine configuration so it can be reconfigured when the host is re-installed to a self-hosted engine environment.Log in to the host and remove the configuration file:rm /etc/ovirt-hosted-engine/hosted-engine.conf
# rm /etc/ovirt-hosted-engine/hosted-engine.conf
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 7. Installing Additional Hosts to a Self-Hosted Environment Copy linkLink copied to clipboard!
Prerequisites
- For a RHEL-based self-hosted engine environment, you must have prepared a freshly installed Red Hat Enterprise Linux system on a physical host, and subscribed the host to the required entitlements. See Subscribing to the Required Entitlements in the Installation Guide for more information on subscriptions.
- For a RHVH-based self-hosted engine environment, you must have prepared a freshly installed Red Hat Virtualization Host system on a physical host. See Red Hat Virtualization Hosts in the Installation Guide.
- If you are reusing a self-hosted engine node, remove its existing self-hosted engine configuration. See Section 5.2, “Removing a Host from a Self-Hosted Engine Environment”.
Procedure 7.1. Adding an Additional Self-Hosted Engine Node
- In the Administration Portal, click the Hosts resource tab.
- Click.For information on additional host settings, see Explanation of Settings and Controls in the New Host and Edit Host Windows in the Administration Guide.
- Use the drop-down list to select the Data Center and Host Cluster for the new host.
- Enter the Name and the Address of the new host. The standard SSH port, port 22, is auto-filled in the SSH Port field.
- Select an authentication method to use for the Manager to access the host.
- Enter the root user's password to use password authentication.
- Alternatively, copy the key displayed in the SSH PublicKey field to
/root/.ssh/authorized_keys
on the host to use public key authentication.
- Optionally, configure power management, where the host has a supported power management card. For information on power management configuration, see Host Power Management Settings Explained in the Administration Guide.
- Click the Hosted Engine sub-tab.
- Select the Deploy radio button.
- Click.
Chapter 8. Migrating the Self-Hosted Engine Database to a Remote Server Database Copy linkLink copied to clipboard!
engine
database of a self-hosted engine to a remote database server after the Red Hat Virtualization Manager has been initially configured. Use engine-backup
to create a database backup and restore it on the new database server. This procedure assumes that the new database server has Red Hat Enterprise Linux 7 installed and the appropriate subscriptions configured. See Subscribing to the Required Entitlements in the Installation Guide.
Procedure 8.1. Migrating the Database
- Log in to a self-hosted engine node and place the environment into
global
maintenance mode. This disables the High Availability agents and prevents the Manager virtual machine from being migrated during the procedure:hosted-engine --set-maintenance --mode=global
# hosted-engine --set-maintenance --mode=global
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Log in to the Red Hat Virtualization Manager machine and stop the
ovirt-engine
service so that it does not interfere with the engine backup:systemctl stop ovirt-engine.service
# systemctl stop ovirt-engine.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Create the
engine
database backup:engine-backup --scope=files --scope=db --mode=backup --file=file_name --log=backup_log_name
# engine-backup --scope=files --scope=db --mode=backup --file=file_name --log=backup_log_name
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Copy the backup file to the new database server:
scp /tmp/engine.dump root@new.database.server.com:/tmp
# scp /tmp/engine.dump root@new.database.server.com:/tmp
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Log in to the new database server and install
engine-backup
:yum install ovirt-engine-tools-backup
# yum install ovirt-engine-tools-backup
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Restore the database on the new database server. file_name is the backup file copied from the Manager.
engine-backup --mode=restore --scope=files --scope=db --file=file_name --log=restore_log_name --provision-db --no-restore-permissions
# engine-backup --mode=restore --scope=files --scope=db --file=file_name --log=restore_log_name --provision-db --no-restore-permissions
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Now that the database has been migrated, start the
ovirt-engine
service:systemctl start ovirt-engine.service
# systemctl start ovirt-engine.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Log in to a self-hosted engine node and turn off maintenance mode, enabling the High Availability agents:
hosted-engine --set-maintenance --mode=none
# hosted-engine --set-maintenance --mode=none
Copy to Clipboard Copied! Toggle word wrap Toggle overflow