Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 4. Configuring a Red Hat high availability cluster on Google Cloud

A high availability (HA) cluster offers to group RHEL nodes so that workloads are automatically redistributed if a node fails. You can deploy HA clusters on public cloud platforms, including Google Cloud. The process for setting up HA clusters on Google Cloud is comparable to configuring them in traditional, non-cloud environments.

To configure a Red Hat HA cluster on Google Cloud that uses Google Compute Engine (GCE) instances as cluster nodes, see the following sections. You have several options for obtaining RHEL images for the cluster.

These provide information on:

  • Prerequisite procedures for setting up your environment for Google Cloud. Once you have set up your environment, you can create and configure VM instances.
  • Procedures specific to the creation of HA clusters, which transform individual nodes into a cluster of HA nodes on Google Cloud. These include procedures for installing the High Availability packages and agents on each cluster node, configuring fencing, and installing network resource agents.

Prerequisites

  • Red Hat Enterprise Linux 9 Server: rhel-9-server-rpms/8Server/x86_64

  • Red Hat Enterprise Linux 9 Server (High Availability): rhel-9-server-ha-rpms/8Server/x86_64

    • You must belong to an active Google Cloud project and have sufficient permissions to create resources in the project.
    • Your project should have a service account that belongs to a VM instance and not an individual user. See Using the Compute Engine Default Service Account for information about using the default service account instead of creating a separate service account.

If you or your project administrator create a custom service account, the service account should be configured for the following roles.

  • Cloud Trace Agent
  • Compute Admin
  • Compute Network Admin
  • Cloud Datastore User
  • Logging Admin
  • Monitoring Editor
  • Monitoring Metric Writer
  • Service Account Administrator
  • Storage Admin

A high-availability (HA) cluster is a set of computers, also known as nodes, linked together to run a specific workload. The purpose of HA clusters is to offer redundancy in case of a hardware or software failure. If a node in the HA cluster fails, the Pacemaker cluster resource manager distributes the workload to other nodes. No noticeable downtime occurs in the services that are running on the cluster.

You can also run HA clusters on public cloud platforms. In this case, you would use virtual machine (VM) instances in the cloud as the individual cluster nodes. Using HA clusters on a public cloud platform has the following benefits:

  • Improved availability: In case of a VM failure, the workload is quickly redistributed to other nodes, so running services are not disrupted.
  • Scalability: You can start additional nodes when demand is high and stop them when demand is low.
  • Cost-effectiveness: With the pay-as-you-go pricing, you pay only for nodes that are running.
  • Simplified management: Some public cloud platforms offer management interfaces to make configuring HA clusters easier.

To enable HA on your Red Hat Enterprise Linux (RHEL) systems, Red Hat offers a High Availability Add-On. The High Availability Add-On provides all necessary components for creating HA clusters on RHEL systems. The components include high availability service management and cluster administration tools.

4.2. Required system packages
Copy link

To create and configure a base image of RHEL, your host system must have the following packages installed.

Expand
Table 4.1. System packages
PackageRepositoryDescription

libvirt

rhel-9-for-x86_64-appstream-rpms

Open source API, daemon, and management tool for managing platform virtualization

virt-install

rhel-9-for-x86_64-appstream-rpms

A command-line utility for building VMs

libguestfs

rhel-9-for-x86_64-appstream-rpms

A library for accessing and modifying VM file systems

guestfs-tools

rhel-9-for-x86_64-appstream-rpms

System administration tools for VMs; includes the virt-customize utility

4.3. Creating a Google Cloud image bucket
Copy link

The following document includes the minimum requirements for creating a multi-regional bucket in your default location.

Prerequisites

  • Google Cloud storage utility (gsutil)

Procedure

  1. If you are not already logged in to Google Cloud, log in with the following command. 

    # gcloud auth login

  2. Create a storage bucket. 

    $ gsutil mb gs://BucketName

    Example: 

    $ gsutil mb gs://rhel-ha-bucket

A custom virtual private cloud (VPC) network and subnet are required for a cluster to be configured with a High Availability (HA) function.

Procedure

  1. Launch the Google Cloud Console.
  2. Select VPC networks under Networking in the left navigation pane.
  3. Click Create VPC Network.
  4. Enter a name for the VPC network.
  5. Under the New subnet, create a Custom subnet in the region where you want to create the cluster.
  6. Click Create.

Before a local RHEL 9 image can be deployed in Google Cloud, you must first convert and upload the image to your Google Cloud Bucket.

Procedure

  1. Convert the file. Images uploaded to Google Cloud must be in raw format and named disk.raw. 

    $ qemu-img convert -f qcow2 ImageName.qcow2 -O raw disk.raw

  2. Compress the raw file. Images uploaded to Google Cloud must be compressed. 

    $ tar -Sczf ImageName.tar.gz disk.raw

  3. Import the compressed image to the bucket created earlier. 

    $ gsutil cp ImageName.tar.gz gs://BucketName

To create and configure a Google Cloud instance that complies with Google Cloud operating and security requirements, complete the following steps.

Procedure

  1. Create an image from the compressed file in the bucket. 

    $ gcloud compute images create BaseImageName --source-uri gs://BucketName/BaseImageName.tar.gz

    Example: 

    [admin@localhost ~] $ gcloud compute images create rhel-76-server --source-uri gs://user-rhelha/rhel-server-76.tar.gz
Created [https://www.googleapis.com/compute/v1/projects/MyProject/global/images/rhel-server-76].
NAME            PROJECT                 FAMILY  DEPRECATED  STATUS
rhel-76-server  rhel-ha-testing-on-gcp                      READY

  2. Create a template instance from the image. The minimum size required for a base RHEL instance is n1-standard-2. See gcloud compute instances create for additional configuration options. 

    $ gcloud compute instances create BaseInstanceName --can-ip-forward --machine-type n1-standard-2 --image BaseImageName --service-account ServiceAccountEmail

    Example: 

    [admin@localhost ~] $ gcloud compute instances create rhel-76-server-base-instance --can-ip-forward --machine-type n1-standard-2 --image rhel-76-server --service-account account@project-name-on-gcp.iam.gserviceaccount.com
Created [https://www.googleapis.com/compute/v1/projects/rhel-ha-testing-on-gcp/zones/us-east1-b/instances/rhel-76-server-base-instance].
NAME   ZONE   MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP     STATUS
rhel-76-server-base-instance  us-east1-bn1-standard-2          10.10.10.3   192.227.54.211  RUNNING

  3. Connect to the instance with an SSH terminal session. 

    $ ssh root@PublicIPaddress

  4. Update the RHEL software.

    1. Register with Red Hat Subscription Manager (RHSM).
    2. Enable a Subscription Pool ID.

    3. Disable all repositories. 

      # subscription-manager repos --disable=*

    4. Enable the following repository. 

      # subscription-manager repos --enable=rhel-9-server-rpms

    5. Run the dnf update command. 

      # dnf update -y

  5. Install the Google Cloud Linux Guest Environment on the running instance (in-place installation).

    See Install the guest environment in-place for instructions.

  6. Select the CentOS/RHEL option.
  7. Copy the command script and paste it at the command prompt to run the script immediately.

  8. Make the following configuration changes to the instance. These changes are based on Google Cloud recommendations for custom images. See gcloudcompute images list for more information.

    1. Edit the /etc/chrony.conf file and remove all NTP servers.

    2. Add the following NTP server. 

      metadata.google.internal iburst Google NTP server

    3. Remove any persistent network device rules. 

      # rm -f /etc/udev/rules.d/70-persistent-net.rules

# rm -f /etc/udev/rules.d/75-persistent-net-generator.rules

    4. Set the network service to start automatically. 

      # chkconfig network on

    5. Set the sshd service to start automatically. 

      # systemctl enable sshd
# systemctl is-enabled sshd

    6. Set the time zone to UTC. 

      # ln -sf /usr/share/zoneinfo/UTC /etc/localtime

    7. Optional: Edit the /etc/ssh/ssh_config file and add the following lines to the end of the file. This keeps your SSH session active during longer periods of inactivity. 

      # Server times out connections after several minutes of inactivity.
# Keep alive ssh connections by sending a packet every 7 minutes.
ServerAliveInterval 420

    8. Edit the /etc/ssh/sshd_config file and make the following changes, if necessary. The ClientAliveInterval 420 setting is optional; this keeps your SSH session active during longer periods of inactivity. 

      PermitRootLogin no
PasswordAuthentication no
AllowTcpForwarding yes
X11Forwarding no
PermitTunnel no
# Compute times out connections after 10 minutes of inactivity.
# Keep ssh connections alive by sending a packet every 7 minutes.
ClientAliveInterval 420

  9. Disable password access. 

    ssh_pwauth from 1 to 0.
ssh_pwauth: 0
    Important

    Previously, you enabled password access to allow SSH session access to configure the instance. You must disable password access. All SSH session access must be passwordless.

  10. Unregister the instance from the subscription manager. 

    # subscription-manager unregister

  11. Clean the shell history. Keep the instance running for the next procedure. 

    # export HISTSIZE=0

4.7. Creating a snapshot image
Copy link

To preserve the configuration and disk data of a Google Cloud HA instance, create a snapshot of it.

Procedure

  1. On the running instance, synchronize data to disk. 

    # sync

  2. On your host system, create the snapshot. 

    $ gcloud compute disks snapshot InstanceName --snapshot-names SnapshotName

  3. On your host system, create the configured image from the snapshot. 

    $ gcloud compute images create ConfiguredImageFromSnapshot --source-snapshot SnapshotName

After you have configured an image from the snapshot, you can create a node template. Then, you can use this template to create all HA nodes.

Procedure

  1. Create an instance template: 

    $ gcloud compute instance-templates create InstanceTemplateName --can-ip-forward --machine-type n1-standard-2 --image ConfiguredImageFromSnapshot --service-account ServiceAccountEmailAddress
     
    $ gcloud compute instance-templates create rhel-9-instance-template --can-ip-forward --machine-type n1-standard-2 --image rhel-9-gcp-image --service-account account@project-name-on-gcp.iam.gserviceaccount.com
     
    Created [https://www.googleapis.com/compute/v1/projects/project-name-on-gcp/global/instanceTemplates/rhel-91-instance-template].
NAME  MACHINE_TYPE   PREEMPTIBLE  CREATION_TIMESTAMP
rhel-9-instance-template   n1-standard-2     2018-07-25T11:09:30.506-07:00

  2. Create multiple nodes in one zone: 

    # gcloud compute instances create NodeName01 NodeName02 --source-instance-template InstanceTemplateName --zone RegionZone --network=NetworkName --subnet=SubnetName

    Example: 

    $ gcloud compute instances create rhel-9-node-01 rhel-9-node-02 rhel-9-node-03 --source-instance-template rhel-9-instance-template --zone us-west1-b --network=projectVPC --subnet=range0
     
    Created [https://www.googleapis.com/compute/v1/projects/project-name-on-gcp/zones/us-west1-b/instances/rhel-9-node-01].
Created [https://www.googleapis.com/compute/v1/projects/project-name-on-gcp/zones/us-west1-b/instances/rhel-9-node-02].
Created [https://www.googleapis.com/compute/v1/projects/project-name-on-gcp/zones/us-west1-b/instances/rhel-9-node-03].
NAME            ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS
rhel-9-node-01  us-west1-b  n1-standard-2               10.10.10.4   192.230.25.81   RUNNING
rhel-9-node-02  us-west1-b  n1-standard-2               10.10.10.5   192.230.81.253  RUNNING
rhel-9-node-03  us-east1-b  n1-standard-2               10.10.10.6   192.230.102.15  RUNNING

4.9. Installing HA packages and agents
Copy link

On each of your nodes, you need to install the High Availability packages and agents to be able to configure a Red Hat High Availability cluster on Google Cloud.

Procedure

  1. In the Google Cloud Console, select Compute Engine and then select VM instances.
  2. Select the instance, click the arrow next to SSH, and select the View gcloud command option.
  3. Paste this command at a command prompt for passwordless access to the instance.
  4. Enable sudo account access and register with Red Hat Subscription Manager.
  5. Enable a Subscription Pool ID.

  6. Disable all repositories. 

    # subscription-manager repos --disable=*

  7. Enable the following repositories. 

    # subscription-manager repos --enable=rhel-9-server-rpms
# subscription-manager repos --enable=rhel-9-for-x86_64-highavailability-rpms

  8. Install pcs pacemaker, the fence agents, and the resource agents. 

    # dnf install -y pcs pacemaker fence-agents-gce resource-agents-cloud

  9. Update all packages. 

    # dnf update -y

4.10. Configuring high availability services
Copy link

On each of your nodes, configure the HA services.

Prerequisites

  • You have enabled the firewalld service.

Procedure

  1. The user hacluster was created during the pcs and pacemaker installation in the earlier step. Create a password for the user hacluster on all cluster nodes. Use the same password for all nodes. 

    # passwd hacluster

  2. Add the high availability service to the firewalld service: 

    # firewall-cmd --permanent --add-service=high-availability

  3. Reload the firewalld service: 

    # firewall-cmd --reload

  4. Start the pcs service and enable it to start on boot: 

    # systemctl start pcsd.service
     
    # systemctl enable pcsd.service
     
    Created symlink from /etc/systemd/system/multi-user.target.wants/pcsd.service to /usr/lib/systemd/system/pcsd.service.

Verification

  1. Ensure the pcsd service is running: 

    # systemctl status pcsd.service
     
    pcsd.service - PCS GUI and remote configuration interface
Loaded: loaded (/usr/lib/systemd/system/pcsd.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-06-25 19:21:42 UTC; 15s ago
Docs: man:pcsd(8)
man:pcs(8)
Main PID: 5901 (pcsd)
CGroup: /system.slice/pcsd.service
└─5901 /usr/bin/ruby /usr/lib/pcsd/pcsd > /dev/null &
  2. Edit the /etc/hosts file and add Red Hat Enterprise Linux (RHEL) host names and internal IP addresses for all nodes.

4.11. Creating a cluster
Copy link

To convert multiple nodes into a cluster, use the following steps.

Procedure

  1. On one of the nodes, authenticate the pcs user. Specify the name of each node in the cluster in the command. 

    # pcs host auth hostname1 hostname2 hostname3
Username: hacluster
Password:
hostname1: Authorized
hostname2: Authorized
hostname3: Authorized

  2. Create the cluster. 

    # pcs cluster setup cluster-name hostname1 hostname2 hostname3

Verification

  1. Run the following command to enable nodes to join the cluster automatically when started. 

    # pcs cluster enable --all

  2. Start the cluster. 

    # pcs cluster start --all

4.12. Creating a fencing device
Copy link

A fencing device in High Availability (HA) environments ensures to isolate malfunctioning nodes and keep the cluster available if an outage occurs. Note that for most default configurations, the Google Cloud instance names and the RHEL host names are the same.

Procedure

  1. Obtain Google Cloud instance names. Note that the output of the following command also shows the internal ID for the instance. 

    # fence_gce --zone us-west1-b --project=rhel-ha-on-gcp -o list

    Example: 

    [root@rhel81-node-01 ~]# fence_gce --zone us-west1-b --project=rhel-ha-testing-on-gcp -o list
     
    4435801234567893181,InstanceName-3
4081901234567896811,InstanceName-1
7173601234567893341,InstanceName-2

  2. Create a fence device. 

    # pcs stonith create FenceDeviceName fence_gce zone=Region-Zone project=MyProject
  3. To ensure immediate and complete fencing, disable ACPI soft-off on all cluster nodes. For information about disabling ACPI soft-off, see Testing fence devices.

Verification

  • Verify that the fence devices started: 

    # pcs status

    Example: 

    [root@rhel81-node-01 ~]# pcs status
     
    Cluster name: gcp-cluster
Stack: corosync
Current DC: rhel81-node-02 (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum
Last updated: Fri Jul 27 12:53:25 2018
Last change: Fri Jul 27 12:51:43 2018 by root via cibadmin on rhel81-node-01

3 nodes configured
3 resources configured

Online: [ rhel81-node-01 rhel81-node-02 rhel81-node-03 ]

Full list of resources:

us-west1-b-fence    (stonith:fence_gce):    Started rhel81-node-01

Daemon Status:
corosync: active/enabled
pacemaker: active/enabled
pcsd: active/enabled

The gcp-vpc-move-vip resource agent attaches a secondary IP address (alias IP) to a running instance. You can assign this floating IP address between different nodes in the cluster. 

# pcs resource describe gcp-vpc-move-vip

You can configure the resource agent to use a primary subnet address range or a secondary subnet address range.

If you need to automate or manage the assigned IP addresses allocation for VM or other resources within a subnet, you can use the primary subnet address range. It ensures that the primary address range is correctly set and configured to use as stable IP addresses for the primary virtual private network (VPC) subnet.

Procedure

  1. Create the aliasip resource by including an unused internal IP address and the CIDR block: 

    # pcs resource create aliasip gcp-vpc-move-vip alias_ip=UnusedIPaddress/CIDRblock

    Example: 

    [root@rhel81-node-01 ~]# pcs resource create aliasip gcp-vpc-move-vip alias_ip=10.10.10.200/32

  2. Create an IPaddr2 resource for managing the IP on the node: 

    # pcs resource create vip IPaddr2 nic=interface ip=AliasIPaddress cidr_netmask=32

    Example: 

    [root@rhel81-node-01 ~]# pcs resource create vip IPaddr2 nic=eth0 ip=10.10.10.200 cidr_netmask=32

  3. Group the network resources under vipgrp: 

    # pcs resource group add vipgrp aliasip vip

Verification

  1. Verify the active resources and under the vipgrp group: 

    # pcs status

  2. Verify the movable resources across the nodes: 

    # pcs resource move vip Node

    Example: 

    [root@rhel81-node-01 ~]# pcs resource move vip rhel81-node-03

  3. Verify if the vip successfully started on a different node: 

    # pcs status

You can use the secondary subnet address range if you need to assign IP addresses from additional and predefined ranges within the same subnet, without creating a new subnet. It is useful for specific purposes such as custom routing. With a secondary subnet address range, you can manage network traffic in a single subnet with multiple IP address ranges.

Prerequisites

Procedure

  1. Create a secondary subnet address range: 

    # gcloud compute networks subnets update SubnetName --region RegionName --add-secondary-ranges SecondarySubnetName=SecondarySubnetRange

    Example: 

    # gcloud compute networks subnets update range0 --region us-west1 --add-secondary-ranges range1=10.10.20.0/24

  2. Create the aliasip resource. Create an unused internal IP address in the secondary subnet address range. Include the CIDR block in the command. 

    # pcs resource create aliasip gcp-vpc-move-vip alias_ip=UnusedIPaddress/CIDRblock

    Example: 

    [root@rhel81-node-01 ~]# pcs resource create aliasip gcp-vpc-move-vip alias_ip=10.10.20.200/32

  3. Create an IPaddr2 resource for managing the IP on the node. 

    # pcs resource create vip IPaddr2 nic=interface ip=AliasIPaddress cidr_netmask=32

    Example: 

    [root@rhel81-node-01 ~]# pcs resource create vip IPaddr2 nic=eth0 ip=10.10.20.200 cidr_netmask=32

  4. Group the network resources under vipgrp: 

    # pcs resource group add vipgrp aliasip vip

Verification

  1. Verify that the resources have started and grouped under the vipgrp group: 

    # pcs status

  2. Verify that the resource can move to a different node: 

    # pcs resource move vip Node

    Example: 

    [root@rhel81-node-01 ~]# pcs resource move vip rhel81-node-03

  3. Verify that the vip successfully started on a different node. 

    # pcs status
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top