Chapter 25. Configuring a high-availability cluster by using the ha_cluster RHEL System Role

Procedure

1. Create an inventory file specifying the nodes in the cluster, as described in Specifying an inventory for the ha_cluster System Role.

2. Create a playbook file, for example new-cluster.yml.

   Note

   When creating your playbook file for production, vault encrypt the password, as described in Encrypting content with Ansible Vault.

   The following example playbook file configures a cluster running the firewalld and selinux services and creates a self-signed pcsd certificate and private key files in /var/lib/pcsd. The pcsd certificate has the file name FILENAME.crt and the key file is named FILENAME.key.

   - hosts: node1 node2
     vars:
       ha_cluster_cluster_name: my-new-cluster
       ha_cluster_hacluster_password: password
       ha_cluster_manage_firewall: true
       ha_cluster_manage_selinux: true
       ha_cluster_pcsd_certificates:
         - name: FILENAME
           common_name: "{{ ansible_hostname }}"
           ca: self-sign
     roles:
       - linux-system-roles.ha_cluster

   Copy to Clipboard Toggle word wrap
3. Save the file.

4. Run the playbook, specifying the path to the inventory file inventory you created in Step 1.

   # ansible-playbook -i inventory new-cluster.yml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create an inventory file specifying the nodes in the cluster, as described in Specifying an inventory for the ha_cluster System Role.

2. Create a playbook file, for example new-cluster.yml.

   Note

   When creating your playbook file for production, vault encrypt the password, as described in Encrypting content with Ansible Vault.

   The following example playbook file configures a cluster running the firewalld and selinux services with no fencing configured and which runs no resources.

   - hosts: node1 node2
     vars:
       ha_cluster_cluster_name: my-new-cluster
       ha_cluster_hacluster_password: password
       ha_cluster_manage_firewall: true
       ha_cluster_manage_selinux: true
   
     roles:
       - rhel-system-roles.ha_cluster

   Copy to Clipboard Toggle word wrap
3. Save the file.

4. Run the playbook, specifying the path to the inventory file inventory you created in Step 1.

   # ansible-playbook -i inventory new-cluster.yml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create an inventory file specifying the nodes in the cluster, as described in Specifying an inventory for the ha_cluster System Role.

2. Create a playbook file, for example new-cluster.yml.

   Note

   When creating your playbook file for production, vault encrypt the password, as described in Encrypting content with Ansible Vault.

   The following example playbook file configures a cluster running the firewalld and selinux services. The cluster includes fencing, several resources, and a resource group. It also includes a resource clone for the resource group.

   - hosts: node1 node2
     vars:
       ha_cluster_cluster_name: my-new-cluster
       ha_cluster_hacluster_password: password
       ha_cluster_manage_firewall: true
       ha_cluster_manage_selinux: true
       ha_cluster_resource_primitives:
         - id: xvm-fencing
           agent: 'stonith:fence_xvm'
           instance_attrs:
             - attrs:
                 - name: pcmk_host_list
                   value: node1 node2
         - id: simple-resource
           agent: 'ocf:pacemaker:Dummy'
         - id: resource-with-options
           agent: 'ocf:pacemaker:Dummy'
           instance_attrs:
             - attrs:
                 - name: fake
                   value: fake-value
                 - name: passwd
                   value: passwd-value
           meta_attrs:
             - attrs:
                 - name: target-role
                   value: Started
                 - name: is-managed
                   value: 'true'
           operations:
             - action: start
               attrs:
                 - name: timeout
                   value: '30s'
             - action: monitor
               attrs:
                 - name: timeout
                   value: '5'
                 - name: interval
                   value: '1min'
         - id: dummy-1
           agent: 'ocf:pacemaker:Dummy'
         - id: dummy-2
           agent: 'ocf:pacemaker:Dummy'
         - id: dummy-3
           agent: 'ocf:pacemaker:Dummy'
         - id: simple-clone
           agent: 'ocf:pacemaker:Dummy'
         - id: clone-with-options
           agent: 'ocf:pacemaker:Dummy'
       ha_cluster_resource_groups:
         - id: simple-group
           resource_ids:
             - dummy-1
             - dummy-2
           meta_attrs:
             - attrs:
                 - name: target-role
                   value: Started
                 - name: is-managed
                   value: 'true'
         - id: cloned-group
           resource_ids:
             - dummy-3
       ha_cluster_resource_clones:
         - resource_id: simple-clone
         - resource_id: clone-with-options
           promotable: yes
           id: custom-clone-id
           meta_attrs:
             - attrs:
                 - name: clone-max
                   value: '2'
                 - name: clone-node-max
                   value: '1'
         - resource_id: cloned-group
           promotable: yes
   
     roles:
       - rhel-system-roles.ha_cluster

   Copy to Clipboard Toggle word wrap
3. Save the file.

4. Run the playbook, specifying the path to the inventory file inventory you created in Step 1.

   # ansible-playbook -i inventory new-cluster.yml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create an inventory file specifying the nodes in the cluster, as described in Specifying an inventory for the ha_cluster System Role.

2. Create a playbook file, for example new-cluster.yml.

   Note

   When creating your playbook file for production, vault encrypt the password, as described in Encrypting content with Ansible Vault.

   The following example playbook file configures a cluster running the firewalld and selinux services. The cluster includes resource location constraints, resource colocation constraints, resource order constraints, and resource ticket constraints.

   - hosts: node1 node2
     vars:
       ha_cluster_cluster_name: my-new-cluster
       ha_cluster_hacluster_password: password
       ha_cluster_manage_firewall: true
       ha_cluster_manage_selinux: true
       # In order to use constraints, we need resources the constraints will apply
       # to.
       ha_cluster_resource_primitives:
         - id: xvm-fencing
           agent: 'stonith:fence_xvm'
           instance_attrs:
             - attrs:
                 - name: pcmk_host_list
                   value: node1 node2
         - id: dummy-1
           agent: 'ocf:pacemaker:Dummy'
         - id: dummy-2
           agent: 'ocf:pacemaker:Dummy'
         - id: dummy-3
           agent: 'ocf:pacemaker:Dummy'
         - id: dummy-4
           agent: 'ocf:pacemaker:Dummy'
         - id: dummy-5
           agent: 'ocf:pacemaker:Dummy'
         - id: dummy-6
           agent: 'ocf:pacemaker:Dummy'
       # location constraints
       ha_cluster_constraints_location:
         # resource ID and node name
         - resource:
             id: dummy-1
           node: node1
           options:
             - name: score
               value: 20
         # resource pattern and node name
         - resource:
             pattern: dummy-\d+
           node: node1
           options:
             - name: score
               value: 10
         # resource ID and rule
         - resource:
             id: dummy-2
           rule: '#uname eq node2 and date in_range 2022-01-01 to 2022-02-28'
         # resource pattern and rule
         - resource:
             pattern: dummy-\d+
           rule: node-type eq weekend and date-spec weekdays=6-7
       # colocation constraints
       ha_cluster_constraints_colocation:
         # simple constraint
         - resource_leader:
             id: dummy-3
           resource_follower:
             id: dummy-4
           options:
             - name: score
               value: -5
         # set constraint
         - resource_sets:
             - resource_ids:
                 - dummy-1
                 - dummy-2
             - resource_ids:
                 - dummy-5
                 - dummy-6
               options:
                 - name: sequential
                   value: "false"
           options:
             - name: score
               value: 20
       # order constraints
       ha_cluster_constraints_order:
         # simple constraint
         - resource_first:
             id: dummy-1
           resource_then:
             id: dummy-6
           options:
             - name: symmetrical
               value: "false"
         # set constraint
         - resource_sets:
             - resource_ids:
                 - dummy-1
                 - dummy-2
               options:
                 - name: require-all
                   value: "false"
                 - name: sequential
                   value: "false"
             - resource_ids:
                 - dummy-3
             - resource_ids:
                 - dummy-4
                 - dummy-5
               options:
                 - name: sequential
                   value: "false"
       # ticket constraints
       ha_cluster_constraints_ticket:
         # simple constraint
         - resource:
             id: dummy-1
           ticket: ticket1
           options:
             - name: loss-policy
               value: stop
         # set constraint
         - resource_sets:
             - resource_ids:
                 - dummy-3
                 - dummy-4
                 - dummy-5
           ticket: ticket2
           options:
             - name: loss-policy
               value: fence
   
     roles:
       - linux-system-roles.ha_cluster

   Copy to Clipboard Toggle word wrap
3. Save the file.

4. Run the playbook, specifying the path to the inventory file inventory you created in Step 1.

   # ansible-playbook -i inventory new-cluster.yml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create an inventory file specifying the nodes in the cluster, as described in Specifying an inventory for the ha_cluster System Role.

2. Create a playbook file, for example new-cluster.yml.

   Note

   When creating your playbook file for production, Vault encrypt the password, as described in Encrypting content with Ansible Vault.

   The following example playbook file configures a cluster running the firewalld and selinux services that configures Corosync properties.

   - hosts: node1 node2
     vars:
       ha_cluster_cluster_name: my-new-cluster
       ha_cluster_hacluster_password: password
       ha_cluster_manage_firewall: true
       ha_cluster_manage_selinux: true
       ha_cluster_transport:
         type: knet
         options:
           - name: ip_version
             value: ipv4-6
           - name: link_mode
             value: active
         links:
           -
             - name: linknumber
               value: 1
             - name: link_priority
               value: 5
           -
             - name: linknumber
               value: 0
             - name: link_priority
               value: 10
         compression:
           - name: level
             value: 5
           - name: model
             value: zlib
         crypto:
           - name: cipher
             value: none
           - name: hash
             value: none
       ha_cluster_totem:
         options:
           - name: block_unlisted_ips
             value: 'yes'
           - name: send_join
             value: 0
       ha_cluster_quorum:
         options:
           - name: auto_tie_breaker
             value: 1
           - name: wait_for_all
             value: 1
   
     roles:
       - linux-system-roles.ha_cluster

   Copy to Clipboard Toggle word wrap
3. Save the file.

4. Run the playbook, specifying the path to the inventory file inventory you created in Step 1.

   # ansible-playbook -i inventory new-cluster.yml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create an inventory file specifying the nodes in the cluster, as described in Specifying an inventory for the ha_cluster System Role. You can optionally configure watchdog and SBD devices for each node in the cluster in an inventory file.

2. Create a playbook file, for example new-cluster.yml.

   Note

   When creating your playbook file for production, vault encrypt the password, as described in Encrypting content with Ansible Vault.

   The following example playbook file configures a cluster running the firewalld and selinux services that uses SBD fencing.

   - hosts: node1 node2
     vars:
       ha_cluster_cluster_name: my-new-cluster
       ha_cluster_hacluster_password: password
       ha_cluster_manage_firewall: true
       ha_cluster_manage_selinux: true
       ha_cluster_sbd_enabled: yes
       ha_cluster_sbd_options:
         - name: delay-start
           value: 'no'
         - name: startmode
           value: always
         - name: timeout-action
           value: 'flush,reboot'
         - name: watchdog-timeout
           value: 5
   
     roles:
       - linux-system-roles.ha_cluster

   Copy to Clipboard Toggle word wrap
3. Save the file.

4. Run the playbook, specifying the path to the inventory file inventory you created in Step 1.

   # ansible-playbook -i inventory new-cluster.yml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create an inventory file specifying the nodes in the cluster, as described in Specifying an inventory for the ha_cluster System Role.

2. Create a playbook file, for example http-cluster.yml.

   Note

   When creating your playbook file for production, vault encrypt the password, as described in Encrypting content with Ansible Vault.

   The following example playbook file configures a previously-created Apache HTTP server in an active/passive two-node HA cluster running the firewalld and selinux services.

   This example uses an APC power switch with a host name of zapc.example.com. If the cluster does not use any other fence agents, you can optionally list only the fence agents your cluster requires when defining the ha_cluster_fence_agent_packages variable, as in this example.

   - hosts: z1.example.com z2.example.com
     roles:
       - rhel-system-roles.ha_cluster
     vars:
       ha_cluster_hacluster_password: password
       ha_cluster_cluster_name: my_cluster
       ha_cluster_manage_firewall: true
       ha_cluster_manage_selinux: true
       ha_cluster_fence_agent_packages:
         - fence-agents-apc-snmp
       ha_cluster_resource_primitives:
         - id: myapc
           agent: stonith:fence_apc_snmp
           instance_attrs:
             - attrs:
                 - name: ipaddr
                   value: zapc.example.com
                 - name: pcmk_host_map
                   value: z1.example.com:1;z2.example.com:2
                 - name: login
                   value: apc
                 - name: passwd
                   value: apc
         - id: my_lvm
           agent: ocf:heartbeat:LVM-activate
           instance_attrs:
             - attrs:
                 - name: vgname
                   value: my_vg
                 - name: vg_access_mode
                   value: system_id
         - id: my_fs
           agent: Filesystem
           instance_attrs:
             - attrs:
                 - name: device
                   value: /dev/my_vg/my_lv
                 - name: directory
                   value: /var/www
                 - name: fstype
                   value: xfs
         - id: VirtualIP
           agent: IPaddr2
           instance_attrs:
             - attrs:
                 - name: ip
                   value: 198.51.100.3
                 - name: cidr_netmask
                   value: 24
         - id: Website
           agent: apache
           instance_attrs:
             - attrs:
                 - name: configfile
                   value: /etc/httpd/conf/httpd.conf
                 - name: statusurl
                   value: http://127.0.0.1/server-status
       ha_cluster_resource_groups:
         - id: apachegroup
           resource_ids:
             - my_lvm
             - my_fs
             - VirtualIP
             - Website

   Copy to Clipboard Toggle word wrap
3. Save the file.

4. Run the playbook, specifying the path to the inventory file inventory you created in Step 1.

   # ansible-playbook -i inventory http-cluster.yml

   Copy to Clipboard Toggle word wrap

5. When you use the apache resource agent to manage Apache, it does not use systemd. Because of this, you must edit the logrotate script supplied with Apache so that it does not use systemctl to reload Apache.

   Remove the following line in the /etc/logrotate.d/httpd file on each node in the cluster.

   /bin/systemctl reload httpd.service > /dev/null 2>/dev/null || true

   Copy to Clipboard Toggle word wrap

   • Replace the line you removed with the following three lines, specifying /var/run/httpd-website.pid as the PID file path where website is the name of the Apache resource. In this example, the Apache resource name is Website.

     /usr/bin/test -f /var/run/httpd-Website.pid >/dev/null 2>/dev/null &&
     /usr/bin/ps -q $(/usr/bin/cat /var/run/httpd-Website.pid) >/dev/null 2>/dev/null &&
     /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf -c "PidFile /var/run/httpd-Website.pid" -k graceful > /dev/null 2>/dev/null || true

     Copy to Clipboard Toggle word wrap

Verification steps

1. From one of the nodes in the cluster, check the status of the cluster. Note that all four resources are running on the same node, z1.example.com.

   If you find that the resources you configured are not running, you can run the pcs resource debug-start resource command to test the resource configuration.

   [root@z1 ~]# pcs status
   Cluster name: my_cluster
   Last updated: Wed Jul 31 16:38:51 2013
   Last change: Wed Jul 31 16:42:14 2013 via crm_attribute on z1.example.com
   Stack: corosync
   Current DC: z2.example.com (2) - partition with quorum
   Version: 1.1.10-5.el7-9abe687
   2 Nodes configured
   6 Resources configured
   
   Online: [ z1.example.com z2.example.com ]
   
   Full list of resources:
    myapc  (stonith:fence_apc_snmp):       Started z1.example.com
    Resource Group: apachegroup
        my_lvm     (ocf::heartbeat:LVM-activate):   Started z1.example.com
        my_fs      (ocf::heartbeat:Filesystem):    Started z1.example.com
        VirtualIP  (ocf::heartbeat:IPaddr2):       Started z1.example.com
        Website    (ocf::heartbeat:apache):        Started z1.example.com

   Copy to Clipboard Toggle word wrap

2. Once the cluster is up and running, you can point a browser to the IP address you defined as the IPaddr2 resource to view the sample display, consisting of the simple word "Hello".

   Hello

   Copy to Clipboard Toggle word wrap

3. To test whether the resource group running on z1.example.com fails over to node z2.example.com, put node z1.example.com in standby mode, after which the node will no longer be able to host resources.

   [root@z1 ~]# pcs node standby z1.example.com

   Copy to Clipboard Toggle word wrap

4. After putting node z1 in standby mode, check the cluster status from one of the nodes in the cluster. Note that the resources should now all be running on z2.

   [root@z1 ~]# pcs status
   Cluster name: my_cluster
   Last updated: Wed Jul 31 17:16:17 2013
   Last change: Wed Jul 31 17:18:34 2013 via crm_attribute on z1.example.com
   Stack: corosync
   Current DC: z2.example.com (2) - partition with quorum
   Version: 1.1.10-5.el7-9abe687
   2 Nodes configured
   6 Resources configured
   
   Node z1.example.com (1): standby
   Online: [ z2.example.com ]
   
   Full list of resources:
   
    myapc  (stonith:fence_apc_snmp):       Started z1.example.com
    Resource Group: apachegroup
        my_lvm     (ocf::heartbeat:LVM-activate):   Started z2.example.com
        my_fs      (ocf::heartbeat:Filesystem):    Started z2.example.com
        VirtualIP  (ocf::heartbeat:IPaddr2):       Started z2.example.com
        Website    (ocf::heartbeat:apache):        Started z2.example.com

   Copy to Clipboard Toggle word wrap

   The web site at the defined IP address should still display, without interruption.

5. To remove z1 from standby mode, enter the following command.

   [root@z1 ~]# pcs node unstandby z1.example.com

   Copy to Clipboard Toggle word wrap
   Note

   Removing a node from standby mode does not in itself cause the resources to fail back over to that node. This will depend on the resource-stickiness value for the resources. For information about the resource-stickiness meta attribute, see Configuring a resource to prefer its current node.