Chapter 5. Configuring the Pacemaker cluster

Procedure

1. Create the directory /etc/systemd/system/pacemaker.service.d/ for the pacemaker service drop-in file:

   [root]# mkdir /etc/systemd/system/pacemaker.service.d/

2. Create the systemd drop-in file for the pacemaker service with the following content:

   [root]# cat << EOF > /etc/systemd/system/pacemaker.service.d/00-pacemaker.conf
   [Unit]
   Description=Pacemaker needs the SAP HANA instance service
   Wants=SAP<SID>_<instance>.service
   After=SAP<SID>_<instance>.service
   EOF

   • Replace <SID> with your HANA SID.
   • Replace <instance> with your HANA instance number.

3. Reload the systemctl daemon to activate the drop-in file:

   [root]# systemctl daemon-reload

4. Repeat steps 1-3 on the other HANA cluster nodes.

Verification

1. Check the systemd service of your HANA instance and verify that it is loaded:

   [root]# systemctl status SAPRH1_02.service
   ● SAPRH1_02.service - SAP Instance SAPRH1_02
        Loaded: loaded (/etc/systemd/system/SAPRH1_02.service; disabled; preset: disabled)
        Active: active (running) since xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
      Main PID: 5825 (sapstartsrv)
         Tasks: 841
        Memory: 88.6G
           CPU: 4h 50min 2.033s
        CGroup: /SAP.slice/SAPRH1_02.service
                ├─ 5825 /usr/sap/RH1/HDB02/exe/sapstartsrv pf=/usr/sap/RH1/SYS/profile/RH1_HDB02_dc1hana1
                ├─ 5986 sapstart pf=/usr/sap/RH1/SYS/profile/RH1_HDB02_dc1hana1
                ├─ 5993 /usr/sap/RH1/HDB02/dc1hana1/trace/hdb.sapRH1_HDB02 -d -nw -f /usr/sap/RH1/HDB02/dc1hana1/daemon.ini pf=/usr/sap/RH1/SYS/profile/RH1_HDB02_dc1hana1
   ...

2. Verify that the SAP HANA instance service is known to the pacemaker service now:

   [root]# systemctl show pacemaker.service | grep -E 'Wants=|After=|SAP.{6}.service'
   Wants=SAPRH1_02.service resource-agents-deps.target dbus-broker.service
   After=... SAPRH1_02.service …

   Make sure that the SAP<SID>_<instance>.service is listed in the After= and Wants= lists.

Procedure

1. Stop the HANA instances on all nodes. Run this as the <sid>adm user on one HANA instance per site:

   rh1adm $ sapcontrol -nr ${TINSTANCE} -function StopSystem HDB

2. Verify as <sid>adm on all sites that the HANA instances are stopped completely and their status is GRAY in the instance list. Run this on one host on each site:

   rh1adm $ sapcontrol -nr ${TINSTANCE} -function GetSystemInstanceList
   ...
   hostname, instanceNr, httpPort, httpsPort, startPriority, features, dispstatus
   dc1hana2, 0, 50013, 50014, 0.3, HDB|HDB_WORKER, GRAY
   dc1hana1, 0, 50013, 50014, 0.3, HDB|HDB_WORKER, GRAY

3. Change to the HANA configuration directory, as the <sid>adm user, using the command alias cdcoc, which is built into the <sid>adm user shell. This automatically changes into the /hana/shared/<SID>/global/hdb/custom/config/ path:

   rh1adm $ cdcoc

4. Update the global.ini file of the SAP HANA site to configure the HanaSR hook. Edit the configuration file on one node of each HANA site and add the following configuration:

   [ha_dr_provider_hanasr]
   provider = HanaSR
   path = /usr/share/sap-hana-ha/
   execution_order = 1
   
   [trace]
   ha_dr_hanasr = info

   Set execution_order to 1 to ensure that the HanaSR hook is always executed with the highest priority.

   Due to the shared /hana/shared filesystem between the nodes of each HANA site, you only adjust the configuration once per site. Do not try to edit the same file on the shared filesystem simultaneously on more than one node of the same site.

5. Optional: When you also want to configure the optional ChkSrv hook for taking action on an hdbindexserver failure, you can add the changes to the global.ini at the same time, see step 1 in Configuring the ChkSrv HA/DR provider for the srServiceStateChanged() hook method:

   [ha_dr_provider_hanasr]
   provider = HanaSR
   path = /usr/share/sap-hana-ha/
   execution_order = 1
   
   [ha_dr_provider_chksrv]
   provider = ChkSrv
   path = /usr/share/sap-hana-ha/
   execution_order = 2
   action_on_lost = stop
   
   [trace]
   ha_dr_hanasr = info
   ha_dr_chksrv = info

6. Create the file /etc/sudoers.d/20-saphana, as the root user, on each cluster node with the following content. These command privileges allow the <sid>adm user to update certain cluster node attributes as part of the HanaSR hook execution:

   [root]# visudo -f /etc/sudoers.d/20-saphana
   Defaults:<sid>adm !requiretty
   <sid>adm ALL=(ALL) NOPASSWD: /usr/sbin/crm_attribute -n hana_*

   • For further information on why the Defaults setting is needed, refer to The srHook attribute is set to SFAIL in a Pacemaker cluster managing SAP HANA system replication, even though replication is in a healthy state.

7. Start the HANA instances on all cluster nodes manually without starting the HA cluster. Run as <sid>adm on one HANA instance per site:

   rh1adm $ sapcontrol -nr ${TINSTANCE} -function StartSystem HDB

Verification

1. Change to the SAP HANA directory, as the <sid>adm user, where trace log files are stored. Use the command alias cdtrace, which is built into the <sid>adm user shell:

   rh1adm $ cdtrace

2. Check the HANA nameserver process logs for the HA/DR provider loading message:

   1. If only the HanaSR provider is configured:

      rh1adm $ grep -he "loading HA/DR Provider.*" nameserver_*
      loading HA/DR Provider 'HanaSR' from /usr/share/sap-hana-ha/

   2. If the optional ChkSrv provider is also implemented:

      rh1adm $ grep -he "loading HA/DR Provider.*" nameserver_*
      loading HA/DR Provider 'ChkSrv' from /usr/share/sap-hana-ha/
      loading HA/DR Provider 'HanaSR' from /usr/share/sap-hana-ha/

3. Verify as user root in the system secure log that the sudo command executed successfully. If the sudoers file is not correct, an error is logged when the sudo command is executed. Check this on the primary node on which the HANA master nameserver is running, for example, dc1hana1:

   [root]# grep -e 'sudo.*crm_attribute.*' /var/log/secure
    sudo[4267]:  rh1adm : PWD=/hana/shared/RH1/HDB02/dc1hana1 ; USER=root ; COMMAND=/usr/sbin/crm_attribute -n hana_rh1_site_srHook_DC2 -v SFAIL -t crm_config -s SAPHanaSR
    ...
    sudo[4319]:  rh1adm : PWD=/hana/shared/RH1/HDB02/dc1hana1 ; USER=root ; COMMAND=/usr/sbin/crm_attribute -n hana_rh1_site_srHook_DC2 -v SOK -t crm_config -s SAPHanaSR

   After the HANA instance starts on both nodes, you can usually see several srHook attribute updates. At first it is setting SFAIL, because immediately after the primary starts, it is not yet in sync with the secondary, which is still synchronizing at this time.

   The last update to SOK is triggered by the HANA event after the system replication status finally changes to fully in sync.

4. Repeat the verification steps 1-2 on the second site, if not already done at the same time. The sudo log messages of step 3 are only visible on the primary instance master nameserver node, on which the system replication events are logged.

5. Check the cluster attributes on any node and verify that the value for the hana_<sid>_site_srHook_<site2> attribute is updated as expected:

   [root]# cibadmin --query | grep -e 'SAPHanaSR.*srHook'
           <nvpair id="SAPHanaSR-hana_rh1_site_srHook_DC2" name="hana_rh1_site_srHook_DC2" value="SOK"/>

   • SOK is set when the HANA system replication is in ACTIVE state, which means established and fully in sync.
   • SFAIL is set when the system replication is in any other state.

Procedure

1. Change to the HANA configuration directory as the <sid>adm user. Use the command alias cdcoc, which is built into the <sid>adm user shell. This automatically changes into the /hana/shared/<SID>/global/hdb/custom/config/ path:

   rh1adm $ cdcoc

2. Update the global.ini file of the HANA site to configure the hook script. Edit the configuration file one node of each HANA site and add the following content in addition to the HanaSR provider definition:

   [ha_dr_provider_chksrv]
   provider = ChkSrv
   path = /usr/share/sap-hana-ha/
   execution_order = 2
   action_on_lost = stop
   
   [trace]
   ha_dr_hanasr = info
   ha_dr_chksrv = info

   Due to the shared /hana/shared filesystem between the nodes of each HANA site, you only adjust the configuration once per site. Do not try to edit the same file on the shared filesystem simultaneously on more than one node of the same site.

3. Optional: If you want to use the fence option for action_on_lost, you must add the SAPHanaSR-hookHelper to the sudo configuration of the <sid>adm user:

   [root]# visudo -f /etc/sudoers.d/20-saphana
   …
   <sid>adm ALL=(ALL) NOPASSWD: /usr/bin/SAPHanaSR-hookHelper

4. Optional: Activate the ChkSrv provider while HANA is running by reloading the HA/DR providers. Skip this step when configuring the hook script while the instance is down, the HA/DR provider is loaded automatically at the next instance start.

   rh1adm $ hdbnsutil -reloadHADRProviders

Verification

1. Change to the SAP HANA directory, as the <sid>adm user, where trace log files are stored. Use the command alias cdtrace, which is built into the <sid>adm user shell:

   rh1adm $ cdtrace

2. Check that the changes are loaded:

   rh1adm $ grep -e "loading HA/DR Provider.*ChkSrv.*" nameserver_*
   loading HA/DR Provider 'ChkSrv' from /usr/share/sap-hana-ha/

3. Check that the dedicated trace log file is created and the provider loaded with the correct configuration parameters:

   rh1adm $ cat nameserver_chksrv.trc
   init called
   ChkSrv.init() version 1.001.1, parameter info: action_on_lost=stop stop_timeout=20 kill_signal=9
   ...

Procedure

1. Create the SAPHanaTopology resource as a clone resource, which means it runs on all cluster nodes at the same time:

   [root]# pcs resource create rsc_SAPHanaTop_<SID>_HDB<instance> \
   ocf:heartbeat:SAPHanaTopology \
   SID=<SID> \
   InstanceNumber=<instance> \
   op start timeout=600 \
   op stop timeout=300 \
   op monitor interval=30 timeout=300 \
   clone cln_SAPHanaTop_<SID>_HDB<instance> \
   meta clone-node-max=1 interleave=true --future

   • Replace <SID> with your HANA SID.

   • Replace <instance> with your HANA instance number.

     Note

     Since RHEL 9.3, a deprecation warning is displayed when not using the meta keyword in the clone command and the attributes are automatically assigned to the base resource.

     In the future, the meta keyword for clone attributes will be required for assigning attributes to the clone resource. Until then, add the --future parameter to already apply this behavior now.

     See also New pcs parsing requires meta keyword when specifying clone meta attributes.

2. Create the SAPHanaController resource as a promotable clone resource. This means it runs on all cluster nodes at the same time, but on one node it functions as the active, or primary, resource:

   [root]# pcs resource create rsc_SAPHanaCon_<SID>_HDB<instance> \
   ocf:heartbeat:SAPHanaController \
   SID=<SID> \
   InstanceNumber=<instance> \
   PREFER_SITE_TAKEOVER=true \
   DUPLICATE_PRIMARY_TIMEOUT=7200 \
   AUTOMATED_REGISTER=false \
   op stop timeout=3600 \
   op monitor interval=59 role=Promoted timeout=700 \
   op monitor interval=61 role=Unpromoted timeout=700 \
   meta priority=100 \
   promotable cln_SAPHanaCon_<SID>_HDB<instance> \
   meta clone-node-max=1 interleave=true --future

   We recommend that you create the resource with AUTOMATED_REGISTER=false and then verify the correct behavior and data consistency through tests to complete the setup. For more information, see Testing the setup. You can enable this already at creation time by setting the parameter to true.

   See SAPHanaController resource parameters for more details.

3. Optional: If you want the cluster to fence the node when the SAPHanaController resource fails, then update the resource with the ON_FAIL_ACTION parameter and set it to fence:

   [root]# pcs resource update rsc_SAPHanaCon_<SID>_HDB<instance> ON_FAIL_ACTION=fence

   In the case that you want to enable the fencing action on failure of the SAPHanaController resource, you must also configure a sudo entry for the helper script, which the resource agent calls to trigger the fencing:

   [root]# visudo -f /etc/sudoers.d/20-saphana
   …
   <sid>adm ALL=(ALL) NOPASSWD: /usr/bin/SAPHanaSR-hookHelper

4. You must start the SAPHanaTopology resource before the SAPHanaController resource, because it collects HANA landscape information, which the SAPHanaController resource requires to start correctly. Create the cluster constraint that enforces the correct start order of the two resources:

   [root]# pcs constraint order cln_SAPHanaTop_<SID>_HDB<instance> \
   then cln_SAPHanaCon_<SID>_HDB<instance> symmetrical=false

   Setting symmetrical=false indicates that the constraint only influences the start order of the resources, but it does not apply to the stop order.

5. Optional: Create the SAPHanaFilesystem resource as a clone resource:

   [root]# pcs resource create rsc_SAPHanaFil_<SID>_HDB<instance> \
   ocf:heartbeat:SAPHanaFilesystem \
   SID=<SID> \
   InstanceNumber=<instance> \
   ON_FAIL_ACTION="fence" \
   op start interval=0 timeout=10 \
   op stop interval=0 timeout=20 \
   op monitor interval=120 timeout=120 \
   clone cln_SAPHanaFil_<SID>_HDB<instance> \
   meta clone-node-max=1 interleave=true --future

   The SAPHanaFilesystem resource agent monitors the filesystem for the /hana/shared/<SID>/ path. If you want to change the path or configure additional SAPHanaFilesystem resources to monitor other paths you can use the DIRECTORY parameter to define the path.

   Instead of setting ON_FAIL_ACTION=fence you can set it to ignore. This can be useful for testing the functionality first. The resource writes information to the system logs, which you can examine for evaluating if the resource takes the wanted action when activated to use the fence action.

Verification

1. Review the SAPHanaTopology resource clone. For operations that you do not define at resource creation, pcs automatically apply default values. Example resource configuration:

   [root]# pcs resource config cln_SAPHanaTop_RH1_HDB02
   Clone: cln_SAPHanaTop_RH1_HDB02
     Meta Attributes: cln_SAPHanaTop_RH1_HDB02-meta_attributes
       clone-node-max=1
       interleave=true
     Resource: rsc_SAPHanaTop_RH1_HDB02 (class=ocf provider=heartbeat type=SAPHanaTopology)
       Attributes: rsc_SAPHanaTop_RH1_HDB02-instance_attributes
         InstanceNumber=02
         SID=RH1
       Operations:
         methods: rsc_SAPHanaTop_RH1_HDB02-methods-interval-0s
           interval=0s timeout=5
         monitor: rsc_SAPHanaTop_RH1_HDB02-monitor-interval-30
           interval=30 timeout=300
         reload: rsc_SAPHanaTop_RH1_HDB02-reload-interval-0s
           interval=0s timeout=5
         start: rsc_SAPHanaTop_RH1_HDB02-start-interval-0s
           interval=0s timeout=600
         stop: rsc_SAPHanaTop_RH1_HDB02-stop-interval-0s
           interval=0s timeout=300

2. Review the SAPHanaController resource clone. Example resource configuration:

   [root]# pcs resource config cln_SAPHanaCon_RH1_HDB02
   Clone: cln_SAPHanaCon_RH1_HDB02
     Meta Attributes: cln_SAPHanaCon_RH1_HDB02-meta_attributes
       clone-node-max=1
       interleave=true
       promotable=true
     Resource: rsc_SAPHanaCon_RH1_HDB02 (class=ocf provider=heartbeat type=SAPHanaController)
       Attributes: rsc_SAPHanaCon_RH1_HDB02-instance_attributes
         AUTOMATED_REGISTER=false
         DUPLICATE_PRIMARY_TIMEOUT=7200
         InstanceNumber=02
         PREFER_SITE_TAKEOVER=true
         SID=RH1
       Meta Attributes: rsc_SAPHanaCon_RH1_HDB02-meta_attributes
         priority=100
       Operations:
         demote: rsc_SAPHanaCon_RH1_HDB02-demote-interval-0s
           interval=0s timeout=320
         methods: rsc_SAPHanaCon_RH1_HDB02-methods-interval-0s
           interval=0s timeout=5
         monitor: rsc_SAPHanaCon_RH1_HDB02-monitor-interval-59
           interval=59 timeout=700 role=Promoted
         monitor: rsc_SAPHanaCon_RH1_HDB02-monitor-interval-61
           interval=61 timeout=700 role=Unpromoted
         promote: rsc_SAPHanaCon_RH1_HDB02-promote-interval-0s
           interval=0s timeout=900
         reload: rsc_SAPHanaCon_RH1_HDB02-reload-interval-0s
           interval=0s timeout=5
         start: rsc_SAPHanaCon_RH1_HDB02-start-interval-0s
           interval=0s timeout=3600
         stop: rsc_SAPHanaCon_RH1_HDB02-stop-interval-0s
           interval=0s timeout=3600

3. Check that the start order constraint is in place:

   [root]# pcs constraint order
   Order Constraints:
     start resource 'cln_SAPHanaTop_RH1_HDB02' then start resource 'cln_SAPHanaCon_RH1_HDB02'
       symmetrical=0

4. Optional: Review the SAPHanaFilesystem resource clone if you created it. Example resource configuration:

   [root]# pcs resource config cln_SAPHanaFil_RH1_HDB02
   Clone: cln_SAPHanaFil_RH1_HDB02
     Meta Attributes: cln_SAPHanaFil_RH1_HDB02-meta_attributes
       clone-node-max=1
       interleave=true
     Resource: rsc_SAPHanaFil_RH1_HDB02 (class=ocf provider=heartbeat type=SAPHanaFilesystem)
       Attributes: rsc_SAPHanaFil_RH1_HDB02-instance_attributes
         InstanceNumber=02
         ON_FAIL_ACTION=fence
         SID=RH1
       Operations:
         methods: rsc_SAPHanaFil_RH1_HDB02-methods-interval-0s
           interval=0s timeout=5
         monitor: rsc_SAPHanaFil_RH1_HDB02-monitor-interval-120
           interval=120 timeout=120
         reload: rsc_SAPHanaFil_RH1_HDB02-reload-interval-0s
           interval=0s timeout=5
         start: rsc_SAPHanaFil_RH1_HDB02-start-interval-0
           interval=0 timeout=10
         stop: rsc_SAPHanaFil_RH1_HDB02-stop-interval-0
           interval=0 timeout=20

5. Check the cluster status. Use --full to include node attributes, which are updated by the HANA resources:

   [root]# pcs status --full
   ...
   
   Full List of Resources:
     * Clone Set: cln_SAPHanaTop_RH1_HDB02 [rsc_SAPHanaTop_RH1_HDB02]:
       * rsc_SAPHanaTop_RH1_HDB02  (ocf:heartbeat:SAPHanaTopology):         Started dc1hana2
       * rsc_SAPHanaTop_RH1_HDB02  (ocf:heartbeat:SAPHanaTopology):         Started dc2hana1
       * rsc_SAPHanaTop_RH1_HDB02  (ocf:heartbeat:SAPHanaTopology):         Started dc2hana2
       * rsc_SAPHanaTop_RH1_HDB02  (ocf:heartbeat:SAPHanaTopology):         Started dc1hana1
     * Clone Set: cln_SAPHanaCon_RH1_HDB02 [rsc_SAPHanaCon_RH1_HDB02] (promotable):
       * rsc_SAPHanaCon_RH1_HDB02  (ocf:heartbeat:SAPHanaController):       Unpromoted dc1hana2
       * rsc_SAPHanaCon_RH1_HDB02  (ocf:heartbeat:SAPHanaController):       Unpromoted dc2hana1
       * rsc_SAPHanaCon_RH1_HDB02  (ocf:heartbeat:SAPHanaController):       Unpromoted dc2hana2
       * rsc_SAPHanaCon_RH1_HDB02  (ocf:heartbeat:SAPHanaController):       Promoted dc1hana1
     * Clone Set: cln_SAPHanaFil_RH1_HDB02 [rsc_SAPHanaFil_RH1_HDB02]:
       * rsc_SAPHanaFil_RH1_HDB02  (ocf:heartbeat:SAPHanaFilesystem):       Started dc1hana2
       * rsc_SAPHanaFil_RH1_HDB02  (ocf:heartbeat:SAPHanaFilesystem):       Started dc2hana1
       * rsc_SAPHanaFil_RH1_HDB02  (ocf:heartbeat:SAPHanaFilesystem):       Started dc2hana2
       * rsc_SAPHanaFil_RH1_HDB02  (ocf:heartbeat:SAPHanaFilesystem):       Started dc1hana1
   
   Node Attributes:
     * Node: dc1hana1 (1):
       * hana_rh1_clone_state              : PROMOTED
       * hana_rh1_roles                    : master1:master:worker:master
       * hana_rh1_site                     : DC1
       * hana_rh1_srah                     : -
       * hana_rh1_version                  : 2.00.083.00
       * hana_rh1_vhost                    : dc1hana1
       * master-rsc_SAPHanaCon_RH1_HDB02   : 150
     * Node: dc1hana2 (2):
       * hana_rh1_clone_state              : DEMOTED
       * hana_rh1_roles                    : slave:slave:worker:slave
       * hana_rh1_site                     : DC1
       * hana_rh1_srah                     : -
       * hana_rh1_version                  : 2.00.083.00
       * hana_rh1_vhost                    : dc1hana2
       * master-rsc_SAPHanaCon_RH1_HDB02   : -10000
     * Node: dc2hana1 (3):
       * hana_rh1_clone_state              : DEMOTED
       * hana_rh1_roles                    : master1:master:worker:master
       * hana_rh1_site                     : DC2
       * hana_rh1_srah                     : -
       * hana_rh1_version                  : 2.00.083.00
       * hana_rh1_vhost                    : dc2hana1
       * master-rsc_SAPHanaCon_RH1_HDB02   : 100
     * Node: dc2hana2 (4):
       * hana_rh1_clone_state              : DEMOTED
       * hana_rh1_roles                    : slave:slave:worker:slave
       * hana_rh1_site                     : DC2
       * hana_rh1_srah                     : -
       * hana_rh1_version                  : 2.00.083.00
       * hana_rh1_vhost                    : dc2hana2
       * master-rsc_SAPHanaCon_RH1_HDB02   : -12200
   
   ...

Procedure

1. Use the appropriate resource agent for managing the virtual IP address based on the platform on which the HA cluster is running. Adjust the parameters according to the resource agent you are using. Create the cluster resource for the primary virtual IP, for example, using the IPaddr2 agent:

   [root]# pcs resource create rsc_vip_<SID>_HDB<instance>_primary \
   ocf:heartbeat:IPaddr2 ip=<address> cidr_netmask=<netmask> nic=<device>

   • Replace <SID> with your HANA SID.
   • Replace <instance> with your HANA instance number.
   • Replace <address>, <netmask> and <device> with the details of your primary virtual IP address.

2. Create a cluster constraint that places the VIP resource with the SAPHanaController resource on the HANA primary node:

   [root]# pcs constraint colocation add rsc_vip_<SID>_HDB<instance>_primary \
   with promoted cln_SAPHanaCon_<SID>_HDB<instance> 2000

   The constraint applies a score of 2000 instead of the default INFINITY. This softens the resource dependency and allows the virtual IP resource to stay active in the case when there is no promoted SAPHanaController resource. This way it is still possible to use tools like the SAP Management Console (MMC) or SAP Landscape Management (LaMa) that can reach this IP address to query the status information of the HANA instance.

Verification

1. Check the resource configuration of the virtual IP resource, for example::

   [root]# pcs resource config rsc_vip_RH1_HDB02_primary
   Resource: rsc_vip_RH1_HDB02_primary (class=ocf provider=heartbeat type=IPaddr2)
     Attributes: rsc_vip_RH1_HDB02_primary-instance_attributes
       cidr_netmask=32
       ip=192.168.1.100
       nic=eth0
     Operations:
       monitor: rsc_vip_RH1_HDB02_primary-monitor-interval-10s
         interval=10s timeout=20s
       start: rsc_vip_RH1_HDB02_primary-start-interval-0s
         interval=0s timeout=20s
       stop: rsc_vip_RH1_HDB02_primary-stop-interval-0s
         interval=0s timeout=20s

2. Check that the constraint is defined correctly:

   [root]# pcs constraint colocation
   Colocation Constraints:
     Started resource 'rsc_vip_RH1_HDB02_primary' with Promoted resource 'cln_SAPHanaCon_RH1_HDB02'
       score=2000

3. Check that the resource is running on the promoted primary node, for example, dc1hana1:

   [root]# pcs status resources rsc_vip_RH1_HDB02_primary
     * rsc_vip_RH1_HDB02_primary   (ocf:heartbeat:IPaddr2):         Started dc1hana1

Procedure

1. Use the appropriate resource agent for managing the virtual IP address based on the platform on which the HA cluster is running. Adjust the parameters according to the resource agent you are using. Create the cluster resource for the secondary virtual IP, for example, using the IPaddr2 agent:

   [root]# pcs resource create rsc_vip_<SID>_HDB<site>_readonly \
   ocf:heartbeat:IPaddr2 ip=<address> cidr_netmask=<netmask> nic=<device>

   • Replace <SID> with your HANA SID.
   • Replace <site> with your HANA site number.
   • Replace <address>, <netmask> and <device> with the details of your read-only secondary virtual IP address.

2. Create a location constraint rule to ensure that the secondary virtual IP is assigned to the secondary site during normal operations:

   [root]# pcs constraint location rsc_vip_<SID>_HDB<site>_readonly \
   rule score=INFINITY master-rsc_SAPHanaCon_<SID>_HDB<site> eq 100 \
   and hana_<sid>_clone_state eq DEMOTED

   • Replace <SID> with your HANA SID.
   • Replace <sid> with the lower-case HANA SID.
   • Replace <site> with your HANA site number.

3. Create a location constraint rule to ensure that the secondary virtual IP runs on the primary site as an alternative whenever necessary:

   [root]# pcs constraint location rsc_vip_<SID>_HDB<site>_readonly \
   rule score=2000 master-rsc_SAPHanaCon_<SID>_HDB<site> eq 150 \
   and hana_<sid>_clone_state eq PROMOTED

Verification

1. Check the resource configuration of the secondary virtual IP resource, for example:

   [root]# pcs resource config rsc_vip_RH1_HDB02_readonly
   Resource: rsc_vip_RH1_HDB02_readonly (class=ocf provider=heartbeat type=IPaddr2)
     Attributes: rsc_vip_RH1_HDB02_readonly-instance_attributes
       cidr_netmask=32
       ip=192.168.1.200
       nic=eth0
     Operations:
       monitor: rsc_vip_RH1_HDB02_readonly-monitor-interval-10s
         interval=10s timeout=20s
       start: rsc_vip_RH1_HDB02_readonly-start-interval-0s
         interval=0s timeout=20s
       stop: rsc_vip_RH1_HDB02_readonly-stop-interval-0s
         interval=0s timeout=20s

2. Check that the constraints are part of the cluster configuration:

   [root]#  pcs constraint location
   Location Constraints:
     resource 'rsc_vip_RH1_HDB02_readonly'
       Rules:
         Rule: boolean-op=and score=INFINITY
           Expression: master-rsc_SAPHanaCon_RH1_HDB02 eq 100
           Expression: hana_rh1_clone_state eq DEMOTED
     resource 'rsc_vip_RH1_HDB02_readonly'
       Rules:
         Rule: boolean-op=and score=2000
           Expression: master-rsc_SAPHanaCon_RH1_HDB02 eq 150
           Expression: hana_rh1_clone_state eq PROMOTED

3. Check that the resource is running on the main secondary node, for example, dc2hana1:

   [root]# pcs status resources rsc_vip_RH1_HDB02_readonly
     * rsc_vip_RH1_HDB02_readonly  (ocf:heartbeat:IPaddr2):         Started dc2hana1

1. Create the alert in the Pacemaker cluster:

   [root]# pcs alert create id=alert-hana-1 \
   path=/usr/bin/SAPHanaSR-alert-fencing \
   options alert_uptime_threshold=300

   • id is the name of the alert, for example, alert-hana-1.
   • alert_uptime_threshold is an alert option that defines the uptime in seconds after a system reboot, during which the cluster does not trigger the alert. This prevents potential fencing loops. The default is 300 seconds.

2. Add a filter to the alert configuration, which limits the cluster to only check cluster node fencing events for alert trigger activity. This prevents processing for unrelated events and reduces logging overhead. Use the cibadmin command to add the alert filter in XML format:

   [root]# cibadmin --modify --scope alerts \
   --xml-text '
   <alert id="alert-hana-1">
    <select>
     <select_nodes />
     <select_fencing />
    </select>
   </alert>'

   • alert id must be the alert name that you used in the previous step, for example, alert-hana-1.
   • Ensure that you surround the xml tags with single quotes '…​'.

1. Check that the alert is defined:

   [root]# pcs alert
   Alerts:
    Alert: alert-hana-1 (path=/usr/bin/SAPHanaSR-alert-fencing)
     Options: alert_uptime_threshold=300

2. Check that the alert filters are set in the cluster configuration and the XML output looks similar to this example between the <select> tags:

   [root]# pcs cluster cib scope=alerts
   <alerts>
     <alert id="alert-hana-1" path="/usr/bin/SAPHanaSR-alert-fencing">
       <instance_attributes id="alert-hana-1-instance_attributes">
         <nvpair id="alert-hana-1-instance_attributes-alert_uptime_threshold" name="alert_uptime_threshold" value="300"/>
       </instance_attributes>
       <select>
         <select_nodes/>
         <select_fencing/>
       </select>
     </alert>
   </alerts>