Chapter 4. Installation

You must check that the other databases are running and the system replication is working properly. Refer to:

You can discover the primary HANA instance with:

az1n1:rh2adm> hdbnsutil -sr_state | egrep -e "primary masters|^mode"
mode: primary

Before you are able to register a new secondary HANA instance, the database keys of the primary HANA instance need to be copied to the new additional HANA replication site. In our example, the hostname of the third site is az3n1.

For example, on the primary node az1n1, run:

az1n1:rh2adm> scp -rp
/usr/sap/${SAPSYSTEMNAME}/SYS/global/security/rsecssfs/data/SSFS_${SAPSYSTEMNAME}.DAT  az3n1:/usr/sap/${SAPSYSTEMNAME}/SYS/global/security/rsecssfs/data/SSFS_${SAPSYSTEMNAME}.DAT
az1n1:rh2adm> scp -rp
/usr/sap/${SAPSYSTEMNAME}/SYS/global/security/rsecssfs/key/SSFS_${SAPSYSTEMNAME}.KEY az3n1:/usr/sap/${SAPSYSTEMNAME}/SYS/global/security/rsecssfs/key/SSFS_${SAPSYSTEMNAME}.KEY

You need to know the name of the node that runs the primary database. For more details refer to Discovering primary database.
To monitor the registration, you can run the following command in a separate terminal on the primary node:

az1n1:rh2adm> watch python
/usr/sap/${SAPSYSTEMNAME}/HDB${TINSTANCE}/python_support/systemReplicationStatus.py

This shows you the progress and any errors if they occur.

To register the HANA instance on the third site (DC3) as an additional secondary SAP HANA instance, run the following command on the third site host az3n1:

az3n1:rh2adm> hdbnsutil -sr_register --name=DC3
--remoteHost=az1n1 --remoteInstance=${TINSTANCE}
--replicationMode=async --operationMode=logreplay --online

In this example, DC3 is the name of the third site, az1n1 is the name of the primary node.

If the database instance is already running, you don’t have to stop it, you can use the option --online, which registers the instance while it is online. The hdbnsutil itself initiates the necessary restart (stop and start) of the instance.

If the HANA instance is offline, you have to start it after the third node is registered. You can find additional information in Verifying installation.

We are using a SAP HANA system replication option called register_secondaries_on_takeover = true. This automatically re-registers with the new primary site in case of a failover between the previous primary site and the other secondary site. This option must be added to the global.ini file on all potential primary sites, which are all database nodes. As the global.ini is stored in /hana/shared you have to edit the global.ini file only once per site.

All HANA instances should have this entry in their global.ini:

[system_replication]
register_secondaries_on_takeover = true

The following two chapters describe the global.ini configuration in detail.

Add the option register_secondaries_on_takeover = true to the [system_replication] section in global.ini of the SAP HANA nodes of site 1 and site 2. The pacemaker cluster manages these. Edit the file global.ini always on the respective node, and do not copy the file from another node.

Edit the global.ini as the rh2adm user:

az1n1:rh2adm> vi
/usr/sap/${SAPSYSTEMNAME}/SYS/global/hdb/custom/config/global.ini

Example:

# global.ini last modified 2023-07-14 16:31:14.120444 by hdbnsutil -sr_register --remoteHost=hana07 --remoteInstance=02 --replicationMode=syncmem --operationMode=logreplay --name=DC2
[multidb]
mode = multidb
database_isolation = low
singletenant = yes

[ha_dr_provider_SAPHanaSR]
provider = SAPHanaSR
path = /usr/share/SAPHanaSR/srhook
execution_order = 1

[persistence]
basepath_datavolumes = /hana/data/RH2
basepath_logvolumes = /hana/log/RH2
log_mode = normal
enable_auto_log_backup = true

[system_replication]
register_secondaries_on_takeover = true
timetravel_logreplay_mode = auto
operation_mode = logreplay
mode = primary
actual_mode = syncmem
site_id = 1
site_name = DC2

[system_replication_site_masters]
2 = az1n1:30201

[trace]
ha_dr_saphanasr = info

This option is active as soon as the SAP HANA database instance is started.

Edit the global.ini as a <sid>adm user:

az3n1:rh2adm>% vi /usr/sap/${SAPSYSTEMNAME}/SYS/global/hdb/custom/config/global.ini

On site 3 (az3n1 and az3n2) , the ha_dr_provider_SAPHanaSR section is not used.

Example of global.ini on az3n1:

# global.ini last modified 2023-06-22 17:22:54.154508 by hdbnameserver
[multidb]
mode = multidb
database_isolation = low
singletenant = yes

[persistence]
basepath_datavolumes = /hana/data/RH2
basepath_logvolumes = /hana/log/RH2
log_mode = normal
enable_auto_log_backup = true

[system_replication]
operation_mode = logreplay
register_secondaries_on_takeover = true
reconnect_time_interval = 5
timetravel_logreplay_mode = auto
site_id = 3
mode = syncmem
actual_mode = syncmem
site_name = DC3

[system_replication_site_masters]
2 = az1n1:30201

After the installation, you have to check if all HANA instances are out and running and that HANA system replication is working between them. The easiest way is to check the systemReplicationStatus on the primary database nodes, as described in more detail in Checking the system replication status. Also refer to the Checking database for further information.

For HANA system replication to work correctly, ensure that the “log_mode” parameter is set to “normal”. Refer to Checking the log_mode of the SAP HANA database for more information.

To verify that the setout is working as expected,

run the xref:asmb_test_cases_configuring-hana-scale-out-multitarget-system-replication-disaster-recovery[Test cases] as described in the following chapter.