Chapter 4. Configuring the SAP HANA system replication

Procedure

1. Optional: Create a dedicated directory for the backup in a suitable path, for example:

   [root]# mkdir <path>/<SID>-backup

   Replace <path> with a path on your system, which has enough free space for the initial backup files.

2. Change the owner of the backup path to user <sid>adm if the target directory is not already owned or writable by the HANA user, for example:

   [root]# chown <sid>adm:sapsys <path>/<SID>-backup

3. Change to the <sid>adm user for the remaining steps:

   [root]# su - <sid>adm

4. Create a backup of the SYSTEMDB as the <sid>adm user. Specify the path to the files the backups will be stored in. Ensure that the target filesystem has enough free space left, then create the backup:

   rh1adm $ hdbsql -i ${TINSTANCE} -u system -d SYSTEMDB \
   "BACKUP DATA USING FILE ('<path>/${SAPSYSTEMNAME}-backup/bkp-SYS')"
   Password: <HANA_SYSTEM_PASSWORD>
   0 rows affected (overall time xx.xxx sec; server time xx.xxx sec)

   • $TINSTANCE and $SAPSYSTEMNAME are environment variables that are part of the <sid>adm user shell environment. $TINSTANCE is the instance number and $SAPSYSTEMNAME is the SID. Both are automatically set to the instance values related to the <sid>adm user.
   • Replace <path> with the path where the <sid>adm user has write access and where there is enough free space left.

5. Create a backup of all tenant databases as the <sid>adm user. Specify the path to the files the backups will be stored in. Ensure that the target filesystem has enough free space left. Create the tenant DB backup:

   rh1adm $ hdbsql -i ${TINSTANCE} -u system -d SYSTEMDB \
   "BACKUP DATA FOR ${SAPSYSTEMNAME} USING FILE ('<path>/${SAPSYSTEMNAME}-backup/bkp-${SAPSYSTEMNAME}')"
   Password: <HANA_SYSTEM_PASSWORD>
   0 rows affected (overall time xx.xxx sec; server time xx.xxx sec)

   Replace <path> with the path where the <sid>adm user has write access and where there is enough free space left.

Verification

1. List the resulting backup files. Example when using /hana/log/RH1-backup as the directory to store the initial backup:

   rh1adm $ ls -lh /hana/log/RH1-backup/
   total 7.3G
   -rw-r-----. 1 rh1adm sapsys 156K Sep 15 10:58 bkp-RH1_databackup_0_1
   -rw-r-----. 1 rh1adm sapsys  81M Sep 15 10:58 bkp-RH1_databackup_2_1
   -rw-r-----. 1 rh1adm sapsys 3.6G Sep 15 10:58 bkp-RH1_databackup_3_1
   -rw-r-----. 1 rh1adm sapsys  81M Sep 15 10:58 bkp-RH1_databackup_4_1
   -rw-r-----. 1 rh1adm sapsys 164K Sep 15 10:56 bkp-SYS_databackup_0_1
   -rw-r-----. 1 rh1adm sapsys 3.6G Sep 15 10:57 bkp-SYS_databackup_1_1

2. Use the HANA command hdbbackupcheck to confirm the sanity of each backup file you created:

   rh1adm $ for i in $(ls /hana/log/RH1-backup/*); do hdbbackupcheck $i; done
   ...
   Loaded library 'libhdblivecache'
   Backup '/hana/log/RH1-backup/RH1_databackup_0_1' successfully checked.
   Loaded library 'libhdbcsaccessor'
   ...
   Loaded library 'libhdblivecache'
   Backup '/hana/log/RH1-backup/system_databackup_0_1' successfully checked.
   Loaded library 'libhdbcsaccessor'
   Loaded library 'libhdblivecache'
   Backup '/hana/log/RH1-backup/system_databackup_1_1' successfully checked.

Procedure

1. Enable the system replication on the HANA site that becomes the initial primary. Run the command as <sid>adm on the first, or primary, node:

   rh1adm $ hdbnsutil -sr_enable --name=<site1>
   nameserver is active, proceeding ...
   successfully enabled system as system replication source site
   done.

   • Replace <site1> with your primary HANA site name, for example, DC1.

Procedure

1. Stop the secondary HANA instances. Run as the <sid>adm user on one secondary instance, for example, on dc2hana1:

   rh1adm $ sapcontrol -nr ${TINSTANCE} -function StopSystem HDB

2. Run this and the following steps only on one node on the secondary site, for example, on dc2hana1. Change to the directory on the shared filesystem where HANA stores the keys of the system replication encryption on one node of the secondary site:

   [root]# cd /hana/shared/<SID>/global/security/rsecssfs

3. Copy the HANA system PKI file SSFS_<SID>.KEY from the primary HANA site to the secondary site on one secondary node:

   [root]# rsync -av <node1>:$PWD/key/SSFS_<SID>.KEY key/

   • Replace <node1> with a primary node, for example, dc1hana1.
   • Replace <SID> with your HANA SID, for example, RH1.

4. Copy the PKI file SSFS_<SID>.DAT from the primary HANA site to the secondary site on one secondary node in the same way as the previous step:

   [root]# rsync -av <node1>:$PWD/data/SSFS_<SID>.DAT data/

5. Register the secondary HANA site to the primary site. Run this in the <sid>adm user terminal on a secondary node:

   rh1adm $ hdbnsutil -sr_register --remoteHost=<node1> \
   --remoteInstance=${TINSTANCE} --replicationMode=sync \
   --operationMode=logreplay --name=<site2>
   adding site ...
   collecting information ...
   updating local ini files ...
   done.

   • Replace <node1> with a primary node, for example, dc1hana1.
   • Replace <site2> with your secondary HANA site name, for example, DC2.
   • Choose the values for replicationMode and operationMode according to your requirements for the system replication.
   • $TINSTANCE is an environment variable that is set automatically for user <sid>adm by reading the HANA instance profile. The variable value is the HANA instance number.

6. Start the secondary HANA instances. Run as <sid>adm on one HANA instance on the secondary site:

   rh1adm $ sapcontrol -nr ${TINSTANCE} -function StartSystem HDB

Verification

1. Check that the system replication is running on the secondary site and the mode matches the value you used for the replicationMode parameter in the hdbnsutil -sr_register command. Run as <sid>adm on one node on the secondary site, for example, dc2hana1:

   rh1adm $ hdbnsutil -sr_stateConfiguration
   
   System Replication State
   ~~~~~~~~
   
   mode: sync
   site id: 2
   site name: DC2
   active primary site: 1
   
   primary masters: dc1hana1

2. Change to the Python script directory of the HANA instance as user <sid>adm, on both sites. The easiest way to do this is to use cdpy, which is an alias built into the <sid>adm user shell that SAP HANA populates during the instance installation:

   rh1adm $ cdpy

   In our example this command changes the current directory to /usr/sap/RH1/HDB02/exe/python_support/.

3. Show the current status of the established HANA system replication, on both HANA sites.

   1. On the primary site the system replication status is always displayed with all details:

      rh1adm $ python systemReplicationStatus.py
      |Database |Host     |Port  |Service Name |Volume ID |Site ID |Site Name |Secondary |Secondary |Secondary |Secondary |Secondary     |Replication |Replication |Replication    |Secondary    |
      |         |         |      |             |          |        |          |Host      |Port      |Site ID   |Site Name |Active Status |Mode        |Status      |Status Details |Fully Synced |
      |-------- |-------- |----- |------------ |--------- |------- |--------- |--------- |--------- |--------- |--------- |------------- |----------- |----------- |-------------- |------------ |
      |RH1      |dc1hana2 |30003 |indexserver  |        4 |      1 |DC1       |dc2hana2  |    30003 |        2 |DC2       |YES           |SYNC        |ACTIVE      |               |        True |
      |SYSTEMDB |dc1hana1 |30001 |nameserver   |        1 |      1 |DC1       |dc2hana1  |    30001 |        2 |DC2       |YES           |SYNC        |ACTIVE      |               |        True |
      |RH1      |dc1hana1 |30007 |xsengine     |        2 |      1 |DC1       |dc2hana1  |    30007 |        2 |DC2       |YES           |SYNC        |ACTIVE      |               |        True |
      |RH1      |dc1hana1 |30003 |indexserver  |        3 |      1 |DC1       |dc2hana1  |    30003 |        2 |DC2       |YES           |SYNC        |ACTIVE      |               |        True |
      
      status system replication site "2": ACTIVE
      overall system replication status: ACTIVE
      
      Local System Replication State
      ~~~~~~~~~~
      
      mode: PRIMARY
      site id: 1
      site name: DC1

      • ACTIVE means that the HANA system replication is in a healthy state and fully synced.
      • SYNCING is displayed while the system replication is being updated on the secondary site, for example after a takeover of the secondary site to become the new primary.
      • INITIALIZING is shown after the system replication has first been enabled or a full sync has been triggered.

   2. On the secondary site, the output of systemReplicationStatus.py is less detailed. Check the status on one secondary node:

      rh1adm $ python systemReplicationStatus.py
      this system is either not running or not primary system replication site
      
      Local System Replication State
      ~~~~~~~~~~
      
      mode: SYNC
      site id: 2
      site name: DC2
      active primary site: 1
      primary masters: dc1hana1