Installation Guide

1. Ensure that the network cable is attached. The link indicator light on the network socket should be lit, even if the computer is not switched on.
2. Switch on the computer.
3. A menu screen appears. Press the number key that corresponds to the preferred option.

1. You must install Red Hat Gluster Storage Server 3.0.4 on the target server.

   Warning

   • For layered installation of Red Hat Gluster Storage, ensure to have only the default Red Hat Enterprise Linux server installation, without the Samba or CTDB packages installed from Red Hat Enterprise Linux.
   • The Samba version 3 is being deprecated from Red Hat Gluster Storage 3.0 Update 4. Further updates will not be provided for samba-3.x. It is recommended that you upgrade to Samba-4.x, which is provided in a separate channel or repository, for all updates including the security updates.
   • CTDB version 2.5 is not supported from Red Hat Gluster Storage 3.1 Update 2. To use CTDB in Red Hat Gluster Storage 3.1.2 and later, you must upgrade the system to CTDB 4.x, which is provided in the Samba channel of Red Hat Gluster Storage.
   • Downgrade of Samba from Samba 4.x to Samba 3.x is not supported.
   • Ensure that Samba is upgraded on all the nodes simultaneously, as running different versions of Samba in the same cluster will lead to data corruption.
2. Enable the channel where the Samba packages are available:
   For Red Hat Gluster Storage 3.3 on Red Hat Enterprise Linux 6.x

   1. If you have registered your machine using Red Hat Subscription Manager, enable the repository by running the following command:

      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-6-server-rpms

   2. If you have registered your machine using Satellite server, enable the channel by running the following command:

      # rhn-channel --add --channel rhel-x86_64-server-6-rh-gluster-3-samba

   For Red Hat Gluster Storage 3.3 on Red Hat Enterprise Linux 7.x

   1. If you have registered your machine using Red Hat Subscription Manager, enable the repository by running the following command:

      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-7-server-rpms

   2. If you have registered your machine using Satellite server, enable the channel by running the following command:

      # rhn-channel --add --channel rhel-x86_64-server-7-rh-gluster-3-samba

1. To install the basic Samba packages, execute the following command:

   # yum install samba

2. If you require the smbclient on the server, then execute the following command:

   # yum install samba-client

3. If you require an Active directory setup, then execute the following commands:

   # yum install samba-winbind
   # yum install samba-winbind-clients
   # yum install samba-winbind-krb5-locator

4. Verify if the following packages are installed.

   samba-libs
   samba-winbind-krb5-locator
   samba-winbind-modules
   samba-vfs-glusterfs
   samba-winbind
   samba-client
   samba-common
   samba-winbind-clients
   samba

1. If you have registered your machine using Red Hat Subscription Manager, enable the repository by running the following command:

   # subscription-manager repos --enable=rh-gluster-3-nfs-for-rhel-7-server-rpms

   To add the HA repository, execute the following command:

   # subscription-manager repos --enable=rhel-7-server-rpms --enable=rhel-ha-for-rhel-7-server-rpms

2. If you have registered your machine using Satellite server, enable the channel by running the following command:

   # rhn-channel --add --channel rhel-x86_64-server-7-rh-gluster-3-nfs

   To subscribe to the HA channel, execute the following command:

   # rhn-channel --add --channel rhel-x86_64-server-ha-7

1. While installing Red Hat Storage using an ISO, in the Customizing the Software Selection screen, select RH-Gluster-NFS-Ganesha and click Next.
2. Proceed with the remaining installation steps for installing Red Hat Gluster Storage. For more information on how to install Red Hat Storage using an ISO, see Installing from an ISO Image.

1. The glusterfs-ganesha package can be installed using the following command:

   # yum install glusterfs-ganesha

   NFS-Ganesha is installed along with the above package. nfs-ganesha-gluster and HA packages are also installed.

1. While installing Red Hat Storage using an ISO, in the Customizing the Software Selection screen, select Red Hat Storage Tools Group and click Optional Packages.
2. From the list of packages, select gstatus and click Close.
3. Proceed with the remaining installation steps for installing Red Hat Storage. For more information on how to install Red Hat Storage using an ISO, see Installing from an ISO Image.

1. You can verify the docker service is running with the following command:

   # systemctl status docker
   docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled)
   Active: active (running) since Fri 2017-01-22 12:03:34 CEST; 3h 19min ago
   Docs: http://docs.docker.com
   Main PID: 843 (docker)
   CGroup: /system.slice/docker.service
              └─843 /usr/bin/docker -d --selinux-enabled
   
   

2. Download the Red Hat Gluster Storage Container images from the Red Hat Docker Registry using the following command:

   # docker pull registry.access.redhat.com/rhgs3/rhgs-server-rhel7

3. Verify that you have a Red Hat Gluster Storage image on a Red Hat Enterprise Linux Atomic Host 7.2 by checking the contents of the /etc/redhat-release file. First, on the host system:

   # cat /etc/redhat-release
   Red Hat Enterprise Linux Atomic Host 7.3

4. And next, you can check the Red Hat Gluster Storage Container Image, again from the host system (no need to log into the container image):

   # docker run rhgs3/rhgs-server-rhel7 cat /etc/redhat-storage-release
   Red Hat Gluster Storage Server 3.1 Update 2 (Container)

Updating Red Hat Gluster Storage 3.3 in the offline mode

1. Ensure that you have a working backup, as described in Section 7.1, “Before you update”.
2. Stop all volumes.

   # for vol in `gluster volume list`; do gluster --mode=script volume stop $vol; sleep 2s; done

3. Run the following commands on one server at a time.
   1. Stop all gluster services.
      On Red Hat Enterprise Linux 7:

      # systemctl stop glusterd
      # pkill glusterfs
      # pkill glusterfsd

      On Red Hat Enterprise Linux 6:

      # service glusterd stop
      # pkill glusterfs
      # pkill glusterfsd

   2. If you want to migrate from Gluster NFS to NFS Ganesha as part of this update, perform the following additional steps.
      1. Stop and disable CTDB. This ensures that multiple versions of Samba do not run in the cluster during the update process, and avoids data corruption.

         # systemctl stop ctdb
         # systemctl disable ctdb

      2. Verify that the CTDB and NFS services are stopped:

         ps axf | grep -E '(ctdb|nfs)[d]'

      3. Delete the CTDB volume by executing the following command:

         # gluster vol delete <ctdb_vol_name>

   3. Update the system.

      # yum update

      Review the packages to be updated, and enter y to proceed with the update when prompted.
      Wait for the update to complete.
   4. If updates to the kernel package occurred, or if you are migrating from Gluster NFS to NFS Ganesha as part of this update, reboot the system.
   5. Start glusterd.
      On Red Hat Enterprise Linux 7:

      # systemctl start glusterd

      On Red Hat Enterprise Linux 6:

      # service glusterd start

4. When all servers have been updated, run the following command to update the cluster operating version. This helps to prevent any compatibility issues within the cluster.

   # gluster volume set all cluster.op-version 31102

   Note

   31102 is the cluster.op-version value for the latest Red Hat Gluster Storage 3.3.1 glusterfs Async. Refer to Section 1.5, “Supported Versions of Red Hat Gluster Storage” for the correct cluster.op-version value for other versions.
5. If you want to migrate from Gluster NFS to NFS Ganesha as part of this update, install the NFS-Ganesha packages as described in Chapter 4, Deploying NFS-Ganesha on Red Hat Gluster Storage, and use the information in the NFS Ganesha section of the Red Hat Gluster Storage 3.3 Administration Guide to configure the NFS Ganesha cluster.
6. Start all volumes.

   # for vol in `gluster volume list`; do gluster --mode=script volume start $vol; done

7. If you did not reboot as part of the update process, run the following command to remount the meta volume:

   # mount /var/run/gluster/shared_storage/

   If this command does not work, review the content of /etc/fstab and ensure that the entry for the shared storage is configured correctly and re-run the mount command. The line for the meta volume in the /etc/fstab file should look like the following:

   hostname:/gluster_shared_storage   /var/run/gluster/shared_storage/   glusterfs   defaults   0 0

8. If you use Gluster NFS to access volumes, enable Gluster NFS using the following command:

   # gluster volume set volname nfs.disable off

   For example:

   # gluster volume set testvol nfs.disable off
   volume set: success

9. If you use geo-replication, restart geo-replication sessions when upgrade is complete.

   # gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL start

   You may need to append the force parameter to successfully restart in some circumstances. See BZ#1347625 for details.

Updating Red Hat Gluster Storage 3.3 in in-service mode

1. Ensure that you have a working backup, as described in Section 7.1, “Before you update”.
2. If you have a replicated configuration, perform these steps on all nodes of a replica set.
   If you have a distributed-replicated configuration, perform these steps on one replica set at a time, for all replica sets.
   • Stop any geo-replication sessions.

     # gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL stop

   • If this node is part of an NFS-Ganehsa cluster, place the node in standby mode.

     # pcs cluster standby

   • Verify that there are no pending self-heals:

     # gluster volume heal volname info

     Wait for any self-heal operations to complete before continuing.
   • If this node is part of an NFS-Ganesha cluster:
     1. Disable the PCS cluster and verify that it has stopped.

        # pcs cluster disable
        # pcs status

     2. Stop the nfs-ganesha service.

        # systemctl stop nfs-ganesha

   • If you need to update an erasure coded (dispersed) volume, set the disperse.optimistic-change-log and disperse.eager-lock options to off. Wait for at least two minutes after disabling these options before attempting to upgrade to ensure that these configuration changes take effect for I/O operations.

     # gluster volume set volname disperse.optimistic-change-log off
     # gluster volume set volname disperse.eager-lock off

   • Stop the gluster services on the storage server using the following commands:
     On Red Hat Enterprise Linux 7:

     # systemctl stop glusterd
     # pkill glusterfs
     # pkill glusterfsd

     On Red Hat Enterprise Linux 6:

     # service glusterd stop
     # pkill glusterfs
     # pkill glusterfsd

   • If you use Samba:
     1. Enable the required repository.
        On Red Hat Enterprise Linux 6.7 or later:

        # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-6-server-rpms

        On Red Hat Enterprise Linux 7:

        # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-7-server-rpms

     2. Stop the CTDB and SMB services across all nodes in the Samba cluster using the following command. Stopping the CTDB service also stops the SMB service.
        On Red Hat Enterprise Linux 7:

        # systemctl stop ctdb
        # systemctl disable ctdb

        On Red Hat Enterprise Linux 6:

        # service ctdb stop
        # chkconfig ctdb off

        This ensures different versions of Samba do not run in the same Samba cluster until all Samba nodes are updated.
     3. Verify that the CTDB and SMB services are stopped by running the following command:

        ps axf | grep -E '(ctdb|smb|winbind|nmb)[d]'

   • Update the server using the following command:

     # yum update

     Take note of the packages being updated, and wait for the update to complete.
   • If a kernel update was included as part of the update process in the previous step, reboot the server.
   • If a reboot of the server was not required, then start the gluster services on the storage server using the following command.
     On Red Hat Enterprise Linux 7:

     # systemctl start glusterd

     On Red Hat Enterprise Linux 6:

     # service glusterd start

   • Verify that you have updated to the latest version of the Red Hat Gluster Storage server.

     # gluster --version

     Compare output with the desired version in Section 1.5, “Supported Versions of Red Hat Gluster Storage”.
   • Ensure that all bricks are online. To check the status, execute the following command:

     # gluster volume status

   • Start self-heal on the volume.

     # gluster volume heal volname

   • Ensure self-heal is complete on the replica using the following command:

     # gluster volume heal volname info

   • Verify that shared storage is mounted.

     # mount | grep /run/gluster/shared_storage

3. When all nodes in the volume have been updated, run the following command to update the op-version of the cluster. This helps to prevent any compatibility issues within the cluster.

   # gluster volume set all cluster.op-version 31102

   Note

   31102 is the cluster.op-version value for the latest Red Hat Gluster Storage 3.3.1 glusterfs Async. Refer to Section 1.5, “Supported Versions of Red Hat Gluster Storage” for the correct cluster.op-version value for other versions.
4. If you use Samba:
   1. Mount /gluster/lock before starting CTDB by executing the following command:

      # mount -a

   2. If all servers that host volumes accessed via SMB have been updated, then start and re-enable the CTDB and Samba services by executing the following commands.
      On Red Hat Enterprise Linux 7:

      # systemctl start ctdb
      # systemctl enable ctdb

      On Red Hat Enterprise Linux 6:

      # service ctdb start
      # chkconfig ctdb on

   3. To verify that the CTDB and SMB services have started, execute the following command:

      ps axf | grep -E '(ctdb|smb|winbind|nmb)[d]'

5. If you had a meta volume configured prior to this upgrade, and you did not reboot as part of the upgrade process, mount the meta volume:

   # mount /var/run/gluster/shared_storage/

   If this command does not work, review the content of /etc/fstab and ensure that the entry for the shared storage is configured correctly and re-run the mount command. The line for the meta volume in the /etc/fstab file should look like the following:

   hostname:/gluster_shared_storage   /var/run/gluster/shared_storage/   glusterfs   defaults   0 0

6. If this node is part of an NFS-Ganesha cluster:
   1. If SELinux is in use, set the ganesha_use_fusefs Boolean to on.

      # setsebool -P ganesha_use_fusefs on

   2. Start the nfs-ganesha service:

      # systemctl start nfs-ganesha

   3. Enable and start the cluster.

      # pcs cluster enable
      # pcs cluster start

   4. Release the node from standby mode.

      # pcs cluster unstandby

   5. Verify that the PCS cluster is running and that the volume is exporting correctly.

      # pcs status
      # showmount -e

      NFS-ganesha enters a short grace period after performing these steps. I/O operations halt during this grace period. Wait until you see NFS Server Now NOT IN GRACE in the ganesha.log file before continuing.
7. If you use geo-replication, restart geo-replication sessions when upgrade is complete.

   # gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL start

   Note

   As a result of BZ#1347625, you may need to use the force parameter to successfully restart in some circumstances.

   # gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL start force

8. If you disabled the disperse.optimistic-change-log and disperse.eager-lock options in order to update an erasure-coded (dispersed) volume, re-enable these settings.

   # gluster volume set volname disperse.optimistic-change-log on
   # gluster volume set volname disperse.eager-lock on