Cluster Administration
Configuring and Managing a Red Hat Cluster
Edition 1.0
Abstract
Introduction
- Red Hat Enterprise Linux Installation Guide — Provides information regarding installation.
- Red Hat Enterprise Linux Introduction to System Administration — Provides introductory information for new Red Hat Enterprise Linux system administrators.
- Red Hat Enterprise Linux System Administration Guide — Provides more detailed information about configuring Red Hat Enterprise Linux to suit your particular needs as a user.
- Red Hat Enterprise Linux Reference Guide — Provides detailed information suited for more experienced users to reference when needed, as opposed to step-by-step instructions.
- Red Hat Enterprise Linux Security Guide — Details the planning and the tools involved in creating a secured computing environment for the data center, workplace, and home.
- Red Hat Cluster Suite Overview — Provides a high level overview of the Red Hat Cluster Suite.
- LVM Administrator's Guide: Configuration and Administration — Provides a description of the Logical Volume Manager (LVM), including information on running LVM in a clustered environment.
- Global File System: Configuration and Administration — Provides information about installing, configuring, and maintaining Red Hat GFS (Red Hat Global File System).
- Using Device-Mapper Multipath — Provides information about using the Device-Mapper Multipath feature of Red Hat Enterprise Linux 4.7.
- Using GNBD with Global File System — Provides an overview on using Global Network Block Device (GNBD) with Red Hat GFS.
- Linux Virtual Server Administration — Provides information on configuring high-performance systems and services with the Linux Virtual Server (LVS).
- Red Hat Cluster Suite Release Notes — Provides information about the current release of Red Hat Cluster Suite.
1. Feedback
rh-cs
.
Cluster_Administration(EN)-4.8 (2009-5-13T12:45)
Chapter 1. Red Hat Cluster Configuration and Management Overview
1.1. Configuration Basics
- Setting up hardware. Refer to Section 1.1.1, “Setting Up Hardware”.
- Installing Red Hat Cluster software. Refer to Section 1.1.2, “Installing Red Hat Cluster software”.
- Configuring Red Hat Cluster Software. Refer to Section 1.1.3, “Configuring Red Hat Cluster Software”.
1.1.1. Setting Up Hardware
- Cluster nodes — Computers that are capable of running Red Hat Enterprise Linux 4 software, with at least 1GB of RAM.
- Ethernet switch or hub for public network — This is required for client access to the cluster.
- Ethernet switch or hub for private network — This is required for communication among the cluster nodes and other cluster hardware such as network power switches and Fibre Channel switches.
- Network power switch — A network power switch is recommended to perform fencing in an enterprise-level cluster.
- Fibre Channel switch — A Fibre Channel switch provides access to Fibre Channel storage. Other options are available for storage according to the type of storage interface; for example, iSCSI or GNBD. A Fibre Channel switch can be configured to perform fencing.
- Storage — Some type of storage is required for a cluster. The type required depends on the purpose of the cluster.
Figure 1.1. Red Hat Cluster Hardware Overview
1.1.2. Installing Red Hat Cluster software
1.1.3. Configuring Red Hat Cluster Software
Figure 1.2. Cluster Configuration Structure
- Conga — This is a comprehensive user interface for installing, configuring, and managing Red Hat clusters, computers, and storage attached to clusters and computers.
system-config-cluster
— This is a user interface for configuring and managing a Red Hat cluster.- Command line tools — This is a set of command line tools for configuring and managing a Red Hat cluster.
system-config-cluster
is provided in subsequent chapters of this document. Information about the command line tools is available in the man pages for the tools.
1.2. Conga
- One Web interface for managing cluster and storage
- Automated Deployment of Cluster Data and Supporting Packages
- Easy Integration with Existing Clusters
- No Need to Re-Authenticate
- Integration of Cluster Status and Logs
- Fine-Grained Control over User Permissions
- luci lists clusters that have been set up with that luci. A system administrator can administer all clusters listed on this tab. Other users can administer only clusters that the user has permission to manage (granted by an administrator).— Provides tools for creating and configuring clusters. Each instance of
Figure 1.3. luci Tab
Figure 1.4. luci Tab
Figure 1.5. luci Tab
1.3. system-config-cluster
Cluster Administration GUI
system-config-cluster
. It is for use with the cluster infrastructure and the high-availability service management components. system-config-cluster
consists of two major functions: the Cluster Configuration Tool and the Cluster Status Tool. The Cluster Configuration Tool provides the capability to create, edit, and propagate the cluster configuration file (/etc/cluster/cluster.conf
). The Cluster Status Tool provides the capability to manage high-availability services. The following sections summarize those functions.
Note
system-config-cluster
provides several convenient tools for configuring and managing a Red Hat Cluster, the newer, more comprehensive tool, Conga, provides more convenience and flexibility than system-config-cluster
.
1.3.1. Cluster Configuration Tool
Figure 1.6. Cluster Configuration Tool
/etc/cluster/cluster.conf
) with a hierarchical graphical display in the left panel. A triangle icon to the left of a component name indicates that the component has one or more subordinate components assigned to it. Clicking the triangle icon expands and collapses the portion of the tree below a component. The components displayed in the GUI are summarized as follows:
- Cluster Nodes — Displays cluster nodes. Nodes are represented by name as subordinate elements under Cluster Nodes. Using configuration buttons at the bottom of the right frame (below Properties), you can add nodes, delete nodes, edit node properties, and configure fencing methods for each node.
- Fence Devices — Displays fence devices. Fence devices are represented as subordinate elements under Fence Devices. Using configuration buttons at the bottom of the right frame (below Properties), you can add fence devices, delete fence devices, and edit fence-device properties. Fence devices must be defined before you can configure fencing (with the button) for each node.
- Managed Resources — Displays failover domains, resources, and services.
- Failover Domains — For configuring one or more subsets of cluster nodes used to run a high-availability service in the event of a node failure. Failover domains are represented as subordinate elements under Failover Domains. Using configuration buttons at the bottom of the right frame (below Properties), you can create failover domains (when Failover Domains is selected) or edit failover domain properties (when a failover domain is selected).
- Resources — For configuring shared resources to be used by high-availability services. Shared resources consist of file systems, IP addresses, NFS mounts and exports, and user-created scripts that are available to any high-availability service in the cluster. Resources are represented as subordinate elements under Resources. Using configuration buttons at the bottom of the right frame (below Properties), you can create resources (when Resources is selected) or edit resource properties (when a resource is selected).
Note
The Cluster Configuration Tool provides the capability to configure private resources, also. A private resource is a resource that is configured for use with only one service. You can configure a private resource within a Service component in the GUI. - Services — For creating and configuring high-availability services. A service is configured by assigning resources (shared or private), assigning a failover domain, and defining a recovery policy for the service. Services are represented as subordinate elements under Services. Using configuration buttons at the bottom of the right frame (below Properties), you can create services (when Services is selected) or edit service properties (when a service is selected).
1.3.2. Cluster Status Tool
Figure 1.7. Cluster Status Tool
/etc/cluster/cluster.conf
). You can use the Cluster Status Tool to enable, disable, restart, or relocate a high-availability service.
1.4. Command Line Administration Tools
system-config-cluster
Cluster Administration GUI, command line tools are available for administering the cluster infrastructure and the high-availability service management components. The command line tools are used by the Cluster Administration GUI and init scripts supplied by Red Hat. Table 1.1, “Command Line Tools” summarizes the command line tools.
Command Line Tool | Used With | Purpose |
---|---|---|
ccs_tool — Cluster Configuration System Tool | Cluster Infrastructure | ccs_tool is a program for making online updates to the cluster configuration file. It provides the capability to create and modify cluster infrastructure components (for example, creating a cluster, adding and removing a node). For more information about this tool, refer to the ccs_tool(8) man page. |
cman_tool — Cluster Management Tool | Cluster Infrastructure | cman_tool is a program that manages the CMAN cluster manager. It provides the capability to join a cluster, leave a cluster, kill a node, or change the expected quorum votes of a node in a cluster. cman_tool is available with DLM clusters only. For more information about this tool, refer to the cman_tool(8) man page. |
gulm_tool — Cluster Management Tool | Cluster Infrastructure | gulm_tool is a program used to manage GULM. It provides an interface to lock_gulmd , the GULM lock manager. gulm_tool is available with GULM clusters only. For more information about this tool, refer to the gulm_tool(8) man page. |
fence_tool — Fence Tool | Cluster Infrastructure | fence_tool is a program used to join or leave the default fence domain. Specifically, it starts the fence daemon (fenced ) to join the domain and kills fenced to leave the domain. fence_tool is available with DLM clusters only. For more information about this tool, refer to the fence_tool(8) man page. |
clustat — Cluster Status Utility | High-availability Service Management Components | The clustat command displays the status of the cluster. It shows membership information, quorum view, and the state of all configured user services. For more information about this tool, refer to the clustat(8) man page. |
clusvcadm — Cluster User Service Administration Utility | High-availability Service Management Components | The clusvcadm command allows you to enable, disable, relocate, and restart high-availability services in a cluster. For more information about this tool, refer to the clusvcadm(8) man page. |
Chapter 2. Before Configuring a Red Hat Cluster
2.1. Compatible Hardware
2.2. Enabling IP Ports
iptables
rules for enabling the ports:
2.2.1. Enabling IP Ports on Cluster Nodes
iptables
rule examples. At each cluster node, enable IP ports according to Table 2.1, “Enabled IP Ports on Red Hat Cluster Nodes”. (All examples are in Section 2.2.3, “Examples of iptables
Rules”.)
IP Port Number | Protocol | Component | Reference to Example of iptables Rules |
---|---|---|---|
6809 | UDP | cman (Cluster Manager), for use in clusters with Distributed Lock Manager (DLM) selected | Example 2.1, “Port 6809: cman” |
11111 | TCP | ricci (part of Conga remote agent) | Example 2.3, “Port 11111: ricci (Cluster Node and Computer Running luci)” |
14567 | TCP | gnbd (Global Network Block Device) | Example 2.4, “Port 14567: gnbd” |
16851 | TCP | modclusterd (part of Conga remote agent) | Example 2.5, “Port 16851: modclusterd” |
21064 | TCP | dlm (Distributed Lock Manager), for use in clusters with Distributed Lock Manager (DLM) selected | Example 2.6, “Port 21064: dlm” |
40040, 40042, 41040 | TCP | lock_gulmd (GULM daemon), for use in clusters with Grand Unified Lock Manager (GULM) selected | Example 2.7, “Ports 40040, 40042, 41040: lock_gulmd” |
41966, 41967, 41968, 41969 | TCP | rgmanager (high-availability service management) | Example 2.8, “Ports 41966, 41967, 41968, 41969: rgmanager” |
50006, 50008, 50009 | TCP | ccsd (Cluster Configuration System daemon) | Example 2.9, “Ports 50006, 50008, 50009: ccsd (TCP)” |
50007 | UDP | ccsd (Cluster Configuration System daemon) | Example 2.10, “Port 50007: ccsd (UDP)” |
2.2.2. Enabling IP Ports on Computers That Run luci
iptables
rule examples. At each computer that runs luci, enable IP ports according to Table 2.1, “Enabled IP Ports on Red Hat Cluster Nodes”. (All examples are in Section 2.2.3, “Examples of iptables
Rules”.)
Note
IP Port Number | Protocol | Component | Reference to Example of iptables Rules |
---|---|---|---|
8084 | TCP | luci (Conga user interface server) | Example 2.2, “Port 8084: luci (Cluster Node or Computer Running luci)” |
11111 | TCP | ricci (Conga remote agent) | Example 2.3, “Port 11111: ricci (Cluster Node and Computer Running luci)” |
2.2.3. Examples of iptables
Rules
iptables
rule examples for enabling IP ports on Red Hat Cluster nodes and computers that run luci. The examples enable IP ports for a computer having an IP address of 10.10.10.200, using a subnet mask of 10.10.10.0/24.
Note
Example 2.1. Port 6809: cman
-A INPUT -i 10.10.10.200 -m state --state NEW -p udp -s 10.10.10.0/24 -d 10.10.10.0/24 --dport 6809 -j ACCEPT
Example 2.2. Port 8084: luci (Cluster Node or Computer Running luci)
-A INPUT -i 10.10.10.200 -m state --state NEW -m multiport -p tcp -s 10.10.10.0/24 -d 10.10.10.0/24 --dports 8084 -j ACCEPT
Example 2.3. Port 11111: ricci (Cluster Node and Computer Running luci)
-A INPUT -i 10.10.10.200 -m state --state NEW -m multiport -p tcp -s 10.10.10.0/24 -d 10.10.10.0/24 --dports 11111 -j ACCEPT
Example 2.4. Port 14567: gnbd
-A INPUT -i 10.10.10.200 -m state --state NEW -m multiport -p tcp -s 10.10.10.0/24 -d 10.10.10.0/24 --dports 14567 -j ACCEPT
Example 2.5. Port 16851: modclusterd
-A INPUT -i 10.10.10.200 -m state --state NEW -m multiport -p tcp -s 10.10.10.0/24 -d 10.10.10.0/24 --dports 16851 -j ACCEPT
Example 2.6. Port 21064: dlm
-A INPUT -i 10.10.10.200 -m state --state NEW -m multiport -p tcp -s 10.10.10.0/24 -d 10.10.10.0/24 --dports 21064 -j ACCEPT
Example 2.7. Ports 40040, 40042, 41040: lock_gulmd
-A INPUT -i 10.10.10.200 -m state --state NEW -m multiport -p tcp -s 10.10.10.0/24 -d 10.10.10.0/24 --dports 40040,40042,41040 -j ACCEPT
Example 2.8. Ports 41966, 41967, 41968, 41969: rgmanager
-A INPUT -i 10.10.10.200 -m state --state NEW -m multiport -p tcp -s 10.10.10.0/24 -d 10.10.10.0/24 --dports 41966,41967,41968,41969 -j ACCEPT
Example 2.9. Ports 50006, 50008, 50009: ccsd (TCP)
-A INPUT -i 10.10.10.200 -m state --state NEW -m multiport -p tcp -s 10.10.10.0/24 -d 10.10.10.0/24 --dports 50006,50008,50009 -j ACCEPT
Example 2.10. Port 50007: ccsd (UDP)
-A INPUT -i 10.10.10.200 -m state --state NEW -m multiport -p udp -s 10.10.10.0/24 -d 10.10.10.0/24 --dports 50007 -j ACCEPT
2.3. Configuring ACPI For Use with Integrated Fence Devices
Note
shutdown -h now
). Otherwise, if ACPI Soft-Off is enabled, an integrated fence device can take four or more seconds to turn off a node (refer to note that follows). In addition, if ACPI Soft-Off is enabled and a node panics or freezes during shutdown, an integrated fence device may not be able to turn off the node. Under those circumstances, fencing is delayed or unsuccessful. Consequently, when a node is fenced with an integrated fence device and ACPI Soft-Off is enabled, a cluster recovers slowly or requires administrative intervention to recover.
Note
chkconfig
management and verify that the node turns off immediately when fenced. The preferred way to disable ACPI Soft-Off is with chkconfig
management: however, if that method is not satisfactory for your cluster, you can disable ACPI Soft-Off with one of the following alternate methods:
- Changing the BIOS setting to "instant-off" or an equivalent setting that turns off the node without delay
Note
Disabling ACPI Soft-Off with the BIOS may not be possible with some computers. - Appending
acpi=off
to the kernel boot command line of the/boot/grub/grub.conf
fileImportant
This method completely disables ACPI; some computers do not boot correctly if ACPI is completely disabled. Use this method only if the other methods are not effective for your cluster.
- Section 2.3.1, “Disabling ACPI Soft-Off with
chkconfig
Management” — Preferred method - Section 2.3.2, “Disabling ACPI Soft-Off with the BIOS” — First alternate method
- Section 2.3.3, “Disabling ACPI Completely in the
grub.conf
File” — Second alternate method
2.3.1. Disabling ACPI Soft-Off with chkconfig
Management
chkconfig
management to disable ACPI Soft-Off either by removing the ACPI daemon (acpid
) from chkconfig
management or by turning off acpid
.
Note
chkconfig
management at each cluster node as follows:
- Run either of the following commands:
chkconfig --del acpid
— This command removesacpid
fromchkconfig
management.— OR —chkconfig --level 2345 acpid off
— This command turns offacpid
.
- Reboot the node.
- When the cluster is configured and running, verify that the node turns off immediately when fenced.
Note
You can fence the node with thefence_node
command or Conga.
2.3.2. Disabling ACPI Soft-Off with the BIOS
chkconfig
management (Section 2.3.1, “Disabling ACPI Soft-Off with chkconfig
Management”). However, if the preferred method is not effective for your cluster, follow the procedure in this section.
Note
- Reboot the node and start the
BIOS CMOS Setup Utility
program. - Navigate to themenu (or equivalent power management menu).
- At the Example 2.11, “menu, set the function (or equivalent) to (or the equivalent setting that turns off the node via the power button without delay).
BIOS CMOS Setup Utility
: set to ” shows a menu with set to and set to .Note
The equivalents to, , and may vary among computers. However, the objective of this procedure is to configure the BIOS so that the computer is turned off via the power button without delay. - Exit the
BIOS CMOS Setup Utility
program, saving the BIOS configuration. - When the cluster is configured and running, verify that the node turns off immediately when fenced.
Note
You can fence the node with thefence_node
command or Conga.
Example 2.11. BIOS CMOS Setup Utility
: set to
+-------------------------------------------------|------------------------+ | ACPI Function [Enabled] | Item Help | | ACPI Suspend Type [S1(POS)] |------------------------| | x Run VGABIOS if S3 Resume Auto | Menu Level * | | Suspend Mode [Disabled] | | | HDD Power Down [Disabled] | | | Soft-Off by PWR-BTTN [Instant-Off] | | | CPU THRM-Throttling [50.0%] | | | Wake-Up by PCI card [Enabled] | | | Power On by Ring [Enabled] | | | Wake Up On LAN [Enabled] | | | x USB KB Wake-Up From S3 Disabled | | | Resume by Alarm [Disabled] | | | x Date(of Month) Alarm 0 | | | x Time(hh:mm:ss) Alarm 0 : 0 : 0 | | | POWER ON Function [BUTTON ONLY] | | | x KB Power ON Password Enter | | | x Hot Key Power ON Ctrl-F1 | | | | | | | | +-------------------------------------------------|------------------------+
2.3.3. Disabling ACPI Completely in the grub.conf
File
chkconfig
management (Section 2.3.1, “Disabling ACPI Soft-Off with chkconfig
Management”). If the preferred method is not effective for your cluster, you can disable ACPI Soft-Off with the BIOS power management (Section 2.3.2, “Disabling ACPI Soft-Off with the BIOS”). If neither of those methods is effective for your cluster, you can disable ACPI completely by appending acpi=off
to the kernel boot command line in the grub.conf
file.
Important
grub.conf
file of each cluster node as follows:
- Open
/boot/grub/grub.conf
with a text editor. - Append
acpi=off
to the kernel boot command line in/boot/grub/grub.conf
(refer to Example 2.12, “Kernel Boot Command Line withacpi=off
Appended to It”). - Reboot the node.
- When the cluster is configured and running, verify that the node turns off immediately when fenced.
Note
You can fence the node with thefence_node
command or Conga.
Example 2.12. Kernel Boot Command Line with acpi=off
Appended to It
# grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00 # initrd /initrd-version.img #boot=/dev/hda default=0 timeout=5 serial --unit=0 --speed=115200 terminal --timeout=5 serial console title Red Hat Enterprise Linux Server (2.6.18-36.el5) root (hd0,0) kernel /vmlinuz-2.6.18-36.el5 ro root=/dev/VolGroup00/LogVol00 console=ttyS0,115200n8 acpi=off initrd /initrd-2.6.18-36.el5.img
acpi=off
has been appended to the kernel boot command line — the line starting with "kernel /vmlinuz-2.6.18-36.el5".
2.4. Configuring max_luns
max_luns
in the /etc/modprobe.conf
file of each node as follows:
- Open
/etc/modprobe.conf
with a text editor. - Append the following line to
/etc/modprobe.conf
. Set N to the highest numbered LUN that is presented by RAID storage.options scsi_mod max_luns=N
For example, with the following line appended to the/etc/modprobe.conf
file, a node can access LUNs numbered as high as 255:options scsi_mod max_luns=255
- Save
/etc/modprobe.conf
. - Run
mkinitrd
to rebuildinitrd
for the currently running kernel as follows. Set the kernel variable to the currently running kernel:#
cd /boot
#mkinitrd -f -v initrd-kernel.img kernel
For example, the currently running kernel in the followingmkinitrd
command is 2.6.9-34.0.2.EL:#
mkinitrd -f -v initrd-2.6.9-34.0.2.EL.img 2.6.9-34.0.2.EL
Note
You can determine the currently running kernel by runninguname -r
. - Restart the node.
2.5. Considerations for Using Quorum Disk
qdiskd
, that provides supplemental heuristics to determine node fitness. With heuristics you can determine factors that are important to the operation of the node in the event of a network partition. For example, in a four-node cluster with a 3:1 split, ordinarily, the three nodes automatically "win" because of the three-to-one majority. Under those circumstances, the one node is fenced. With qdiskd
however, you can set up heuristics that allow the one node to win based on access to a critical resource (for example, a critical network path). If your cluster requires additional methods of determining node health, then you should configure qdiskd
to meet those needs.
Note
qdiskd
is not required unless you have special requirements for node health. An example of a special requirement is an "all-but-one" configuration. In an all-but-one configuration, qdiskd
is configured to provide enough quorum votes to maintain quorum even though only one node is working.
Important
qdiskd
parameters for your Red Hat Cluster depend on the site environment and special requirements needed. To understand the use of heuristics and other qdiskd
parameters, refer to the qdisk(5) man page. If you require assistance understanding and using qdiskd
for your site, contact an authorized Red Hat support representative.
qdiskd
, you should take into account the following considerations:
- Cluster node votes
- Each cluster node should have the same number of votes.
- CMAN membership timeout value
- The CMAN membership timeout value (the time a node needs to be unresponsive before CMAN considers that node to be dead, and not a member) should be at least two times that of the
qdiskd
membership timeout value. The reason is because the quorum daemon must detect failed nodes on its own, and can take much longer to do so than CMAN. The default value for CMAN membership timeout is 10 seconds. Other site-specific conditions may affect the relationship between the membership timeout values of CMAN andqdiskd
. For assistance with adjusting the CMAN membership timeout value, contact an authorized Red Hat support representative. - Fencing
- To ensure reliable fencing when using
qdiskd
, use power fencing. While other types of fencing (such as watchdog timers and software-based solutions to reboot a node internally) can be reliable for clusters not configured withqdiskd
, they are not reliable for a cluster configured withqdiskd
. - Maximum nodes
- A cluster configured with
qdiskd
supports a maximum of 16 nodes. The reason for the limit is because of scalability; increasing the node count increases the amount of synchronous I/O contention on the shared quorum disk device. - Quorum disk device
- A quorum disk device should be a shared block device with concurrent read/write access by all nodes in a cluster. The minimum size of the block device is 10 Megabytes. Examples of shared block devices that can be used by
qdiskd
are a multi-port SCSI RAID array, a Fibre Channel RAID SAN, or a RAID-configured iSCSI target. You can create a quorum disk device withmkqdisk
, the Cluster Quorum Disk Utility. For information about using the utility refer to the mkqdisk(8) man page.Note
Using JBOD as a quorum disk is not recommended. A JBOD cannot provide dependable performance and therefore may not allow a node to write to it quickly enough. If a node is unable to write to a quorum disk device quickly enough, the node is falsely evicted from a cluster.
2.6. Red Hat Cluster Suite and SELinux
SELINUX=disabled
in the /etc/selinux/config
file.
2.7. Considerations for Using Conga
2.8. General Configuration Considerations
- No-single-point-of-failure hardware configuration
- Clusters can include a dual-controller RAID array, multiple bonded network channels, multiple paths between cluster members and storage, and redundant un-interruptible power supply (UPS) systems to ensure that no single failure results in application down time or loss of data.Alternatively, a low-cost cluster can be set up to provide less availability than a no-single-point-of-failure cluster. For example, you can set up a cluster with a single-controller RAID array and only a single Ethernet channel.Certain low-cost alternatives, such as host RAID controllers, software RAID without cluster support, and multi-initiator parallel SCSI configurations are not compatible or appropriate for use as shared cluster storage.
- Data integrity assurance
- To ensure data integrity, only one node can run a cluster service and access cluster-service data at a time. The use of power switches in the cluster hardware configuration enables a node to power-cycle another node before restarting that node's cluster services during a failover process. This prevents two nodes from simultaneously accessing the same data and corrupting it. It is strongly recommended that fence devices (hardware or software solutions that remotely power, shutdown, and reboot cluster nodes) are used to guarantee data integrity under all failure conditions. Watchdog timers provide an alternative way to to ensure correct operation of cluster service failover.
- Ethernet channel bonding
- Cluster quorum and node health is determined by communication of messages among cluster nodes via Ethernet. In addition, cluster nodes use Ethernet for a variety of other critical cluster functions (for example, fencing). With Ethernet channel bonding, multiple Ethernet interfaces are configured to behave as one, reducing the risk of a single-point-of-failure in the typical switched Ethernet connection among cluster nodes and other cluster hardware.
Chapter 3. Configuring Red Hat Cluster With Conga
3.1. Configuration Tasks
- Configuring and running the Conga configuration user interface — the luci server. Refer to Section 3.2, “Starting luci and ricci”.
- Creating a cluster. Refer to Section 3.3, “Creating A Cluster”.
- Configuring global cluster properties. Refer to Section 3.4, “Global Cluster Properties”.
- Configuring fence devices. Refer to Section 3.5, “Configuring Fence Devices”.
- Configuring cluster members. Refer to Section 3.6, “Configuring Cluster Members”.
- Creating failover domains. Refer to Section 3.7, “Configuring a Failover Domain”.
- Creating resources. Refer to Section 3.8, “Adding Cluster Resources”.
- Creating cluster services. Refer to Section 3.9, “Adding a Cluster Service to the Cluster”.
- Configuring storage. Refer to Section 3.10, “Configuring Cluster Storage”.
3.2. Starting luci and ricci
- At each node to be administered by Conga, install the ricci agent. For example:
#
up2date -i ricci
- At each node to be administered by Conga, start ricci. For example:
#
service ricci start
Starting ricci: [ OK ] - Select a computer to host luci and install the luci software on that computer. For example:
#
up2date -i luci
Note
Typically, a computer in a server cage or a data center hosts luci; however, a cluster computer can host luci. - At the computer running luci, initialize the luci server using the
luci_admin init
command. For example:#
luci_admin init
Initializing the Luci server Creating the 'admin' user Enter password: <Type password and press ENTER.> Confirm password: <Re-type password and press ENTER.> Please wait... The admin password has been successfully set. Generating SSL certificates... Luci server has been successfully initialized Restart the Luci server for changes to take effect eg. service luci restart - Start luci using
service luci restart
. For example:#
service luci restart
Shutting down luci: [ OK ] Starting luci: generating https SSL certificates... done [ OK ] Please, point your web browser to https://nano-01:8084 to access luci - At a Web browser, place the URL of the luci server into the URL address box and click Go (or the equivalent). The URL syntax for the luci server is
https://luci_server_hostname:8084
. The first time you access luci, two SSL certificate dialog boxes are displayed. Upon acknowledging the dialog boxes, your Web browser displays the luci login page.
3.3. Creating A Cluster
- As administrator of luci, select the tab.
- Click.
- At thetext box, enter a cluster name. The cluster name cannot exceed 15 characters. Add the node name and password for each cluster node. Enter the node name for each node in the column; enter the root password for each node in the in the column. Check the checkbox if clustered storage is required.
- Click Create a new cluster page to be displayed again, showing the parameters entered in the preceding step, and parameters. The parameters consist of the lock manager option buttons, and , and text boxes in the group box. Configure parameters for either DLM or GULM as follows:. Clicking causes the the
- For DLM — Clickor confirm that it is set.
- For GULM — Clickor confirm that it is set. At the group box, enter the FQDN or the IP address of each lock server in a text box.
Note
You must enter the FQDN or the IP address of one, three, or five GULM lock servers.
- Re-enter enter the root password for each node in the in thecolumn.
- Click. Clicking causes the following actions:
- Cluster software packages to be downloaded onto each cluster node.
- Cluster software to be installed onto each cluster node.
- Cluster configuration file to be created and propagated to each node in the cluster.
- Starting the cluster.
A progress page shows the progress of those actions for each node in the cluster.When the process of creating a new cluster is complete, a page is displayed providing a configuration interface for the newly created cluster.
3.4. Global Cluster Properties
- Thetext box displays the cluster name; it does not accept a cluster name change. You cannot change the cluster name. The only way to change the name of a Red Hat cluster is to create a new cluster configuration with the new name.
- Thevalue is set to
1
by default and is automatically incremented each time you modify your cluster configuration. However, if you need to set it to another value, you can specify it at the text box. - You can enter advanced cluster properties by clicking. Clicking reveals a list of advanced properties. You can click any advanced property for online help about the property.
Enter the values required and clickfor changes to take effect.- Theparameter is the number of seconds the fence daemon (
fenced
) waits before fencing a node (a member of the fence domain) after the node has failed. The default value is0
. Its value may be varied to suit cluster and network performance. - Theparameter is the number of seconds the fence daemon (
fenced
) waits before fencing a node after the node joins the fence domain. The default value is3
. A typical setting for is between 20 and 30 seconds, but can vary according to cluster and network performance.
Enter values required and Clickfor changes to take effect.Note
For more information about fenced(8) man page.and , refer to theImportant
The number of nodes that can be configured as GULM lock servers is limited to either one, three, or five.- Table 3.1, “Quorum-Disk Parameters” describes the parameters. If you need to use a quorum disk, click , enter quorum disk parameters, click , and restart the cluster for the changes to take effect.tab (DLM clusters only) — This tab provides an interface for configuring these parameters: , , , , , , , , and . The parameter is enabled by default.
Important
Quorum-disk parameters and heuristics depend on the site environment and the special requirements needed. To understand the use of quorum-disk parameters and heuristics, refer to the qdisk(5) man page. If you require assistance understanding and using quorum disk, contact an authorized Red Hat support representative.Note
Clickingon the tab propagates changes to the cluster configuration file (/etc/cluster/cluster.conf
) in each cluster node. However, for the quorum disk to operate, you must restart the cluster (refer to Section 4.1, “Starting, Stopping, and Deleting Clusters”).
Parameter | Description | |||
---|---|---|---|---|
Disables quorum partition. Disables quorum-disk parameters in the | tab.||||
Enables quorum partition. Enables quorum-disk parameters in the | tab.||||
The frequency of read/write cycles, in seconds. | ||||
The number of votes the quorum daemon advertises to CMAN when it has a high enough score. | ||||
The number of cycles a node must miss to be declared dead. | ||||
The minimum score for a node to be considered "alive". If omitted or set to 0, the default function, floor((n+1)/2) , is used, where n is the sum of the heuristics scores. The value must never exceed the sum of the heuristic scores; otherwise, the quorum disk cannot be available. | ||||
The storage device the quorum daemon uses. The device must be the same on all nodes. | ||||
Specifies the quorum disk label created by the mkqdisk utility. If this field contains an entry, the label overrides the field. If this field is used, the quorum daemon reads /proc/partitions and checks for qdisk signatures on every block device found, comparing the label against the specified label. This is useful in configurations where the quorum device name differs among nodes. | ||||
| ||||
Propagates the changes to the cluster configuration file (/etc/cluster/cluster.conf ) in each cluster node. |
3.5. Configuring Fence Devices
Note
- APC Power Switch
- Brocade Fabric Switch
- Bull PAP
- Egenera SAN Controller
- GNBD
- IBM Blade Center
- McData SAN Switch
- QLogic SANbox2
- SCSI Fencing
- Virtual Machine Fencing
- Vixel SAN Switch
- WTI Power Switch
- Dell DRAC
- HP iLO
- IBM RSA II
- IPMI LAN
- RPS10 Serial Switch
- Creating shared fence devices — Refer to Section 3.5.1, “Creating a Shared Fence Device”. The procedures apply only to creating shared fence devices. You can create non-shared (and shared) fence devices while configuring nodes (refer to Section 3.6, “Configuring Cluster Members”).
- Modifying or deleting fence devices — Refer to Section 3.5.2, “Modifying or Deleting a Fence Device”. The procedures apply to both shared and non-shared fence devices.
3.5.1. Creating a Shared Fence Device
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of the fence devices for a cluster and causes the display of menu items for fence device configuration: and .
Note
If this is an initial cluster configuration, no fence devices have been created, and therefore none are displayed. - Click Add a Sharable Fence Device page to be displayed (refer to Figure 3.1, “Fence Device Configuration”).. Clicking causes the
Figure 3.1. Fence Device Configuration
- At the Add a Sharable Fence Device page, click the drop-down box under and select the type of fence device to configure.
- Specify the information in the Fencing Type dialog box according to the type of fence device. Refer to Appendix B, Fence Device Parameters for more information about fence device parameters.
- Click.
- Clickingcauses a progress page to be displayed temporarily. After the fence device has been added, the detailed cluster properties menu is updated with the fence device under .
3.5.2. Modifying or Deleting a Fence Device
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of the fence devices for a cluster and causes the display of menu items for fence device configuration: and .
- Click. Clicking causes the display of a list of fence devices under .
- Click a fence device in the list. Clicking a fence device in the list causes the display of a Fence Device Form page for the fence device selected from the list.
- Either modify or delete the fence device as follows:
- To modify the fence device, enter changes to the parameters displayed. Refer to Appendix B, Fence Device Parameters for more information about fence device parameters. Click and wait for the configuration to be updated.
- To delete the fence device, clickand wait for the configuration to be updated.
Note
You can create shared fence devices on the node configuration page, also. However, you can only modify or delete a shared fence device viaat the detailed menu for the cluster (below the menu).
3.6. Configuring Cluster Members
3.6.1. Initially Configuring Members
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of an element and a element with a list of the nodes already configured in the cluster.
- Click a link for a node at either the list in the center of the page or in the list in the detailed menu under themenu. Clicking a link for a node causes a page to be displayed for that link showing how that node is configured.
- At the bottom of the page, under, click .
- Select a fence device and provide parameters for the fence device (for example port number).
Note
You can choose from an existing fence device or create a new fence device. - Clickand wait for the change to take effect.
3.6.2. Adding a Member to a Running Cluster
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of an element and a element with a list of the nodes already configured in the cluster. (In addition, a list of the cluster nodes is displayed in the center of the page.)
- Click Add a node to cluster name page.. Clicking causes the display of the
- At that page, enter the node name in thetext box; enter the root password in the text box. Check the checkbox if clustered storage is required. If you want to add more nodes, click and enter node name and password for the each additional node.
- Click. Clicking causes the following actions:
- Cluster software packages to be downloaded onto the added node.
- Cluster software to be installed (or verification that the appropriate software packages are installed) onto the added node.
- Cluster configuration file to be updated and propagated to each node in the cluster — including the added node.
- Joining the added node to cluster.
A progress page shows the progress of those actions for each added node. - When the process of adding a node is complete, a page is displayed providing a configuration interface for the cluster.
- At the detailed menu for the cluster (below themenu), click . Clicking causes the following displays:
- A list of cluster nodes in the center of the page
- Theelement and the element with a list of the nodes configured in the cluster at the detailed menu for the cluster (below the menu)
- Click the link for an added node at either the list in the center of the page or in the list in the detailed menu under themenu. Clicking the link for the added node causes a page to be displayed for that link showing how that node is configured.
- At the bottom of the page, under, click .
- Select a fence device and provide parameters for the fence device (for example port number).
Note
You can choose from an existing fence device or create a new fence device. - Clickand wait for the change to take effect.
3.6.3. Deleting a Member from a Cluster
- Click the link of the node to be deleted. Clicking the link of the node to be deleted causes a page to be displayed for that link showing how that node is configured.
Note
To allow services running on a node to fail over when the node is deleted, skip the next step. - Disable or relocate each service that is running on the node to be deleted:
Note
Repeat this step for each service that needs to be disabled or started on another node.- Under, click the link for a service. Clicking that link cause a configuration page for that service to be displayed.
- On that page, at thedrop-down box, choose to either disable the service are start it on another node and click .
- Upon confirmation that the service has been disabled or started on another node, click the Choose a cluster to administer page to be displayed.tab. Clicking the tab causes the
- At the Choose a cluster to administer page, click the link of the node to be deleted. Clicking the link of the node to be deleted causes a page to be displayed for that link showing how that node is configured.
- On that page, at thedrop-down box, choose and click . When the node is deleted, a page is displayed that lists the nodes in the cluster. Check the list to make sure that the node has been deleted.
3.7. Configuring a Failover Domain
- Unrestricted — Allows you to specify that a subset of members are preferred, but that a cluster service assigned to this domain can run on any available member.
- Restricted — Allows you to restrict the members that can run a particular cluster service. If none of the members in a restricted failover domain are available, the cluster service cannot be started (either manually or by the cluster software).
- Unordered — When a cluster service is assigned to an unordered failover domain, the member on which the cluster service runs is chosen from the available failover domain members with no priority ordering.
- Ordered — Allows you to specify a preference order among the members of a failover domain. The member at the top of the list is the most preferred, followed by the second member in the list, and so on.
Note
Note
httpd
), which requires you to set up the configuration identically on all members that run the cluster service). Instead of setting up the entire cluster to run the cluster service, you must set up only the members in the restricted failover domain that you associate with the cluster service.
Note
3.7.1. Adding a Failover Domain
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of failover domains with related services and the display of menu items for failover domains: and .
- Click Add a Failover Domain page.. Clicking causes the display of the
- At the Add a Failover Domain page, specify a failover domain name at the text box.
Note
The name should be descriptive enough to distinguish its purpose relative to other names used in your cluster. - To enable setting failover priority of the members in the failover domain, click thecheckbox. With checked, you can set the priority value, , for each node selected as members of the failover domain.
- To restrict failover to members in this failover domain, click the checkbox next to. With checked, services assigned to this failover domain fail over only to nodes in this failover domain.
- Configure members for this failover domain. Under, click the checkbox for each node that is to be a member of the failover domain. If is checked, set the priority in the text box for each member of the failover domain.
- Click Failover Domain Form page. That page displays the added resource and includes the failover domain in the cluster menu to the left under .. Clicking causes a progress page to be displayed followed by the display of the
- To make additional changes to the failover domain, continue modifications at the Failover Domain Form page and click when you are done.
3.7.2. Modifying a Failover Domain
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of failover domains with related services and the display of menu items for failover domains: and .
- Click. Clicking causes the display of failover domains under at the detailed menu for the cluster (below the menu).
- At the detailed menu for the cluster (below the Failover Domain Form page. At the Failover Domain Form page, you can modify the failover domain name, prioritize failover, restrict failover to this domain, and modify failover domain membership.menu), click the failover domain to modify. Clicking the failover domain causes the display of the
- Modifying failover name — To change the failover domain name, modify the text at thetext box.
Note
The name should be descriptive enough to distinguish its purpose relative to other names used in your cluster. - Failover priority — To enable or disable prioritized failover in this failover domain, click the not checked, setting priority levels is disabled for this failover domain.checkbox. With checked, you can set the priority value, , for each node selected as members of the failover domain. With
- Restricted failover — To enable or disable restricted failover for members in this failover domain, click the checkbox next to not checked, services assigned to this failover domain can fail over to nodes outside this failover domain.. With checked, services assigned to this failover domain fail over only to nodes in this failover domain. With
- Modifying failover domain membership — Under, click the checkbox for each node that is to be a member of the failover domain. A checked box for a node means that the node is a member of the failover domain. If is checked, you can adjust the priority in the text box for each member of the failover domain.
- Click Failover Domain Form page. That page displays the added resource and includes the failover domain in the cluster menu to the left under .. Clicking causes a progress page to be displayed followed by the display of the
- To make additional changes to the failover domain, continue modifications at the Failover Domain Form page and click when you are done.
3.8. Adding Cluster Resources
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of resources in the center of the page and causes the display of menu items for resource configuration: and .
- Click Add a Resource page to be displayed.. Clicking causes the
- At the Add a Resource page, click the drop-down box under and select the type of resource to configure. The resource options are described as follows:
- GFS
- not unmounted at service tear-down unless this box is checked.checkbox — If checked, forces the file system to unmount. The default setting is unchecked. kills all processes using the mount point to free up the mount when it tries to unmount. With GFS resources, the mount point is
- File System
- File System Resource Configuration dialog box. If you need to assign a file system ID explicitly, specify it in this field.— When creating a new file system resource, you can leave this field blank. Leaving the field blank causes a file system ID to be assigned automatically after you click at theCheckboxes — Specify mount and unmount actions when a service is stopped (for example, when disabling or relocating a service):
fsck
to be run on the file system before mounting it. The default setting is unchecked.
- IP Address
- NFS Mount
- nfs(5) man page.— Mount options. For more information, refer to the
- NFS Client
- exports(5) man page, General Options— Additional client access rights. For more information, refer to the
- NFS Export
- Script
/etc/init.d/userscript
)- Samba Service
Note
When creating or editing a cluster service, connect a Samba-service resource directly to service, not to a resource within a service.
- Click Resources forcluster name page. That page displays the added resource (and other resources).. Clicking causes a progress page to be displayed followed by the display of
3.9. Adding a Cluster Service to the Cluster
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of services in the center of the page and causes the display of menu items for services configuration: and .
- Click Add a Service page to be displayed.. Clicking causes the
- On the Add a Service page, at the text box, type the name of the service. Below the text box is an checkbox labeled . The checkbox is checked by default. When the checkbox is checked, the service is started automatically when a cluster is started and running. If the checkbox is not checked, the service must be started manually any time the cluster comes up from the stopped state.
Note
Use a descriptive name that clearly distinguishes the service from other services in the cluster. - Add a resource to the service; click only to this service. The process of adding a local resource is the same as adding a global resource described in Section 3.8, “Adding Cluster Resources”. Adding a global resource adds a resource that has been previously added as a global resource (refer to Section 3.8, “Adding Cluster Resources”).. Clicking causes the display of two drop-down boxes: and . Adding a new local resource adds a resource that is available
- At the drop-down box of either Section 3.8, “Adding Cluster Resources”.)or , select the resource to add and configure it according to the options presented. (The options are the same as described in
Note
If you are adding a Samba-service resource, connect a Samba-service resource directly to the service, not to a resource within a service. - If you want to add resources to that resource, click. Clicking causes the display of additional options to local and global resources. You can continue adding children resources to the resource to suit your requirements. To view children resources, click the triangle icon to the left of .
- When you have completed adding resources to the service, and have completed adding children resources to resources, click. Clicking causes a progress page to be displayed followed by a page displaying the added service (and other services).
Note
/sbin/ip addr list
command on a cluster node. The following output shows the /sbin/ip addr list
command executed on a node running a cluster service:
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP> mtu 1356 qdisc pfifo_fast qlen 1000 link/ether 00:05:5d:9a:d8:91 brd ff:ff:ff:ff:ff:ff inet 10.11.4.31/22 brd 10.11.7.255 scope global eth0 inet6 fe80::205:5dff:fe9a:d891/64 scope link inet 10.11.4.240/22 scope global secondary eth0 valid_lft forever preferred_lft forever
3.10. Configuring Cluster Storage
- A computer is unreachable via the network.
- A computer has been re-imaged and the luci server admin must re-authenticate with the ricci agent on the computer.
Note
clvmd
) or the High Availability Logical Volume Management agents (HA-LVM). If you are not able to use either the clvmd
daemon or HA-LVM for operational reasons or because you do not have the correct entitlements, you must not use single-instance LVM on the shared disk as this may result in data corruption. If you have any concerns please contact your Red Hat service representative.
Chapter 4. Managing Red Hat Cluster With Conga
4.1. Starting, Stopping, and Deleting Clusters
- Restart a cluster.
- Start a cluster.
- Stop a cluster.
- Delete a cluster.
- At the right of the Choose a cluster to administer page is a drop-down box. By default, the drop-down box is set to . Clicking the drop-down box box reveals all the selections available: , / , and . The actions of each function are summarized as follows:for each cluster listed on the
- Selectingshuts down cluster software in all cluster nodes.Selectingstarts cluster software.
- Select one of the functions and click.
- Clickingcauses a progress page to be displayed. When the action is complete, a page is displayed showing either of the following pages according to the action selected:
- Forand / — Displays a page with the list of nodes for the cluster.
- For Choose a cluster to administer page in the tab, showing a list of clusters.— Displays the
4.2. Managing Cluster Nodes
- Make a node leave or join a cluster.
- Fence a node.
- Reboot a node.
- Delete a node.
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of nodes in the center of the page and causes the display of an element and a element with a list of the nodes already configured in the cluster.
- At the right of each node listed on the page displayed from the preceding step, click thedrop-down box. Clicking drop-down box reveals the following selections: / , , , and . The actions of each function are summarized as follows:
- Selectingshuts down cluster software and makes the node leave the cluster. Making a node leave a cluster prevents the node from automatically joining the cluster when it is rebooted.Selectingstarts cluster software and makes the node join the cluster. Making a node join a cluster allows the node to automatically join the cluster when it is rebooted.
cluster.conf
file from/etc/cluster/
.
- Select one of the functions and click.
- Clickingcauses a progress page to be displayed. When the action is complete, a page is displayed showing the list of nodes for the cluster.
4.3. Managing High-Availability Services
- Configure a service.
- Stop or start a service.
- Restart a service.
- Delete a service
- At the detailed menu for the cluster (below themenu), click . Clicking causes the display of services for the cluster in the center of the page.
- At the right of each service listed on the page, click thedrop-down box. Clicking drop-down box reveals the following selections depending on if the service is running:
- If service is running —, , and .
- If service is not running —, , and .
The actions of each function are summarized as follows:- Section 3.8, “Adding Cluster Resources” and Section 3.9, “Adding a Cluster Service to the Cluster”.) In addition, a drop-down box on the page provides other functions depending on if the service is running.— is available when the service is running or not running. Selecting causes the services configuration page for the service to be displayed. On that page, you can change the configuration of the service. For example, you can add a resource to the service. (For more information about adding resources and services, refer toWhen a service is running, the drop-down box provides the following functions: restarting, disabling, and relocating the service.When a service is not running, the drop-down box on the configuration page provides the following functions: enabling and deleting the service.If you are making configuration changes, save the changes by clicking. Clicking causes a progress page to be displayed. When the change is complete, another page is displayed showing a list of services for the cluster.If you have selected one of the functions in the drop-down box on the configuration page, click. Clicking causes a progress page to be displayed. When the change is complete, another page is displayed showing a list of services for the cluster.
Chapter 5. Configuring Red Hat Cluster With system-config-cluster
system-config-cluster
, and consists of the following sections:
Note
system-config-cluster
provides several convenient tools for configuring and managing a Red Hat Cluster, the newer, more comprehensive tool, Conga, provides more convenience and flexibility than system-config-cluster
. You may want to consider using Conga instead (refer to Chapter 3, Configuring Red Hat Cluster With Conga and Chapter 4, Managing Red Hat Cluster With Conga).
5.1. Configuration Tasks
system-config-cluster
consists of the following steps:
- Starting the Cluster Configuration Tool,
system-config-cluster
. Refer to Section 5.2, “Starting the Cluster Configuration Tool”. - Configuring cluster properties. Refer to Section 5.3, “Configuring Cluster Properties”.
- Creating fence devices. Refer to Section 5.4, “Configuring Fence Devices”.
- Creating cluster members. Refer to Section 5.5, “Adding and Deleting Members”.
- Creating failover domains. Refer to Section 5.6, “Configuring a Failover Domain”.
- Creating resources. Refer to Section 5.7, “Adding Cluster Resources”.
- Creating cluster services.
- Propagating the configuration file to the other nodes in the cluster.
- Starting the cluster software. Refer to Section 5.10, “Starting the Cluster Software”.
5.2. Starting the Cluster Configuration Tool
ssh -Y
command and issuing the system-config-cluster
command. For example, to start the Cluster Configuration Tool on cluster node nano-01, do the following:
- Log in to a cluster node and run
system-config-cluster
. For example:$
ssh -Y root@nano-01
. . . #system-config-cluster
- If this is the first time you have started the Cluster Configuration Tool, the program prompts you to either open an existing configuration or create a new one. Click to start a new configuration file (refer to Figure 5.1, “Starting a New Configuration File”).
Figure 5.1. Starting a New Configuration File
Note
The Cluster Configuration Tool, exit, and restart the the Red Hat Cluster Suite management GUI (tab for the Red Hat Cluster Suite management GUI is available after you save the configuration file with thesystem-config-cluster
). (The tab displays the status of the cluster service manager, cluster nodes, and resources, and shows statistics concerning cluster service operation. To manage the cluster system further, choose the tab.) - Clicking New Configuration dialog box to be displayed (refer to Figure 5.2, “Creating A New Configuration”). The New Configuration dialog box provides a text box for a cluster name and group boxes for the following configuration options: , (DLM clusters only), and (DLM clusters only). In most circumstances you only need to configure a cluster name and a lock method. is the default lock method. To configure a GULM cluster, select . (Selecting disables and , which are applicable only to DLM clusters). specifies whether a multicast address is used for cluster management communication among cluster nodes. is disabled (checkbox unchecked) by default. To use a multicast address for cluster management communication among cluster nodes, click the checkbox (enabled when checked). When is enabled, the text boxes are enabled; enter the multicast address into the text boxes. To use a quorum disk, click the checkbox and enter quorum disk parameters. The following quorum-disk parameters are available in the dialog box if you enable : , , , , , , and . Table 5.1, “Quorum-Disk Parameters” describes the parameters.causes the
Important
Quorum-disk parameters and heuristics depend on the site environment and special requirements needed. To understand the use of quorum-disk parameters and heuristics, refer to the qdisk(5) man page. If you require assistance understanding and using quorum disk, contact an authorized Red Hat support representative.Note
It is probable that configuring a quorum disk requires changing quorum-disk parameters after the initial configuration. The Cluster Configuration Tool (system-config-cluster
) provides only the display of quorum-disk parameters after initial configuration. If you need to configure quorum disk, consider using Conga instead; Conga allows modification of quorum disk parameters.Overall:Whilesystem-config-cluster
provides several convenient tools for configuring and managing a Red Hat Cluster, the newer, more comprehensive tool, Conga, provides more convenience and flexibility thansystem-config-cluster
. You may want to consider using Conga instead (refer to Chapter 3, Configuring Red Hat Cluster With Conga and Chapter 4, Managing Red Hat Cluster With Conga).Figure 5.2. Creating A New Configuration
- When you have completed entering the cluster name and other parameters in the New Configuration dialog box, click . Clicking starts the Cluster Configuration Tool, displaying a graphical representation of the configuration (Figure 5.3, “The Cluster Configuration Tool”).
Figure 5.3. The Cluster Configuration Tool
Parameter | Description | |||
---|---|---|---|---|
Enables quorum disk. Enables quorum-disk parameters in the New Configuration dialog box. | ||||
The frequency of read/write cycles, in seconds. | ||||
The number of cycles a node must miss in order to be declared dead. | ||||
The number of votes the quorum daemon advertises to CMAN when it has a high enough score. | ||||
The minimum score for a node to be considered "alive". If omitted or set to 0, the default function, floor((n+1)/2) , is used, where n is the sum of the heuristics scores. The value must never exceed the sum of the heuristic scores; otherwise, the quorum disk cannot be available. | ||||
The storage device the quorum daemon uses. The device must be the same on all nodes. | ||||
Specifies the quorum disk label created by the mkqdisk utility. If this field contains an entry, the label overrides the field. If this field is used, the quorum daemon reads /proc/partitions and checks for qdisk signatures on every block device found, comparing the label against the specified label. This is useful in configurations where the quorum device name differs among nodes. | ||||
|
5.3. Configuring Cluster Properties
- At the left frame, click.
- At the bottom of the right frame (labeled Cluster Properties dialog box to be displayed. The Cluster Properties dialog box presents text boxes for , and , and two parameters (DLM clusters only): and .), click the button. Clicking that button causes a
- (Optional) At the Section 5.2, “Starting the Cluster Configuration Tool”). The cluster alias should be descriptive enough to distinguish it from other clusters and systems on your network (for example,text box, specify a cluster alias for the cluster. The default cluster alias is set to the true cluster name provided when the cluster is set up (refer to
nfs_cluster
orhttpd_cluster
). The cluster alias cannot exceed 15 characters. - (Optional) Thevalue is set to
1
by default and is automatically incremented each time you save your cluster configuration. However, if you need to set it to another value, you can specify it at the text box. - Specify theparameters (DLM clusters only): and .
- Theparameter is the number of seconds the fence daemon (
fenced
) waits before fencing a node after the node joins the fence domain. The default value is3
. A typical setting for is between 20 and 30 seconds, but can vary according to cluster and network performance. - Theparameter is the number of seconds the fence daemon (
fenced
) waits before fencing a node (a member of the fence domain) after the node has failed.The default value is0
. Its value may be varied to suit cluster and network performance.
Note
For more information about fenced(8) man page.and , refer to the - Save cluster configuration changes by selecting=> .
5.4. Configuring Fence Devices
- Click Fence Device Configuration dialog box to be displayed (refer to Figure 5.4, “Fence Device Configuration”).. At the bottom of the right frame (labeled ), click the button. Clicking causes the
Figure 5.4. Fence Device Configuration
- At the Fence Device Configuration dialog box, click the drop-down box under and select the type of fence device to configure.
- Specify the information in the Fence Device Configuration dialog box according to the type of fence device. Refer to Appendix B, Fence Device Parameters for more information about fence device parameters.
- Click.
- Choose=> to save the changes to the cluster configuration.
5.5. Adding and Deleting Members
5.5.1. Adding a Member to a New Cluster
- At
system-config-cluster
, in the Cluster Configuration Tool tab, click . - At the bottom of the right frame (labeled Node Properties dialog box to be displayed. For a DLM cluster, the Node Properties dialog box presents text boxes for and (refer to Figure 5.5, “Adding a Member to a New DLM Cluster”). For a GULM cluster, the Node Properties dialog box presents text boxes for and , and presents a checkbox for (refer to Figure 5.6, “Adding a Member to a New GULM Cluster”)), click the button. Clicking that button causes a
Important
The number of nodes that can be configured as GULM lock servers is limited to either one, three, or five.Figure 5.5. Adding a Member to a New DLM Cluster
Figure 5.6. Adding a Member to a New GULM Cluster
- At thetext box, specify a node name. The entry can be a name or an IP address of the node on the cluster subnet.
Note
Each node must be on the same subnet as the node from which you are running the Cluster Configuration Tool and must be defined either in DNS or in the/etc/hosts
file of each cluster node.Note
The node on which you are running the Cluster Configuration Tool must be explicitly added as a cluster member; the node is not automatically added to the cluster configuration as a result of running the Cluster Configuration Tool. - Optionally, at thetext box, you can specify a value; however in most configurations you can leave it blank. Leaving the text box blank causes the quorum votes value for that node to be set to the default value of
1
. - Click.
- Configure fencing for the node:
- Click the node that you added in the previous step.
- At the bottom of the right frame (below Fence Configuration dialog box to be displayed.), click . Clicking causes the
- At the Fence Configuration dialog box, bottom of the right frame (below ), click . Clicking causes a fence-level element (for example, , , and so on) to be displayed below the node in the left frame of the Fence Configuration dialog box.
- Click the fence-level element.
- At the bottom of the right frame (below Fence Properties dialog box to be displayed.), click . Clicking causes the
- At the Fence Properties dialog box, click the drop-down box and select the fence device for this node. Also, provide additional information required (for example, and for an APC Power Device).
- At the Fence Properties dialog box, click . Clicking causes a fence device element to be displayed below the fence-level element.
- To create additional fence devices at this fence level, return to step 6d. Otherwise, proceed to the next step.
- To create additional fence levels, return to step 6c. Otherwise, proceed to the next step.
- If you have configured all the fence levels and fence devices for this node, click.
- Choose=> to save the changes to the cluster configuration.
5.5.2. Adding a Member to a Running DLM Cluster
- For clusters with only two nodes —
- For clusters with more than two nodes —
5.5.2.1. Adding a Member to a Running DLM Cluster That Contains Only Two Nodes
- Add the node and configure fencing for it as in Section 5.5.1, “Adding a Member to a New Cluster”.
- Clickto propagate the updated configuration to other running nodes in the cluster.
- Use the
scp
command to send the updated/etc/cluster/cluster.conf
file from one of the existing cluster nodes to the new node. - At
system-config-cluster
, in the Cluster Status Tool tab, disable each service listed under . - Stop the cluster software on the two running nodes by running the following commands at each node in this order:
service rgmanager stop
, if the cluster is running high-availability services (rgmanager
)service gfs stop
, if you are using Red Hat GFSservice clvmd stop
, if CLVM has been used to create clustered volumesservice fenced stop
service cman stop
service ccsd stop
- Start cluster software on all cluster nodes (including the added one) by running the following commands in this order:
service ccsd start
service cman start
service fenced start
service clvmd start
, if CLVM has been used to create clustered volumesservice gfs start
, if you are using Red Hat GFSservice rgmanager start
, if the cluster is running high-availability services (rgmanager
)
- Start
system-config-cluster
(refer to Section 5.2, “Starting the Cluster Configuration Tool”). At the Cluster Configuration Tool tab, verify that the configuration is correct. At the Cluster Status Tool tab verify that the nodes and services are running as expected.
Note
5.5.2.2. Adding a Member to a Running DLM Cluster That Contains More Than Two Nodes
- Add the node and configure fencing for it as in Section 5.5.1, “Adding a Member to a New Cluster”.
- Clickto propagate the updated configuration to other running nodes in the cluster.
- Use the
scp
command to send the updated/etc/cluster/cluster.conf
file from one of the existing cluster nodes to the new node. - Start cluster services on the new node by running the following commands in this order:
service ccsd start
service cman start
service fenced start
service clvmd start
, if CLVM has been used to create clustered volumesservice gfs start
, if you are using Red Hat GFSservice rgmanager start
, if the cluster is running high-availability services (rgmanager
)
- Start
system-config-cluster
(refer to Section 5.2, “Starting the Cluster Configuration Tool”). At the Cluster Configuration Tool tab, verify that the configuration is correct. At the Cluster Status Tool tab verify that the nodes and services are running as expected.
Note
5.5.3. Deleting a Member from a DLM Cluster
- At one of the running nodes (not at a node to be deleted), start
system-config-cluster
(refer to Section 5.2, “Starting the Cluster Configuration Tool”). At the Cluster Status Tool tab, under , disable or relocate each service that is running on the node to be deleted. - Stop the cluster software on the node to be deleted by running the following commands at that node in this order:
service rgmanager stop
, if the cluster is running high-availability services (rgmanager
)service gfs stop
, if you are using Red Hat GFSservice clvmd stop
, if CLVM has been used to create clustered volumesservice fenced stop
service cman stop
service ccsd stop
- At
system-config-cluster
(running on a node that is not to be deleted), in the Cluster Configuration Tool tab, delete the member as follows:- If necessary, click the triangle icon to expand theproperty.
- Select the cluster node to be deleted. At the bottom of the right frame (labeled), click the button.
- Clicking the Figure 5.7, “Confirm Deleting a Member”).button causes a warning dialog box to be displayed requesting confirmation of the deletion (
Figure 5.7. Confirm Deleting a Member
- At that dialog box, clickto confirm deletion.
- Propagate the updated configuration by clicking thebutton. (Propagating the updated configuration automatically saves the configuration.)
- Stop the cluster software on the remaining running nodes by running the following commands at each node in this order:
service rgmanager stop
, if the cluster is running high-availability services (rgmanager
)service gfs stop
, if you are using Red Hat GFSservice clvmd stop
, if CLVM has been used to create clustered volumesservice fenced stop
service cman stop
service ccsd stop
- Start cluster software on all remaining cluster nodes by running the following commands in this order:
service ccsd start
service cman start
service fenced start
service clvmd start
, if CLVM has been used to create clustered volumesservice gfs start
, if you are using Red Hat GFSservice rgmanager start
, if the cluster is running high-availability services (rgmanager
)
- At
system-config-cluster
(running on a node that was not deleted), in the Cluster Configuration Tool tab, verify that the configuration is correct. At the Cluster Status Tool tab verify that the nodes and services are running as expected.
Note
5.5.4. Adding a GULM Client-only Member
- At one of the running members, start
system-config-cluster
(refer to Section 5.2, “Starting the Cluster Configuration Tool”). At the Cluster Configuration Tool tab, add the node and configure fencing for it as in Section 5.5.1, “Adding a Member to a New Cluster”. - Clickto propagate the updated configuration to other running nodes in the cluster.
- Use the
scp
command to send the updated/etc/cluster/cluster.conf
file from one of the existing cluster nodes to the new node. - Start cluster services on the new node by running the following commands in this order:
service ccsd start
service lock_gulmd start
service clvmd start
, if CLVM has been used to create clustered volumesservice gfs start
, if you are using Red Hat GFSservice rgmanager start
, if the cluster is running high-availability services (rgmanager
)
- At
system-config-cluster
, in the Cluster Configuration Tool tab, verify that the configuration is correct. At the Cluster Status Tool tab verify that the nodes and services are running as expected.
Note
5.5.5. Deleting a GULM Client-only Member
- At one of the running nodes (not at a node to be deleted), start
system-config-cluster
(refer to Section 5.2, “Starting the Cluster Configuration Tool”). At the Cluster Status Tool tab, under , disable or relocate each service that is running on the node to be deleted. - Stop the cluster software on the node to be deleted by running the following commands at that node in this order:
service rgmanager stop
, if the cluster is running high-availability services (rgmanager
)service gfs stop
, if you are using Red Hat GFSservice clvmd stop
, if CLVM has been used to create clustered volumesservice lock_gulmd stop
service ccsd stop
- At
system-config-cluster
(running on a node that is not to be deleted), in the Cluster Configuration Tool tab, delete the member as follows:- If necessary, click the triangle icon to expand theproperty.
- Select the cluster node to be deleted. At the bottom of the right frame (labeled), click the button.
- Clicking the Figure 5.8, “Confirm Deleting a Member”).button causes a warning dialog box to be displayed requesting confirmation of the deletion (
Figure 5.8. Confirm Deleting a Member
- At that dialog box, clickto confirm deletion.
- Propagate the updated configuration by clicking thebutton. (Propagating the updated configuration automatically saves the configuration.)
- Stop the cluster software on the remaining running nodes by running the following commands at each node in this order:
service rgmanager stop
, if the cluster is running high-availability services (rgmanager
)service gfs stop
, if you are using Red Hat GFSservice clvmd stop
, if CLVM has been used to create clustered volumesservice lock_gulmd stop
service ccsd stop
- Start cluster software on all remaining cluster nodes by running the following commands in this order:
service ccsd start
service lock_gulmd start
service clvmd start
, if CLVM has been used to create clustered volumesservice gfs start
, if you are using Red Hat GFSservice rgmanager start
, if the cluster is running high-availability services (rgmanager
)
- At
system-config-cluster
(running on a node that was not deleted), in the Cluster Configuration Tool tab, verify that the configuration is correct. At the Cluster Status Tool tab verify that the nodes and services are running as expected.
Note
5.5.6. Adding or Deleting a GULM Lock Server Member
Important
- At one of the running members (running on a node that is not to be deleted), start
system-config-cluster
(refer to Section 5.2, “Starting the Cluster Configuration Tool”). At the Cluster Status Tool tab, disable each service listed under . - Stop the cluster software on each running node by running the following commands at each node in this order:
service rgmanager stop
, if the cluster is running high-availability services (rgmanager
)service gfs stop
, if you are using Red Hat GFSservice clvmd stop
, if CLVM has been used to create clustered volumesservice lock_gulmd stop
service ccsd stop
- To add a a GULM lock server member, at
system-config-cluster
, in the Cluster Configuration Tool tab, add each node and configure fencing for it as in Section 5.5.1, “Adding a Member to a New Cluster”. Make sure to select in the Node Properties dialog box (refer to Figure 5.6, “Adding a Member to a New GULM Cluster”). - To delete a GULM lock server member, at
system-config-cluster
(running on a node that is not to be deleted), in the Cluster Configuration Tool tab, delete each member as follows:- If necessary, click the triangle icon to expand theproperty.
- Select the cluster node to be deleted. At the bottom of the right frame (labeled), click the button.
- Clicking the Figure 5.9, “Confirm Deleting a Member”).button causes a warning dialog box to be displayed requesting confirmation of the deletion (
Figure 5.9. Confirm Deleting a Member
- At that dialog box, clickto confirm deletion.
- Propagate the configuration file to the cluster nodes as follows:
- Log in to the node where you created the configuration file (the same node used for running
system-config-cluster
). - Using the
scp
command, copy the/etc/cluster/cluster.conf
file to all nodes in the cluster.Note
Propagating the cluster configuration file this way is necessary under these circumstances because the cluster software is not running, and therefore not capable of propagating the configuration. Once a cluster is installed and running, the cluster configuration file is propagated using the Red Hat cluster management GUI Section 6.3, “Modifying the Cluster Configuration”.button. For more information about propagating the cluster configuration using the GUI button, refer to - After you have propagated the cluster configuration to the cluster nodes you can either reboot each node or start the cluster software on each cluster node by running the following commands at each node in this order:
service ccsd start
service lock_gulmd start
service clvmd start
, if CLVM has been used to create clustered volumesservice gfs start
, if you are using Red Hat GFSservice rgmanager start
, if the node is also functioning as a GULM client and the cluster is running cluster services (rgmanager
)
- At
system-config-cluster
(running on a node that was not deleted), in the Cluster Configuration Tool tab, verify that the configuration is correct. At the Cluster Status Tool tab verify that the nodes and services are running as expected.
Note
5.6. Configuring a Failover Domain
- Unrestricted — Allows you to specify that a subset of members are preferred, but that a cluster service assigned to this domain can run on any available member.
- Restricted — Allows you to restrict the members that can run a particular cluster service. If none of the members in a restricted failover domain are available, the cluster service cannot be started (either manually or by the cluster software).
- Unordered — When a cluster service is assigned to an unordered failover domain, the member on which the cluster service runs is chosen from the available failover domain members with no priority ordering.
- Ordered — Allows you to specify a preference order among the members of a failover domain. The member at the top of the list is the most preferred, followed by the second member in the list, and so on.
Note
Note
httpd
), which requires you to set up the configuration identically on all members that run the cluster service). Instead of setting up the entire cluster to run the cluster service, you must set up only the members in the restricted failover domain that you associate with the cluster service.
Note
5.6.1. Adding a Failover Domain
- At the left frame of the the Cluster Configuration Tool, click .
- At the bottom of the right frame (labeled Add Failover Domain dialog box to be displayed.), click the button. Clicking the button causes the
- At the Add Failover Domain dialog box, specify a failover domain name at the text box and click . Clicking causes the Failover Domain Configuration dialog box to be displayed (Figure 5.10, “). : Configuring a Failover Domain”
Note
The name should be descriptive enough to distinguish its purpose relative to other names used in your cluster.Figure 5.10.
: Configuring a Failover Domain - Click thedrop-down box and select the members for this failover domain.
- To restrict failover to members in this failover domain, click (check) thecheckbox. (With checked, services assigned to this failover domain fail over only to nodes in this failover domain.)
- To prioritize the order in which the members in the failover domain assume control of a failed cluster service, follow these steps:
- Click (check) the Figure 5.11, “). Clicking : Adjusting Priority” causes the column to be displayed next to the column.checkbox (
Figure 5.11.
: Adjusting Priority - For each node that requires a priority adjustment, click the node listed in thecolumns and adjust priority by clicking one of the arrows. Priority is indicated by the position in the column and the value in the column. The node priorities are listed highest to lowest, with the highest priority node at the top of the column (having the lowest number).
- Clickto create the domain.
- At the Cluster Configuration Tool, perform one of the following actions depending on whether the configuration is for a new cluster or for one that is operational and running:
- New cluster — If this is a new cluster, choose=> to save the changes to the cluster configuration.
- Running cluster — If this cluster is operational and running, and you want to propagate the change immediately, click thebutton. Clicking automatically saves the configuration change. If you do not want to propagate the change immediately, choose => to save the changes to the cluster configuration.
5.6.2. Removing a Failover Domain
- At the left frame of the the Cluster Configuration Tool, click the failover domain that you want to delete (listed under ).
- At the bottom of the right frame (labeled Cluster Configuration Tool.), click the button. Clicking the button causes a warning dialog box do be displayed asking if you want to remove the failover domain. Confirm that the failover domain identified in the warning dialog box is the one you want to delete and click . Clicking causes the failover domain to be removed from the list of failover domains under in the left frame of the
- At the Cluster Configuration Tool, perform one of the following actions depending on whether the configuration is for a new cluster or for one that is operational and running:
- New cluster — If this is a new cluster, choose=> to save the changes to the cluster configuration.
- Running cluster — If this cluster is operational and running, and you want to propagate the change immediately, click thebutton. Clicking automatically saves the configuration change. If you do not want to propagate the change immediately, choose => to save the changes to the cluster configuration.
5.6.3. Removing a Member from a Failover Domain
- At the left frame of the the Cluster Configuration Tool, click the failover domain that you want to change (listed under ).
- At the bottom of the right frame (labeled Failover Domain Configuration dialog box to be displayed (Figure 5.10, “). : Configuring a Failover Domain”), click the button. Clicking the button causes the
- At the Failover Domain Configuration dialog box, in the column, click the node name that you want to delete from the failover domain and click the button. Clicking removes the node from the column. Repeat this step for each node that is to be deleted from the failover domain. (Nodes must be deleted one at a time.)
- When finished, click.
- At the Cluster Configuration Tool, perform one of the following actions depending on whether the configuration is for a new cluster or for one that is operational and running:
- New cluster — If this is a new cluster, choose=> to save the changes to the cluster configuration.
- Running cluster — If this cluster is operational and running, and you want to propagate the change immediately, click thebutton. Clicking automatically saves the configuration change. If you do not want to propagate the change immediately, choose => to save the changes to the cluster configuration.
5.7. Adding Cluster Resources
- On the Cluster Configuration Tool, click the button. Clicking the button causes the Resource Configuration dialog box to be displayed.property of the
- At the Resource Configuration dialog box, under , click the drop-down box. At the drop-down box, select a resource to configure. The resource options are described as follows:
- GFS
- Resource Configuration dialog box. If you need to assign a file system ID explicitly, specify it in this field.— When creating a new file system resource, you can leave this field blank. Leaving the field blank causes a file system ID to be assigned automatically after you click at thenot unmounted at service tear-down unless this box is checked.checkbox — If checked, forces the file system to unmount. The default setting is unchecked. kills all processes using the mount point to free up the mount when it tries to unmount. With GFS resources, the mount point is
- File System
- Resource Configuration dialog box. If you need to assign a file system ID explicitly, specify it in this field.— When creating a new file system resource, you can leave this field blank. Leaving the field blank causes a file system ID to be assigned automatically after you click at theCheckboxes — Specify mount and unmount actions when a service is stopped (for example, when disabling or relocating a service):
fsck
to be run on the file system before mounting it. The default setting is unchecked.
- IP Address
- NFS Mount
- nfs(5) man page.— Mount options. For more information, refer to the
- NFS Client
- exports(5) man page, General Options— Additional client access rights. For more information, refer to the
- NFS Export
- Script
/etc/init.d/userscript
)- Samba Service
Note
When creating or editing a cluster service, connect a Samba-service resource directly to the service, not to a resource within a service. That is, at the Service Management dialog box, use either or ; do not use or .
- When finished, click.
- Choose=> to save the change to the
/etc/cluster/cluster.conf
configuration file.
5.8. Adding a Cluster Service to the Cluster
- At the left frame, click.
- At the bottom of the right frame (labeled Add a Service dialog box to be displayed.), click the button. Clicking causes the
- At the Add a Service dialog box, type the name of the service in the text box and click . Clicking causes the Service Management dialog box to be displayed (refer to Figure 5.12, “Adding a Cluster Service”).
Note
Use a descriptive name that clearly distinguishes the service from other services in the cluster.Figure 5.12. Adding a Cluster Service
- If you want to restrict the members on which this cluster service is able to run, choose a failover domain from the Section 5.6, “Configuring a Failover Domain” for instructions on how to configure a failover domain.)drop-down box. (Refer to
- not checked, the service must be started manually any time the cluster comes up from stopped state.checkbox — This is checked by default. If is checked, the service is started automatically when a cluster is started and running. If is
- no other services running on them. For example, for a very busy web server that is clustered for high availability, it would would be advisable to keep that service on a node alone with no other services competing for his resources — that is, checked. On the other hand, services that consume few resources (like NFS and Samba), can run together on the same node without little concern over contention for resources. For those types of services you can leave the unchecked.checkbox — This sets a policy wherein the service only runs on nodes that have
Note
Circumstances that require enablingare rare. Enabling can render a service offline if the node it is running on fails and no other nodes are empty. - Select a recovery policy to specify how the resource manager should recover from a service failure. At the upper right of the Service Management dialog box, there are three options available:
- Click the Section 5.7, “Adding Cluster Resources”.button and choose the a resource listed that you have configured in
Note
If you are adding a Samba-service resource, connect a Samba-service resource directly to the service, not to a resource within a service. That is, at the Service Management dialog box, use either or ; do not use or . - If needed, you may also create a private resource that you can create that becomes a subordinate resource by clicking on the Section 5.7, “Adding Cluster Resources”. The private resource will appear as a child to the shared resource to which you associated with the shared resource. Click the triangle icon next to the shared resource to display any private resources associated.button. The process is the same as creating a shared resource described in
- When finished, click.
- Choose=> to save the changes to the cluster configuration.
Note
/sbin/ip addr list
command on a cluster node. The following output shows the /sbin/ip addr list
command executed on a node running a cluster service:
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP> mtu 1356 qdisc pfifo_fast qlen 1000 link/ether 00:05:5d:9a:d8:91 brd ff:ff:ff:ff:ff:ff inet 10.11.4.31/22 brd 10.11.7.255 scope global eth0 inet6 fe80::205:5dff:fe9a:d891/64 scope link inet 10.11.4.240/22 scope global secondary eth0 valid_lft forever preferred_lft forever
5.9. Propagating The Configuration File: New Cluster
- Log in to the node where you created the configuration file.
- Using the
scp
command, copy the/etc/cluster/cluster.conf
file to all nodes in the cluster.Note
Propagating the cluster configuration file this way is necessary for the first time a cluster is created. Once a cluster is installed and running, the cluster configuration file is propagated using the Red Hat cluster management GUI Section 6.3, “Modifying the Cluster Configuration”.button. For more information about propagating the cluster configuration using the GUI button, refer to
5.10. Starting the Cluster Software
service ccsd start
service cman start
(orservice lock_gulmd start
for GULM clusters)service fenced start
(DLM clusters only)service clvmd start
, if CLVM has been used to create clustered volumesNote
Shared storage for use in Red Hat Cluster Suite requires that you be running the cluster logical volume manager daemon (clvmd
) or the High Availability Logical Volume Management agents (HA-LVM). If you are not able to use either theclvmd
daemon or HA-LVM for operational reasons or because you do not have the correct entitlements, you must not use single-instance LVM on the shared disk as this may result in data corruption. If you have any concerns please contact your Red Hat service representative.service gfs start
, if you are using Red Hat GFSservice rgmanager start
, if the cluster is running high-availability services (rgmanager
)- Start the Red Hat Cluster Suite management GUI. At the Cluster Configuration Tool tab, verify that the configuration is correct. At the Cluster Status Tool tab verify that the nodes and services are running as expected.
Chapter 6. Managing Red Hat Cluster With system-config-cluster
6.1. Starting and Stopping the Cluster Software
service ccsd start
service cman start
(orservice lock_gulmd start
for GULM clusters)service fenced start
(DLM clusters only)service clvmd start
, if CLVM has been used to create clustered volumesservice gfs start
, if you are using Red Hat GFSservice rgmanager start
, if the cluster is running high-availability services (rgmanager
)
service rgmanager stop
, if the cluster is running high-availability services (rgmanager
)service gfs stop
, if you are using Red Hat GFSservice clvmd stop
, if CLVM has been used to create clustered volumesservice fenced stop
(DLM clusters only)service cman stop
(orservice lock_gulmd stop
for GULM clusters)service ccsd stop
6.2. Managing High-Availability Services
Figure 6.1. Cluster Status Tool
Members Status | Description | ||
---|---|---|---|
| |||
The node is unable to participate as a cluster member. The most basic cluster software is not running on the node. |
Services Status | Description |
---|---|
The service resources are configured and available on the cluster system that owns the service. | |
The service has failed on a member and is pending start on another member. | |
The service has been disabled, and does not have an assigned owner. A disabled service is never restarted automatically by the cluster. | |
The service is not running; it is waiting for a member capable of starting the service. A service remains in the stopped state if autostart is disabled. | |
The service has failed to start on the cluster and cannot successfully stop the service. A failed service is never restarted automatically by the cluster. |
6.3. Modifying the Cluster Configuration
/etc/cluster/cluster.conf
), use the Cluster Configuration Tool. For more information about using the Cluster Configuration Tool, refer to Chapter 5, Configuring Red Hat Cluster With system-config-cluster
.
Warning
/etc/cluster/cluster.conf
file without guidance from an authorized Red Hat representative or unless you fully understand the consequences of editing the /etc/cluster/cluster.conf
file manually.
Important
1
. For more information about using the Cluster Configuration Tool, refer to Chapter 5, Configuring Red Hat Cluster With system-config-cluster
.
Important
- Make changes to cluster elements (for example, create a service).
- Propagate the updated configuration file throughout the cluster by clicking.
Note
The Cluster Configuration Tool does not display the button if the cluster is new and has not been started yet, or if the node from which you are running the Cluster Configuration Tool is not a member of the cluster. If the button is not displayed, you can still use the Cluster Configuration Tool; however, you cannot propagate the configuration. You can still save the configuration file. For information about using the Cluster Configuration Tool for a new cluster configuration, refer to Chapter 5, Configuring Red Hat Cluster Withsystem-config-cluster
. - Clicking Warning dialog box to be displayed. Click to save and propagate the configuration.causes a
- Clicking Information dialog box to be displayed, confirming that the current configuration has been propagated to the cluster. Click .causes an
- Click thetab and verify that the changes have been propagated to the cluster members.
6.4. Backing Up and Restoring the Cluster Database
/etc/cluster/cluster.conf.bak.1
, /etc/cluster/cluster.conf.bak.2
, and /etc/cluster/cluster.conf.bak.3
. The backup file /etc/cluster/cluster.conf.bak.1
is the newest backup, /etc/cluster/cluster.conf.bak.2
is the second newest backup, and /etc/cluster/cluster.conf.bak.3
is the third newest backup.
- At the Cluster Configuration Tool tab of the Red Hat Cluster Suite management GUI, click => .
- Clicking system-config-cluster dialog box to be displayed.causes the
- At the the system-config-cluster dialog box, select a backup file (for example,
/etc/cluster/cluster.conf.bak.1
). Verify the file selection in the box and click . - Click=> .
- Clicking system-config-cluster dialog box to be displayed.=> causes the
- At the the system-config-cluster dialog box, select
/etc/cluster/cluster.conf
and click . (Verify the file selection in the box.) - Clicking Information dialog box to be displayed. At that dialog box, click .causes an
- Propagate the updated configuration file throughout the cluster by clicking.
Note
The Cluster Configuration Tool does not display the button if the cluster is new and has not been started yet, or if the node from which you are running the Cluster Configuration Tool is not a member of the cluster. If the button is not displayed, you can still use the Cluster Configuration Tool; however, you cannot propagate the configuration. You can still save the configuration file. For information about using the Cluster Configuration Tool for a new cluster configuration, refer to Chapter 5, Configuring Red Hat Cluster Withsystem-config-cluster
. - Clicking Warning dialog box to be displayed. Click to propagate the configuration.causes a
- Click thetab and verify that the changes have been propagated to the cluster members.
6.5. Disabling the Cluster Software
/sbin/chkconfig
command to stop the member from joining the cluster at boot-up as follows:
#chkconfig --level 2345 rgmanager off
#chkconfig --level 2345 gfs off
#chkconfig --level 2345 clvmd off
#chkconfig --level 2345 fenced off
#chkconfig --level 2345 lock_gulmd off
#chkconfig --level 2345 cman off
#chkconfig --level 2345 ccsd off
#chkconfig --level 2345 rgmanager on
#chkconfig --level 2345 gfs on
#chkconfig --level 2345 clvmd on
#chkconfig --level 2345 fenced on
#chkconfig --level 2345 lock_gulmd on
#chkconfig --level 2345 cman on
#chkconfig --level 2345 ccsd on
service ccsd start
service cman start
(orservice lock_gulmd start
for GULM clusters)service fenced start
(DLM clusters only)service clvmd start
, if CLVM has been used to create clustered volumesservice gfs start
, if you are using Red Hat GFSservice rgmanager start
, if the cluster is running high-availability services (rgmanager
)
Appendix A. Example of Setting Up Apache HTTP Server
Note
system-config-cluster
). You can use comparable Conga functions to make an Apache HTTP Server highly available on a Red Hat Cluster.
A.1. Apache HTTP Server Setup Overview
httpd
RPM package on all cluster nodes (or on nodes in the failover domain, if used) and configuring a shared GFS shared resource for the Web content.
# chkconfig --del httpd
httpd
daemon, the cluster infrastructure initializes the service on the active cluster node. This ensures that the corresponding IP address and file system mounts are active on only one cluster node at a time.
httpd
service, a floating IP address must be assigned to the service so that the IP address will transfer from one cluster node to another in the event of failover or service relocation. The cluster infrastructure binds this IP address to the network interface on the cluster system that is currently running the Apache HTTP Server. This IP address ensures that the cluster node running httpd
is transparent to the clients accessing the service.
httpd
service is started and stopped. This prevents the cluster systems from accessing the same data simultaneously, which may result in data corruption. Therefore, do not include the file systems in the /etc/fstab
file.
A.3. Installing and Configuring the Apache HTTP Server
httpd
RPM package. For example:
rpm -Uvh httpd-<version>.<arch>.rpm
- Edit the
/etc/httpd/conf/httpd.conf
configuration file and customize the file according to your configuration. For example:- Specify the directory that contains the HTML files. Also specify this mount point when adding the service to the cluster configuration. It is only required to change this field if the mount point for the web site's content differs from the default setting of
/var/www/html/
. For example:DocumentRoot "/mnt/httpdservice/html"
- Specify a unique IP address to which the service will listen for requests. For example:
Listen 192.168.1.100:80
This IP address then must be configured as a cluster resource for the service using the Cluster Configuration Tool. - If the script directory resides in a non-standard location, specify the directory that contains the CGI programs. For example:
ScriptAlias /cgi-bin/ "/mnt/httpdservice/cgi-bin/"
- Specify the path that was used in the previous step, and set the access permissions to default to that directory. For example:
<Directory /mnt/httpdservice/cgi-bin"> AllowOverride None Options None Order allow,deny Allow from all </Directory>
Additional changes may need to be made to tune the Apache HTTP Server or add module functionality. For information on setting up other options, refer to the Red Hat Enterprise Linux System Administration Guide and the Red Hat Enterprise Linux Reference Guide.
- The standard Apache HTTP Server start script,
/etc/rc.d/init.d/httpd
is also used within the cluster framework to start and stop the Apache HTTP Server on the active cluster node. Accordingly, when configuring the service, specify this script by adding it as a resource in the Cluster Configuration Tool. - Copy the configuration file over to the other nodes of the cluster (or nodes of the failover domain, if configured).
httpd-domain
was created for this service.
- Add the init script for the Apache HTTP Server service.
- Select the Resources Configuration properties dialog box is displayed.tab and click . The
- Selectform the drop down menu.
- Enter ato be associated with the Apache HTTP Server service.
- Specify the path to the Apache HTTP Server init script (for example,
/etc/rc.d/init.d/httpd
) in the field. - Click.
- Add a device for the Apache HTTP Server content files and/or custom scripts.
- Click.
- In the Resource Configuration dialog, select from the drop-down menu.
- Enter thefor the resource (for example,
httpd-content
. - Choosefrom the drop-down menu.
- Enter the mount point in thefield (for example,
/var/www/html/
). - Enter the device special file name in thefield (for example,
/dev/sda3
).
- Add an IP address for the Apache HTTP Server service.
- Click.
- Choosefrom the drop-down menu.
- Enter theto be associated with the Apache HTTP Server service.
- Make sure that thecheckbox is left checked.
- Click.
- Click theproperty.
- Create the Apache HTTP Server service.
- Click Add a Service dialog.. Type a for the service in the
- In the Service Management dialog, select a from the drop-down menu or leave it as .
- Click thebutton. From the available list, choose each resource that you created in the previous steps. Repeat this step until all resources have been added.
- Click.
- Choose=> to save your changes.
Appendix B. Fence Device Parameters
Note
/etc/cluster/cluster.conf
).
Field | Description |
---|---|
Name | A name for the APC device connected to the cluster. |
IP Address | The IP address assigned to the device. |
Login | The login name used to access the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Port | The switch outlet number. |
Switch (optional) | The switch number for the APC switch that connects to the node when you have multiple daisy-chained switches. |
Use SSH | (Rhel 4.8 and later) Indicates that system will use SSH to access the device. |
Field | Description |
---|---|
Name | A name for the Brocade device connected to the cluster. |
IP Address | The IP address assigned to the device. |
Login | The login name used to access the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Port | The switch outlet number. |
Field | Description |
---|---|
Name | A name for the Bull PAP system connected to the cluster. |
IP Address | The IP address assigned to the PAP console. |
Login | The login name used to access the PAP console. |
Password | The password used to authenticate the connection to the PAP console. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Domain | Domain of the Bull PAP system to power cycle. |
Field | Description |
---|---|
Name | The name assigned to the DRAC. |
IP Address | The IP address assigned to the DRAC. |
Login | The login name used to access the DRAC. |
Password | The password used to authenticate the connection to the DRAC. |
Module name | (optional) The module name for the DRAC when you have multiple DRAC modules. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Use SSH (DRAC5 only) | (Rhel 4.8 and later) Indicates that system will use SSH to access the device. |
Field | Description |
---|---|
Name | A name for the BladeFrame device connected to the cluster. |
CServer | The hostname (and optionally the username in the form of username@hostname ) assigned to the device. Refer to the fence_egenera(8) man page for more information. |
ESH Path (optional) | The path to the esh command on the cserver (default is /opt/pan- mgr/bin/esh) |
lpan | The logical process area network (LPAN) of the device. |
pserver | The processing blade (pserver) name of the device. |
Field | Description |
---|---|
Name | A name for the RSB to use as a fence device. |
Hostname | The hostname assigned to the device. |
Login | The login name used to access the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Field | Description |
---|---|
Name | A name for the GNBD device used to fence the cluster. Note that the GFS server must be accessed via GNBD for cluster node fencing support. |
Server | The hostname of the server to fence the client from, in either IP address or hostname form. For multiple hostnames, separate each hostname with a space. |
IP address | The cluster name of the node to be fenced. Refer to the fence_gnbd (8) man page for more information. |
Field | Description |
---|---|
Name | A name for the server with HP iLO support. |
Hostname | The hostname assigned to the device. |
Login | The login name used to access the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Use SSH | (Rhel 4.8 and later) Indicates that system will use SSH to access the device. |
Field | Description |
---|---|
Name | A name for the IBM BladeCenter device connected to the cluster. |
IP Address | The IP address assigned to the device. |
Login | The login name used to access the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Blade | The blade of the device. |
Use SSH | (Rhel 4.8 and later) Indicates that system will use SSH to access the device. |
Field | Description |
---|---|
Name | A name for the RSA device connected to the cluster. |
Hostname | The hostname assigned to the device. |
Login | The login name used to access the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Field | Description |
---|---|
Name | A name for the IPMI LAN device connected to the cluster. |
IP Address | The IP address assigned to the IPMI port. |
Login | The login name of a user capable of issuing power on/off commands to the given IPMI port. |
Password | The password used to authenticate the connection to the IPMI port. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Authentication Type | none , password , md2 , or md5 |
Use Lanplus | True or 1 . If blank, then value is False . |
Field | Description |
---|---|
Name | A name for the LPAR to use as a fence device. |
Hostname | The hostname assigned to the device. |
Login | The login name used to access the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Partition | Name of LPAR partition to use as a fence device when there are multiple LPARs. |
hmc-version | Version 3 or 4. A value of 4 is the default. |
Managed | Name of the managed system. |
Field | Description |
---|---|
Name | A name to assign the Manual fencing agent. Refer to the fence_manual (8) man page for more information. |
Warning
Field | Description |
---|---|
Name | A name for the McData device connected to the cluster. |
IP Address | The IP address assigned to the device. |
Login | The login name used to access the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Port | The switch outlet number. |
Field | Description |
---|---|
Name | A name for the SANBox2 device connected to the cluster. |
IP Address | The IP address assigned to the device. |
Login | The login name used to access the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Port | The switch outlet number. |
Field | Description |
---|---|
Name | A name for the WTI RPS-10 power switch connected to the cluster. |
Device Name | The device name of the device the switch is connected to on the controlling host (for example, /dev/ttys2 ). |
Port | The switch outlet number. |
Field | Description |
---|---|
Name | A name for the SCSI fence device. |
Node name | Name of the node to be fenced. Refer to the fence_scsi (8) man page for more information. |
Field | Description |
---|---|
Name | Name of the virtual machine fencing device. |
Domain | Unique domain name of the guest to be fenced. |
Field | Description |
---|---|
Name | A name for the Vixel switch connected to the cluster. |
IP Address | The IP address assigned to the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Port | The switch outlet number. |
Field | Description |
---|---|
Name | A name for the WTI power switch connected to the cluster. |
IP Address | The IP address assigned to the device. |
Password | The password used to authenticate the connection to the device. |
Password Script (optional) | The script that supplies a password for access to the fence device. Using this supersedes the | parameter.
Port | The switch outlet number. |
Use SSH | (Rhel 4.8 and later) Indicates that system will use SSH to access the device. |
Appendix C. Revision History
Revision History | |||
---|---|---|---|
Revision 1.0-10.400 | 2013-10-31 | ||
| |||
Revision 1.0-10 | 2012-07-18 | ||
| |||
Revision 1.0-0 | Mon Apr 13 2009 | ||
|
Index
A
- ACPI
- Apache HTTP Server
- httpd.conf, Installing and Configuring the Apache HTTP Server
- setting up service, Example of Setting Up Apache HTTP Server
C
- cluster
- administration, Before Configuring a Red Hat Cluster, Managing Red Hat Cluster With Conga, Managing Red Hat Cluster With system-config-cluster
- diagnosing and correcting problems, Diagnosing and Correcting Problems in a Cluster, Diagnosing and Correcting Problems in a Cluster
- disabling the cluster software, Disabling the Cluster Software
- displaying status, Cluster Status Tool, Managing High-Availability Services
- managing node, Managing Cluster Nodes
- starting, Starting the Cluster Software
- starting, stopping, restarting, and deleting, Starting, Stopping, and Deleting Clusters
- cluster administration, Before Configuring a Red Hat Cluster, Managing Red Hat Cluster With Conga, Managing Red Hat Cluster With system-config-cluster
- backing up the cluster database, Backing Up and Restoring the Cluster Database
- compatible hardware, Compatible Hardware
- configuring ACPI, Configuring ACPI For Use with Integrated Fence Devices
- configuring iptables, Enabling IP Ports
- configuring max_luns, Configuring max_luns
- Conga considerations, Considerations for Using Conga
- considerations for using qdisk, Considerations for Using Quorum Disk
- considerations for using quorum disk, Considerations for Using Quorum Disk
- diagnosing and correcting problems in a cluster, Diagnosing and Correcting Problems in a Cluster, Diagnosing and Correcting Problems in a Cluster
- disabling the cluster software, Disabling the Cluster Software
- displaying cluster and service status, Cluster Status Tool, Managing High-Availability Services
- enabling IP ports, Enabling IP Ports
- general considerations, General Configuration Considerations
- managing cluster node, Managing Cluster Nodes
- managing high-availability services, Managing High-Availability Services
- modifying the cluster configuration, Modifying the Cluster Configuration
- restoring the cluster database, Backing Up and Restoring the Cluster Database
- SELinux, Red Hat Cluster Suite and SELinux
- starting and stopping the cluster software, Starting and Stopping the Cluster Software
- starting, stopping, restarting, and deleting a cluster, Starting, Stopping, and Deleting Clusters
- cluster configuration, Configuring Red Hat Cluster With Conga
- modifying, Modifying the Cluster Configuration
- Cluster Configuration Tool
- accessing, Cluster Configuration Tool
- cluster database
- backing up, Backing Up and Restoring the Cluster Database
- restoring, Backing Up and Restoring the Cluster Database
- cluster service
- displaying status, Cluster Status Tool, Managing High-Availability Services
- cluster service managers
- cluster services, Adding a Cluster Service to the Cluster, Adding a Cluster Service to the Cluster
- (see also adding to the cluster configuration)
- Apache HTTP Server, setting up, Example of Setting Up Apache HTTP Server
- cluster software
- configuration, Configuring Red Hat Cluster With Conga
- disabling, Disabling the Cluster Software
- installation and configuration, Configuring Red Hat Cluster With system-config-cluster
- starting and stopping, Starting and Stopping the Cluster Software
- cluster software installation and configuration, Configuring Red Hat Cluster With system-config-cluster
- cluster storage
- configuration, Configuring Cluster Storage
- command line tools table, Command Line Administration Tools
- configuration file
- propagation of, Propagating The Configuration File: New Cluster
- configuring cluster storage , Configuring Cluster Storage
- Conga
- accessing, Configuring Red Hat Cluster Software
- considerations for cluster administration, Considerations for Using Conga
- overview, Conga
- Conga overview, Conga
F
- feedback, Feedback
G
- general
- considerations for cluster administration, General Configuration Considerations
H
- hardware
- compatible, Compatible Hardware
- HTTP services
- Apache HTTP Server
- httpd.conf, Installing and Configuring the Apache HTTP Server
- setting up, Example of Setting Up Apache HTTP Server
I
- integrated fence devices
- configuring ACPI, Configuring ACPI For Use with Integrated Fence Devices
- introduction, Introduction
- other Red Hat Enterprise Linux documents, Introduction
- IP ports
- enabling, Enabling IP Ports
- iptables
- configuring, Enabling IP Ports
M
- max_luns
- configuring, Configuring max_luns
P
- parameters, fence device, Fence Device Parameters
- power controller connection, configuring, Fence Device Parameters
- power switch, Fence Device Parameters
- (see also power controller)
Q
- qdisk
- considerations for using, Considerations for Using Quorum Disk
- quorum disk
- considerations for using, Considerations for Using Quorum Disk
S
- SELinux
- configuring, Red Hat Cluster Suite and SELinux
- starting the cluster software, Starting the Cluster Software
- System V init, Starting and Stopping the Cluster Software
T
- table
- command line tools, Command Line Administration Tools
- tables
- power controller connection, configuring, Fence Device Parameters
- troubleshooting
- diagnosing and correcting problems in a cluster, Diagnosing and Correcting Problems in a Cluster, Diagnosing and Correcting Problems in a Cluster