4.4.4. Fencing delays
In two-node clusters, simultaneous communication loss can cause nodes to fence each other, shutting down the entire cluster. Configure a fencing delay to prevent this race condition. Delays are unnecessary in larger clusters, where quorum determines fencing authority.
You can set different types of fencing delays, depending on your system requirements.
static fencing delays
A static fencing delay is a fixed, predetermined delay. Setting a static delay on one node makes that node more likely to be fenced because it increases the chances that the other node will initiate fencing first after detecting lost communication. In an active/passive cluster, setting a delay on a passive node makes it more likely that the passive node will be fenced when communication breaks down. You configure a static delay by using the
pcmk_delay_baseinstance attribute. To configure different delays, set this attribute on a separate fence device for each node. If you use a single fence device for all nodes, you can set this attribute to a host map, for example,node1:0s;node2:5s.dynamic fencing delays
A dynamic fencing delay is random. It can vary and is determined at the time fencing is needed. You configure a random delay and specify a maximum value for the combined base delay and random delay with the
pcmk_delay_maxinstance attribute. When the fencing delay for each node is random, which node is fenced is also random. You might find this feature useful if your cluster is configured with a single fence device for all nodes in an active/active design.priority fencing delays
A priority fencing delay is based on active resource priorities. If all resources have the same priority, the node with the fewest resources running is the node that gets fenced. In most cases, you use only one delay-related parameter, but it is possible to combine them. Combining delay-related parameters adds the priority values for the resources together to create a total delay. You configure a priority fencing delay with the
priority-fencing-delaycluster property. You may find this feature useful in an active/active cluster design because it can make the node running the fewest resources more likely to be fenced when communication between the nodes is lost.
The pcmk_delay_base instance attribute
The pcmk_delay_base instance attribute specifies a base delay value for fencing.
When you set the pcmk_delay_max instance attribute in addition to the pcmk_delay_base instance attribute, the overall delay combines a random delay value with this static delay, keeping the sum below the maximum delay. When you set pcmk_delay_base but do not set pcmk_delay_max, there is no random component to the delay, and the delay is the value of pcmk_delay_base.
You can specify different values for different nodes with the pcmk_delay_base instance attribute. This allows a single fence device to be used in a two-node cluster, with a different delay for each node. You do not need to configure two separate devices to use separate delays. To specify different values for different nodes, you map the host names to the delay value for that node using a similar syntax to pcmk_host_map. For example, node1:0;node2:10s would use no delay when fencing node1 and a 10-second delay when fencing node2.
- The
pcmk_delay_maxinstance attribute Use the
pcmk_delay_baseinstance attribute to specify a base delay for fencing actions and the maximum delay, which is the maximum value of the combined base delay and random delay. For example, if the base delay is 3 andpcmk_delay_maxis 10, the random delay will be between 3 and 10.When you set the
pcmk_delay_baseinstance attribute in addition to thepcmk_delay_maxinstance attribute, the overall delay is derived from a random delay value added to this static delay so that the sum is kept below the maximum delay. When you setpcmk_delay_maxbut do not setpcmk_delay_basethere is no static component to the delay.
The priority-fencing-delay cluster property
Setting the priority-fencing-delay cluster property allows you to configure a two-node cluster so that in a split-brain situation the node with the fewest or least important resources running is the node that gets fenced.
The priority-fencing-delay property can be set to a time duration. The default value for this property is 0 (disabled). If this property is set to a non-zero value, and the priority meta-attribute is configured for at least one resource, then in a split-brain situation the node with the highest combined priority of all resources running on it will be more likely to remain operational. For example, if you set pcs resource defaults update priority=1 and pcs property set priority-fencing-delay=15s and no other priorities are set, then the node running the most resources will be more likely to remain operational because the other node will wait 15 seconds before initiating fencing. If a particular resource is more important than the rest, you can give it a higher priority.
The node running the promoted role of a promotable clone gets an extra 1 point if a priority has been configured for that clone.
Interaction of fencing delays
Setting more than one type of fencing delay yields the following results:
-
Any delay set with the
priority-fencing-delayproperty is added to any delay from thepcmk_delay_baseandpcmk_delay_maxinstance attributes. This behavior allows some delay when both nodes have equal priority, or both nodes need to be fenced for some reason other than node loss, as whenon-fail=fencingis set for a resource monitor operation. When setting these delays in combination, set thepriority-fencing-delayproperty to a value that is significantly greater than the maximum delay frompcmk_delay_baseandpcmk_delay_maxto be sure the prioritized node is preferred. Setting this property to twice this value is always safe. -
Only fencing scheduled by Pacemaker itself observes fencing delays. Fencing scheduled by external code such as
dlm_controldand fencing implemented by thepcs stonith fencecommand do not provide the necessary information to the fence device. -
Some individual fence agents implement a delay parameter, with a name determined by the agent. This delay is independent of delays configured with a
pcmk_delay_*instance attribute. If both of these delays are configured, they are added together and would generally not be used in conjunction.