第8章 Troubleshooting resource problems
In case of resource failure, you must investigate the cause and location of the problem, fix the failed resource, and optionally clean up the resource. There are many possible causes of resource failures depending on your deployment, and you must investigate the resource to determine how to fix the problem.
For example, you can check the resource constraints to ensure that the resources are not interrupting each other, and that the resources can connect to each other. You can also examine a Controller node that is fenced more often than other Controller nodes to identify possible communication problems.
8.1. Viewing resource constraints
You can view constraints on how services are launched, including constraints related to where each resource is located, the order in which the resource starts, and whether the resource must be colocated with another resource.
View all resource constraints
On any Controller node, run the pcs constraint show command.
$ sudo pcs constraint show
The following example shows a truncated output from the pcs constraint show command on a Controller node:
Location Constraints:
Resource: galera-bundle
Constraint: location-galera-bundle (resource-discovery=exclusive)
Rule: score=0
Expression: galera-role eq true
[...]
Resource: ip-192.168.24.15
Constraint: location-ip-192.168.24.15 (resource-discovery=exclusive)
Rule: score=0
Expression: haproxy-role eq true
[...]
Resource: my-ipmilan-for-controller-0
Disabled on: overcloud-controller-0 (score:-INFINITY)
Resource: my-ipmilan-for-controller-1
Disabled on: overcloud-controller-1 (score:-INFINITY)
Resource: my-ipmilan-for-controller-2
Disabled on: overcloud-controller-2 (score:-INFINITY)
Ordering Constraints:
start ip-172.16.0.10 then start haproxy-bundle (kind:Optional)
start ip-10.200.0.6 then start haproxy-bundle (kind:Optional)
start ip-172.19.0.10 then start haproxy-bundle (kind:Optional)
start ip-192.168.1.150 then start haproxy-bundle (kind:Optional)
start ip-172.16.0.11 then start haproxy-bundle (kind:Optional)
start ip-172.18.0.10 then start haproxy-bundle (kind:Optional)
Colocation Constraints:
ip-172.16.0.10 with haproxy-bundle (score:INFINITY)
ip-172.18.0.10 with haproxy-bundle (score:INFINITY)
ip-10.200.0.6 with haproxy-bundle (score:INFINITY)
ip-172.19.0.10 with haproxy-bundle (score:INFINITY)
ip-172.16.0.11 with haproxy-bundle (score:INFINITY)
ip-192.168.1.150 with haproxy-bundle (score:INFINITY)This output displays the following main constraint types:
- Location Constraints
Lists the locations to which resources can be assigned:
-
The first constraint defines a rule that sets the galera-bundle resource to run on nodes with the
galera-roleattribute set totrue. -
The second location constraint specifies that the IP resource ip-192.168.24.15 runs only on nodes with the
haproxy-roleattribute set totrue. This means that the cluster associates the IP address with thehaproxyservice, which is necessary to make the services reachable. - The third location constraint shows that the ipmilan resource is disabled on each of the Controller nodes.
-
The first constraint defines a rule that sets the galera-bundle resource to run on nodes with the
- Ordering Constraints
Lists the order in which resources can launch. This example shows a constraint that sets the virtual IP address resources IPaddr2 to start before the HAProxy service.
注記Ordering constraints only apply to IP address resources and to HAproxy. Systemd manages all other resources, because services such as Compute are expected to withstand an interruption of a dependent service, such as Galera.
- Colocation Constraints
- Lists which resources must be located together. All virtual IP addresses are linked to the haproxy-bundle resource.
View Galera location constraints
On any Controller node, run the pcs property show command.
$ sudo pcs property showExample output:
Cluster Properties: cluster-infrastructure: corosync cluster-name: tripleo_cluster dc-version: 1.1.16-12.el7_4.5-94ff4df have-watchdog: false redis_REPL_INFO: overcloud-controller-0 stonith-enabled: false Node Attributes: overcloud-controller-0: cinder-volume-role=true galera-role=true haproxy-role=true rabbitmq-role=true redis-role=true rmq-node-attr-last-known-rabbitmq=rabbit@overcloud-controller-0 overcloud-controller-1: cinder-volume-role=true galera-role=true haproxy-role=true rabbitmq-role=true redis-role=true rmq-node-attr-last-known-rabbitmq=rabbit@overcloud-controller-1 overcloud-controller-2: cinder-volume-role=true galera-role=true haproxy-role=true rabbitmq-role=true redis-role=true rmq-node-attr-last-known-rabbitmq=rabbit@overcloud-controller-2
In this output, you can verify that the galera-role attribute is true for all Controller nodes. This means that the galera-bundle resource runs only on these nodes. The same concept applies to the other attributes associated with the other location constraints.
