3.2. Missing Node Host

When you run the oo-mco ping command on the broker host, all node hosts should be listed in the output. Although applications on an unlisted node host can continue to operate without problems, the unlisted node hosts are not controlled by the broker host.

Node hosts may not respond to the oo-mco ping command if the clock on the broker host is not synchronized with the clock on the node host. MCollective messages have a TTL of 60 seconds. Therefore, if the clocks are not synchronized the MCollective messages can be dropped, causing communication issues. Verify that the broker host and node host clocks are synchronized, and the ntpd service is enabled. All configured hosts must use the same NTP server.

Clock synchronization issues are logged in the /var/log/openshift/node/ruby193-mcollective.log file on the node host, and could look like the following sample screen output:

W, [2012-08-10T14:27:01.526544 #12179]  WARN -- : runner.rb:62:in `run' Message 8beea9354f9784de939ec5693940d5ce from uid=48@broker.example.com created at 1344622854 is 367 seconds old, TTL is 60

The node host also may not respond to the oo-mco ping command if ActiveMQ on the broker host cannot communicate with MCollective on the node host. Verify that the ruby193-mcollective service is running on the node host, and it can communicate with ActiveMQ on the broker host. If a configuration has been modified recently, use the following command to restart the ruby193-mcollective service:

# service ruby193-mcollective restart