红帽全球峰会此内容没有您所选择的语言版本。
Monitoring Ceph for Ubuntu with Nagios
Monitoring Ceph for Ubuntu with Nagios Core.
Abstract
Chapter 1. Introduction
Nagios Core is an open source solution for monitoring hosts. Large Ceph Storage clusters benefit from distributed monitoring systems such as Nagios Core that check each host in a cluster, including the health of the underlying operating system, as well as the health of the Ceph Storage Cluster daemons.
To deploy Nagios Core with Ceph requires:
- A running Ceph cluster.
- A running Nagios core server.
In lieu of Nagios Core, you may also substitute the more feature rich commercial version, Nagios XI.
Red Hat does not provide the Nagios packages.
Red Hat works with our technology partners to provide this documentation as a service to our customers. However, Red Hat does not provide support for this product. If you need technical assistance for this product, then contact Nagios for support.
Chapter 2. Installing Nagios Core
Installing Nagios Core involves downloading the Nagios Core source code; then, configuring, making and installing it on the host that will run Nagios Core instance.
The following sections describe the process for Ubuntu 16.04 and later releases.
For Ubuntu 16.04, execute all commands as root:
sudo -i
[user@nagios]$ sudo -i2.1. Installing Nagios Prerequisites
Install the prerequisites.
yum install -y httpd php php-cli gcc glibc glibc-common gd gd-devel net-snmp openssl openssl-devel wget unzip
[user@nagios]# yum install -y httpd php php-cli gcc glibc glibc-common gd gd-devel net-snmp openssl openssl-devel wget unzipapt-get install wget build-essential apache2 php apache2-mod-php7.0 openssl libssl-dev gcc make php-gd libgd-dev unzip
[user@nagios]# apt-get install wget build-essential apache2 php apache2-mod-php7.0 openssl libssl-dev gcc make php-gd libgd-dev unzip
Open port 80 for httpd.
firewall-cmd --zone=public --add-port=80/tcp firewall-cmd --zone=public --add-port=80/tcp --permanent
[user@nagios]# firewall-cmd --zone=public --add-port=80/tcp
[user@nagios]# firewall-cmd --zone=public --add-port=80/tcp --permanent
Open port 80 for apache2:
iptables -A INPUT -p tcp --dport 80 -j ACCEPT apt-get install iptables-persistent
[user@nagios]# iptables -A INPUT -p tcp --dport 80 -j ACCEPT
[user@nagios]# apt-get install iptables-persistent2.2. Creating a Nagios User and Group
Create a user and group for Nagios Core.
useradd nagios passwd nagios groupadd nagcmd usermod -a -G nagcmd nagios
[user@nagios]# useradd nagios
[user@nagios]# passwd nagios
[user@nagios]# groupadd nagcmd
[user@nagios]# usermod -a -G nagcmd nagiosThen, execute the following:
usermod -a -G nagcmd apache
[user@nagios]# usermod -a -G nagcmd apacheusermod -a -G nagios,nagcmd www-data
[user@nagios]# usermod -a -G nagios,nagcmd www-data2.3. Download Nagios Source Code and Plug-ins
Download the latest version of Nagios Core and Plug-ins.
wget --inet4-only https://assets.nagios.com/downloads/nagioscore/releases/nagios-4.3.1.tar.gz wget --inet4-only http://www.nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz tar zxf nagios-4.3.1.tar.gz tar zxf nagios-plugins-2.2.1.tar.gz cd nagios-4.3.1
[user@nagios]# wget --inet4-only https://assets.nagios.com/downloads/nagioscore/releases/nagios-4.3.1.tar.gz
[user@nagios]# wget --inet4-only http://www.nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz
[user@nagios]# tar zxf nagios-4.3.1.tar.gz
[user@nagios]# tar zxf nagios-plugins-2.2.1.tar.gz
[user@nagios]# cd nagios-4.3.12.4. Make and Install Nagios Core
To make and install Nagios Core, first run ./configure.
./configure --with-command-group=nagcmd
[user@nagios]# ./configure --with-command-group=nagcmd./configure --with-command-group=nagcmd --with-httpd-conf=/etc/apache2/
[user@nagios]# ./configure --with-command-group=nagcmd --with-httpd-conf=/etc/apache2/
After running ./configure, compile the Nagios Core source code.
make all
[user@nagios]# make allAfter making Nagios Core, install it.
make install make install-init make install-config make install-commandmode make install-webconf
[user@nagios]# make install
[user@nagios]# make install-init
[user@nagios]# make install-config
[user@nagios]# make install-commandmode
[user@nagios]# make install-webconfCopy the event handlers and change their ownership.
cp -R contrib/eventhandlers/ /usr/local/nagios/libexec/ chown -R nagios:nagios /usr/local/nagios/libexec/eventhandlers
[user@nagios]# cp -R contrib/eventhandlers/ /usr/local/nagios/libexec/
[user@nagios]# chown -R nagios:nagios /usr/local/nagios/libexec/eventhandlersFinally, run the pre-flight check.
/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
[user@nagios]# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg2.5. Make and Install Nagios Core Plug-ins
Make and install the Nagios Core plug-ins.
cd ../nagios-plugins-2.2.1 ./configure --with-nagios-user=nagios --with-nagios-group=nagios make make install
[user@nagios]# cd ../nagios-plugins-2.2.1
[user@nagios]# ./configure --with-nagios-user=nagios --with-nagios-group=nagios
[user@nagios]# make
[user@nagios]# make install2.6. Create a Default Nagios Core User
Create a user for the Nagios Core user interface.
sudo htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
[user@nagios]$ sudo htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
If adding a user other than nagiosadmin, ensure the /usr/local/nagios/etc/cgi.cfg file gets updated with the username too.
Also modify the /usr/local/nagios/etc/objects/contacts.cfg file with the user name, full name and email address as needed.
2.7. Start Nagios
Add Nagios Core as a service and enable it. Then start the Nagios Core daemon and Apache.
chkconfig --add nagios chkconfig --level 35 nagios on systemctl start nagios systemctl enable httpd systemctl start httpd
[user@nagios]# chkconfig --add nagios
[user@nagios]# chkconfig --level 35 nagios on
[user@nagios]# systemctl start nagios
[user@nagios]# systemctl enable httpd
[user@nagios]# systemctl start httpda2ensite nagios a2enmod rewrite cgi sudo cp /etc/init.d/skeleton /etc/init.d/nagios
[user@nagios]# a2ensite nagios
[user@nagios]# a2enmod rewrite cgi
[user@nagios]# sudo cp /etc/init.d/skeleton /etc/init.d/nagiosThen, edit the initialization file:
vi /etc/init.d/nagios
[user@nagios]# vi /etc/init.d/nagiosAdd the following lines:
DESC="Nagios" NAME=nagios DAEMON=/usr/local/nagios/bin/$NAME DAEMON_ARGS="-d /usr/local/nagios/etc/nagios.cfg" PIDFILE=/usr/local/nagios/var/$NAME.lock
DESC="Nagios"
NAME=nagios
DAEMON=/usr/local/nagios/bin/$NAME
DAEMON_ARGS="-d /usr/local/nagios/etc/nagios.cfg"
PIDFILE=/usr/local/nagios/var/$NAME.lockFinally, start Nagios:
systemctl restart apache2 systemctl start nagios
[user@nagios]# systemctl restart apache2
[user@nagios]# systemctl start nagios2.8. Log in to Nagios Core
With Nagios up and running, log in to the web user interface.
http://<ip-address>/nagios
http://<ip-address>/nagiosNagios Core will prompt for a user name and password. Input the login and password of the default Nagios Core user.
Chapter 3. Installing Nagios Remote Plug-in Executor (NRPE)
To monitor Ceph Storage cluster hosts, install Nagios Plug-ins, the Ceph plug-ins and the NRPE add-on to each of the Ceph cluster’s hosts.
For demonstration purposes, this section adds NRPE to a Ceph monitor node with the hostname mon. Repeat the remaining procedures on all Ceph nodes that Nagios should monitor.
3.1. Install Prerequisites
NRPE requires OpenSSL. Install the following libraries first.
Execute the following:
yum install openssl openssl-devel gcc make git
[user@host]# yum install openssl openssl-devel gcc make gitapt install openssl libssl-dev gcc make git
[user@host]# apt install openssl libssl-dev gcc make git3.2. Create a Nagios User
NRPE installation requires a Nagios user. So create the user first.
useradd nagios passwd nagios
[user@mon]# useradd nagios
[user@mon]# passwd nagios3.3. Download, Make and Install the Nagios Plug-ins
Download the latest version of the Nagios plug-ins. Then, make and install them.
wget http://www.nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz tar zxf nagios-plugins-2.2.1.tar.gz cd nagios-plugins-2.2.1 ./configure make make install
[user@mon]# wget http://www.nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz
[user@mon]# tar zxf nagios-plugins-2.2.1.tar.gz
[user@mon]# cd nagios-plugins-2.2.1
[user@mon]# ./configure
[user@mon]# make
[user@mon]# make install3.4. Download, Make and Install the Nagios Ceph Plug-ins
Download the latest verion of the Ceph plug-ins. See https://github.com/valerytschopp/ceph-nagios-plugins for details.
cd ~ git clone --recursive https://github.com/valerytschopp/ceph-nagios-plugins.git cd ceph-nagios-plugins make dist make install
[user@mon]# cd ~
[user@mon]# git clone --recursive https://github.com/valerytschopp/ceph-nagios-plugins.git
[user@mon]# cd ceph-nagios-plugins
[user@mon]# make dist
[user@mon]# make install3.5. Install xinetd
NRPE uses xinetd for communication. Install it before installing the NRPE module. Execute the following:
yum install xinetd
[user@mon]# yum install xinetdapt install xinetd
[user@mon]# apt install xinetd3.6. Download, Make and Install Nagios NRPE
cd ~ wget https://github.com/NagiosEnterprises/nrpe/releases/download/nrpe-3.1.0/nrpe-3.1.0.tar.gz tar xvfz nrpe-3.1.0.tar.gz cd nrpe-3.1.0 ./configure make all make install-groups-users make install make install-config make install-init
[user@mon]# cd ~
[user@mon]# wget https://github.com/NagiosEnterprises/nrpe/releases/download/nrpe-3.1.0/nrpe-3.1.0.tar.gz
[user@mon]# tar xvfz nrpe-3.1.0.tar.gz
[user@mon]# cd nrpe-3.1.0
[user@mon]# ./configure
[user@mon]# make all
[user@mon]# make install-groups-users
[user@mon]# make install
[user@mon]# make install-config
[user@mon]# make install-init
Then, add nrpe 5666/tcp to the /etc/services file.
3.7. Enable, Restart and Reload xinetd.
systemctl enable xinetd systemctl restart xinetd systemctl reload xinetd
[user@mon]# systemctl enable xinetd
[user@mon]# systemctl restart xinetd
[user@mon]# systemctl reload xinetd3.8. Enable and Start NRPE
systemctl enable nrpe systemctl start nrpe
[user@mon]# systemctl enable nrpe
[user@mon]# systemctl start nrpe3.9. Open Port 5666
Open port 5666 to allow communication with NRPE.
firewall-cmd --zone=public --add-port=5666/tcp firewall-cmd --zone=public --add-port=5666/tcp --permanent
[user@mon]# firewall-cmd --zone=public --add-port=5666/tcp
[user@mon]# firewall-cmd --zone=public --add-port=5666/tcp --permanentiptables -A INPUT -p tcp --dport 5666 -j ACCEPT apt-get install iptables-persistent
[user@mon]# iptables -A INPUT -p tcp --dport 5666 -j ACCEPT
[user@mon]# apt-get install iptables-persistent3.10. Add the Nagios Core Server IP Address
In order for the Nagios Core server to access NRPE on a remote machine, the remote machine’s xinetd and NRPE configurations must be updated with the IP address of the Nagios Core server.
Edit the xinetd configuration with the Nagios server’s IP address.
vim /etc/xinetd.d/nrpe
[user@mon]# vim /etc/xinetd.d/nrpedefault: off description: NRPE (Nagios Remote Plugin Executor)
# default: off
# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
disable = yes
socket_type = stream
port = 5666
wait = no
user = nagios
group = nagios
server = /usr/local/nagios/bin/nrpe
server_args = -c /usr/local/nagios/etc/nrpe.cfg --inetd
only_from = 127.0.0.1,<ip-address-of-nagios-core>
log_on_success =
}
Add the IP address of the Nagios Core server to the only_from setting. Then, restart xinetd.
systemctl restart xinetd
[user@mon]# systemctl restart xinetdEdit the NRPE configuration with the Nagios server’s IP address.
vim /usr/local/nagios/etc/nrpe.cfg
[user@mon]# vim /usr/local/nagios/etc/nrpe.cfgallowed_hosts=127.0.0.1,<ip-address-of-nagios-core>
allowed_hosts=127.0.0.1,<ip-address-of-nagios-core>
Add the IP address of the Nagios Core server to the allowed_hosts setting. Then, restart nrpe.
systemctl restart nrpe
[user@mon]# systemctl restart nrpe3.11. Test the Installation
Ensure that the make and install procedures worked.
/usr/local/nagios/libexec/check_nrpe -H localhost
[user@host]# /usr/local/nagios/libexec/check_nrpe -H localhost
The check should echo NRPE v3.1.0-rc1 if it is working correctly.
Chapter 4. Configure the Nagios Core Server
After configuring NRPE on a Ceph host, configure the Nagios Core Server to recognize and monitor the host.
4.1. Install the check_nrpe Plug-in
cd ~ wget https://github.com/NagiosEnterprises/nrpe/releases/download/nrpe-3.1.0/nrpe-3.1.0.tar.gz tar xvfz nrpe-3.1.0.tar.gz cd nrpe-3.1.0 ./configure make check_nrpe make install-plugin
[user@nagios]# cd ~
[user@nagios]# wget https://github.com/NagiosEnterprises/nrpe/releases/download/nrpe-3.1.0/nrpe-3.1.0.tar.gz
[user@nagios]# tar xvfz nrpe-3.1.0.tar.gz
[user@nagios]# cd nrpe-3.1.0
[user@nagios]# ./configure
[user@nagios]# make check_nrpe
[user@nagios]# make install-plugin4.2. Check to Ensure Connectivity
Ensure that the make and install procedures worked and that there is connectivity between the Nagios Core server and the remote host containing NRPE.
/usr/local/nagios/libexec/check_nrpe -H <IP-address-of-remote-host>
[user@nagios]# /usr/local/nagios/libexec/check_nrpe -H <IP-address-of-remote-host>
It should echo NRPE v3.1.0-rc1 if it is working correctly.
4.3. Create a Configuration for the Remote Host
cd /usr/local/nagios/etc/objects cp localhost.cfg mon.cfg
[user@nagios]# cd /usr/local/nagios/etc/objects
[user@nagios]# cp localhost.cfg mon.cfg
Replace localhost with the hostname of the remote host, and the loopback IP address with the IP address of the remote host. Finally, delete or comment out the Host Group definition.
Change the file ownership to nagios.
chown nagios:nagios mon.cfg
[user@nagios]# chown nagios:nagios mon.cfg
Add a cfg_file= reference to the mon.cfg file in /usr/local/nagios/etc/nagios.cfg.
vim /usr/local/nagios/etc/nagios.cfg
[user@nagios]# vim /usr/local/nagios/etc/nagios.cfgFor example:
cfg_file=/usr/local/nagios/etc/objects/mon.cfg
cfg_file=/usr/local/nagios/etc/objects/mon.cfgThen, restart the Nagios server.
systemctl restart nagios
[user@nagios]# systemctl restart nagiosChapter 5. Configure Ceph Plug-ins
There are some open source Ceph plug-ins provided at https://github.com/valerytschopp/ceph-nagios-plugins. They include:
-
check_ceph_df: This plug-in outputs messages related toceph dffor the entire cluster or for individual pools. This plug-in only needs to run on Ceph monitor hosts. Multiple instances may be configured to monitor individual pools. -
check_ceph_health: This plug-in outputs the result ofceph health. This plug-in only needs to run on Ceph monitor hosts. -
check_ceph_mon: This plug-in checks a single monitor and returnsOKif the monitor is up and running orWARNif it is down or missing. This plug-in only needs to run on Ceph monitor hosts. -
check_ceph_osd: This plug-in checks an OSD host or a single OSD and returnsOKif the OSD is up and running orWARNif it is down. This plug-in only needs to run on Ceph OSD hosts. -
check_ceph_rgw: This plug-in checks a single Ceph Object Gateway and returnsOKand the buckets and data usage if it is up and running orWARNif it is down or missing. This plug-in only needs to run on Ceph Object Gateway hosts. -
check_ceph_mds: This plug-in checks a single metadata server and returnsOKif it is up and running,WARNif it is laggy andErrorif it is down or missing. This plug-in only needs to run on Ceph metadata server hosts. These plug-ins get installed on the appropriate Ceph hosts. The following sections describe how to configure theceph healthplug-in on a monitor host.
5.1. Create Keyring and Key
Log in to the monitor server and create a Ceph key and keyring for Nagios.
ssh mon cd /etc/ceph ceph auth get-or-create client.nagios mon 'allow r' > client.nagios.keyring
[user@mon]# ssh mon
[user@mon]# cd /etc/ceph
[user@mon]# ceph auth get-or-create client.nagios mon 'allow r' > client.nagios.keyringEach plug-in will require authentication. Repeat this procedure for each host that contains a plug-in.
5.2. Test the Ceph Plug-in Installation
Before proceeding with additional configuration, ensure that the plug-ins are working. For example:
/usr/lib/nagios/plugins/check_ceph_health --id nagios --keyring /etc/ceph/client.nagios.keyring
[user@mon]# /usr/lib/nagios/plugins/check_ceph_health --id nagios --keyring /etc/ceph/client.nagios.keyring
The check_ceph_health plug-in performs the the equivalent of:
ceph health
[user@mon]# ceph health5.3. Add a Command for the Ceph Plug-in
Add a command for the check_ceph_health plug-in.
vim /usr/local/nagios/etc/nrpe.cfg
[user@mon]# vim /usr/local/nagios/etc/nrpe.cfgFor example:
command[check_ceph_health]=/usr/lib/nagios/plugins/check_ceph_health --id nagios --keyring /etc/ceph/client.nagios.keyring
command[check_ceph_health]=/usr/lib/nagios/plugins/check_ceph_health --id nagios --keyring /etc/ceph/client.nagios.keyringSave and restart NRPE.
systemctl restart nrpe
[user@mon]# systemctl restart nrpeRepeat this procedure for each Ceph plug-in applicable to the host. See https://github.com/valerytschopp/ceph-nagios-plugins for usage.
5.4. Define the check_nrpe Command
Return to the Nagios server and define a check_nrpe command for the NRPE plug-in.
cd /usr/local/nagios/etc/objects vi commands.cfg
[user@nagios]# cd /usr/local/nagios/etc/objects
[user@nagios]# vi commands.cfgdefine command{
command_name check_nrpe
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}
define command{
command_name check_nrpe
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}5.5. Define a Service for the Plug-in
On the Nagios server, edit the configuration file for the host and add a service for the Ceph plug-in. For example:
vim /usr/local/nagios/etc/objects/mon.cfg
[user@nagios]# vim /usr/local/nagios/etc/objects/mon.cfgdefine service {
use generic-service
host_name mon
service_description Ceph Health Check
check_command check_nrpe!check_ceph_health
}
define service {
use generic-service
host_name mon
service_description Ceph Health Check
check_command check_nrpe!check_ceph_health
}
Note that the check_command setting uses check_nrpe!` before the Ceph plug-in name. This tells NRPE to execute the check_ceph_health command on the remote host.
Repeat this procedure for each plug-in applicable to the host.
Then, restart the Nagios server.
systemctl restart nagios
[user@nagios]# systemctl restart nagiosChapter 6. Summary
After completing the foregoing procedures, return to the Nagios web user interface and click on the "Hosts" link. The host should appear in the list of hosts. Click on the host to see additional details. Click on the View Status Detail hyperlink. It should display the checks it performs. In the instant example, there should be a Ceph Health Check service with status information on the Ceph cluster.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}