Ce contenu n'est pas disponible dans la langue sélectionnée.

Chapter 4. Troubleshooting


The following sections explain how to resolve issues that may occur in Metrics Store.

4.1. Information Is Missing from Kibana

If Kibana is not displaying metric or log information as expected, use journalctl to investigate the collectd and rsyslog log files as follows:

  • If only metrics information is missing, check the collectd log files.
  • If only log information is missing, check the rsyslog log files.
  • If both metrics and logs information are missing, check both log files.

    1. To investigate collectd log files, log in as root and run the following command:

      # journalctl -u collectd
      Copy to Clipboard Toggle word wrap
    2. To investigate rsyslog log files, run the following command:

      # journalctl -u rsyslog
      Copy to Clipboard Toggle word wrap

To learn about additional journalctl options, see journalctl in Linux man pages .

4.2. Extracting Elasticsearch logs

To extract Elasticsearch logs

  1. Log in to the Metrics Store virtual machine as root, and run the following command. Where openshift-logging is the namespace for the Elasticsearch pod:

    for espod in $(oc -n openshift-logging get pods -l component=es -o jsonpath='{.items[*].metadata.name}') ; do
        oc -n openshift-logging exec -c elasticsearch $espod -- logs > $espod.log 2>&1
    done
    Copy to Clipboard Toggle word wrap
  2. Optionally, you can use the use the OpenShift logging-dump tool located at logging dump tool script

4.3. Searching Elasticsearch Logs

  1. To search for all RHV/oVirt logs, ordered by timestamp from newest to oldest.Log in to the Metrics Store virtual machine as root, and run the following command, where openshift-logging is the namespace for the Elasticsearch pod:

    oc -n opesnshift-logging exec -c elasticsearch $(oc -n openshift-logging get pods -l component=es -o jsonpath='{.items[0].metadata.name}') -- es_util --query=project.ovirt-logs*/_search?sort=@timestamp:desc​\&pretty | more
    Copy to Clipboard Toggle word wrap
  2. The output is presented in the human readable JSON format.

4.4. Searching log record results

To search the log report results, you can use the get last rec from host search tool.

The search tool shows how long it has been since the Metrics Store virtual machine received a record from each host that it knows about during a given time interval (last 3 hours by default).

For each host that the Metrics Store virtual machine receives logs from over the defined duration (default 3h) duration, it prints out "green", "yellow", or "red", depending on whether the Metrics Store virtual machine received a record from that host recently or not.

This is followed by the number of seconds it has been since the last record was received from that host, and the number of records received.

  1. Log in to the Metrics Store virtual machine as root, and clone the script repository:

    # git clone https://github.com/jcantrill/cluster-logging-tools -b release-3.11
    Copy to Clipboard Toggle word wrap
  2. Run the script:

    # cd cluster-logging-tools/scripts
    [OLDEST=3h] ./get-last-rec-from-host
    Copy to Clipboard Toggle word wrap
  3. To change the interval, change the OLDEST value to a longer or shorter interval in hours.

    For example, to go back 1 day (24 hours), use:

    OLDEST=24h ./get-last-rec-from-host
    Copy to Clipboard Toggle word wrap
Note

Some hosts may not be listed, if no records were received from that host during the given time interval.

Retour au début
Red Hat logoGithubredditYoutubeTwitter

Apprendre

Essayez, achetez et vendez

Communautés

À propos de la documentation Red Hat

Nous aidons les utilisateurs de Red Hat à innover et à atteindre leurs objectifs grâce à nos produits et services avec un contenu auquel ils peuvent faire confiance. Découvrez nos récentes mises à jour.

Rendre l’open source plus inclusif

Red Hat s'engage à remplacer le langage problématique dans notre code, notre documentation et nos propriétés Web. Pour plus de détails, consultez le Blog Red Hat.

À propos de Red Hat

Nous proposons des solutions renforcées qui facilitent le travail des entreprises sur plusieurs plates-formes et environnements, du centre de données central à la périphérie du réseau.

Theme

© 2025 Red Hat