Troubleshooting

Red Hat build of MicroShift 4.12

Troubleshooting common issues

Red Hat OpenShift Documentation Team

Legal Notice

Abstract

This document provides information about how to troubleshoot MicroShift.

Chapter 1. Checking which version you have installed

To begin troubleshooting, determine which version of Red Hat build of MicroShift you have installed.

1.1. Checking the Red Hat build of MicroShift version using the command-line interface

To begin troubleshooting, you must know your Red Hat build of MicroShift version. One way to get this information is by using the CLI.

Procedure

Run the following command to check the version information:

$ microshift version

Example output

Red Hat build of MicroShift Version: 4.12-0.microshift-e6980e25
Base OCP Version: 4.12

1.2. Checking the Red Hat build of MicroShift version using the API

To begin troubleshooting, you must know your Red Hat build of MicroShift version. One way to get this information is by using the API.

Procedure

To get the version number using the OpenShift CLI (oc), view the kube-public/microshift-version config map by running the following command:

$ oc get configmap -n kube-public microshift-version -o yaml

Example output

apiVersion: v1
data:
  major: "4"
  minor: "10"
  version: 4.10.0-0.microshift-e6980e25
kind: ConfigMap
metadata:
  creationTimestamp: "2022-08-08T21:06:11Z"
  name: microshift-version
  namespace: kube-public

Chapter 2. Responsive restarts and security certificates

Red Hat build of MicroShift responds to system configuration changes and restarts after alterations are detected, including IP address changes, clock adjustments, and security certificate age.

2.1. IP address changes or clock adjustments

Red Hat build of MicroShift depends on device IP addresses and system-wide clock settings to remain consistent during its runtime. However, these settings may occasionally change on edge devices, such as DHCP or Network Time Protocol (NTP) updates.

When such changes occur, some Red Hat build of MicroShift components may stop functioning properly. To mitigate this situation, Red Hat build of MicroShift monitors the IP address and system time and restarts if either setting change is detected.

The threshold for clock changes is a time adjustment of greater than 10 seconds in either direction. Smaller drifts on regular time adjustments performed by the Network Time Protocol (NTP) service do not cause a restart.

2.2. Security certificate lifetime

Red Hat build of MicroShift certificates are separated into two basic groups:

Short-lived certificates having certificate validity of one year.
Long-lived certificates having certificate validity of 10 years.

Most server or leaf certificates are short-lived.

An example of a long-lived certificate is the client certificate for system:admin user authentication, or the certificate of the signer of the kube-apiserver external serving certificate.

2.2.1. Certificate rotation

As certificates age, Red Hat build of MicroShift can be restarted to rotate certificates. A certificate that is close to expiring might also automatically cause a restart. Read the following situation overviews to understand the actions at each moment in time:

Green zone:
1. When a short-term certificate is 5 months old, no rotation occurs.
2. When a long-term certificate is 8.5 years old, no rotation occurs.
Yellow zone:
1. When a short-term certificate is 8 months old, it is rotated when Red Hat build of MicroShift starts or restarts.
2. When a long-term certificate is 9 years old, it is rotated when Red Hat build of MicroShift starts or restarts.
Red zone
1. When a short-term certificate is 8 months old, Red Hat build of MicroShift restarts to rotate and apply a new certificate.
2. When a long-term certificate is 9 years old, Red Hat build of MicroShift restarts to rotate and apply a new certificate.

Note

If the rotated certificate is a Certificate Authority, all of the certificates it signed rotate.

Figure 2.1. Stoplight timeline of Red Hat build of MicroShift certificate validity.

<Red Hat build of MicroShift graph with symbolic green-yellow-red stoplight map of certificates>

Chapter 3. Troubleshooting

Read about troubleshooting and possible solutions for known issues.

3.1. Troubleshooting the NodePort service iptable rules

OVN-Kubernetes sets up an iptable chain in the network address translation (NAT) table to handle incoming traffic to the NodePort service. When the NodePort service is not reachable or the connection is refused, check the iptable rules on the host to make sure the relevant rules are properly inserted.

Procedure

View the iptable rules for the NodePort service by running the following command:
```
$ iptables-save | grep NODEPORT
```
Example output
```
-A OUTPUT -j OVN-KUBE-NODEPORT
-A OVN-KUBE-NODEPORT -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 30326 -j DNAT --to-destination 10.43.95.170:80
```
OVN-Kubernetes configures the OVN-KUBE-NODEPORT iptable chain in the NAT table to match the destination port and Destination Network Address Translates (DNATs) the packet to the clusterIP service. The packet is then routed to the OVN network through the gateway bridge br-ex via routing rules on the host.
Route the packet through the network with routing rules by running the following command:
```
$ ip route
```
Example output
```
10.43.0.0/16 via 192.168.122.1 dev br-ex mtu 1400
```
This routing rule matches the Kubernetes service IP address range and forwards the packet to the gateway bridge br-ex. You must enable ip_forward on the host. After the packet is forwarded to the OVS bridge br-ex, it is handled by OpenFlow rules in OVS which steers the packet to the OVN network and eventually to the pod.

Legal Notice

The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.

Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.

Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.

Linux® is the registered trademark of Linus Torvalds in the United States and other countries.

Java® is a registered trademark of Oracle and/or its affiliates.

XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.

MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.

Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.

The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.

All other trademarks are the property of their respective owners.

Troubleshooting