Managing, monitoring, and updating the kernel

Procedure

1. List the available versions of a package:

   $ yum repoquery <package_name>

2. Display the list of files in a package:

   $ yum repoquery -l <package_name>

Procedure

1. To update the kernel, enter the following command:

   # yum update kernel

   This command updates the kernel along with all dependencies to the latest available version.

2. Reboot your system for the changes to take effect.

Procedure

1. Select a kernel module you want to load.

   The modules are located in the /lib/modules/$(uname -r)/kernel/<SUBSYSTEM>/ directory.

2. Load the relevant kernel module:

   # modprobe <MODULE_NAME>

   Note

   When entering the name of a kernel module, do not append the .ko.xz extension to the end of the name. Kernel module names do not have extensions; their corresponding files do.

Procedure

1. List all the loaded kernel modules:

   # lsmod

2. Select the kernel module you want to unload.

   If a kernel module has dependencies, unload those prior to unloading the kernel module. For details on identifying modules with dependencies, see Listing currently loaded kernel modules and Kernel module dependencies.

3. Unload the relevant kernel module:

   # modprobe -r <MODULE_NAME>

   When entering the name of a kernel module, do not append the .ko.xz extension to the end of the name. Kernel module names do not have extensions; their corresponding files do.

Procedure

1. Boot the system into the boot loader.
2. Use the cursor keys to highlight the relevant boot loader entry.

3. Press the e key to edit the entry.

   Figure 2.1. Kernel boot menu

   
4. Use the cursor keys to navigate to the line that starts with linux.

5. Append modprobe.blacklist=module_name to the end of the line.

   Figure 2.2. Kernel boot entry

   

   The serio_raw kernel module illustrates a rogue module to be unloaded early in the boot process.

6. Press Ctrl+X to boot using the modified configuration.

Procedure

1. Select a kernel module you want to load during the boot process.

   The modules are located in the /lib/modules/$(uname -r)/kernel/<SUBSYSTEM>/ directory.

2. Create a configuration file for the module:

   # echo <MODULE_NAME> > /etc/modules-load.d/<MODULE_NAME>.conf

   Note

   When entering the name of a kernel module, do not append the .ko.xz extension to the end of the name. Kernel module names do not have extensions; their corresponding files do.

Verification

1. After reboot, verify the relevant module is loaded:

   $ lsmod | grep <MODULE_NAME>

Procedure

1. List modules loaded to the currently running kernel by using the lsmod command:

   $ lsmod

   Module                  Size  Used by
   tls                   131072  0
   uinput                 20480  1
   snd_seq_dummy          16384  0
   snd_hrtimer            16384  1
   …

   In the output, identify the module you want to prevent from getting loaded.

   • Alternatively, identify an unloaded kernel module you want to prevent from potentially loading in the /lib/modules/<KERNEL-VERSION>/kernel/<SUBSYSTEM>/ directory, for example:

     $ ls /lib/modules/4.18.0-477.20.1.el8_8.x86_64/kernel/crypto/

     ansi_cprng.ko.xz        chacha20poly1305.ko.xz  md4.ko.xz               serpent_generic.ko.xz
     anubis.ko.xz            cmac.ko.xz…

2. Create a configuration file serving as a denylist:

   # touch /etc/modprobe.d/denylist.conf

3. In a text editor of your choice, combine the names of modules you want to exclude from automatic loading to the kernel with the blacklist configuration command, for example:

   # Prevents <KERNEL-MODULE-1> from being loaded
   blacklist <MODULE-NAME-1>
   install <MODULE-NAME-1> /bin/false
   
   # Prevents <KERNEL-MODULE-2> from being loaded
   blacklist <MODULE-NAME-2>
   install <MODULE-NAME-2> /bin/false
   …

   Because the blacklist command does not prevent the module from getting loaded as a dependency for another kernel module that is not in a denylist, you must also define the install line. In this case, the system runs /bin/false instead of installing the module. The lines starting with a hash sign are comments you can use to make the file more readable.

   Note

   When entering the name of a kernel module, do not append the .ko.xz extension to the end of the name. Kernel module names do not have extensions; their corresponding files do.

4. Create a backup copy of the current initial RAM disk image before rebuilding:

   # cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).bak.$(date +%m-%d-%H%M%S).img

   • Alternatively, create a backup copy of an initial RAM disk image which corresponds to the kernel version for which you want to prevent kernel modules from automatic loading:

     # cp /boot/initramfs-<VERSION>.img /boot/initramfs-<VERSION>.img.bak.$(date +%m-%d-%H%M%S)

5. Generate a new initial RAM disk image to apply the changes:

   # dracut -f -v

   • If you build an initial RAM disk image for a different kernel version than your system currently uses, specify both target initramfs and kernel version:

     # dracut -f -v /boot/initramfs-<TARGET-VERSION>.img <CORRESPONDING-TARGET-KERNEL-VERSION>

6. Restart the system:

   $ reboot

Procedure

1. Create the /root/testmodule/test.c file with the following content.

   #include <linux/module.h>
   #include <linux/kernel.h>
   
   int init_module(void)
       { printk("Hello World\n This is a test\n"); return 0; }
   
   void cleanup_module(void)
       { printk("Good Bye World"); }

   The test.c file is a source file that provides the main functionality to the kernel module. The file has been created in a dedicated /root/testmodule/ directory for organizational purposes. After the module compilation, the /root/testmodule/ directory will contain multiple files.

   The test.c file includes from the system libraries:

   • The linux/kernel.h header file is necessary for the printk() function in the example code.
   • The linux/module.h header file contains function declarations and macro definitions that are shared across multiple C source files.
2. Follow the init_module() and cleanup_module() functions to start and end the kernel logging function printk(), which prints text.

3. Create the /root/testmodule/Makefile file with the following content.

   obj-m := test.o

   The Makefile contains instructions for the compiler to produce an object file named test.o. The obj-m directive specifies that the resulting test.ko file is going to be compiled as a loadable kernel module. Alternatively, the obj-y directive can instruct to build test.ko as a built-in kernel module.

4. Compile the kernel module.

   # make -C /lib/modules/$(uname -r)/build M=/root/testmodule modules
   make: Entering directory '/usr/src/kernels/4.18.0-305.el8.x86_64'
     CC [M]  /root/testmodule/test.o
     Building modules, stage 2.
     MODPOST 1 modules
   WARNING: modpost: missing MODULE_LICENSE() in /root/testmodule/test.o
   see include/linux/module.h for more information
     CC      /root/testmodule/test.mod.o
     LD [M]  /root/testmodule/test.ko
   make: Leaving directory '/usr/src/kernels/4.18.0-305.el8.x86_64'

   The compiler creates an object file (test.o) for each source file (test.c) as an intermediate step before linking them together into the final kernel module (test.ko).

   After a successful compilation, /root/testmodule/ contains additional files that relate to the compiled custom kernel module. The compiled module itself is represented by the test.ko file.

Verification

1. Optional: check the contents of the /root/testmodule/ directory:

   # ls -l /root/testmodule/
   total 152
   -rw-r—​r--. 1 root root    16 Jul 26 08:19 Makefile
   -rw-r—​r--. 1 root root    25 Jul 26 08:20 modules.order
   -rw-r—​r--. 1 root root     0 Jul 26 08:20 Module.symvers
   -rw-r—​r--. 1 root root   224 Jul 26 08:18 test.c
   -rw-r—​r--. 1 root root 62176 Jul 26 08:20 test.ko
   -rw-r—​r--. 1 root root    25 Jul 26 08:20 test.mod
   -rw-r—​r--. 1 root root   849 Jul 26 08:20 test.mod.c
   -rw-r—​r--. 1 root root 50936 Jul 26 08:20 test.mod.o
   -rw-r—​r--. 1 root root 12912 Jul 26 08:20 test.o

2. Copy the kernel module to the /lib/modules/$(uname -r)/ directory:

   # cp /root/testmodule/test.ko /lib/modules/$(uname -r)/

3. Update the modular dependency list:

   # depmod -a

4. Load the kernel module:

   # modprobe -v test
   insmod /lib/modules/4.18.0-305.el8.x86_64/test.ko

5. Verify that the kernel module was successfully loaded:

   # lsmod | grep test
   test                   16384  0

6. Read the latest messages from the kernel ring buffer:

   # dmesg
   [74422.545004] Hello World
                   This is a test

Procedure

1. Export your public key to the sb_cert.cer file:

   # certutil -d /etc/pki/pesign \
              -n 'Custom Secure Boot key' \
              -Lr \
              > sb_cert.cer

2. Import your public key into the MOK list:

   # mokutil --import sb_cert.cer

3. Enter a new password for this MOK enrollment request.

4. Reboot the machine.

   The shim boot loader notices the pending MOK key enrollment request and it launches MokManager.efi to enable you to complete the enrollment from the UEFI console.

5. Choose Enroll MOK, enter the password you previously associated with this request when prompted, and confirm the enrollment.

   Your public key is added to the MOK list, which is persistent.

   Once a key is on the MOK list, it will be automatically propagated to the .platform keyring on this and subsequent boots when UEFI Secure Boot is enabled.

Procedure

1. Export your public key to the sb_cert.cer file:

   # certutil -d /etc/pki/pesign \
              -n 'Custom Secure Boot key' \
              -Lr \
              > sb_cert.cer

2. Extract the key from the NSS database as a PKCS #12 file:

   # pk12util -o sb_cert.p12 \
              -n 'Custom Secure Boot key' \
              -d /etc/pki/pesign

3. When the previous command prompts, enter a new password that encrypts the private key.

4. Export the unencrypted private key:

   # openssl pkcs12 \
            -in sb_cert.p12 \
            -out sb_cert.priv \
            -nocerts \
            -nodes

   Important

   Keep the unencrypted private key secure.

5. Sign your kernel module. The following command appends the signature directly to the ELF image in your kernel module file:

   # /usr/src/kernels/$(uname -r)/scripts/sign-file \
             sha256 \
             sb_cert.priv \
             sb_cert.cer \
             my_module.ko

Verification

1. Display information about the kernel module’s signature:

   # modinfo my_module.ko | grep signer

   signer:         Your Name Key

   Check that the signature lists your name as entered during generation.

   Note

   The appended signature is not contained in an ELF image section and is not a formal part of the ELF image. Therefore, utilities such as readelf cannot display the signature on your kernel module.

2. Load the module:

   # insmod my_module.ko

3. Remove (unload) the module:

   # modprobe -r my_module.ko

Procedure

1. Verify that your public keys are on the system keyring:

   # keyctl list %:.platform

2. Copy the kernel module into the extra/ directory of the kernel that you want:

   # cp my_module.ko /lib/modules/$(uname -r)/extra/

3. Update the modular dependency list:

   # depmod -a

4. Load the kernel module:

   # modprobe -v my_module

5. Optional: To load the module on boot, add it to the /etc/modules-loaded.d/my_module.conf file:

   # echo "my_module" > /etc/modules-load.d/my_module.conf

Procedure

1. Boot into the GRUB boot menu.
2. Select the kernel you want to start.
3. Press the e key to edit the kernel parameters.
4. Find the kernel command line by moving the cursor down. The kernel command line starts with linux on 64-Bit IBM Power Series and x86-64 BIOS-based systems, or linuxefi on UEFI systems.

5. Move the cursor to the end of the line.

   Note

   Press Ctrl+a to jump to the start of the line and Ctrl+e to jump to the end of the line. On some systems, Home and End keys might also work.

6. Edit the kernel parameters as required. For example, to run the system in emergency mode, add the emergency parameter at the end of the linux line:

   linux   ($root)/vmlinuz-4.18.0-348.12.2.el8_5.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet emergency

   To enable the system messages, remove the rhgb and quiet parameters.

7. Press Ctrl+x to boot with the selected kernel and the modified command line parameters.

Procedure

1. Add the following two lines to the /etc/default/grub file:

   GRUB_TERMINAL="serial"
   GRUB_SERIAL_COMMAND="serial --speed=9600 --unit=0 --word=8 --parity=no --stop=1"

   The first line disables the graphical terminal. The GRUB_TERMINAL key overrides values of GRUB_TERMINAL_INPUT and GRUB_TERMINAL_OUTPUT keys.

   The second line adjusts the baud rate (--speed), parity and other values to fit your environment and hardware. Note that a much higher baud rate, for example 115200, is preferable for tasks such as following log files.

2. Update the GRUB configuration file.

   • On BIOS-based machines:

     # grub2-mkconfig -o /boot/grub2/grub.cfg

   • On UEFI-based machines:

     # grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

3. Reboot the system for the changes to take effect.

Procedure

1. List all parameters and their values.

   # sysctl -a

   Note

   The # sysctl -a command displays kernel parameters, which can be adjusted at runtime and at boot time.

2. To configure a parameter temporarily, enter:

   # sysctl <TUNABLE_CLASS>.<PARAMETER>=<TARGET_VALUE>

   The sample command above changes the parameter value while the system is running. The changes take effect immediately, without a need for restart.

   Note

   The changes return back to default after your system reboots.

Procedure

1. List all parameters.

   # sysctl -a

   The command displays all kernel parameters that can be configured at runtime.

2. Configure a parameter permanently:

   # sysctl -w <TUNABLE_CLASS>.<PARAMETER>=<TARGET_VALUE> >> /etc/sysctl.conf

   The sample command changes the tunable value and writes it to the /etc/sysctl.conf file, which overrides the default values of kernel parameters. The changes take effect immediately and persistently, without a need for restart.

Procedure

1. Create a new configuration file in /etc/sysctl.d/:

   # vim /etc/sysctl.d/<some_file.conf>

2. Include kernel parameters, one per line:

   <TUNABLE_CLASS>.<PARAMETER>=<TARGET_VALUE>
   <TUNABLE_CLASS>.<PARAMETER>=<TARGET_VALUE>

3. Save the configuration file.

4. Reboot the machine for the changes to take effect.

   • Alternatively, apply changes without rebooting:

     # sysctl -p /etc/sysctl.d/<some_file.conf>

     The command enables you to read values from the configuration file, which you created earlier.

Procedure

1. Identify a kernel parameter you want to configure.

   # ls -l /proc/sys/<TUNABLE_CLASS>/

   The writable files returned by the command can be used to configure the kernel. The files with read-only permissions provide feedback on the current settings.

2. Assign a target value to the kernel parameter.

   # echo <TARGET_VALUE> > /proc/sys/<TUNABLE_CLASS>/<PARAMETER>

   The configuration changes applied by using a command are not permanent and will disappear once the system is restarted.

Verification

1. Verify the value of the newly set kernel parameter.

   # cat /proc/sys/<TUNABLE_CLASS>/<PARAMETER>

Procedure

1. On the GRUB boot screen, press the e key for edit.

2. Add the following parameter at the end of the linux line:

   systemd.unit=rescue.target

   

3. Press Ctrl+x to boot to rescue mode.

   

Procedure

1. On the GRUB boot screen, press the e key for edit.

2. Add the following parameter at the end of the linux line:

   systemd.unit=emergency.target

   

3. Press Ctrl+x to boot to emergency mode.

   

Procedure

1. On the GRUB boot screen, press the e key for edit.

2. Add the following parameter at the end of the linux line:

   systemd.debug-shell

   

3. Optional: Add the debug option.

   Note

   Adding the debug option to the kernel command line increases the number of log messages. For systemd, the kernel command-line option debug is now a shortcut for systemd.log_level=debug.

4. Press Ctrl+x to boot to the debug shell.

Procedure

1. Press Ctrl+Alt+F9 to connect to the debug shell.

   If you work with a virtual machine, sending this key combination requires support from the virtualization application. For example, if you use Virtual Machine Manager, select Send Key → Ctrl+Alt+F9 from the menu.

2. The debug shell does not require authentication, therefore you can see a prompt similar to the following on TTY9:

Procedure

1. Boot the host from an installation source.

2. In the boot menu for the installation media, select the Troubleshooting option.

   

3. In the Troubleshooting menu, select the Rescue a Red Hat Enterprise Linux system option.

   

4. At the Rescue menu, select 1 and press the Enter key to continue.

   

5. Change the file system root as follows:

   sh-4.4# chroot /mnt/sysimage

   

6. Enter the passwd command and follow the instructions displayed on the command line to change the root password.

   

7. Remove the autorelable file to prevent a time consuming SELinux relabel of the disk:

   sh-4.4# rm -f /.autorelabel

8. Enter the exit command to exit the chroot environment.
9. Enter the exit command again to resume the initialization and finish the system boot.

Procedure

1. Start the system and, on the GRUB boot screen, press the e key for edit.

2. Add the rd.break parameter at the end of the linux line:

   

3. Press Ctrl+x to boot the system with the changed parameters.

   

4. Remount the file system as writable.

   switch_root:/# mount -o remount,rw /sysroot

5. Change the file system’s root.

   switch_root:/# chroot /sysroot

6. Enter the passwd command and follow the instructions displayed on the command line.

   

7. Relabel all files on the next system boot.

   sh-4.4# touch /.autorelabel

8. Remount the file system as read only:

   sh-4.4# mount -o remount,ro /

9. Enter the exit command to exit the chroot environment.

10. Enter the exit command again to resume the initialization and finish the system boot.

    Note

    The SELinux relabeling process can take a long time. A system reboot occurs automatically when the process is complete.

Procedure

1. Copy all the kernel arguments from your default kernel to this new kernel entry:

   # grubby --add-kernel=new_kernel --title="entry_title" --initrd="new_initrd" --copy-default

2. Get the list of available boot entries:

   # ls -l /boot/loader/entries/*
   -rw-r--r--. 1 root root 408 May 27 06:18 /boot/loader/entries/67db13ba8cdb420794ef3ee0a8313205-0-rescue.conf
   -rw-r--r--. 1 root root 536 Jun 30 07:53 /boot/loader/entries/67db13ba8cdb420794ef3ee0a8313205-4.18.0-372.9.1.el8.x86_64.conf
   -rw-r--r--  1 root root 336 Aug 15 15:12 /boot/loader/entries/d88fa2c7ff574ae782ec8c4288de4e85-4.18.0-193.el8.x86_64.conf

3. Create a new boot entry. For example, for the 4.18.0-193.el8.x86_64 kernel, issue the command as follows:

   # grubby --grub2 --add-kernel=/boot/vmlinuz-4.18.0-193.el8.x86_64 --title="Red Hat Enterprise 8 Test" --initrd=/boot/initramfs-4.18.0-193.el8.x86_64.img --copy-default

Procedure

1. List the kernel parameters from the kernelopts variable:

   # grub2-editenv - list | grep kernelopts
   kernelopts=root=/dev/mapper/rhel-root ro crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet

2. Make the changes to the kernel command-line parameters. You can add, remove or modify a parameter. For example, to add the debug parameter, enter:

   # grub2-editenv - set "$(grub2-editenv - list | grep kernelopts) <debug>"

3. Optional: Verify the parameter newly added to kernelopts:

   # grub2-editenv - list | grep kernelopts
   kernelopts=root=/dev/mapper/rhel-root ro crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet debug

4. Reboot the system for the changes to take effect.

Procedure

1. Set the GRUB_TIMEOUT_STYLE option in the /etc/default/grub file as follows:

   GRUB_TIMEOUT_STYLE=hidden

2. Rebuild the grub.cfg file for the changes to take effect.

   • On BIOS-based machines, enter:

     # grub2-mkconfig -o /boot/grub2/grub.cfg

   • On UEFI-based machines, enter:

     # grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

3. Press the Esc key to display the list of bootable kernels when booting.

Procedure

1. Specify which operating system or kernel must be loaded by default by passing its index to the grub2-set-default command, for example:

   # grubby --set-default-index=1
   The default is /boot/loader/entries/d5151aa93c444ac89e78347a1504d6c6-4.18.0-348.el8.x86_64.conf with index 1 and kernel /boot/vmlinuz-4.18.0-348.el8.x86_64

   GRUB supports using a numeric value as the key for the saved_entry directive in /boot/grub2/grubenv to change the default order in which the operating systems are loaded.

   Note

   Index counting starts with zero. Therefore, GRUB loads the second entry. With the next installed kernel, the index value will be overwritten.

   Note

   You can also use grubby to find indices for kernels. For more information, see Viewing the GRUB Menu Entry for a Kernel.

2. Optional: Force the system to always use a particular menu entry:

   1. List the available menu entries:

      # grubby --info=ALL

   2. Use the menu entry name or the number of the position of a menu entry in the list as the key to the GRUB_DEFAULT directive in the /etc/default/grub file. For example:

      GRUB_DEFAULT=example-gnu-linux

3. Rebuild the grub.cfg file for the changes to take effect.

   • On BIOS-based machines, enter:

     # grub2-mkconfig -o /boot/grub2/grub.cfg

   • On UEFI-based machines:

     # grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

Procedure

1. Reinstall GRUB on the device where it is installed. For example, if sda is your device:

   # grub2-install /dev/sda

2. Reboot your system for the changes to take effect:

   # reboot

Procedure

1. Reinstall the grub2-efi and shim boot loader files:

   # yum reinstall grub2-efi shim

2. Reboot your system for the changes to take effect:

   # reboot

Procedure

1. Determine the disk partition that stores GRUB:

   # bootlist -m normal -o

   sda1

2. Reinstall GRUB on the disk partition:

   # grub2-install partition

   Replace partition with the identified GRUB partition, such as /dev/sda1.

3. Reboot your system for the changes to take effect:

   # reboot

Procedure

1. Remove the configuration files:

   # rm /etc/grub.d/*
   # rm /etc/sysconfig/grub

2. Reinstall packages.

   • On BIOS-based machines:

     # yum reinstall grub2-tools

   • On UEFI-based machines:

     # yum reinstall grub2-efi shim grub2-tools grub2-common

3. Rebuild the grub.cfg file for the changes to take effect.

   • On BIOS-based machines:

     # grub2-mkconfig -o /boot/grub2/grub.cfg

   • On UEFI-based machines:

     # grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

4. Follow Reinstalling GRUB procedure to restore GRUB on the /boot/ partition.

Procedure

1. Issue the grub2-setpassword command as root:

   # grub2-setpassword

2. Enter the password for the user and press the Enter key to confirm the password:

   Enter password:
   Confirm the password:

Procedure

1. Open the Boot Loader Specification (BLS) file for boot entry you want to modify from the /boot/loader/entries/ directory.
2. Find the line beginning with grub_users. This parameter passes extra arguments to menuentry.
3. Set the grub_users attribute to the user name that is allowed to boot the entry besides the superusers, by default this user is root.

1. Save and close the BLS file:

1. Console log-level, defines the lowest priority of messages printed to the console.
2. Default log-level for messages without an explicit log-level attached to them.
3. Sets the lowest possible log-level configuration for the console log-level.

4. Sets default value for the console log-level at boot time.

   Each of these values defines a different rule for handling error messages.

Procedure

1. On the Anaconda installer, click KDUMP and enable kdump:

   
2. In Kdump Memory Reservation, select Manual` if you must customize the memory reserve.

3. In KDUMP > Memory To Be Reserved (MB), set the required memory reserve for kdump.

   

Procedure

1. Check if kdump is installed on your system:

   # rpm -q kexec-tools

   Output if the package is installed:

   kexec-tools-2.0.17-11.el8.x86_64

   Output if the package is not installed:

   package kexec-tools is not installed

2. Install kdump and other necessary packages:

   # dnf install kexec-tools

Procedure

1. Prepare the crashkernel= option.

   • For example, to reserve 128 MB of memory, use the following:

     crashkernel=128M

   • Alternatively, you can set the amount of reserved memory to a variable depending on the total amount of installed memory. The syntax for memory reservation into a variable is crashkernel=<range1>:<size1>,<range2>:<size2>. For example:

     crashkernel=512M-2G:64M,2G-:128M

     The command reserves 64 MB of memory if the total amount of system memory is in the range of 512 MB and 2 GB. If the total amount of memory is more than 2 GB, the memory reserve is 128 MB.

   • Offset the reserved memory.

     Some systems require to reserve memory with a certain fixed offset because the crashkernel reservation happens early, and you may need to reserve more memory for special usage. When you define an offset, the reserved memory begins there. To offset the reserved memory, use the following syntax:

     crashkernel=128M@16M

     In this example, kdump reserves 128 MB of memory starting at 16 MB (physical address 0x01000000). If you set the offset parameter to 0 or omit entirely, kdump offsets the reserved memory automatically. You can also use this syntax when setting a variable memory reservation. In that case, the offset is always specified last. For example:

     crashkernel=512M-2G:64M,2G-:128M@16M

2. Apply the crashkernel= option to your boot loader configuration:

   # grubby --update-kernel=ALL --args="crashkernel=<value>"

   Replace <value> with the value of the crashkernel= option that you prepared in the previous step.

Procedure

1. As a root, edit the /etc/kdump.conf configuration file and remove the hash sign ("#") from the beginning of the #core_collector makedumpfile -l --message-level 1 -d 31.
2. Enter the following command to enable crash dump file compression:

Procedure

1. As a root user, remove the hash sign (#) from the beginning of the #failure_action line in the /etc/kdump.conf configuration file.

2. Replace the value with a required action.

   failure_action poweroff

Procedure

1. Enable the kdump service:

   # kdumpctl restart

2. Check the status of the kdump service with the kdumpctl:

   # kdumpctl status

   kdump:Kdump is operational

   Optionally, if you use the systemctl command, the output prints in the systemd journal.

3. Start a kernel crash to test the kdump configuration. The sysrq-trigger key combination causes the kernel to crash and might reboot the system if required.

   # echo c > /proc/sysrq-trigger

   On a kernel reboot, the address-YYYY-MM-DD-HH:MM:SS/vmcore file is created at the location you have specified in the /etc/kdump.conf file. The default is /var/crash/.

Procedure

1. Enable the kdump service for multi-user.target:

   # systemctl enable kdump.service

2. Start the service in the current session:

   # systemctl start kdump.service

3. Stop the kdump service:

   # systemctl stop kdump.service

4. Disable the kdump service:

   # systemctl disable kdump.service

Procedure

1. Display the list of modules that are loaded to the currently running kernel. Select the kernel module that you intend to block from loading:

   $ lsmod

   Module                  Size  Used by
   fuse                  126976  3
   xt_CHECKSUM            16384  1
   ipt_MASQUERADE         16384  1
   uinput                 20480  1
   xt_conntrack           16384  1

2. Update the KDUMP_COMMANDLINE_APPEND= variable in the /etc/sysconfig/kdump file. For example:

   KDUMP_COMMANDLINE_APPEND="rd.driver.blacklist=hv_vmbus,hv_storvsc,hv_utils,hv_netvsc,hid-hyperv"

   Also, consider the following example by using the modprobe.blacklist=<modules> configuration option:

   KDUMP_COMMANDLINE_APPEND="modprobe.blacklist=emcp modprobe.blacklist=bnx2fc modprobe.blacklist=libfcoe modprobe.blacklist=fcoe"

3. Restart the kdump service:

   # systemctl restart kdump

Procedure

1. Print the estimate crashkernel= value:

   # *kdumpctl estimate*

   Encrypted kdump target requires extra memory, assuming using the keyslot with minimum memory requirement
      Reserved crashkernel:    256M
      Recommended crashkernel: 652M
   
      Kernel image size:   47M
      Kernel modules size: 8M
      Initramfs size:      20M
      Runtime reservation: 64M
      LUKS required size:  512M
      Large modules: <none>
      WARNING: Current crashkernel size is lower than recommended size 652M.

2. Configure the amount of required memory by increasing the crashkernel= value.
3. Reboot the system.

Procedure

1. In the web console, open the Kernel dump tab and start the kdump service by setting the Kernel crash dump switch to on.

2. Configure the kdump memory usage in the terminal, for example:

   $ sudo grubby --update-kernel ALL --args crashkernel=512M

   Restart the system to apply the changes.

3. In the Kernel dump tab, click Edit at the end of the Crash dump location field.

4. Specify the target directory for saving the vmcore dump file:

   • For a local filesystem, select Local Filesystem from the drop-down menu.

   • For a remote system by using the SSH protocol, select Remote over SSH from the drop-down menu and specify the following fields:

     • In the Server field, enter the remote server address.
     • In the SSH key field, enter the SSH key location.
     • In the Directory field, enter the target directory.

   • For a remote system by using the NFS protocol, select Remote over NFS from the drop-down menu and specify the following fields:

     • In the Server field, enter the remote server address.
     • In the Export field, enter the location of the shared folder of an NFS server.

     • In the Directory field, enter the target directory.

       Note

       You can reduce the size of the vmcore file by selecting the Compression checkbox.

5. Optional: Display the automation script by clicking View automation script.

   A window with the generated script opens. You can browse a shell script and an Ansible Playbook generation options tab.

6. Optional: Copy the script by clicking Copy to clipboard.

   You can use this script to apply the same configuration on multiple machines.

Verification

1. Click Test configuration.

2. Click Crash system under Test kdump settings.

   Warning

   When you start the system crash, the kernel operation stops and results in a system crash with data loss.

Procedure

1. Add the crashkernel= command-line parameter to all installed kernels:

   # grubby --update-kernel=ALL --args="crashkernel=xxM"

   xxM is the required memory in megabytes.

2. Reboot the system:

   # reboot

3. Enable the kdump service:

   # systemctl enable --now kdump.service

Procedure

1. List the kernels installed on the machine.

   # ls -a /boot/vmlinuz-*

   /boot/vmlinuz-0-rescue-2930657cd0dc43c2b75db480e5e5b4a9
   /boot/vmlinuz-4.18.0-330.el8.x86_64
   /boot/vmlinuz-4.18.0-330.rt7.111.el8.x86_64

2. Add a specific kdump kernel to the system’s Grand Unified Bootloader (GRUB) configuration.

   For example:

   # grubby --update-kernel=vmlinuz-4.18.0-330.el8.x86_64 --args="crashkernel=xxM"

   xxM is the required memory reserve in megabytes.

3. Enable the kdump service.

   # systemctl enable --now kdump.service

Procedure

1. To stop the kdump service in the current session:

   # systemctl stop kdump.service

2. To disable the kdump service:

   # systemctl disable kdump.service

Procedure

1. To configure final_action, edit the /etc/kdump.conf file and add one of the following options:

   • final_action reboot
   • final_action halt
   • final_action poweroff

2. Restart the kdump service for the changes to take effect.

   # kdumpctl restart

Procedure

1. To configure an action to take if the dump fails, edit the /etc/kdump.conf file and specify one of the failure_action options:

   • failure_action reboot
   • failure_action halt
   • failure_action poweroff
   • failure_action shell
   • failure_action dump_to_rootfs

2. Restart the kdump service for the changes to take effect.

   # kdumpctl restart

Procedure

1. Install and configure kdump.

2. Enable the fadump=on kernel option:

   # grubby --update-kernel=ALL --args="fadump=on"

3. Optional: If you want to specify reserved boot memory instead of using the defaults, enable the crashkernel=xxM option, where xx is the amount of the memory required in megabytes:

   # grubby --update-kernel=ALL --args="crashkernel=xxM fadump=on"

   Important

   When specifying boot configuration options, test all boot configuration options before you run them. If the kdump kernel fails to boot, increase the value specified in crashkernel= argument gradually to set an appropriate value.

Procedure

1. Add or edit the following lines in the /etc/sysctl.conf file to ensure that kdump starts as expected for sadump:

   kernel.panic=0
   kernel.unknown_nmi_panic=1

   Warning

   In particular, ensure that after kdump, the system does not reboot. If the system reboots after kdump has failed to save the vmcore file, then it is not possible to invoke the sadump.

2. Set the failure_action parameter in /etc/kdump.conf appropriately as halt or shell.

   failure_action shell

Procedure

1. Enable the relevant repositories:

   # subscription-manager repos --enable baseos repository

   # subscription-manager repos --enable appstream repository

   # subscription-manager repos --enable rhel-8-for-x86_64-baseos-debug-rpms

2. Install the crash package:

   # yum install crash

3. Install the kernel-debuginfo package:

   # yum install kernel-debuginfo

   The package kernel-debuginfo corresponds to the running kernel and provides the data necessary for the dump analysis.

Procedure

1. To start the crash utility, pass the following two necessary parameters:

   • The debug-info (a decompressed vmlinuz image), for example /usr/lib/debug/lib/modules/4.18.0-5.el8.x86_64/vmlinux provided through a specific kernel-debuginfo package.

   • The actual vmcore file, for example /var/crash/127.0.0.1-2018-10-06-14:05:33/vmcore.

     The resulting crash command will be as follows:

     # crash /usr/lib/debug/lib/modules/4.18.0-5.el8.x86_64/vmlinux /var/crash/127.0.0.1-2018-10-06-14:05:33/vmcore

     Use the same <kernel> version that was captured by kdump.

     Example 20.1. Running the crash utility

     The following example shows analyzing a core dump created on October 6 2018 at 14:05 PM, using the 4.18.0-5.el8.x86_64 kernel.

     ...
     WARNING: kernel relocated [202MB]: patching 90160 gdb minimal_symbol values
     
           KERNEL: /usr/lib/debug/lib/modules/4.18.0-5.el8.x86_64/vmlinux
         DUMPFILE: /var/crash/127.0.0.1-2018-10-06-14:05:33/vmcore  [PARTIAL DUMP]
             CPUS: 2
             DATE: Sat Oct  6 14:05:16 2018
           UPTIME: 01:03:57
     LOAD AVERAGE: 0.00, 0.00, 0.00
            TASKS: 586
         NODENAME: localhost.localdomain
          RELEASE: 4.18.0-5.el8.x86_64
          VERSION: #1 SMP Wed Aug 29 11:51:55 UTC 2018
          MACHINE: x86_64  (2904 Mhz)
           MEMORY: 2.9 GB
            PANIC: "sysrq: SysRq : Trigger a crash"
              PID: 10635
          COMMAND: "bash"
             TASK: ffff8d6c84271800  [THREAD_INFO: ffff8d6c84271800]
              CPU: 1
            STATE: TASK_RUNNING (SYSRQ)
     
     crash>

2. To exit the interactive prompt and stop crash, type exit or q.

   crash> exit
   ~]#

Procedure

1. Access the Kernel Oops Analyzer tool.

2. To diagnose a kernel crash issue, upload a kernel oops log generated in vmcore.

   • Alternatively, you can diagnose a kernel crash issue by providing a text message or a vmcore-dmesg.txt as an input.
3. Click DETECT to compare the oops message based on information from the makedumpfile against known solutions.

Procedure

1. Verify that the kdump service is enabled and active:

   # systemctl is-enabled kdump.service && systemctl is-active kdump.service
   enabled
   active

   If kdump is not enabled and running, set all required configurations and verify that kdump service is enabled.

2. Rebuild the initramfs image of the booting kernel with the early kdump functionality:

   # dracut -f --add earlykdump

3. Add the rd.earlykdump kernel command line parameter:

   # grubby --update-kernel=/boot/vmlinuz-$(uname -r) --args="rd.earlykdump"

4. Reboot the system to reflect the changes:

   # reboot

1. The kernel patch module is copied to the /var/lib/kpatch/ directory and registered for re-application to the kernel by systemd on next boot.
2. The kpatch module loads into the running kernel and the new functions are registered to the ftrace mechanism with a pointer to the location in memory of the new code.

Procedure

1. Log in to the RHEL 8 web console.

   For details, see Logging in to the web console.

2. Click Software Updates.

3. Check the status of your kernel patching settings.

   1. If the patching is not installed, click Install.

   2. To enable kernel patching, click Enable.

      
   3. Select the checkbox for applying kernel patches.
   4. Select whether you want to apply patches for current and future kernels or the current kernel only. If you decide to subscribe to applying patches for future kernels, the system also applies patches for the upcoming kernel releases.
   5. Click Apply.

Procedure

1. Optional: Check your kernel version:

   # uname -r
   4.18.0-94.el8.x86_64

2. Search for a live patching package that corresponds to the version of your kernel:

   # yum search $(uname -r)

3. Install the live patching package:

   # yum install "kpatch-patch = $(uname -r)"

   The command above installs and applies the latest cumulative live patches for that specific kernel only.

   If the version of a live patching package is 1-1 or higher, the package will contain a patch module. In that case the kernel will be automatically patched during the installation of the live patching package.

   The kernel patch module is also installed into the /var/lib/kpatch/ directory to be loaded by the systemd system and service manager during the future reboots.

   Note

   An empty live patching package will be installed when no live patches are available for a given kernel. An empty live patching package will have a kpatch_version-kpatch_release of 0-0, for example kpatch-patch-4_18_0-94-0-0.el8.x86_64.rpm. The installation of the empty RPM subscribes the system to all future live patches for the given kernel.

Procedure

1. Optional: Check all installed kernels and the kernel you are currently running:

   # yum list installed | grep kernel
   Updating Subscription Management repositories.
   Installed Packages
   ...
   kernel-core.x86_64         4.18.0-240.10.1.el8_3           @rhel-8-for-x86_64-baseos-rpms
   kernel-core.x86_64         4.18.0-240.15.1.el8_3           @rhel-8-for-x86_64-baseos-rpms
   ...
   
   # uname -r
   4.18.0-240.10.1.el8_3.x86_64

2. Install the kpatch-dnf plugin:

   # yum install kpatch-dnf

3. Enable automatic subscription to kernel live patches:

   # yum kpatch auto
   Updating Subscription Management repositories.
   Last metadata expiration check: 19:10:26 ago on Wed 10 Mar 2021 04:08:06 PM CET.
   Dependencies resolved.
   ==================================================
    Package                             Architecture
   ==================================================
   Installing:
    kpatch-patch-4_18_0-240_10_1        x86_64
    kpatch-patch-4_18_0-240_15_1        x86_64
   
   Transaction Summary
   ===================================================
   Install  2 Packages
   …​

   This command subscribes all currently installed kernels to receiving kernel live patches. The command also installs and applies the latest cumulative live patches, if any, for all installed kernels.

   When you update the kernel, live patches are installed automatically during the new kernel installation process.

   The kernel patch module is also installed into the /var/lib/kpatch/ directory to be loaded by the systemd system and service manager during future reboots.

   Note

   An empty live patching package will be installed when no live patches are available for a given kernel. An empty live patching package will have a kpatch_version-kpatch_release of 0-0, for example kpatch-patch-4_18_0-240-0-0.el8.x86_64.rpm. The installation of the empty RPM subscribes the system to all future live patches for the given kernel.