Chapter 4. Using the command-line assistant to debug or troubleshoot system issues

Procedure

1. Check the SSHD status and restart it.

   $ sudo systemctl status ssh
   $ sudo systemctl restart ssh

   Copy to Clipboard Toggle word wrap

2. Enable the optional command-line assistant terminal capture feature:

   $ c shell --enable-capture

   Copy to Clipboard Toggle word wrap

3. Use the -w 1 "your-question" option to specify to include the output from the last command that was run.

   $ c -w 1 "what_is_this"

   Copy to Clipboard Toggle word wrap
   • If you specify the number 2, that references the output from the second previous 2 commands. This is also true for the additional numbers.

   • You can also specify a prompt to run with the command and ask “help me understand the output”, and reference the output with the error, so that the command my assistant understands that you are asking for more details on what is the error.

     The command-line assistant takes some time to process the request, and provide several possible solutions. In the example, you can use the suggestion to run the journalctl -xeu command, so that you can use sshd.service to check the log files.

4. Run that journalctl command. Add the tail command to get the last 30 lines, pipe that output into the command-line assistant, and add a query to understand the error.

   $ journalctl -xeu sshd.service | tail -n 30 | c “here are the logs, please help me understand this”

   Copy to Clipboard Toggle word wrap

   The command-line assistant checks the log files and indicates some potential issues. typing error in the config file.

5. Ask the command-line assistant to generate a command on how to fix this typing error.

   $ c “what is the command that I can use to change “Porrt ”to “Port” in the /etc/ssh/sshd_config file?”

   Copy to Clipboard Toggle word wrap

   Use the command suggested by the command-line assistant.

6. Run the command suggested by the command-line assistant. For example:

   $ sed -i s/Porrt/Port/g /etc/ssh/sshd_config

   Copy to Clipboard Toggle word wrap

   The output found a permission denied error to edit that file. Rerun the previous command as a sudo user.

   $ sudo sed -i s/Porrt/Port/g /etc/ssh/sshd_config

   Copy to Clipboard Toggle word wrap

7. Restart the SSHD service and check the status of the SSHD.

   $ sudo systemctl restart sshd
   $ sudo systemctl status sshd

   Copy to Clipboard Toggle word wrap

Procedure

1. On your terminal, enter the following command to list the httpd package version that you have installed in your system:

   $ sudo rpm -qa httpd
   httpd-2.4.62-2.fc40.x86_64

   Copy to Clipboard Toggle word wrap

2. Show the httpd.conf file content.

   $ sudo cat /etc/httpd/conf/httpd.conf
   # This is the main Apache HTTP server configuration file.  It contains the
   # configuration directives that give the server its instructions.
   # See <URL:http://httpd.apache.org/docs/2.4/> for detailed information.
   # In particular, see<URL:http://httpd.apache.org/docs/2.4/mod/directives.html>
   # for a discussion of each configuration directive.

   Copy to Clipboard Toggle word wrap

3. Query all httpdq packages:

   $ sudo rpm -qa httpdq

   Copy to Clipboard Toggle word wrap

4. Identify the ports on which the web server accepts incoming requests:

   $ cat /etc/httpd/conf/httpd.conf | grep Listen
   # Listen: Allows you to bind Apache to specific IP addresses and/or
   # Change this to Listen on a specific IP address, but note that if
   #Listen 12.34.56.78:80
   Listen 80

   Copy to Clipboard Toggle word wrap

5. Restart the httpd service:

   $ systemctl restart httpd
   
   Job for httpd. Service failed because the control process exited with error code.
   See “systemctl status httpd.service" and “journalctl -xeu httpd.service” for details.

   Copy to Clipboard Toggle word wrap

   1. Run the journalctl command for more details on the failed service:

      $ sudo journalctl -xeu httpd.service

      Copy to Clipboard Toggle word wrap

6. Use the command-line assistant to troubleshoot the issue and ask why the service is failing:

   $ sudo c “why did httpd fail to start”

   Copy to Clipboard Toggle word wrap

   One of the suggestions from the assistant is to query audit logs by using the ausearch tool, and use the AVC and USER_AVC values for the message type parameter. For that, run the following command:

   $ sudo ausearch -m AVC,USER_AVC -ts recent

   Copy to Clipboard Toggle word wrap

   1. Ask the command-line assistant about the selinux httpd port:

      $ c “selinux httpd port”

      Copy to Clipboard Toggle word wrap

      The assistant advises to use the sestatus command to check the current SELinux status and the content of the httpd services with the following command:

      $ sudo sestatus
      
      SELinux status:                 enabled
      SELinuxfs mount:                /sys/fs/selinux
      SELinux root directory:         /etc/selinux
      Loaded policy name:             targeted
      Current mode:                   enforcing
      Mode from config file:          enforcing
      Policy MLS status:              enabled
      Policy deny_unknown status:     allowed
      Memory protection checking:     actual (secure)
      Max kernel policy version:      33

      Copy to Clipboard Toggle word wrap

   2. View the specific SELinux policy for the httpd services by running the following command:

      $ sudo cat /usr/share/selinux/targeted/contexts/httpd_var_run_t
      No such file or directory

      Copy to Clipboard Toggle word wrap

   3. Ask the command-line assistant about contexts.

      $ c “i don't have a httpd_var_run_t contexts”

      Copy to Clipboard Toggle word wrap

      The command-line assistant takes some time to process the request, then provides several possible suggestions.

   4. The assistant says that you might not have context and need to set it with the following command:

      $ sudo chcon -R -t httpd_var_run_t

      Copy to Clipboard Toggle word wrap

   5. Ask the CLA about the port:

      $ c “selinux won't let httpd listen on port 12345”

      Copy to Clipboard Toggle word wrap

   6. Try the following suggestion, run the command:

      $ sudo semage port -a -t httpd_port_t -p tcp 12345
      ValueError: Type httpd_port_t is invalid, must be a port type

      Copy to Clipboard Toggle word wrap

   7. Ask the CLA about the error you see in the output:

      $ c “how do I fix ValueError: Type httpd_port_t is invalid, must be a port type”

      Copy to Clipboard Toggle word wrap

7. Run the steps provided by the CLA:

   $  sudo getenforce
   Enforcing
   $ setenforce 0
   $ sudo systemctl restart httpd
   $ sudo systemctl status httpd
   
   $ sudo ls -Z /usr/sbin/httpd
   system_u:object_r:httpd_exec_t:s0 /usr/sbin/httpd
   
   $ chcon -t httpd_exec_t /usr/sbin/httpd
   
   $ sudo setenforce 1

   Copy to Clipboard Toggle word wrap

   1. Restart the httpd service and check the status of httpd.service:

      $ sudo systemctl restart httpd
      Job failed
      
      $ sudo systemctl status httpd.service
      Failed to start the Apache Server

      Copy to Clipboard Toggle word wrap

8. Ask the CLA how to enable httpd to listen on port 12345:

   $ c “how do I enable httpd to listen on port 12345 selinux”

   Copy to Clipboard Toggle word wrap

   1. Run the command advised by the CLA:

      $ sudo setsebool -P httpd_can_network_connect=1

      Copy to Clipboard Toggle word wrap

9. Restart the httpd service and check the status of httpd.service again:

   $ sudo systemctl status htppd
   $ sudo systemctl restart httpd
   Job failed, see journalctl

   Copy to Clipboard Toggle word wrap

10. Check the journalctl service:

    $ journalctl -xeu httpd
    Output: An ExecStart= process belonging to unit httpd.service has exited.

    Copy to Clipboard Toggle word wrap

11. Use the output to ask the CLA to troubleshoot:

    $ c “An ExecStart= process belonging to unit httpd.service has exited.”

    Copy to Clipboard Toggle word wrap

    1. Run the command that the CLA responds with:

       $ sudo ausearch -m AVC,USER_AVC -ts recent
       Output: “avc:  denied  {name_bind}  for pid=7184 comm=”httpd” src=12345 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r: unreserved_port_t:s0 tclas=tcp_socket permissive=0"

       Copy to Clipboard Toggle word wrap

    2. Copy the output of the previous command:

       $ sudo c ““avc: denied {name_bind} for pid=7184 comm=”httpd” src=12345 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r: unreserved_port_t:s0 tclas=tcp_socket permissive=0”

       Copy to Clipboard Toggle word wrap

    3. Run the following command to resolve the error “SELinux is preventing Apache Server (httpd) from binding to port 12345”.

       $ sudo semage port -a -t http_port_t -p tcp 12345

       Copy to Clipboard Toggle word wrap

12. Restart the httpd service and check the status of httpd.service:

    $ sudo systemctl restart httpd
    No error
    $ sudo systemctl status httpd.service

    Copy to Clipboard Toggle word wrap

    The server is configured, up and running, and listening on port 443, port 12345.