Chapter 5. Using the command-line assistant to debug or troubleshoot system issues

Procedure

1. Check the SSHD status and restart it.

   $ sudo systemctl status ssh
   $ sudo systemctl restart ssh

2. Enable the optional command-line assistant terminal capture feature:

   $ c shell --enable-capture

3. Use the -w 1 "your-question" option to specify to include the output from the last command that was run.

   $ c -w 1 "what_is_this"

   • If you specify the number 2, that references the output from the second previous 2 commands. This is also true for the additional numbers.

   • You can also specify a prompt to run with the command and ask “help me understand the output", and reference the output with the error, so that the command my assistant understands that you are asking for more details on what is the error.

     The command-line assistant takes some time to process the request, and provide several possible solutions. In the example, you can use the suggestion to run the journalctl -xeu command, so that you can use sshd.service to check the log files.

4. Run that journalctl command. Add the tail command to get the last 30 lines, pipe that output into the command-line assistant, and add a query to understand the error.

   $ journalctl -xeu sshd.service | tail -n 30 | c “here are the logs, please help me understand this"

   The command-line assistant checks the log files and indicates some potential issues. typing error in the config file.

5. Ask the command-line assistant to generate a command on how to fix this typing error.

   $ c “what is the command that I can use to change “Porrt "to “Port” in the /etc/ssh/sshd_config file?”

   Use the command suggested by the command-line assistant.

6. Run the command suggested by the command-line assistant. For example:

   $ sed -i s/Porrt/Port/g /etc/ssh/sshd_config

   The output found a permission denied error to edit that file. Rerun the previous command as a sudo user.

   $ sudo sed -i s/Porrt/Port/g /etc/ssh/sshd_config

7. Restart the SSHD service and check the status of the SSHD.

   $ sudo systemctl restart sshd
   $ sudo systemctl status sshd

Procedure

1. On your terminal, enter the following command to list the httpd package version that you have installed in your system:

   $ sudo rpm -qa httpd
   httpd-2.4.62-2.fc40.x86_64

2. Show the httpd.conf file content.

   $ sudo cat /etc/httpd/conf/httpd.conf
   # This is the main Apache HTTP server configuration file.  It contains the
   # configuration directives that give the server its instructions.
   # See <URL:http://httpd.apache.org/docs/2.4/> for detailed information.
   # In particular, see<URL:http://httpd.apache.org/docs/2.4/mod/directives.html>
   # for a discussion of each configuration directive.

3. Query all httpdq packages:

   $ sudo rpm -qa httpdq

4. Identify the ports on which the web server accepts incoming requests:

   $ cat /etc/httpd/conf/httpd.conf | grep Listen
   # Listen: Allows you to bind Apache to specific IP addresses and/or
   # Change this to Listen on a specific IP address, but note that if
   #Listen 12.34.56.78:80
   Listen 80

5. Restart the httpd service:

   $ systemctl restart httpd
   
   Job for httpd. Service failed because the control process exited with error code.
   See “systemctl status httpd.service" and “journalctl -xeu httpd.service" for details.

   1. Run the journalctl command for more details on the failed service:

      $ sudo journalctl -xeu httpd.service

6. Use the command-line assistant to troubleshoot the issue and ask why the service is failing:

   $ sudo c “why did httpd fail to start"

   One of the suggestions from the assistant is to query audit logs by using the ausearch tool, and use the AVC and USER_AVC values for the message type parameter. For that, run the following command:

   $ sudo ausearch -m AVC,USER_AVC -ts recent

   1. Ask the command-line assistant about the selinux httpd port:

      $ c “selinux httpd port"

      The assistant advises to use the sestatus command to check the current SELinux status and the content of the httpd services with the following command:

      $ sudo sestatus
      
      SELinux status:                 enabled
      SELinuxfs mount:                /sys/fs/selinux
      SELinux root directory:         /etc/selinux
      Loaded policy name:             targeted
      Current mode:                   enforcing
      Mode from config file:          enforcing
      Policy MLS status:              enabled
      Policy deny_unknown status:     allowed
      Memory protection checking:     actual (secure)
      Max kernel policy version:      33

   2. View the specific SELinux policy for the httpd services by running the following command:

      $ sudo cat /usr/share/selinux/targeted/contexts/httpd_var_run_t
      No such file or directory

   3. Ask the command-line assistant about contexts.

      $ c “i don't have a httpd_var_run_t contexts"

      The command-line assistant takes some time to process the request, then provides several possible suggestions.

   4. The assistant says that you might not have context and need to set it with the following command:

      $ sudo chcon -R -t httpd_var_run_t

   5. Ask the CLA about the port:

      $ c “selinux won't let httpd listen on port 12345"

   6. Try the following suggestion, run the command:

      $ sudo semage port -a -t httpd_port_t -p tcp 12345
      ValueError: Type httpd_port_t is invalid, must be a port type

   7. Ask the CLA about the error you see in the output:

      $ c “how do I fix ValueError: Type httpd_port_t is invalid, must be a port type"

7. Run the steps provided by the CLA:

   $  sudo getenforce
   Enforcing
   $ setenforce 0
   $ sudo systemctl restart httpd
   $ sudo systemctl status httpd
   
   $ sudo ls -Z /usr/sbin/httpd
   system_u:object_r:httpd_exec_t:s0 /usr/sbin/httpd
   
   $ chcon -t httpd_exec_t /usr/sbin/httpd
   
   $ sudo setenforce 1

   1. Restart the httpd service and check the status of httpd.service:

      $ sudo systemctl restart httpd
      Job failed
      
      $ sudo systemctl status httpd.service
      Failed to start the Apache Server

8. Ask the CLA how to enable httpd to listen on port 12345:

   $ c “how do I enable httpd to listen on port 12345 selinux"

   1. Run the command advised by the CLA:

      $ sudo setsebool -P httpd_can_network_connect=1

9. Restart the httpd service and check the status of httpd.service again:

   $ sudo systemctl status htppd
   $ sudo systemctl restart httpd
   Job failed, see journalctl

10. Check the journalctl service:

    $ journalctl -xeu httpd
    Output: An ExecStart= process belonging to unit httpd.service has exited.

11. Use the output to ask the CLA to troubleshoot:

    $ c “An ExecStart= process belonging to unit httpd.service has exited."

    1. Run the command that the CLA responds with:

       $ sudo ausearch -m AVC,USER_AVC -ts recent
       Output: “avc:  denied  {name_bind}  for pid=7184 comm="httpd” src=12345 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r: unreserved_port_t:s0 tclas=tcp_socket permissive=0"

    2. Copy the output of the previous command:

       $ sudo c ““avc: denied {name_bind} for pid=7184 comm="httpd” src=12345 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r: unreserved_port_t:s0 tclas=tcp_socket permissive=0”

    3. Run the following command to resolve the error “SELinux is preventing Apache Server (httpd) from binding to port 12345".

       $ sudo semage port -a -t http_port_t -p tcp 12345

12. Restart the httpd service and check the status of httpd.service:

    $ sudo systemctl restart httpd
    No error
    $ sudo systemctl status httpd.service

    The server is configured, up and running, and listening on port 443, port 12345.