Chapter 19. Configuring and setting up remote jobs

1. Satellite finds the host’s interfaces that have the Remote execution checkbox selected.
2. Satellite finds the subnets of these interfaces.
3. Satellite finds remote execution Capsules assigned to these subnets.
4. From this set of Capsules, Satellite selects the Capsule that has the least number of running jobs. By doing this, Satellite ensures that the jobs load is balanced between remote execution Capsules.

Verification

1. Determine which version of the yggdrasil package is installed on the host:

   $ rpm --query yggdrasil

2. Check the status of the Yggdrasil services:

   • If your host has yggdrasil version 0.4.z or later installed:

     # systemctl status yggdrasil com.redhat.Yggdrasil1.Worker1.foreman

   • If your host has yggdrasil version 0.2.z or earlier installed:

     # systemctl status yggdrasild

Procedure

1. In the Satellite web UI, navigate to Monitor > Jobs and click Run job.
2. Select the Job category and the Job template you want to use, then click Next.

3. Select hosts on which you want to run the job. If you do not select any hosts, the job will run on all hosts you can see in the current context.

   Note

   If you want to select a host group and all of its subgroups, it is not sufficient to select the host group as the job would only run on hosts directly in that group and not on hosts in subgroups. Instead, you must either select the host group and all of its subgroups or use this search query:

   hostgroup_fullname ~ "My_Host_Group*"

   Replace My_Host_Group with the name of the top-level host group.

4. If required, provide inputs for the job template. Different templates have different inputs and some templates do not have any inputs. After entering all the required inputs, click Next.
5. Optional: To configure advanced settings for the job, fill in the Advanced fields. To learn more about advanced settings, see Section 19.7.1, “Advanced settings in the job wizard”.
6. Click Next.

7. Schedule time for the job.

   • To execute the job immediately, keep the pre-selected Immediate execution.
   • To execute the job in future time, select Future execution.
   • To execute the job on regular basis, select Recurring execution.

8. Optional: If you selected future or recurring execution, select the Query type, otherwise click Next.

   • Static query means that job executes on the exact list of hosts that you provided.
   • Dynamic query means that the list of hosts is evaluated just before the job is executed. If you entered the list of hosts based on some filter, the results can be different from when you first used that filter.

   Click Next after you have selected the query type.

9. Optional: If you selected future or recurring execution, provide additional details:

   • For Future execution, enter the Starts at date and time. You also have the option to select the Starts before date and time. If the job cannot start before that time, it will be canceled.
   • For Recurring execution, select the start date and time, frequency, and the condition for ending the recurring job. You can choose the recurrence to never end, end at a certain time, or end after a given number of repetitions. You can also add Purpose - a special label for tracking the job. There can only be one active job with a given purpose at a time.

   Click Next after you have entered the required information.

10. Review job details. You have the option to return to any part of the job wizard and edit the information.
11. Click Submit to schedule the job for execution.

Procedure

1. Enter the following command on Satellite:

   $ hammer settings set \
   --name remote_execution_global_proxy \
   --value false

2. Find the ID of the job template you want to use:

   $ hammer job-template list

3. Show the template details to see parameters required by your template:

   $ hammer job-template info --id My_Template_ID

4. Execute a remote job with custom parameters:

   $ hammer job-invocation create \
   --inputs My_Key_1="My_Value_1",My_Key_2="My_Value_2",... \
   --job-template "My_Template_Name" \
   --search-query "My_Search_Query"

   Replace My_Search_Query with the filter expression that defines hosts, for example "name ~ My_Pattern".

Procedure

1. In the Satellite web UI, navigate to Administer > Settings.
2. Click Remote Execution.
3. Configure the Fallback to Any Capsule setting.

Procedure

1. In the Satellite web UI, navigate to Administer > Settings.
2. Click Remote Execution.
3. Configure the Enable Global Capsule setting.

Procedure

1. On your host, create a new directory:

   # mkdir /My_Remote_Working_Directory

2. Copy the SELinux context from the default /var/tmp directory:

   # chcon --reference=/var/tmp /My_Remote_Working_Directory

3. Configure your Satellite Server or Capsule Server to use the new directory:

   # satellite-installer \
   --foreman-proxy-plugin-remote-execution-script-remote-working-dir /My_Remote_Working_Directory

Procedure

1. Create a new directory:

   # mkdir /My_Remote_Working_Directory

2. Access the Yggdrasil service configuration:

   • If your host has yggdrasil version 0.4.z or later installed:

     # systemctl edit com.redhat.Yggdrasil1.Worker1.foreman

   • If your host has yggdrasil version 0.2.z or earlier installed:

     # systemctl edit yggdrasild

3. Specify the alternative directory by adding the following line to the configuration:

   Environment=FOREMAN_YGG_WORKER_WORKDIR=/My_Remote_Working_Directory

4. Restart the Yggdrasil services:

   • If your host has yggdrasil version 0.4.z or later installed:

     # systemctl restart yggdrasil com.redhat.Yggdrasil1.Worker1.foreman

   • If your host has yggdrasil version 0.2.z or earlier installed:

     # systemctl restart yggdrasild

Procedure

1. Navigate to Administer > Settings.
2. Select the Remote Execution tab.
3. Click the value of the Effective User Method setting.
4. Select the new value.
5. Click Submit.

Procedure

1. If the ~root/.ssh/authorized_keys file does not exist, create it with restrictive ownership and permissions:

   1. If the ~root/.ssh directory does not exist, create it:

      # mkdir ~root/.ssh

   2. Ensure that the directory is owned by the root user:

      # chown root:root ~root/.ssh

   3. Ensure that the directory is accessible only to the root user:

      # chmod 700 ~root/.ssh

   4. Create the authorized_keys file:

      # touch ~root/.ssh/authorized_keys

   5. Ensure that the authorized_keys file is owned by the root user:

      # chown root:root ~root/.ssh/authorized_keys

   6. Ensure that the authorized_keys file is accessible only to the root user:

      # chmod 600 ~root/.ssh/authorized_keys

2. Add the SSH key of the foreman-proxy user to the authorized_keys file:

   # cat ~foreman-proxy/.ssh/id_rsa_foreman_proxy.pub >>~root/.ssh/authorized_keys

Procedure

1. On the target host, create the ~/.ssh directory to store the SSH key:

   # mkdir ~/.ssh

2. Download the SSH key from Capsule:

   # curl https://capsule.example.com:9090/ssh/pubkey >> ~/.ssh/authorized_keys

3. Configure permissions for the ~/.ssh directory:

   # chmod 700 ~/.ssh

4. Configure permissions for the authorized_keys file:

   # chmod 600 ~/.ssh/authorized_keys

Procedure

1. On the Capsule Server, locate the SSH key pair that Capsule uses for remote execution. You can find the private key location in /etc/foreman-proxy/settings.d/remote_execution_ssh.yml under the :ssh_identity_key_file: setting. By default, the location is /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy.
2. If the CA is on a different server than your Capsule Server, copy the public key file of the Capsule to the CA server.

3. Sign the Capsule SSH public key with your CA.

   Note

   The ssh-keygen command creates the certificate file with the name My_Capsule_Private_Key-cert.pub. Do not change the name of the file. By default, the created SSH certificate is named id_rsa_foreman_proxy-cert.pub.

4. If the CA is on a different server than your Capsule Server, ensure the Capsule Server has the SSH certificate and the CA public key file:

   1. Copy the created SSH certificate file back to the Capsule Server into the same directory as the original SSH key pair.

   2. Copy the CA public key file to the Capsule Server.

      Note

      By default, the foreman-proxy user has read access to the CA public key file. Do not store the file in non-standard directories so that the foreman-proxy user can read the file. Red Hat recommends storing the file in the /etc/ssh/ directory.

5. Enable SSH certificate authentication on your Capsule:

   # satellite-installer \
   --foreman-proxy-plugin-remote-execution-script-ssh-user-ca-public-key-file My_CA_Public_Key_File_Path

Verification

1. Run a remote execution job on a host through your Capsule with SSH certificate authentication enabled.

2. On the host, view the sshd logs:

   # journalctl -u sshd

3. Find the log message that confirms that the SSH certificate was used for authentication:

   Accepted publickey for <SSH USER> from <IP> port 22 ssh2: RSA-CERT ID foreman-proxy CA RSA

Procedure

1. Distribute your Capsule SSH keys to the hosts that use your Capsule for remote execution. For more information, see Section 19.2.3, “Distributing SSH keys for remote execution”.
2. Delete the SSH certificate file of the Capsule. By default, the path of this file is /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy-cert.pub.

3. Disable the SSH certificate authentication:

   # satellite-installer \
   --reset-foreman-proxy-plugin-remote-execution-script-ssh-user-ca-public-key-file

Procedure

1. On the Capsule Server, locate the directory that Capsule uses for remote execution by checking where the configured Capsule SSH private key is stored. You can find the location of the SSH private key in /etc/foreman-proxy/settings.d/remote_execution_ssh.yml under the :ssh_identity_key_file: setting. By default, the directory is /var/lib/foreman-proxy/ssh/.

2. On the Capsule Server, generate a new SSH key pair in the directory:

   # ssh-keygen -f My_Capsule_SSH_Directory/My_New_SSH_Key

3. Set the correct ownership for the new key files:

   # chown foreman-proxy:foreman-proxy My_Capsule_SSH_Directory/My_New_SSH_Key*

4. If the CA is on a different server than your Capsule Server, copy the new public key file to the CA server.
5. Sign the new SSH public key with your CA.
6. If the CA is on a different server than your Capsule Server, copy the created SSH certificate file back to the Capsule Server into the Capsule SSH directory.

7. Set the correct ownership for the certificate file:

   # chown foreman-proxy:foreman-proxy My_Capsule_SSH_Directory/My_New_SSH_Key-cert.pub

8. Configure Capsule to use the new SSH key:

   # satellite-installer \
   --foreman-proxy-plugin-remote-execution-script-ssh-identity-file My_Capsule_SSH_Directory/My_New_SSH_Key

   Note

   If you created the new SSH key pair with the same name as the old one and thus replaced the old key pair, you do not need to update your Capsule configuration with satellite-installer.

9. Optional: Remove the old SSH key files from the Capsule SSH directory.

Verification

1. Run a remote execution job on a host through your Capsule with SSH certificate authentication enabled.

2. On the host, view the sshd logs:

   # journalctl -u sshd

3. Find the log message that confirms that the SSH certificate was used for authentication:

   Accepted publickey for <SSH USER> from <IP> port 22 ssh2: RSA-CERT ID foreman-proxy CA RSA

Procedure

1. Copy the CA public key file to the host.

2. Ensure that the /etc/ssh/sshd_config.d/ directory exists:

   # mkdir --parents /etc/ssh/sshd_config.d/

3. Configure the sshd service to trust the CA:

   # echo 'TrustedUserCAKeys My_CA_Public_Key_File_Path' > /etc/ssh/sshd_config.d/60-foreman-user-ca.conf

4. Set the correct SELinux context for the created file:

   # restorecon /etc/ssh/sshd_config.d/60-foreman-user-ca.conf

5. Restart the sshd service:

   # systemctl restart sshd

6. Remove existing SSH keys of the Capsule from the authorized_keys file of your SSH user. By default, the SSH user is root and the SSH keys are associated with the foreman-proxy user:

   # sed -i '/foreman-proxy/d' ~/.ssh/authorized_keys

   Note

   This command might not work if you configured custom SSH keys on your Capsule. In that case, manually edit the authorized_keys file.

Procedure

1. On each host that uses this Capsule for remote execution, prepare the SSH certificate for the host:

   1. Copy one of the public key files of the host to the CA server. The public keys are located in /etc/ssh/.

   2. Sign the public key with your CA to create an SSH host certificate. Use the -h option when generating the SSH host certificate with ssh-keygen.

      Important

      The ssh-keygen command creates the certificate file with the name My_Host_Private_SSH_Key-cert.pub. Do not change the name of the file.

   3. Copy the SSH certificate file back to the host. The location must be the same as the location of the private key.

   4. Configure the host to use the certificate:

      # mkdir --parents /etc/ssh/sshd_config.d/
      # echo 'HostCertificate My_Host_SSH_Certificate_Path' > /etc/ssh/sshd_config.d/60-host-cert.conf
      # restorecon -R /etc/ssh/

      Note

      If the host SSH public key you used for the certificate is not one of the default keys (/etc/ssh/ssh_host_ecdsa_key.pub, /etc/ssh/ssh_host_ed25519_key.pub, or /etc/ssh/ssh_host_rsa_key.pub), also add the host key configuration:

      # echo 'HostKey My_Host_Private_SSH_Key_Path' >> /etc/ssh/sshd_config.d/60-host-cert.conf

   5. Restart the sshd service:

      # systemctl restart sshd

2. On the Capsule, configure host certificate verification:

   1. If the host CA is not located on the Capsule Server, copy the host CA public key file to your Capsule Server. If you have multiple host CAs, place all of their public keys in a single file, with each key on a separate line.

   2. Enable host certificate verification:

      # satellite-installer \
      --foreman-proxy-plugin-remote-execution-script-ssh-host-ca-public-keys-file My_Host_CA_Public_Keys_File_Path

Procedure

1. Find the ID of the foreman-proxy user:

   # id -u foreman-proxy

2. Modify the umask value so that new files have the permissions 600:

   # umask 077

3. Create the directory for the keytab:

   # mkdir -p "/var/kerberos/krb5/user/My_User_ID"

4. Create a keytab or copy an existing keytab to the directory:

   # cp My_Client.keytab /var/kerberos/krb5/user/My_User_ID/client.keytab

5. Change the directory owner to the foreman-proxy user:

   # chown -R foreman-proxy:foreman-proxy "/var/kerberos/krb5/user/My_User_ID"

6. Ensure that the keytab file is read-only:

   # chmod -wx "/var/kerberos/krb5/user/My_User_ID/client.keytab"

7. Restore the SELinux context:

   # restorecon -RvF /var/kerberos/krb5

Procedure

1. Enable Kerberos authentication for remote execution:

   # satellite-installer --foreman-proxy-plugin-remote-execution-script-ssh-kerberos-auth true

2. To edit the default user for remote execution, in the Satellite web UI, navigate to Administer > Settings and click the Remote Execution tab. In the SSH User row, edit the second column and add the user name for the Kerberos account.
3. Navigate to remote_execution_effective_user and edit the second column to add the user name for the Kerberos account.

Procedure

1. In the Satellite web UI, navigate to Hosts > Templates > Job templates.
2. Click New Job Template.
3. Click the Template tab, and in the Name field, enter a unique name for your job template.
4. Select Default to make the template available for all organizations and locations.
5. Create the template directly in the template editor or upload it from a text file by clicking Import.
6. Optional: In the Audit Comment field, add information about the change.
7. Click the Job tab, and in the Job category field, enter your own category or select from the default categories.
8. Optional: In the Description Format field, enter a description template. For example, Install package %{package_name}. You can also use %{template_name} and %{job_category} in your template.
9. From the Provider Type list, select SSH for shell scripts and Ansible for Ansible tasks or playbooks.
10. Optional: In the Timeout to kill field, enter a timeout value to terminate the job if it does not complete.
11. Optional: Click Add Input to define an input parameter. Parameters are requested when executing the job and do not have to be defined in the template. For examples, see the Help tab.
12. Optional: Click Foreign input set to include other templates in this job.
13. Optional: In the Effective user area, configure a user if the command cannot use the default remote_execution_effective_user setting.
14. Optional: If this template is a snippet to be included in other templates, click the Type tab and select Snippet.

15. Optional: If you use the Ansible provider, click the Ansible tab.

    1. Select Enable Ansible Callback to allow hosts to send facts, which are used to create configuration reports, back to Satellite after a job finishes.
    2. Select Enable Ansible Check Mode to run jobs based on the template in Ansible check mode, which executes Ansible playbooks without making changes to hosts. For more information on Ansible check mode, see Using automation execution in Red Hat Ansible Automation Platform documentation.
16. Click the Location tab and add the locations where you want to use the template.
17. Click the Organizations tab and add the organizations where you want to use the template.
18. Click Submit to save your changes.

Procedure

1. Fetch the available Ansible Playbooks by using the following API request:

   $ curl \
   --header 'Content-Type: application/json' \
   --request GET \
   https://satellite.example.com/ansible/api/v2/ansible_playbooks/fetch?proxy_id=My_Capsule_ID

2. Select the Ansible Playbook you want to import and note its name.

3. Import the Ansible Playbook by its name:

   $ curl \
   --data '{ "playbook_names": ["My_Playbook_Name"] }' \
   --header 'Content-Type: application/json' \
   --request PUT \
   https://satellite.example.com/ansible/api/v2/ansible_playbooks/sync?proxy_id=My_Capsule_ID

   You get a notification in the Satellite web UI after the import completes.

Procedure

1. In the Satellite web UI, navigate to Hosts > Templates > Job templates.
2. To clone a template, in the Actions column, select Clone.
3. Enter a unique name for the clone and click Submit to save the changes.

4. In the list of job templates, click the cloned template to start editing it.

   Note

   An Ansible template must begin with ---. You can embed an Ansible Playbook YAML file into the job template body. You can also add ERB syntax to customize your YAML Ansible template.

Procedure

1. In the Satellite web UI, navigate to Hosts > All Hosts.
2. Select your host.
3. On the Ansible tab, select Jobs.
4. Click Schedule recurring job.
5. In the Repeat list, select the interval.
6. In the Start time field, enter the start time of the recurring job.
7. In the Start date field, enter the start date of the recurring job.
8. Click Submit.

Procedure

1. In the Satellite web UI, navigate to Configure > Host Groups.
2. In the Actions column, select Configure Ansible Job for the host group you want to schedule an Ansible roles run for.
3. Click Schedule recurring job.
4. In the Repeat list, select the interval.
5. In the Start time field, enter the start time of the recurring job.
6. In the Start date field, enter the start date of the recurring job.
7. Click Submit.

Procedure

1. In the Satellite web UI, navigate to Administer > Remote Execution Features.
2. Find each feature whose name contains by_search.
3. Change the job template for these features from Katello Script Default to Katello Ansible Default.

4. Click Submit.

   Satellite now uses Ansible provider templates for remote execution jobs by which you can perform package and errata actions. This applies to job invocations from the Satellite web UI as well as by using hammer job-invocation create with the same remote execution features that you have changed.