Appendix D. Synchronizing Content Between Satellite Servers
Red Hat Satellite 6.2 uses Inter-Satellite Synchronization (ISS) to synchronize content between upstream and downstream servers. In the context of ISS, upstream refers to the server from which content is exported; downstream refers to the server into which content is imported.
ISS is designed to address two scenarios:
- If you have both connected and disconnected Satellite Servers, and want to propagate content from the connected servers to the disconnected servers.
-
If you have a primary Satellite Server and want to propagate some, but not all, content to other Satellite Servers. For example, you might have Content Views (CVs) that are validated by the IT department, and you want to propagate the
yum
content from those CVs to a downstream Satellite.
Be aware that you cannot use ISS to synchronize content from a Satellite Server to a Capsule Server. Capsule Server supports synchronization natively. See the Red Hat Architecture Guide for more details.
Satellite 6.2 supports exports as a set of directories (default) or as ISO files. You can then import the resulting export to another Satellite Server. This replaces the katello-disconnected
script in earlier Satellite versions, which exported repositories into a directory structure that could be later imported into another Satellite Server. In Satellite 6.2, all export and import functions are performed on the command line.
Only RPM, kickstart, and ISO files are exported. Content View metadata, such as package filters, is not transferred. Satellite 6.2 does not support the export of Puppet, Docker, or OSTree content. Imports occur as a normal repository synchronization, and consequently always arrive in the Library environment.
The disconnected use case is used extensively by customers who have air-gapped networks where a Satellite and its clients are on a network that is never connected to the Internet. The only way that these disconnected Satellites are populated with content is by exports from a connected Satellite.
Bidirectional synchronization is NOT used in a disconnected environment. Content never passes from the disconnected server to the connected server.
Figure D.1. Information Flow in ISS in an Air-gapped Network
D.1. Satellite Servers, Capsule Servers, and ISS
ISS serves a distinct purpose in a Red Hat Satellite deployment. If you intend to include ISS as part of your deployment, it equates to keeping a separate Satellite Server that you need to maintain, back up, and so on. ISS does not provide a failover mechanism for clients, and neither is it designed as a backup and recovery system. It is a means of providing information sharing between Satellite Servers. The primary use case for implementing ISS is when you have a Satellite Server that is not connected to the Internet or to external content, and you need to synchronize content with a Satellite that is connected to the Internet. This can apply when you need complete isolation of management infrastructure for security or other purposes.
If you do not want to maintain another management web UI and platform, and you want to perform management and provisioning to local clients, you should consider setting up a Capsule Server.
D.2. Prerequisites
- In Red Hat Satellite 6, ISS is only available in Satellite 6.2 and later, which requires Red Hat Enterprise Linux 6.7, 7.2, or later.
- The export directory needs to be large enough to accommodate at least one Red Hat Enterprise Linux export. By default, the export directory is /var/lib/pulp/katello-export/.
- The /var/lib/pulp/ directory must have free storage space equivalent to the size of the repository being exported for temporary files created during the export process. This is in addition to the space required by the default export directory.
- The downstream Satellite Server must have the required manifests and entitlements for any content that you intend to enable. You cannot enable repositories on a downstream Satellite for which no entitlements exist.
- The repository download policy needs to be set to immediate. This policy specifies whether or not Satellite first downloads metadata and other repository information, and only downloads actual repositories when requested. ISS will not function correctly if this policy is not set to immediate.
Section D.5, “Configuring ISS” describes how to configure required options.
D.3. Supported Synchronization Options
Satellite 6.2 supports the following synchronization options:
- Exporting repositories to a directory or ISO file.
- Exporting all repositories in an Environment or CV version to a directory or ISO file. You can also recreate any custom products during the import process, but Red Hat products are not recreated because they must be created using a manifest.
- Date-based incremental exports of RPM files and errata.
These synchronization options include a range of history details about the export and import, depending on the type of content. For example:
- The repository synchronization history includes upstream source information, as well as the time the export occurred.
- The CV synchronization history includes the export time and version, as well as the import time, version, and upstream source.
D.4. Using Chunked ISO Files
Satellite 6.2 supports exports to chunked ISO files. A chunked ISO is similar to a split ISO but with one significant difference. Satellite tracks the size of the ISO file, and if the total size of the files being added to the ISO exceeds that value, Satellite stops writing to the ISO and creates a new one in the series. The advantage of this is that you can specify the ISO file size (for example, 4.7 GB), and still export larger repositories. The result is multiple 4.7 GB ISO files that you can burn to DVDs.
The difference between this and splitting large files is that the split
utility is not aware of the ISO file format, which means that it will not create a new burnable ISO file for the next file in the series. This method requires that you copy all of the files to one place, concatenate those files, and then mount that single large ISO via loopback.
You can use the --iso-mb-size
parameter to specify the size of ISO export files. The default value is 4380 MB, the size of a single-sided, single-layer DVD.
D.5. Configuring ISS
This section describes how to configure the required settings for ISS. It is important that these settings be configured correctly or your synchronization may fail.
D.5.1. Configuring an Export Destination
Inter-Satellite Synchronization by default uses the /var/lib/pulp/katello-export/ directory, as indicated by the pulp_export_destination setting. To change this directory you must create the new directory and configure the Pulp export destination setting. Only Satellite Administrators can specify this directory, and SELinux and other permissions are also in place to prevent Satellite from writing to arbitrary file systems.
It can be helpful to create symbolic links to commonly used directories after they are exported. Exported repositories and CVs will have the organization name and environment name prepended to the repository directory structure, which might create paths of overlong length.
The directory used in this example is for demonstration purposes only. Confirm that the export directory has adequate space for the required export RPM and ISO files. Section 1.5, “Defining Content Management Storage” provides information on estimating storage requirements. A temporary file will be created during the export process in the /var/lib/pulp/ directory. This means that storage space equal to twice the size of the repository being exported is required during the export process. The temporary file will be deleted when the export is completed.
Create the Export Directory
Create the export directory:
# mkdir /var/www/html/pub/export
Ensure the
foreman
user has read and write permissions on the export directory:# chown foreman:foreman /var/www/html/pub/export
Configure the SELinux context:
# semanage fcontext -a -t httpd_sys_rw_content_t \ "/var/www/html/pub/export(/.*)?" # restorecon -RvF /var/www/html/pub/export # ls -Zd /var/www/html/pub/export \ drwxr-xr-x. foreman foreman \ system_u:object_r:httpd_sys_rw_content_t:s0 /var/www/html/pub/export
Configure the Export Destination
For CLI Users
To change the export destination using the command line, use a hammer
command in the following format:
# hammer settings set \
--name pulp_export_destination \
--value your-export-directory
For example, to specify /var/www/html/pub/export/ as the export destination, enter:
# hammer settings set \ --name pulp_export_destination \ --value /var/www/html/pub/export
For Web UI Users
- In the web UI, navigate to Administer > Settings, and click the Katello tab.
-
Locate the pulp_export_destination variable in the
Name
column, and click theValue
field. -
Enter the export destination, for example /var/www/html/pub/export, in the
Value
field and clickSave
.
D.5.2. Configuring the Download Policy
ISS requires that the Download Policy be set to immediate. You can set this globally so that it applies to new repositories created in all Organizations, or you can set it individually for every repository. Changing the default value will not change existing settings.
For CLI Users
To change the global default Download Policy using the command line, use a command as follows:
# hammer settings set \ --name default_download_policy \ --value immediate
If required to change the policy for a specific repository, you can list the repositories for an organization as follows:
# hammer repository list \
--organization-label <organization-label>
To change the download policy for an existing repository, use a command as follows:
# hammer repository update \
--organization-label <organization-label> \
--product "Red Hat Enterprise Linux Server" \
--name "Red Hat Enterprise Linux 7 Server Kickstart x86_64 7.2" \
--download-policy immediate
For Web UI Users
To change the global default Download Policy using the web UI:
- Go to Administer > Settings.
-
On the Katello tab, locate
default_download_policy
. - In the Value field, click the edit icon.
-
Set the value to immediate, and then click
Save
.
To change the Download Policy for an existing repository using the web UI:
- In the web UI, navigate to Content > Products, and click the required product name.
- On the Repositories tab, click the required repository, locate the Download Policy field, and click the edit icon.
-
From the drop-down list, select Immediate, and then click
Save
.
D.6. Exporting Content
This section describes how to export different types of content from your upstream server and import it to one or more downstream servers. Synchronization between upstream and downstream servers is currently supported in disconnected deployments, for example in an air-gapped environment, where complete isolation is required.
D.6.1. Exporting Repositories
Use the hammer repository export
command to export content from your upstream server. This command exports content to the directory specified in pulp_export_destination. ISS exports to a directory by default; you can use the --export-to-iso 1 parameter to export to an ISO file instead. For example:
# hammer repository export --id 1 [--export-to-iso 1]
If you use the --export-to-iso parameter, you need to specify either 1 (ISO) or 0 (directory). This parameter does not have a default value.
D.6.2. Exporting Content View Version to a Directory
You can export a specific version of a Content View to a directory. That means that you can label a particular version of a CV to suit your requirements. This way you will be able to curate and track your exports and facilitate updates.
To Create a Content View for Exporting
- Ensure all repositories within the CV have their download policy set to Immediate. You cannot export repositories with policies other than Immediate.
- Ensure Products are synchronized to the required date.
For Web UI Users
- Ensure Products are synchronized to the required date.
Go to Content
Content Views. Click Create New View. Enter following details to create a CV: - Name — A plain text name for the CV. Enter Export_CV.
- Label — An internal ID for the CV. Red Hat Satellite 6 automatically completes this field based on what you have entered for Name.
- Description — An optional plain text description of the CV.
- Composite View — Defines whether or not to use a Composite Content View. Leave this option unselected.
- Click Save to submit your changes.
- On the Repository Selection screen, select the repositories to be added to the new CV from the Repository Selection table. Click Add Repositories to add selected packages to the CV.
Go to Yum Content
Filters and click New Filter. Enter following details to create a filter for including non-errata packages: - Name — A plain text name for the filter. Enter Non-errata Products.
- Content type — A drop-down menu listing types of content to be included into the filter. Choose Package.
- Inclusion type — A drop-down menu defining whether the content will be included or excluded from the CV. Choose Include.
- Description — An optional plain text description of the filter. Enter Include all non-errata Products.
- Click Save.
- On the Include RPM screen, select the Include all RPMs with no errata check box.
Go to Yum Content
Filters and click New Filter. Enter following details to create a filter for including errata packages in accordance with the required date range: - Name — A plain text name for the filter. Enter Erratas untill YYYY-MM-DD.
- Content type — A drop-down menu listing types of content to be included into the filter. Choose Erratum - Date and Type.
- Inclusion type — A drop-down menu defining whether the content will be included or excluded from the CV. Choose Include.
- Description — An optional plain text description of the filter. Enter Include errata products untill YYYY-MM-DD.
- Click Save.
- On the Erratum Date Range screen, select all Security, Enhancement and Bugfix errata types.
- Select the Updated On check box in Data type.
- Fill in the Start Date and End Date menus to configure the date range of Products for the filter.
- Click Save.
- Click Publish New Version, fill in a short description of a version into the Description field. Ensure the Force Yum Metadata Regeneration check box is not selected.
- Click Save to publish the CV version ready for exporting.
For CLI Users
Use the
hammer content-view create
command to create a new CV:# hammer content-view create \ --name "Export_CV" \ --organization "Default Organization"
Use the
hammer content-view add-repository
command to add repositories to the CV:# hammer content-view add-repository \ --name "Export_CV" \ --product "Red Hat Satellite" \ --repository "Red Hat Satellite Tools 6.3 for RHEL 7 Server RPMs x86_64" \ --organization "Default Organization" # hammer content-view add-repository \ --name "Export_CV" \ --product "Red Hat Satellite Capsule" \ --repository "Red Hat Satellite Capsule Tools 6.2 for RHEL 7 Server RPMs x86_64" \ --organization "Default Organization"
Create filters for the new CV:
Use the
hammer content-view filter create
command to create a filter for including non-errata packages:# hammer content-view filter create \ --content-view "Export_CV" \ --inclusion true \ --name "Non-errata_Products" \ --type rpm \ --original-packages true \ --organization "Default Organization"
Use the
hammer content-view filter create
command to create a filter for including errata packages:# hammer content-view filter create \ --content-view "Export_CV" \ --inclusion true \ --name "Erratas until YYYY-MM-DD" \ --type erratum \ --organization "Default Organization"
Use the
hammer content-view filter rule create
command to create a rule defining the date range:# hammer content-view filter rule create \ --content-view "Export_CV" \ --content-view-filter "Erratas until YYYY-MM-DD" \ --end-date YYYY-MM-DD \ --types security,enhancement,bugfix \ --organization "Default Organization"
Use the
hammer content-view publish
command to publish the CV version ready for exporting. It is recommended to fill in the date ranges of repositories under the--description
option.# hammer content-view publish \ --name "Export_CV" \ --description "Repositories until YYYY-MM-DD" \ --force-yum-metadata-regeneration true \ --async \ --organization "Default Organization"
To Determine Which Content View Version to Export:
Use the
hammer content-view version list
command to determine which version of a Content View to export. For example:$ hammer content-view version list \ --organization "Default Organization" ---|-------------------------------|---------|----------------------- ID | NAME | VERSION | LIFECYCLE ENVIRONMENTS ---|-------------------------------|---------|----------------------- 3 | Export_CV 2.0 | 2.0 | Library 2 | Export_CV 1.0 | 1.0 | Library 1 | Default Organization View 1.0 | 1.0 | Library ---|-------------------------------|---------|-----------------------
To Export a Content View Version:
Use the
hammer content-view version export
command to export a version of a Content View:# hammer content-view version export --id 3
On a downstream server, use the
hammer organization update
command to add new repositories to an organization. Set the address to the directory corresponding to the version you want within the exported Content View as shown:$ hammer organization update \ --name "Default Organization" \ --redhat-repository-url \ http://satellite.example.com/var/www/html/pub/export/Default_Organization-Export_CV-v2.0/Default_Organization/content_views/Export_CV/2.0 Organization updated
D.6.3. Incremental Updates
To avoid exporting large repositories from a Satellite Server in order to get recent updates you can make make use of incremental updates. Incremental updates are made by exporting changes made to a local repository, by one or more synchronization events, since a particular date and time.
To make an incremental-update repository, use the hammer repository export
command with the --since
option. For example:
# hammer repository export \
--id 1 [--export-to-iso 1] \
--since [ <ISO_Date> ]
Where ISO_Date is in ISO 8601 format. For example, 2010-01-01T12:00:00Z
.
The time stamp used for the calculations is the time that the RPMs were synchronized on the Satellite Server. For example, if Red Hat adds RPMs to a repository on a Monday and then again on a Wednesday, you cannot synchronize your local repository on Thursday and then use a date of Tuesday to get only the Wednesday update.
In addition to exporting changes to a repository, this feature is useful with the Default Organization View content view, but is not as useful for published CVs.
Ensure you use the --incremental
option to the hammer repository synchronize
command when synchronizing from an incremental export. If you do not use this option and also have "Mirror on Sync" enabled for the repository, Satellite Server will treat the import as if it is a full import and will erase all data that was not in the incremental export. Recovering from such a scenario requires the time consuming process of doing a full export and then synchronizing from that.
D.7. Importing Content
Red Hat Satellite 6.2 currently supports importing content that has been exported from an upstream Satellite Server in a disconnected environment. This method is used for disconnected Satellite Servers that have no Internet access, and requires the physical transfer of content between the servers, for example using a DVD.
D.7.1. Importing a Repository
Prerequisites
- Export the repository from the upstream Satellite Server. For more details, see Section D.6.1, “Exporting Repositories”.
To Import a Repository:
-
Make the data available for a repository over HTTP, not HTTPS. For example, copy the exported directory to the
/var/www/html/pub/export/
directory on the downstream server, which is available over HTTP by default. -
In the web UI, navigate to Content
Red Hat Subscriptions. - Select Manage Manifests.
On the Import/Remove Manifest tab, set the Red Hat CDN URL address field to match the location of a
content
directory and alisting
file within the exported repository.For example, if the exported repository is located in
/var/www/html/pub/export/7f6a4f46-c77b-4781-bbd9-871764f8c8b8
, set the URL to behttp://satellite.example.com/pub/export/7f6a4f46-c77b-4781-bbd9-871764f8c8b8/Default_Organization/Library/
.- Click Save.
-
Navigate to Content
Red Hat Repositories and select the check box of the repository you have exported.
D.7.2. Importing a Content View as a Red Hat Repository
Prerequisites
- You have a Content View with Red Hat repositories on the upstream Satellite Server.
- Export the Content View from the upstream Satellite Server. For more details, see Section D.6.2, “Exporting Content View Version to a Directory”.
A custom repository can also be imported to a custom product, for example a disconnected Satellite. For more information about Red Hat Content Delivery Network (CDN) see Content Delivery Network (CDN) Structure in the Architecture Guide.
To Import a Content View:
-
Make the data available for a repository over HTTP, not HTTPS. For example, copy it to the
/var/www/html/pub/export/
directory on the downstream server. -
In the web UI, navigate to Content
Red Hat Subscriptions. - Select Manage Manifests.
On the Import/Remove Manifest tab, set the Red Hat CDN URL address field to match the location of a
content
directory and alisting
file within the exported Content View.For example, if the exported CV is located in
/var/www/html/pub/export/Default_Organization-Export_CV-v1.0
, set the URL to behttp://satellite.example.com/pub/export/Default_Organization-Export_CV-v1.0/Default_Organization/content_views/Export_CV/1.0/
- Click Save.
-
Navigate to Content
Red Hat Repositories and check the repository you have exported. On a downstream server, use the
hammer organization update
command to add new repositories to an organization. Set the address to the directory corresponding to the version you want within the exported Content View as shown:$ hammer organization update \ --name "Default Organization" \ --redhat-repository-url \ http://satellite.example.com/pub/export/Default_Organization-Export_CV-v1.0/Default_Organization/content_views/Export_CV/1.0/ Organization updated