This documentation is for a release that is no longer maintained
See documentation for the latest supported version 3 or the latest supported version 4.此内容没有您所选择的语言版本。
Chapter 7. Docker tasks
OpenShift Container Platform uses Docker to run applications in pods that are composed by any number of containers.
As a cluster administrator, sometimes Docker requires some extra configuration in order to efficiently run elements of the OpenShift Container Platform installation.
7.1. Increasing Docker storage 复制链接链接已复制到粘贴板!
Increasing the amount of storage available ensures continued deployment without any outages. To do so, a free partition must be made available that contains an appropriate amount of free capacity.
7.1.1. Evacuating the node 复制链接链接已复制到粘贴板!
Procedure
From a master instance, or as a cluster administrator, allow the evacuation of any pod from the node and disable scheduling of other pods on that node:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteIf there are containers running with local volumes that will not migrate, run the following command:
oc adm drain ${NODE} --ignore-daemonsets --delete-local-data
.List the pods on the node to verify that they have been removed:
oc adm manage-node ${NODE} --list-pods
$ oc adm manage-node ${NODE} --list-pods Listing matched pods on node: ose-app-node01.example.com NAME READY STATUS RESTARTS AGE
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
For more information on evacuating and draining pods or nodes, see Node maintenance.
7.1.2. Increasing storage 复制链接链接已复制到粘贴板!
You can increase Docker storage in two ways: attaching a new disk, or extending the existing disk.
Increasing storage with a new disk
Prerequisites
A new disk must be available to the existing instance that requires more storage. In the following steps, the original disk is labeled
/dev/xvdb
, and the new disk is labeled/dev/xvdd
, as shown in the /etc/sysconfig/docker-storage-setup file:vi /etc/sysconfig/docker-storage-setup
# vi /etc/sysconfig/docker-storage-setup DEVS="/dev/xvdb /dev/xvdd"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteThe process may differ depending on the underlying OpenShift Container Platform infrastructure.
Procedure
Stop the
docker
andatomic-openshift-node
services:systemctl stop docker atomic-openshift-node
# systemctl stop docker atomic-openshift-node
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
docker-storage-setup
command to extend the volume groups and logical volumes associated with container storage:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Start the Docker services:
systemctl start docker vgs
# systemctl start docker # vgs VG #PV #LV #SN Attr VSize VFree docker_vol 2 1 0 wz--n- 64.99g <55.00g
Copy to Clipboard Copied! Toggle word wrap Toggle overflow A benefit in adding a disk compared to creating a new volume group and re-running
docker-storage-setup
is that the images that were used on the system still exist after the new storage has been added:Copy to Clipboard Copied! Toggle word wrap Toggle overflow With the increase in storage capacity, enable the node to be schedulable in order to accept new incoming pods.
As a cluster administrator, run the following from a master instance:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Increasing storage with a new disk
- Evacuate the node following the previous steps.
Stop the
docker
andatomic-openshift-node
services:systemctl stop docker atomic-openshift-node
# systemctl stop docker atomic-openshift-node
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Resize the existing disk as desired. This can can depend on your environment:
If you are using LVM (Logical Volume Manager):
lvremove /dev/docker_vg/docker/lv
# lvremove /dev/docker_vg/docker/lv
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Remove the Docker volume group:
vgremove docker_vg
# vgremove docker_vg
Copy to Clipboard Copied! Toggle word wrap Toggle overflow pvremove /dev/<my_previous_disk_device>
# pvremove /dev/<my_previous_disk_device>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- If you are using a cloud provider, you can detach the disk, destroy the disk, then create a new bigger disk, and attach it to the instance.
For a non-cloud environment, the disk and file system can be resized. See the following solution for more information:
- Verify that the /etc/sysconfig/container-storage-setup file is correctly configured for the new disk by checking the device name, size, etc.
Run
docker-storage-setup
to reconfigure the new disk:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Start the Docker services:
systemctl start docker vgs
# systemctl start docker # vgs VG #PV #LV #SN Attr VSize VFree docker_vol 2 1 0 wz--n- 64.99g <55.00g
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Start the
atomic-openshift-node
service:systemctl start atomic-openshift-node
# systemctl start atomic-openshift-node
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
7.1.3. Changing the storage backend 复制链接链接已复制到粘贴板!
With the advancements of services and file systems, changes in a storage backend may be necessary to take advantage of new features. The following steps provide an example of changing a device mapper backend to an overlay2
storage backend. overlay2
offers increased speed and density over traditional device mapper.
7.1.3.1. Evacuating the node 复制链接链接已复制到粘贴板!
From a master instance, or as a cluster administrator, allow the evacuation of any pod from the node and disable scheduling of other pods on that node:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteIf there are containers running with local volumes that will not migrate, run the following command:
oc adm drain ${NODE} --ignore-daemonsets --delete-local-data
List the pods on the node to verify that they have been removed:
oc adm manage-node ${NODE} --list-pods
$ oc adm manage-node ${NODE} --list-pods Listing matched pods on node: ose-app-node01.example.com NAME READY STATUS RESTARTS AGE
Copy to Clipboard Copied! Toggle word wrap Toggle overflow For more information on evacuating and draining pods or nodes, see Node maintenance.
With no containers currently running on the instance, stop the
docker
andatomic-openshift-node service
services:systemctl stop docker atomic-openshift-node
# systemctl stop docker atomic-openshift-node
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the name of the volume group, logical volume name, and physical volume name:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Modify the
docker-storage-setup
file to specify theSTORAGE_DRIVER
.NoteWhen a system is upgraded from Red Hat Enterprise Linux version 7.3 to 7.4, the
docker
service attempts to use/var
with theSTORAGE_DRIVER
of extfs. The use of extfs as theSTORAGE_DRIVER
causes errors. See the following bug for more info regarding the error:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set up the storage:
docker-storage-setup
# docker-storage-setup
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Start the
docker
andatomic-openshift-node
services:systemctl start docker atomic-openshift-node
# systemctl start docker atomic-openshift-node
Copy to Clipboard Copied! Toggle word wrap Toggle overflow With the storage modified to use
overlay2
, enable the node to be schedulable in order to accept new incoming pods.From a master instance, or as a cluster administrator:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
7.2. Managing Docker certificates 复制链接链接已复制到粘贴板!
An OpenShift Container Platform internal registry is created as a pod. However, containers may be pulled from external registries if desired. By default, registries listen on TCP port 5000. Registries provide the option of securing exposed images via TLS or running a registry without encrypting traffic.
Docker interprets .crt
files as CA certificates and .cert
files as client certificates. Any CA extensions must be .crt
.
In order to use OpenShift Container Platform with an external registry, the registry certificate authority (CA) certificate must be trusted for all the nodes that can pull images from the registry.
Depending on the Docker version, the process to trust a Docker registry varies. The latest versions of Docker’s root certificate authorities are merged with system defaults. Prior to docker
version 1.13, the system default certificate is used only when no other custom root certificates exist.
Procedure
Copy the CA certificate to
/etc/pki/ca-trust/source/anchors/
:sudo cp myregistry.example.com.crt /etc/pki/ca-trust/source/anchors/
$ sudo cp myregistry.example.com.crt /etc/pki/ca-trust/source/anchors/
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Extract and add the CA certificate to the list of trusted certificates authorities:
sudo update-ca-trust extract
$ sudo update-ca-trust extract
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the SSL certificate using the
openssl
command:openssl verify myregistry.example.com.crt
$ openssl verify myregistry.example.com.crt myregistry.example.com.crt: OK
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Once the certificate is in place and the trust is updated, restart the
docker
service to ensure the new certificates are properly set:sudo systemctl restart docker.service
$ sudo systemctl restart docker.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
For Docker versions prior to 1.13, perform the following additional steps for trusting certificates of authority:
On every node create a new directory in
/etc/docker/certs.d
where the name of the directory is the host name of the Docker registry:sudo mkdir -p /etc/docker/certs.d/myregistry.example.com
$ sudo mkdir -p /etc/docker/certs.d/myregistry.example.com
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteThe port number is not required unless the Docker registry cannot be accessed without a port number. Addressing the port to the original Docker registry is as follows:
myregistry.example.com:port
Accessing the Docker registry via IP address requires the creation of a new directory within
/etc/docker/certs.d
on every node where the name of the directory is the IP of the Docker registry:sudo mkdir -p /etc/docker/certs.d/10.10.10.10
$ sudo mkdir -p /etc/docker/certs.d/10.10.10.10
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the CA certificate to the newly created Docker directories from the previous steps:
sudo cp myregistry.example.com.crt \ /etc/docker/certs.d/myregistry.example.com/ca.crt sudo cp myregistry.example.com.crt /etc/docker/certs.d/10.10.10.10/ca.crt
$ sudo cp myregistry.example.com.crt \ /etc/docker/certs.d/myregistry.example.com/ca.crt $ sudo cp myregistry.example.com.crt /etc/docker/certs.d/10.10.10.10/ca.crt
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Once the certificates have been copied, restart the
docker
service to ensure the new certificates are used:sudo systemctl restart docker.service
$ sudo systemctl restart docker.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
7.2.2. Docker certificates backup 复制链接链接已复制到粘贴板!
When performing a node host backup, ensure to include the certificates for external registries.
Procedure
If using
/etc/docker/certs.d
, copy all the certificates included in the directory and store the files:sudo tar -czvf docker-registry-certs-$(hostname)-$(date +%Y%m%d).tar.gz /etc/docker/certs.d/
$ sudo tar -czvf docker-registry-certs-$(hostname)-$(date +%Y%m%d).tar.gz /etc/docker/certs.d/
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If using a system trust, store the certificates prior to adding them within the system trust. Once the store is complete, extract the certificate for restoration using the
trust
command. Identify the system trust CAs and note thepkcs11
ID:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Extract the certificate in
pem
format and provide it a name. For example,myca.crt
.trust extract --format=pem-bundle \ --filter="%a5%b3%e1%2b%2b%49%b6%d7%73%a1%aa%94%f5%01%e7%73%65%4c%ac%50;type=cert" myca.crt
$ trust extract --format=pem-bundle \ --filter="%a5%b3%e1%2b%2b%49%b6%d7%73%a1%aa%94%f5%01%e7%73%65%4c%ac%50;type=cert" myca.crt
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the certificate has been properly extracted via
openssl
:openssl verify myca.crt
$ openssl verify myca.crt
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Repeat the procedure for all the required certificates and store the files in a remote location.
7.2.3. Docker certificates restore 复制链接链接已复制到粘贴板!
In the event of the deletion or corruption of the Docker certificates for the external registries, the restore mechanism uses the same steps as the installation method using the files from the backups performed previously.
7.3. Managing Docker registries 复制链接链接已复制到粘贴板!
You can configure OpenShift Container Platform to use external docker
registries to pull images. However, you can use configuration files to allow or deny certain images or registries.
If the external registry is exposed using certificates for the network traffic, it can be named as a secure registry. Otherwise, traffic between the registry and host is plain text and not encrypted, meaning it is an insecure registry.
7.3.1. Docker search external registries 复制链接链接已复制到粘贴板!
By default, the docker
daemon has the ability to pull images from any registry, but the search operation is performed against docker.io/
and registry.access.redhat.com
. The daemon can be be configured to search images from other registries using the --add-registry
option with the docker
daemon.
The ability to search images from the Red Hat Registry registry.access.redhat.com
exists by default in the Red Hat Enterprise Linux docker
package.
Procedure
To allow users to search for images using
docker search
with other registries, add those registries to the/etc/containers/registries.conf
file under theregistries
parameter:registries: - registry.access.redhat.com - my.registry.example.com
registries: - registry.access.redhat.com - my.registry.example.com
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Prior to OpenShift Container Platform version 3.6, this was accomplished using
/etc/sysconfig/docker
with the following options:ADD_REGISTRY="--add-registry=registry.access.redhat.com --add-registry=my.registry.example.com"
ADD_REGISTRY="--add-registry=registry.access.redhat.com --add-registry=my.registry.example.com"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The first registry added is the first registry searched.
Restart the
docker
daemon to allow formy.registry.example.com
to be used:sudo systemctl restart docker.service
$ sudo systemctl restart docker.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Restarting the
docker
daemon causes thedocker
containers to restart.Using the Ansible installer, this can be configured using the
openshift_docker_additional_registries
variable in the Ansible hosts file:openshift_docker_additional_registries=registry.access.redhat.com,my.registry.example.com
openshift_docker_additional_registries=registry.access.redhat.com,my.registry.example.com
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Docker can be configured to block operations from external registries by configuring the registries
and block_registries
flags for the docker
daemon.
Procedure
Add the allowed registries to the
/etc/containers/registries.conf
file with theregistries
flag:registries: - registry.access.redhat.com - my.registry.example.com
registries: - registry.access.redhat.com - my.registry.example.com
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Prior to 3.6, the
/etc/sysconfig/docker
file is modified instead:ADD_REGISTRY="--add-registry=registry.access.redhat.com --add-registry=my.registry.example.com"
ADD_REGISTRY="--add-registry=registry.access.redhat.com --add-registry=my.registry.example.com"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteThe
docker.io
registry can be added using the same method.Block the rest of the registries:
block_registries: - all
block_registries: - all
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Block the rest of the registries in older versions:
BLOCK_REGISTRY='--block-registry=all'
BLOCK_REGISTRY='--block-registry=all'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Restart the
docker
daemon:sudo systemctl restart docker.service
$ sudo systemctl restart docker.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Restarting the
docker
daemon causes thedocker
containers to restart.In this example, the
docker.io
registry has been blacklisted, so any operation regarding that registry fails:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Add
docker.io
back to theregistries
variable by modifying the file again and restarting the service.Copy to Clipboard Copied! Toggle word wrap Toggle overflow or
ADD_REGISTRY="--add-registry=registry.access.redhat.com --add-registry=my.registry.example.com --add-registry=docker.io" BLOCK_REGISTRY='--block-registry=all'
ADD_REGISTRY="--add-registry=registry.access.redhat.com --add-registry=my.registry.example.com --add-registry=docker.io" BLOCK_REGISTRY='--block-registry=all'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Restart the Docker service:
sudo systemctl restart docker
$ sudo systemctl restart docker
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To verify that the image is now available to be pulled:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If using an external registry is required, for example to modify the
docker
daemon configuration file in all the node hosts that require to use that registry, create a blacklist on those nodes to avoid malicious containers from being executed.Using the Ansible installer, this can be configured using the
openshift_docker_additional_registries
andopenshift_docker_blocked_registries
variables in the Ansible hosts file:openshift_docker_additional_registries=registry.access.redhat.com,my.registry.example.com openshift_docker_blocked_registries=all
openshift_docker_additional_registries=registry.access.redhat.com,my.registry.example.com openshift_docker_blocked_registries=all
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
7.3.3. Secure registries 复制链接链接已复制到粘贴板!
In order to be able to pull images from an external registry, it is required to trust the registry certificates, otherwise the pull image operation fails.
In order to do so, see the Installing a Certificate Authority Certificate for External Registries section.
If using a whitelist, the external registries should be added to the registries
variable, as explained above.
7.3.4. Insecure registries 复制链接链接已复制到粘贴板!
External registries that use non-trusted certificates, or without certificates at all, should be avoided.
However, any insecure registries should be added using the --insecure-registry
option to allow for the docker
daemon to pull images from the repository. This is the same as the --add-registry
option, but the docker
operation is not verified.
The registry should be added using both options to enable search, and, if there is a blacklist, to perform other operations, such as pulling images.
For testing purposes, an example is shown on how to add a localhost
insecure registry.
Procedure
Modify
/etc/containers/registries.conf
configuration file to add the localhost insecure registry:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Prior to 3.6, modify the
/etc/sysconfig/docker
configuration file to add the localhost:ADD_REGISTRY="--add-registry=registry.access.redhat.com --add-registry=my.registry.example.com --add-registry=docker.io --add-registry=localhost:5000" INSECURE_REGISTRY="--insecure-registry=localhost:5000" BLOCK_REGISTRY='--block-registry=all'
ADD_REGISTRY="--add-registry=registry.access.redhat.com --add-registry=my.registry.example.com --add-registry=docker.io --add-registry=localhost:5000" INSECURE_REGISTRY="--insecure-registry=localhost:5000" BLOCK_REGISTRY='--block-registry=all'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Restart the
docker
daemon to use the registry:sudo systemctl restart docker.service
$ sudo systemctl restart docker.service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Restarting the
docker
daemon causes thedocker
containers to be restarted.Run a Docker registry pod at
localhost
:sudo docker run -p 5000:5000 registry:2
$ sudo docker run -p 5000:5000 registry:2
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Pull an image:
sudo docker pull openshift/hello-openshift
$ sudo docker pull openshift/hello-openshift
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Tag the image:
sudo docker tag docker.io/openshift/hello-openshift:latest localhost:5000/hello-openshift-local:latest
$ sudo docker tag docker.io/openshift/hello-openshift:latest localhost:5000/hello-openshift-local:latest
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Push the image to the local registry:
sudo docker push localhost:5000/hello-openshift-local:latest
$ sudo docker push localhost:5000/hello-openshift-local:latest
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Using the Ansible installer, this can be configured using the
openshift_docker_additional_registries
,openshift_docker_blocked_registries
, andopenshift_docker_insecure_registries
variables in theAnsible
hosts file:openshift_docker_additional_registries=registry.access.redhat.com,my.registry.example.com,localhost:5000 openshift_docker_insecure_registries=localhost:5000 openshift_docker_blocked_registries=all
openshift_docker_additional_registries=registry.access.redhat.com,my.registry.example.com,localhost:5000 openshift_docker_insecure_registries=localhost:5000 openshift_docker_blocked_registries=all
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteYou can also set the
openshift_docker_insecure_registries
variable to the IP address of the host.0.0.0.0/0
is not a valid setting.
7.3.5. Authenticated registries 复制链接链接已复制到粘贴板!
Using authenticated registries with docker
requires the docker
daemon to log in to the registry. With OpenShift Container Platform, a different set of steps must be performed, because the users can not run docker login
commands on the host. Authenticated registries can be used to limit the images users can pull or who can access the external registries.
If an external docker
registry requires authentication, create a special secret in the project that uses that registry and then use that secret to perform the docker
operations.
Procedure
Create a
dockercfg
secret in the project where the user is going to log in to thedocker
registry:oc project <my_project> oc create secret docker-registry <my_registry> --docker-server=<my.registry.example.com> --docker-username=<username> --docker-password=<my_password> --docker-email=<me@example.com>
$ oc project <my_project> $ oc create secret docker-registry <my_registry> --docker-server=<my.registry.example.com> --docker-username=<username> --docker-password=<my_password> --docker-email=<me@example.com>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If a
.dockercfg
file exists, create the secret using theoc
command:oc create secret generic <my_registry> --from-file=.dockercfg=<path/to/.dockercfg> --type=kubernetes.io/dockercfg
$ oc create secret generic <my_registry> --from-file=.dockercfg=<path/to/.dockercfg> --type=kubernetes.io/dockercfg
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Populate the
$HOME/.docker/config.json
file:oc create secret generic <my_registry> --from-file=.dockerconfigjson=<path/to/.dockercfg> --type=kubernetes.io/dockerconfigjson
$ oc create secret generic <my_registry> --from-file=.dockerconfigjson=<path/to/.dockercfg> --type=kubernetes.io/dockerconfigjson
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
dockercfg
secret to pull images from the authenticated registry by linking the secret to the service account performing the pull operations. The default service account to pull images is nameddefault
:oc secrets link default <my_registry> --for=pull
$ oc secrets link default <my_registry> --for=pull
Copy to Clipboard Copied! Toggle word wrap Toggle overflow For pushing images using the S2I feature, the
dockercfg
secret is mounted in the S2I pod, so it needs to be linked to the proper service account that performs the build. The default service account used to build images is namedbuilder
.oc secrets link builder <my_registry>
$ oc secrets link builder <my_registry>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow In the
buildconfig
, the secret should be specified for push or pull operations:Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the external registry delegates authentication to external services, create both
dockercfg
secrets: the registry one using the registry URL and the external authentication system using its own URL. Both secrets should be added to the service accounts.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
7.3.6. ImagePolicy admission plug-in 复制链接链接已复制到粘贴板!
An admission control plug-in intercepts requests to the API, and performs checks depending on the configured rules and allows or denies certain actions based on those rules. OpenShift Container Platform can limit the allowed images running in the environment using the ImagePolicy
admission plug-in where it can control:
- The source of images: which registries can be used to pull images
- Image resolution: force pods to run with immutable digests to ensure the image does not change due to a re-tag
- Container image label restrictions: force an image to have or not have particular labels
- Image annotation restrictions: force an image in the integrated container registry to have or not have particular annotations
ImagePolicy
admission plug-in is currently considered beta.
Procedure
If the
ImagePolicy
plug-in is enabled, it needs to be modified to allow the external registries to be used by modifying the/etc/origin/master/master-config.yaml
file on every master node:Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteEnabling
ImagePolicy
requires users to specify the registry when deploying an application likeoc new-app docker.io/kubernetes/guestbook
insteadoc new-app kubernetes/guestbook
, otherwise it fails.To enable the admission plug-ins at installation time, the
openshift_master_admission_plugin_config
variable can be used with ajson
formatted string including all thepluginConfig
configuration:openshift_master_admission_plugin_config={"openshift.io/ImagePolicy":{"configuration":{"kind":"ImagePolicyConfig","apiVersion":"v1","executionRules":[{"name":"allow-images-from-other-registries","onResources":[{"resource":"pods"},{"resource":"builds"}],"matchRegistries":["docker.io","*my.registry.example.com*","registry.access.redhat.com"]}]}}}
openshift_master_admission_plugin_config={"openshift.io/ImagePolicy":{"configuration":{"kind":"ImagePolicyConfig","apiVersion":"v1","executionRules":[{"name":"allow-images-from-other-registries","onResources":[{"resource":"pods"},{"resource":"builds"}],"matchRegistries":["docker.io","*my.registry.example.com*","registry.access.redhat.com"]}]}}}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow WarningThere is a current issue to be fixed in OpenShift Container Platform 3.6.1 where
ImagePolicy
pods can not be deployed using default templates, and give the following error messageFailed create | Error creating: Pod "" is invalid: spec.containers[0].\image: Forbidden: this image is prohibited by policy
.See the Image Policy is not working as expected Red Hat Knowledgebase article for a workaround.
7.3.7. Import images from external registries 复制链接链接已复制到粘贴板!
Application developers can import images to create imagestreams
using the oc import-image
command, and OpenShift Container Platform can be configured to allow or deny image imports from external registries.
Procedure
To configure the allowed registries where users can import images, add the following to the
/etc/origin/master/master-config.yaml
file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - To import images from an external authenticated registry, create a secret within the desired project.
Even if not recommended, if the external authenticated registry is insecure or the certificates can not be trusted, the
oc import-image
command can be used with the--insecure=true
option.If the external authenticated registry is secure, the registry certificate should be trusted in the master hosts as they run the registry import controller as:
Copy the certificate in the
/etc/pki/ca-trust/source/anchors/
:sudo cp <my.registry.example.com.crt> /etc/pki/ca-trust/source/anchors/<my.registry.example.com.crt>
$ sudo cp <my.registry.example.com.crt> /etc/pki/ca-trust/source/anchors/<my.registry.example.com.crt>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run
update-ca-trust
command:sudo update-ca-trust
$ sudo update-ca-trust
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Restart the master services on all the master hosts:
sudo systemctl restart atomic-openshift-master-api sudo systemctl restart atomic-openshift-master-controllers
$ sudo systemctl restart atomic-openshift-master-api $ sudo systemctl restart atomic-openshift-master-controllers
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The certificate for the external registry should be trusted in the OpenShift Container Platform registry:
for i in pem openssl java; do oc create configmap ca-trust-extracted-${i} --from-file /etc/pki/ca-trust/extracted/${i} oc set volume dc/docker-registry --add -m /etc/pki/ca-trust/extracted/${i} --configmap-name=ca-trust-extracted-${i} --name ca-trust-extracted-${i} done
$ for i in pem openssl java; do oc create configmap ca-trust-extracted-${i} --from-file /etc/pki/ca-trust/extracted/${i} oc set volume dc/docker-registry --add -m /etc/pki/ca-trust/extracted/${i} --configmap-name=ca-trust-extracted-${i} --name ca-trust-extracted-${i} done
Copy to Clipboard Copied! Toggle word wrap Toggle overflow WarningThere is no official procedure currently for adding the certificate to the registry pod, but the above workaround can be used.
This workaround creates
configmaps
with all the trusted certificates from the system running those commands, so the recommendation is to run it from a clean system where just the required certificates are trusted.Alternatively, modify the registry image in order to trust the proper certificates rebuilding the image using a
Dockerfile
as:FROM registry.access.redhat.com/openshift3/ose-docker-registry:v3.6 ADD <my.registry.example.com.crt> /etc/pki/ca-trust/source/anchors/ USER 0 RUN update-ca-trust extract USER 1001
FROM registry.access.redhat.com/openshift3/ose-docker-registry:v3.6 ADD <my.registry.example.com.crt> /etc/pki/ca-trust/source/anchors/ USER 0 RUN update-ca-trust extract USER 1001
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Rebuild the image, push it to a
docker
registry, and use that image asspec.template.spec.containers["name":"registry"].image
in the registrydeploymentconfig
:oc patch dc docker-registry -p '{"spec":{"template":{"spec":{"containers":[{"name":"registry","image":"*myregistry.example.com/openshift3/ose-docker-registry:latest*"}]}}}}'
$ oc patch dc docker-registry -p '{"spec":{"template":{"spec":{"containers":[{"name":"registry","image":"*myregistry.example.com/openshift3/ose-docker-registry:latest*"}]}}}}'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
To add the imagePolicyConfig
configuration at installation, the openshift_master_image_policy_config
variable can be used with a json
formatted string including all the imagePolicyConfig
configuration, like:
openshift_master_image_policy_config={"imagePolicyConfig":{"allowedRegistriesForImport":[{"domainName":"docker.io"},{"domainName":"\*.docker.io"},{"domainName":"*.redhat.com"},{"domainName":"*my.registry.example.com*"}]}}
openshift_master_image_policy_config={"imagePolicyConfig":{"allowedRegistriesForImport":[{"domainName":"docker.io"},{"domainName":"\*.docker.io"},{"domainName":"*.redhat.com"},{"domainName":"*my.registry.example.com*"}]}}
For more information about the ImagePolicy
, see the ImagePolicy
admission plug-in section.
7.3.8. OpenShift Container Platform registry integration 复制链接链接已复制到粘贴板!
You can install OpenShift Container Platform as a stand-alone container registry to provide only the registry capabilities, but with the advantages of running in an OpenShift Container Platform platform.
For more information about the OpenShift Container Platform registry, see Installing a Stand-alone Deployment of OpenShift Container Registry.
To integrate the OpenShift Container Platform registry, all previous sections apply. From the OpenShift Container Platform point of view, it is treated as an external registry, but there are some extra tasks that need to be performed, because it is a multi-tenant registry and the authorization model from OpenShift Container Platform applies so when a new project is created, the registry does not create a project within its environment as it is independent.
7.3.8.1. Connect the registry project with the cluster 复制链接链接已复制到粘贴板!
As the registry is a full OpenShift Container Platform environment with a registry pod and a web interface, the process to create a new project in the registry is performed using the oc new-project
or oc create
command line or via the web interface.
Once the project has been created, the usual service accounts (builder
, default
, and deployer
) are created automatically, as well as the project administrator user is granted permissions. Different users can be authorized to push/pull images as well as "anonymous" users.
There can be several use cases, such as allowing all the users to pull images from this new project within the registry, but if you want to have a 1:1 project relationship between OpenShift Container Platform and the registry, where the users can push and pull images from that specific project, some steps are required.
The registry web console shows a token to be used for pull/push operations, but the token showed there is a session token, so it expires. Creating a service account with specific permissions allows the administrator to limit the permissions for the service account, so that, for example, different service accounts can be used for push or pull images. Then, a user does not have to configure for token expiration, secret recreation, and other tasks, as the service account tokens will not expire.
Procedure
Create a new project:
oc new-project <my_project>
$ oc new-project <my_project>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a registry project:
oc new-project <registry_project>
$ oc new-project <registry_project>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a service account in the registry project:
oc create serviceaccount <my_serviceaccount> -n <registry_project>
$ oc create serviceaccount <my_serviceaccount> -n <registry_project>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Give permissions to push and pull images using the
registry-editor
role:oc adm policy add-role-to-user registry-editor -z <my_serviceaccount> -n <registry_project>
$ oc adm policy add-role-to-user registry-editor -z <my_serviceaccount> -n <registry_project>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If only pull permissions are required, the
registry-viewer
role can be used.Get the service account token:
TOKEN=$(oc sa get-token <my_serviceaccount> -n <registry_project>)
$ TOKEN=$(oc sa get-token <my_serviceaccount> -n <registry_project>)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Use the token as the password to create a
dockercfg
secret:oc create secret docker-registry <my_registry> \ --docker-server=<myregistry.example.com> --docker-username=<notused> --docker-password=${TOKEN} --docker-email=<me@example.com>
$ oc create secret docker-registry <my_registry> \ --docker-server=<myregistry.example.com> --docker-username=<notused> --docker-password=${TOKEN} --docker-email=<me@example.com>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
dockercfg
secret to pull images from the registry by linking the secret to the service account performing the pull operations. The default service account to pull images is nameddefault
:oc secrets link default <my_registry> --for=pull
$ oc secrets link default <my_registry> --for=pull
Copy to Clipboard Copied! Toggle word wrap Toggle overflow For pushing images using the S2I feature, the
dockercfg
secret is mounted in the S2I pod, so it needs to be linked to the proper service account that performs the build. The default service account used to build images is namedbuilder
:oc secrets link builder <my_registry>
$ oc secrets link builder <my_registry>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow In the
buildconfig
, the secret should be specified for push or pull operations:Copy to Clipboard Copied! Toggle word wrap Toggle overflow