第二天的操作指南

创建一个新项目

$ oc new-project sdn-test

部署 httpd 应用程序：

$ oc new-app centos/httpd-24-centos7~https://github.com/sclorg/httpd-ex

等待构建完成：

$ oc get pods
NAME               READY     STATUS      RESTARTS   AGE
httpd-ex-1-205hz   1/1       Running     0          34s
httpd-ex-1-build   0/1       Completed   0          1m

连接到正在运行的 pod：

$ oc rsh po/<pod-name>

例如：

$ oc rsh po/httpd-ex-1-205hz

检查内部 registry 服务的 healthz 路径：

$ curl -kv https://docker-registry.default.svc.cluster.local:5000/healthz
* About to connect() to docker-registry.default.svc.cluster.locl port 5000 (#0)
*   Trying 172.30.150.7...
* Connected to docker-registry.default.svc.cluster.local (172.30.150.7) port 5000 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: CN=172.30.150.7
* 	start date: Nov 30 17:21:51 2017 GMT
* 	expire date: Nov 30 17:21:52 2019 GMT
* 	common name: 172.30.150.7
* 	issuer: CN=openshift-signer@1512059618
> GET /healthz HTTP/1.1
> User-Agent: curl/7.29.0
> Host: docker-registry.default.svc.cluster.local:5000
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: no-cache
< Date: Mon, 04 Dec 2017 16:26:49 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host docker-registry.default.svc.cluster.local left intact

sh-4.2$ *exit*

HTTP/1.1 200 OK 响应表示节点正确连接。

清理 test 项目：

$ oc delete project sdn-test
project "sdn-test" deleted

节点主机正在侦听 TCP 端口 10250。此端口需要可以被任何节点上的所有 master 主机访问，如果在集群中部署了监控，基础架构节点也必须可以访问所有实例上的所有端口。可以通过以下命令检测到这个端口上的通信中断：

$ oc get nodes
NAME                  STATUS                     AGE       VERSION
ocp-infra-node-1clj   Ready                      4d        v1.6.1+5115d708d7
ocp-infra-node-86qr   Ready                      4d        v1.6.1+5115d708d7
ocp-infra-node-g8qw   Ready                      4d        v1.6.1+5115d708d7
ocp-master-94zd       Ready,SchedulingDisabled   4d        v1.6.1+5115d708d7
ocp-master-gjkm       Ready,SchedulingDisabled   4d        v1.6.1+5115d708d7
ocp-master-wc8w       Ready,SchedulingDisabled   4d        v1.6.1+5115d708d7
ocp-node-c5dg         Ready                      4d        v1.6.1+5115d708d7
ocp-node-ghxn         Ready                      4d        v1.6.1+5115d708d7
ocp-node-w135         NotReady                   4d        v1.6.1+5115d708d7

在上面的输出中，master 服务无法访问 ocp-node-w135 节点上的节点服务，该服务由其 NotReady 状态表示。

最后一个服务是路由器，它负责将连接路由到 OpenShift Container Platform 集群中运行的正确服务。路由器在基础架构节点上的 TCP 端口 80 和 443 上侦听入口流量。在路由器可以开始工作前，必须配置 DNS：

$ dig *.apps.example.com

; <<>> DiG 9.11.1-P3-RedHat-9.11.1-8.P3.fc27 <<>> *.apps.example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45790
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;*.apps.example.com.	IN	A

;; ANSWER SECTION:
*.apps.example.com. 3571	IN	CNAME	apps.example.com.
apps.example.com.	3561	IN	A	35.xx.xx.92

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Dec 05 16:03:52 CET 2017
;; MSG SIZE  rcvd: 105

IP 地址（本例中为 35.xx.xx.92）应指向将入口流量分发到所有基础架构节点的负载均衡器。要验证路由器的功能，请再次检查 registry 服务，但这次它们来自集群外部：

$ curl -kv https://docker-registry-default.apps.example.com/healthz
*   Trying 35.xx.xx.92...
* TCP_NODELAY set
* Connected to docker-registry-default.apps.example.com (35.xx.xx.92) port 443 (#0)
...
< HTTP/2 200
< cache-control: no-cache
< content-type: text/plain; charset=utf-8
< content-length: 0
< date: Tue, 05 Dec 2017 15:13:27 GMT
<
* Connection #0 to host docker-registry-default.apps.example.com left intact

创建 master 主机配置文件的备份：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo mkdir -p ${MYBACKUPDIR}/etc/sysconfig
$ sudo cp -aR /etc/origin ${MYBACKUPDIR}/etc
$ sudo cp -aR /etc/sysconfig/ ${MYBACKUPDIR}/etc/sysconfig/

在规划备份时需要考虑的其他重要文件包括：

创建这些文件的备份：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo mkdir -p ${MYBACKUPDIR}/etc/sysconfig
$ sudo mkdir -p ${MYBACKUPDIR}/etc/pki/ca-trust/source/anchors
$ sudo cp -aR /etc/sysconfig/{iptables,docker-*,flanneld} \
    ${MYBACKUPDIR}/etc/sysconfig/
$ sudo cp -aR /etc/dnsmasq* /etc/cni ${MYBACKUPDIR}/etc/
$ sudo cp -aR /etc/pki/ca-trust/source/anchors/* \
    ${MYBACKUPDIR}/etc/pki/ca-trust/source/anchors/

如果软件包被意外删除，或者您需要重新整合 rpm 软件包中所含的文件，则系统中安装的 rhel 软件包列表会很有用。

要创建系统中当前 rhel 软件包的列表：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo mkdir -p ${MYBACKUPDIR}
$ rpm -qa | sort | sudo tee $MYBACKUPDIR/packages.txt

如果您使用前面的步骤，则备份目录中存在以下文件：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo find ${MYBACKUPDIR} -mindepth 1 -type f -printf '%P\n'
etc/sysconfig/flanneld
etc/sysconfig/iptables
etc/sysconfig/docker-network
etc/sysconfig/docker-storage
etc/sysconfig/docker-storage-setup
etc/sysconfig/docker-storage-setup.rpmnew
etc/origin/master/ca.crt
etc/origin/master/ca.key
etc/origin/master/ca.serial.txt
etc/origin/master/ca-bundle.crt
etc/origin/master/master.proxy-client.crt
etc/origin/master/master.proxy-client.key
etc/origin/master/service-signer.crt
etc/origin/master/service-signer.key
etc/origin/master/serviceaccounts.private.key
etc/origin/master/serviceaccounts.public.key
etc/origin/master/openshift-master.crt
etc/origin/master/openshift-master.key
etc/origin/master/openshift-master.kubeconfig
etc/origin/master/master.server.crt
etc/origin/master/master.server.key
etc/origin/master/master.kubelet-client.crt
etc/origin/master/master.kubelet-client.key
etc/origin/master/admin.crt
etc/origin/master/admin.key
etc/origin/master/admin.kubeconfig
etc/origin/master/etcd.server.crt
etc/origin/master/etcd.server.key
etc/origin/master/master.etcd-client.key
etc/origin/master/master.etcd-client.csr
etc/origin/master/master.etcd-client.crt
etc/origin/master/master.etcd-ca.crt
etc/origin/master/policy.json
etc/origin/master/scheduler.json
etc/origin/master/htpasswd
etc/origin/master/session-secrets.yaml
etc/origin/master/openshift-router.crt
etc/origin/master/openshift-router.key
etc/origin/master/registry.crt
etc/origin/master/registry.key
etc/origin/master/master-config.yaml
etc/origin/generated-configs/master-master-1.example.com/master.server.crt
...[OUTPUT OMITTED]...
etc/origin/cloudprovider/openstack.conf
etc/origin/node/system:node:master-0.example.com.crt
etc/origin/node/system:node:master-0.example.com.key
etc/origin/node/ca.crt
etc/origin/node/system:node:master-0.example.com.kubeconfig
etc/origin/node/server.crt
etc/origin/node/server.key
etc/origin/node/node-dnsmasq.conf
etc/origin/node/resolv.conf
etc/origin/node/node-config.yaml
etc/origin/node/flannel.etcd-client.key
etc/origin/node/flannel.etcd-client.csr
etc/origin/node/flannel.etcd-client.crt
etc/origin/node/flannel.etcd-ca.crt
etc/pki/ca-trust/source/anchors/openshift-ca.crt
etc/pki/ca-trust/source/anchors/registry-ca.crt
etc/dnsmasq.conf
etc/dnsmasq.d/origin-dns.conf
etc/dnsmasq.d/origin-upstream-dns.conf
etc/dnsmasq.d/node-dnsmasq.conf
packages.txt

如果需要，您可以压缩文件以节省空间：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo tar -zcvf /backup/$(hostname)-$(date +%Y%m%d).tar.gz $MYBACKUPDIR
$ sudo rm -Rf ${MYBACKUPDIR}

恢复 /etc/origin/master/master-config.yaml 文件：

# MYBACKUPDIR=*/backup/$(hostname)/$(date +%Y%m%d)*
# cp /etc/origin/master/master-config.yaml /etc/origin/master/master-config.yaml.old
# cp /backup/$(hostname)/$(date +%Y%m%d)/origin/master/master-config.yaml /etc/origin/master/master-config.yaml
# master-restart api
# master-restart controllers

如果因为缺少软件包而无法重启 OpenShift Container Platform，请重新安装软件包。

通过将证书复制到 /etc/pki/ca-trust/source/anchors/ 目录并执行 update-ca-trust 来恢复系统证书：

$ MYBACKUPDIR=*/backup/$(hostname)/$(date +%Y%m%d)*
$ sudo cp ${MYBACKUPDIR}/etc/pki/ca-trust/source/anchors/<certificate> /etc/pki/ca-trust/source/anchors/ 1
$ sudo update-ca-trust

列出所有可用的节点，以查找要弃用的节点：

$ oc get nodes
NAME                  STATUS                     AGE       VERSION
ocp-infra-node-b7pl   Ready                      23h       v1.6.1+5115d708d7
ocp-infra-node-p5zj   Ready                      23h       v1.6.1+5115d708d7
ocp-infra-node-rghb   Ready                      23h       v1.6.1+5115d708d7
ocp-master-dgf8       Ready,SchedulingDisabled   23h       v1.6.1+5115d708d7
ocp-master-q1v2       Ready,SchedulingDisabled   23h       v1.6.1+5115d708d7
ocp-master-vq70       Ready,SchedulingDisabled   23h       v1.6.1+5115d708d7
ocp-node-020m         Ready                      23h       v1.6.1+5115d708d7
ocp-node-7t5p         Ready                      23h       v1.6.1+5115d708d7
ocp-node-n0dd         Ready                      23h       v1.6.1+5115d708d7

例如，本节弃用 ocp-infra-node-b7pl 基础架构节点。

描述节点及其运行的服务：

$ oc describe node ocp-infra-node-b7pl
Name:			ocp-infra-node-b7pl
Role:
Labels:			beta.kubernetes.io/arch=amd64
			beta.kubernetes.io/instance-type=n1-standard-2
			beta.kubernetes.io/os=linux
			failure-domain.beta.kubernetes.io/region=europe-west3
			failure-domain.beta.kubernetes.io/zone=europe-west3-c
			kubernetes.io/hostname=ocp-infra-node-b7pl
			role=infra
Annotations:		volumes.kubernetes.io/controller-managed-attach-detach=true
Taints:			<none>
CreationTimestamp:	Wed, 22 Nov 2017 09:36:36 -0500
Phase:
Conditions:
  ...
Addresses:		10.156.0.11,ocp-infra-node-b7pl
Capacity:
 cpu:		2
 memory:	7494480Ki
 pods:		20
Allocatable:
 cpu:		2
 memory:	7392080Ki
 pods:		20
System Info:
 Machine ID:			bc95ccf67d047f2ae42c67862c202e44
 System UUID:			9762CC3D-E23C-AB13-B8C5-FA16F0BCCE4C
 Boot ID:			ca8bf088-905d-4ec0-beec-8f89f4527ce4
 Kernel Version:		3.10.0-693.5.2.el7.x86_64
 OS Image:			Employee SKU
 Operating System:		linux
 Architecture:			amd64
 Container Runtime Version:	docker://1.12.6
 Kubelet Version:		v1.6.1+5115d708d7
 Kube-Proxy Version:		v1.6.1+5115d708d7
ExternalID:			437740049672994824
Non-terminated Pods:		(2 in total)
  Namespace			Name				CPU Requests	CPU Limits	Memory Requests	Memory Limits
  ---------			----				------------	----------	---------------	-------------
  default			docker-registry-1-5szjs		100m (5%)	0 (0%)		256Mi (3%)0 (0%)
  default			router-1-vzlzq			100m (5%)	0 (0%)		256Mi (3%)0 (0%)
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests	CPU Limits	Memory Requests	Memory Limits
  ------------	----------	---------------	-------------
  200m (10%)	0 (0%)		512Mi (7%)	0 (0%)
Events:		<none>

上面的输出显示节点正在运行两个 pod： router-1-vzlzq 和 docker-registry-1-5szjs。有两个基础架构节点可用于迁移这两个 pod。

将节点标记为不可调度并撤离其所有 pod：

$ oc adm drain ocp-infra-node-b7pl --delete-local-data
node "ocp-infra-node-b7pl" cordoned
WARNING: Deleting pods with local storage: docker-registry-1-5szjs
pod "docker-registry-1-5szjs" evicted
pod "router-1-vzlzq" evicted
node "ocp-infra-node-b7pl" drained

如果 pod 已附加了本地存储（例如 EmptyDir），则必须提供 --delete-local-data 选项。通常，在生产环境中运行的 pod 应该只将本地存储用于临时或缓存文件，但不适用于任何重要或持久的。对于常规存储，应用程序应使用对象存储或持久性卷。在这种情况下，docker-registry pod 的本地存储为空，因为对象存储被用来存储容器镜像。

以下示例显示了 registry 的复制控制器的输出：

$ oc describe rc/docker-registry-1
Name:		docker-registry-1
Namespace:	default
Selector:	deployment=docker-registry-1,deploymentconfig=docker-registry,docker-registry=default
Labels:		docker-registry=default
		openshift.io/deployment-config.name=docker-registry
Annotations: ...
Replicas:	3 current / 3 desired
Pods Status:	3 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:		deployment=docker-registry-1
			deploymentconfig=docker-registry
			docker-registry=default
  Annotations:		openshift.io/deployment-config.latest-version=1
			openshift.io/deployment-config.name=docker-registry
			openshift.io/deployment.name=docker-registry-1
  Service Account:	registry
  Containers:
   registry:
    Image:	openshift3/ose-docker-registry:v3.6.173.0.49
    Port:	5000/TCP
    Requests:
      cpu:	100m
      memory:	256Mi
    Liveness:	http-get https://:5000/healthz delay=10s timeout=5s period=10s #success=1 #failure=3
    Readiness:	http-get https://:5000/healthz delay=0s timeout=5s period=10s #success=1 #failure=3
    Environment:
      REGISTRY_HTTP_ADDR:					:5000
      REGISTRY_HTTP_NET:					tcp
      REGISTRY_HTTP_SECRET:					tyGEnDZmc8dQfioP3WkNd5z+Xbdfy/JVXf/NLo3s/zE=
      REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_ENFORCEQUOTA:	false
      REGISTRY_HTTP_TLS_KEY:					/etc/secrets/registry.key
      OPENSHIFT_DEFAULT_REGISTRY:				docker-registry.default.svc:5000
      REGISTRY_CONFIGURATION_PATH:				/etc/registry/config.yml
      REGISTRY_HTTP_TLS_CERTIFICATE:				/etc/secrets/registry.crt
    Mounts:
      /etc/registry from docker-config (rw)
      /etc/secrets from registry-certificates (rw)
      /registry from registry-storage (rw)
  Volumes:
   registry-storage:
    Type:	EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
   registry-certificates:
    Type:	Secret (a volume populated by a Secret)
    SecretName:	registry-certificates
    Optional:	false
   docker-config:
    Type:	Secret (a volume populated by a Secret)
    SecretName:	registry-config
    Optional:	false
Events:
  FirstSeen	LastSeen	Count	From			SubObjectPath	Type		Reason		Message
  ---------	--------	-----	----			-------------	--------	------		-------
  49m		49m		1	replication-controller			Normal		SuccessfulCreate	Created pod: docker-registry-1-dprp5

输出底部的事件显示有关新 pod 创建的信息。因此，当列出所有 pod 时：

$ oc get pods
NAME                       READY     STATUS    RESTARTS   AGE
docker-registry-1-dprp5    1/1       Running   0          52m
docker-registry-1-kr8jq    1/1       Running   0          1d
docker-registry-1-ncpl2    1/1       Running   0          1d
registry-console-1-g4nqg   1/1       Running   0          1d
router-1-2gshr             0/1       Pending   0          52m
router-1-85qm4             1/1       Running   0          1d
router-1-q5sr8             1/1       Running   0          1d

观察新创建的 registry pod 时，以下示例显示了 ocp-infra-node-rghb 节点上已创建了 pod，它与弃用节点的不同：

$ oc describe pod docker-registry-1-dprp5
Name:			docker-registry-1-dprp5
Namespace:		default
Security Policy:	hostnetwork
Node:			ocp-infra-node-rghb/10.156.0.10
...

弃用基础架构和应用程序节点的唯一区别在于，在基础架构节点被撤离后，如果没有计划替换该节点，在基础架构节点上运行的服务可以缩减：

$ oc scale dc/router --replicas 2
deploymentconfig "router" scaled

$ oc scale dc/docker-registry --replicas 2
deploymentconfig "docker-registry" scaled

现在，每个基础架构节点只运行一个 pod 中的一类：

$ oc get pods
NAME                       READY     STATUS    RESTARTS   AGE
docker-registry-1-kr8jq    1/1       Running   0          1d
docker-registry-1-ncpl2    1/1       Running   0          1d
registry-console-1-g4nqg   1/1       Running   0          1d
router-1-85qm4             1/1       Running   0          1d
router-1-q5sr8             1/1       Running   0          1d

$ oc describe po/docker-registry-1-kr8jq | grep Node:
Node:			ocp-infra-node-p5zj/10.156.0.9

$ oc describe po/docker-registry-1-ncpl2 | grep Node:
Node:			ocp-infra-node-rghb/10.156.0.10

验证节点上的调度是否已禁用：

$ oc get nodes
NAME                  STATUS                     AGE       VERSION
ocp-infra-node-b7pl   Ready,SchedulingDisabled   1d        v1.6.1+5115d708d7
ocp-infra-node-p5zj   Ready                      1d        v1.6.1+5115d708d7
ocp-infra-node-rghb   Ready                      1d        v1.6.1+5115d708d7
ocp-master-dgf8       Ready,SchedulingDisabled   1d        v1.6.1+5115d708d7
ocp-master-q1v2       Ready,SchedulingDisabled   1d        v1.6.1+5115d708d7
ocp-master-vq70       Ready,SchedulingDisabled   1d        v1.6.1+5115d708d7
ocp-node-020m         Ready                      1d        v1.6.1+5115d708d7
ocp-node-7t5p         Ready                      1d        v1.6.1+5115d708d7
ocp-node-n0dd         Ready                      1d        v1.6.1+5115d708d7

节点没有包含任何 pod:

$ oc describe node ocp-infra-node-b7pl
Name:			ocp-infra-node-b7pl
Role:
Labels:			beta.kubernetes.io/arch=amd64
			beta.kubernetes.io/instance-type=n1-standard-2
			beta.kubernetes.io/os=linux
			failure-domain.beta.kubernetes.io/region=europe-west3
			failure-domain.beta.kubernetes.io/zone=europe-west3-c
			kubernetes.io/hostname=ocp-infra-node-b7pl
			role=infra
Annotations:		volumes.kubernetes.io/controller-managed-attach-detach=true
Taints:			<none>
CreationTimestamp:	Wed, 22 Nov 2017 09:36:36 -0500
Phase:
Conditions:
  ...
Addresses:		10.156.0.11,ocp-infra-node-b7pl
Capacity:
 cpu:		2
 memory:	7494480Ki
 pods:		20
Allocatable:
 cpu:		2
 memory:	7392080Ki
 pods:		20
System Info:
 Machine ID:			bc95ccf67d047f2ae42c67862c202e44
 System UUID:			9762CC3D-E23C-AB13-B8C5-FA16F0BCCE4C
 Boot ID:			ca8bf088-905d-4ec0-beec-8f89f4527ce4
 Kernel Version:		3.10.0-693.5.2.el7.x86_64
 OS Image:			Employee SKU
 Operating System:		linux
 Architecture:			amd64
 Container Runtime Version:	docker://1.12.6
 Kubelet Version:		v1.6.1+5115d708d7
 Kube-Proxy Version:		v1.6.1+5115d708d7
ExternalID:			437740049672994824
Non-terminated Pods:		(0 in total)
  Namespace			Name		CPU Requests	CPU Limits	Memory Requests	Memory Limits
  ---------			----		------------	----------	---------------	-------------
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests	CPU Limits	Memory Requests	Memory Limits
  ------------	----------	---------------	-------------
  0 (0%)	0 (0%)		0 (0%)		0 (0%)
Events:		<none>

从 /etc/haproxy/haproxy.cfg 配置文件中的 backend 部分删除基础架构实例：

backend router80
    balance source
    mode tcp
    server infra-1.example.com 192.168.55.12:80 check
    server infra-2.example.com 192.168.55.13:80 check

backend router443
    balance source
    mode tcp
    server infra-1.example.com 192.168.55.12:443 check
    server infra-2.example.com 192.168.55.13:443 check

然后，重新启动 haproxy 服务。

$ sudo systemctl restart haproxy

在所有 pod 都被逐出后，使用以下命令从集群中删除节点：

$ oc delete node ocp-infra-node-b7pl
node "ocp-infra-node-b7pl" deleted

$ oc get nodes
NAME                  STATUS                     AGE       VERSION
ocp-infra-node-p5zj   Ready                      1d        v1.6.1+5115d708d7
ocp-infra-node-rghb   Ready                      1d        v1.6.1+5115d708d7
ocp-master-dgf8       Ready,SchedulingDisabled   1d        v1.6.1+5115d708d7
ocp-master-q1v2       Ready,SchedulingDisabled   1d        v1.6.1+5115d708d7
ocp-master-vq70       Ready,SchedulingDisabled   1d        v1.6.1+5115d708d7
ocp-node-020m         Ready                      1d        v1.6.1+5115d708d7
ocp-node-7t5p         Ready                      1d        v1.6.1+5115d708d7
ocp-node-n0dd         Ready                      1d        v1.6.1+5115d708d7

创建节点配置文件的备份：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo mkdir -p ${MYBACKUPDIR}/etc/sysconfig
$ sudo cp -aR /etc/origin ${MYBACKUPDIR}/etc
$ sudo cp -aR /etc/sysconfig/atomic-openshift-node ${MYBACKUPDIR}/etc/sysconfig/

OpenShift Container Platform 使用在规划备份策略时必须考虑的特定文件，包括：

创建这些文件：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo mkdir -p ${MYBACKUPDIR}/etc/sysconfig
$ sudo mkdir -p ${MYBACKUPDIR}/etc/pki/ca-trust/source/anchors
$ sudo cp -aR /etc/sysconfig/{iptables,docker-*,flanneld} \
    ${MYBACKUPDIR}/etc/sysconfig/
$ sudo cp -aR /etc/dnsmasq* /etc/cni ${MYBACKUPDIR}/etc/
$ sudo cp -aR /etc/pki/ca-trust/source/anchors/* \
    ${MYBACKUPDIR}/etc/pki/ca-trust/source/anchors/

如果意外删除软件包，或者应该恢复 rpm 软件包中包含的文件，系统中安装的 rhel 软件包列表很有用。

要创建系统中当前 rhel 软件包的列表：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo mkdir -p ${MYBACKUPDIR}
$ rpm -qa | sort | sudo tee $MYBACKUPDIR/packages.txt

备份目录中现在应存在以下文件：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo find ${MYBACKUPDIR} -mindepth 1 -type f -printf '%P\n'
etc/sysconfig/atomic-openshift-node
etc/sysconfig/flanneld
etc/sysconfig/iptables
etc/sysconfig/docker-network
etc/sysconfig/docker-storage
etc/sysconfig/docker-storage-setup
etc/sysconfig/docker-storage-setup.rpmnew
etc/origin/node/system:node:app-node-0.example.com.crt
etc/origin/node/system:node:app-node-0.example.com.key
etc/origin/node/ca.crt
etc/origin/node/system:node:app-node-0.example.com.kubeconfig
etc/origin/node/server.crt
etc/origin/node/server.key
etc/origin/node/node-dnsmasq.conf
etc/origin/node/resolv.conf
etc/origin/node/node-config.yaml
etc/origin/node/flannel.etcd-client.key
etc/origin/node/flannel.etcd-client.csr
etc/origin/node/flannel.etcd-client.crt
etc/origin/node/flannel.etcd-ca.crt
etc/origin/cloudprovider/openstack.conf
etc/pki/ca-trust/source/anchors/openshift-ca.crt
etc/pki/ca-trust/source/anchors/registry-ca.crt
etc/dnsmasq.conf
etc/dnsmasq.d/origin-dns.conf
etc/dnsmasq.d/origin-upstream-dns.conf
etc/dnsmasq.d/node-dnsmasq.conf
packages.txt

如果需要，可以压缩文件以节省空间：

$ MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
$ sudo tar -zcvf /backup/$(hostname)-$(date +%Y%m%d).tar.gz $MYBACKUPDIR
$ sudo rm -Rf ${MYBACKUPDIR}

恢复 /etc/origin/node/node-config.yaml 文件：

# MYBACKUPDIR=/backup/$(hostname)/$(date +%Y%m%d)
# cp /etc/origin/node/node-config.yaml /etc/origin/node/node-config.yaml.old
# cp /backup/$(hostname)/$(date +%Y%m%d)/etc/origin/node/node-config.yaml /etc/origin/node/node-config.yaml
# reboot

如果因为缺少软件包而无法重启 OpenShift Container Platform，请重新安装软件包。

通过将证书复制到 /etc/pki/ca-trust/source/anchors/ 目录并执行 update-ca-trust 来恢复系统证书：

$ MYBACKUPDIR=*/backup/$(hostname)/$(date +%Y%m%d)*
$ sudo cp ${MYBACKUPDIR}/etc/pki/ca-trust/source/anchors/<certificate> /etc/pki/ca-trust/source/anchors/
$ sudo update-ca-trust

检查当前的 etcd 集群状态，以避免添加新主机到不健康的集群。运行这个命令：

# ETCDCTL_API=3 etcdctl --cert="/etc/etcd/peer.crt" \
          --key=/etc/etcd/peer.key \
          --cacert="/etc/etcd/ca.crt" \
          --endpoints="https://*master-0.example.com*:2379,\
            https://*master-1.example.com*:2379,\
            https://*master-2.example.com*:2379"
            endpoint health
https://master-0.example.com:2379 is healthy: successfully committed proposal: took = 5.011358ms
https://master-1.example.com:2379 is healthy: successfully committed proposal: took = 1.305173ms
https://master-2.example.com:2379 is healthy: successfully committed proposal: took = 1.388772ms

在运行 scaleup playbook 前，请确保新主机已注册到正确的红帽软件频道：

# subscription-manager register \
    --username=*<username>* --password=*<password>*
# subscription-manager attach --pool=*<poolid>*
# subscription-manager repos --disable="*"
# subscription-manager repos \
    --enable=rhel-7-server-rpms \
    --enable=rhel-7-server-extras-rpms

etcd 托管在 rhel-7-server-extras-rpms 软件频道中。

确保从 etcd 集群中删除所有未使用的 etcd 成员。这必须在运行 scaleup playbook 之前完成。

在当前 etcd 节点上升级 etcd 和 iptables：

# yum update etcd iptables-services

从 etcd 集群中删除其他 etcd 主机。为每个 etcd 节点运行以下命令：

# etcdctl3 --endpoints=https://<surviving host IP>:2379
  --cacert=/etc/etcd/ca.crt
  --cert=/etc/etcd/peer.crt
  --key=/etc/etcd/peer.key member remove <failed member ID>

在每个 master 上重启 master API 服务：

# master-restart api restart-master controller

从集群中删除失败的主机：

# etcdctl2 cluster-health
member 5ee217d19001 is healthy: got healthy result from https://192.168.55.12:2379
member 2a529ba1840722c0 is healthy: got healthy result from https://192.168.55.8:2379
failed to check the health of member 8372784203e11288 on https://192.168.55.21:2379: Get https://192.168.55.21:2379/health: dial tcp 192.168.55.21:2379: getsockopt: connection refused
member 8372784203e11288 is unreachable: [https://192.168.55.21:2379] are all unreachable
member ed4f0efd277d7599 is healthy: got healthy result from https://192.168.55.13:2379
cluster is healthy

# etcdctl2 member remove 8372784203e11288 1
Removed member 8372784203e11288 from cluster

# etcdctl2 cluster-health
member 5ee217d19001 is healthy: got healthy result from https://192.168.55.12:2379
member 2a529ba1840722c0 is healthy: got healthy result from https://192.168.55.8:2379
member ed4f0efd277d7599 is healthy: got healthy result from https://192.168.55.13:2379
cluster is healthy

要确保 etcd 配置在 etcd 服务重启时不使用失败的主机，修改所有剩余的 etcd 主机上的 /etc/etcd/etcd.conf 文件，并在 ETCD_INITIAL_CLUSTER 变量的值中删除失败主机：

# vi /etc/etcd/etcd.conf

例如：

ETCD_INITIAL_CLUSTER=master-0.example.com=https://192.168.55.8:2380,master-1.example.com=https://192.168.55.12:2380,master-2.example.com=https://192.168.55.13:2380

成为：

ETCD_INITIAL_CLUSTER=master-0.example.com=https://192.168.55.8:2380,master-1.example.com=https://192.168.55.12:2380

修改 Ansible 清单文件，以反映集群的当前状态，并避免在重新运行 playbook 时出现问题：

[OSEv3:children]
masters
nodes
etcd

... [OUTPUT ABBREVIATED] ...

[etcd]
master-0.example.com
master-1.example.com

如果您使用 Flannel，请修改每个主机上 /etc/sysconfig/flanneld 的 flanneld 服务配置并删除 etcd 主机：

FLANNEL_ETCD_ENDPOINTS=https://master-0.example.com:2379,https://master-1.example.com:2379,https://master-2.example.com:2379

重启 flanneld 服务：

# systemctl restart flanneld.service

删除该文件：

$ oc rsh demo-2-fxx6d
sh-4.2$ ls */opt/app-root/src/uploaded/*
lost+found  ocp_sop.txt
sh-4.2$ *rm -rf /opt/app-root/src/uploaded/ocp_sop.txt*
sh-4.2$ *ls /opt/app-root/src/uploaded/*
lost+found

替换包含 pvc 中文件的 rsync 备份的服务器中的文件：

$ oc rsync uploaded demo-2-fxx6d:/opt/app-root/src/

使用 oc rsh 连接到 pod 并查看目录的内容，验证该文件是否在 pod 上恢复：

$ oc rsh demo-2-fxx6d
sh-4.2$ *ls /opt/app-root/src/uploaded/*
lost+found  ocp_sop.txt

覆盖当前定义的 claim-name:

$ oc set volume dc/demo --add --name=persistent-volume \
		--type=persistentVolumeClaim --claim-name=filestore \ --mount-path=/opt/app-root/src/uploaded --overwrite

验证 pod 是否在使用新 PVC:

$ oc describe dc/demo
Name:		demo
Namespace:	test
Created:	3 hours ago
Labels:		app=demo
Annotations:	openshift.io/generated-by=OpenShiftNewApp
Latest Version:	3
Selector:	app=demo,deploymentconfig=demo
Replicas:	1
Triggers:	Config, Image(demo@latest, auto=true)
Strategy:	Rolling
Template:
  Labels:	app=demo
		deploymentconfig=demo
  Annotations:	openshift.io/container.demo.image.entrypoint=["container-entrypoint","/bin/sh","-c","$STI_SCRIPTS_PATH/usage"]
		openshift.io/generated-by=OpenShiftNewApp
  Containers:
   demo:
    Image:	docker-registry.default.svc:5000/test/demo@sha256:0a9f2487a0d95d51511e49d20dc9ff6f350436f935968b0c83fcb98a7a8c381a
    Port:	8080/TCP
    Volume Mounts:
      /opt/app-root/src/uploaded from persistent-volume (rw)
    Environment Variables:	<none>
  Volumes:
   persistent-volume:
    Type:	PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    *ClaimName:	filestore*
    ReadOnly:	false
...omitted...

现在，部署配置使用新的 pvc，运行 oc rsync 将文件放在新的 pvc 中：

$ oc rsync uploaded demo-3-2b8gs:/opt/app-root/src/
sending incremental file list
uploaded/
uploaded/ocp_sop.txt
uploaded/lost+found/

sent 181 bytes  received 39 bytes  146.67 bytes/sec
total size is 32  speedup is 0.15

使用 oc rsh 连接到 pod 并查看目录的内容，验证该文件是否在 pod 上恢复：

$ oc rsh demo-3-2b8gs
sh-4.2$ ls /opt/app-root/src/uploaded/
lost+found  ocp_sop.txt

从 master 实例或集群管理员，允许从该节点驱除任何 pod，并禁用该节点上调度其他 pod:

$ NODE=ose-app-node01.example.com
$ oc adm manage-node ${NODE} --schedulable=false
NAME                          STATUS                     AGE       VERSION
ose-app-node01.example.com   Ready,SchedulingDisabled   20m       v1.6.1+5115d708d7

$ oc adm drain ${NODE} --ignore-daemonsets
node "ose-app-node01.example.com" already cordoned
pod "perl-1-build" evicted
pod "perl-1-3lnsh" evicted
pod "perl-1-9jzd8" evicted
node "ose-app-node01.example.com" drained

列出节点上的 pod，以验证是否已移除它们：

$ oc adm manage-node ${NODE} --list-pods

Listing matched pods on node: ose-app-node01.example.com

NAME      READY     STATUS    RESTARTS   AGE

停止 docker ：

# systemctl stop docker

通过删除 pod 定义并重启主机来停止节点服务：

# mkdir -p /etc/origin/node/pods-stopped
# mv /etc/origin/node/pods/* /etc/origin/node/pods-stopped/

运行 docker-storage-setup 命令以扩展与容器存储关联的卷组和逻辑卷：

# docker-storage-setup

启动 Docker 服务：

# systemctl start docker
# vgs
  VG         #PV #LV #SN Attr   VSize  VFree
  docker_vol   2   1   0 wz--n- 64.99g <55.00g

恢复 pod 定义：

# mv /etc/origin/node/pods-stopped/* /etc/origin/node/pods/

通过重启主机来重启节点服务：

# systemctl restart atomic-openshift-node.service

在添加新卷组并重新运行 docker-storage-setup 时，添加磁盘的一个优点是，在添加新存储后，系统中使用的镜像仍然存在：

# container images
REPOSITORY                                              TAG                 IMAGE ID            CREATED             SIZE
docker-registry.default.svc:5000/tet/perl               latest              8b0b0106fb5e        13 minutes ago      627.4 MB
registry.redhat.io/rhscl/perl-524-rhel7         <none>              912b01ac7570        6 days ago          559.5 MB
registry.redhat.io/openshift3/ose-deployer      v3.6.173.0.21       89fd398a337d        5 weeks ago         970.2 MB
registry.redhat.io/openshift3/ose-pod           v3.6.173.0.21       63accd48a0d7        5 weeks ago         208.6 MB

随着存储容量增加，使节点可以调度，以接受新的传入的 pod。

作为集群管理员，从 master 实例运行以下 ：

$ oc adm manage-node ${NODE} --schedulable=true

ose-master01.example.com   Ready,SchedulingDisabled   24m       v1.6.1+5115d708d7
ose-master02.example.com   Ready,SchedulingDisabled   24m       v1.6.1+5115d708d7
ose-master03.example.com   Ready,SchedulingDisabled   24m       v1.6.1+5115d708d7
ose-infra-node01.example.com   Ready                      24m       v1.6.1+5115d708d7
ose-infra-node02.example.com   Ready                      24m       v1.6.1+5115d708d7
ose-infra-node03.example.com   Ready                      24m       v1.6.1+5115d708d7
ose-app-node01.example.com   Ready                      24m       v1.6.1+5115d708d7
ose-app-node02.example.com   Ready                      24m       v1.6.1+5115d708d7

停止 docker ：

# systemctl stop docker

通过删除 pod 定义来停止节点服务：

# mkdir -p /etc/origin/node/pods-stopped
# mv /etc/origin/node/pods/* /etc/origin/node/pods-stopped/

根据需要重新定义现有磁盘的大小。这可取决于您的环境：

运行 docker-storage-setup 以重新配置新磁盘：

# docker-storage-setup
INFO: Volume group backing root filesystem could not be determined
INFO: Device /dev/xvdb is already partitioned and is part of volume group docker_vol
INFO: Device node /dev/xvdd1 exists.
  Physical volume "/dev/xvdd1" successfully created.
  Volume group "docker_vol" successfully extended

启动 Docker 服务：

# systemctl start docker
# vgs
  VG         #PV #LV #SN Attr   VSize  VFree
  docker_vol   2   1   0 wz--n- 64.99g <55.00g

恢复 pod 定义：

# mv /etc/origin/node/pods-stopped/* /etc/origin/node/pods/

通过重启主机来重启节点服务：

# systemctl restart atomic-openshift-node.service

将 CA 证书复制到 /etc/pki/ca-trust/source/anchors/ 中：

$ sudo cp myregistry.example.com.crt /etc/pki/ca-trust/source/anchors/

提取 CA 证书并将其添加到可信证书颁发机构列表中：

$ sudo update-ca-trust extract

使用 openssl 命令验证 SSL 证书：

$ openssl verify myregistry.example.com.crt
myregistry.example.com.crt: OK

证书被放置并信任被更新后，重启 docker 服务以确保正确设置新证书：

$ sudo systemctl restart docker.service

在每个节点上在 /etc/docker/certs.d 中创建一个新目录，其中目录的名称是容器镜像 registry 的主机名：

$ sudo mkdir -p /etc/docker/certs.d/myregistry.example.com

通过 IP 地址访问容器镜像 registry 需要在 /etc/docker/certs.d 中创建新目录，其中目录的名称为容器镜像 registry 的 IP：

$ sudo mkdir -p /etc/docker/certs.d/10.10.10.10

将 CA 证书复制到前面步骤中新创建的 Docker 目录中：

$ sudo cp myregistry.example.com.crt \
  /etc/docker/certs.d/myregistry.example.com/ca.crt

$ sudo cp myregistry.example.com.crt /etc/docker/certs.d/10.10.10.10/ca.crt

复制证书后，重启 docker 服务以确保使用新证书：

$ sudo systemctl restart docker.service

如果使用 /etc/docker/certs.d，请复制目录中包含的所有证书并存储这些文件：

$ sudo tar -czvf docker-registry-certs-$(hostname)-$(date +%Y%m%d).tar.gz /etc/docker/certs.d/

如果使用系统信任，请在将证书添加到系统信任之前存储证书。存储完成后，使用 trust 命令提取证书以进行恢复。识别系统信任 CA 并记下 pkcs11 ID：

$ trust list
...[OUTPUT OMMITED]...
pkcs11:id=%a5%b3%e1%2b%2b%49%b6%d7%73%a1%aa%94%f5%01%e7%73%65%4c%ac%50;type=cert
    type: certificate
    label: MyCA
    trust: anchor
    category: authority
...[OUTPUT OMMITED]...

解压 pem 格式证书，并为其提供名称。例如，myca.crt。

$ trust extract --format=pem-bundle \
 --filter="%a5%b3%e1%2b%2b%49%b6%d7%73%a1%aa%94%f5%01%e7%73%65%4c%ac%50;type=cert" myca.crt

使用 openssl 检查已正确解压了证书：

$ openssl verify myca.crt

要允许用户在其他 registry 中使用 docker search 搜索镜像，请将这些 registry 添加到 registry 参数下的 /etc/containers/registries.conf 文件中：

registries:
  - registry.redhat.io
  - my.registry.example.com

重启 docker 守护进程以允许使用 my.registry.example.com ：

$ sudo systemctl restart docker.service

重启 docker 守护进程会导致 docker 容器重启。

使用 Ansible 安装程序，这可以使用 Ansible 主机文件中的 openshift_docker_additional_registries 变量进行配置：

openshift_docker_additional_registries=registry.redhat.io,my.registry.example.com

使用 registries 标记将允许的 registry 添加到 /etc/containers/registries.conf 文件中：

registries:
  - registry.redhat.io
  - my.registry.example.com

阻止其余的 registry：

block_registries:
   - all

阻止旧版本中的 registry 的其余部分：

BLOCK_REGISTRY='--block-registry=all'

重启 docker 守护进程：

$ sudo systemctl restart docker.service

重启 docker 守护进程会导致 docker 容器重启。

在本例中，docker.io registry 已被列入黑名单，因此有关该 registry 的任何操作都会失败：

$ sudo docker pull hello-world
Using default tag: latest
Trying to pull repository registry.redhat.io/hello-world ...
Trying to pull repository my.registry.example.com/hello-world ...
Trying to pull repository registry.redhat.io/hello-world ...
unknown: Not Found
$ sudo docker pull docker.io/hello-world
Using default tag: latest
Trying to pull repository docker.io/library/hello-world ...
All endpoints blocked.

通过再次修改文件并重新启动服务，将 docker.io 添加会 registry 变量。

registries:
  - registry.redhat.io
  - my.registry.example.com
  - docker.io
block_registries:
  - all

或者

ADD_REGISTRY="--add-registry=registry.redhat.io --add-registry=my.registry.example.com --add-registry=docker.io"
BLOCK_REGISTRY='--block-registry=all'

重启 Docker 服务：

$ sudo systemctl restart docker

验证镜像现在可拉取(pull):

$ sudo docker pull docker.io/hello-world
Using default tag: latest
Trying to pull repository docker.io/library/hello-world ...
latest: Pulling from docker.io/library/hello-world

9a0669468bf7: Pull complete
Digest: sha256:0e06ef5e1945a718b02a8c319e15bae44f47039005530bc617a5d071190ed3fc

如果需要使用外部 registry，例如修改所有需要使用该 registry 的节点主机中的 docker 守护进程配置文件，请在这些节点上创建一个黑名单以避免恶意容器被执行。

使用 Ansible 安装程序，这可以使用 Ansible 主机文件中的 openshift_docker_additional_registries 和 openshift_docker_blocked_registries 变量进行配置：

openshift_docker_additional_registries=registry.redhat.io,my.registry.example.com
openshift_docker_blocked_registries=all

修改 /etc/containers/registries.conf 配置文件以添加 localhost 不安全的 registry:

[registries.search]
registries = ['registry.redhat.io', 'my.registry.example.com', 'docker.io', 'localhost:5000' ]

[registries.insecure]
registries = ['localhost:5000']

[registries.block]
registries = ['all']

重启 docker 守护进程以使用 registry：

$ sudo systemctl restart docker.service

重启 docker 守护进程会导致 docker 容器被重启。

在 localhost 运行容器镜像 registry pod:

$ sudo docker run -p 5000:5000 registry:2

拉取镜像：

$ sudo docker pull openshift/hello-openshift

标记镜像：

$ sudo docker tag docker.io/openshift/hello-openshift:latest localhost:5000/hello-openshift-local:latest

将镜像推送到本地 registry：

$ sudo docker push localhost:5000/hello-openshift-local:latest

使用 Ansible 安装程序，这可以使用 Ansible 主机文件中的 openshift_docker_additional_registries、openshift_docker_blocked_registries 和 openshift_docker_insecure_registries 变量进行配置：

openshift_docker_additional_registries=registry.redhat.io,my.registry.example.com,localhost:5000
openshift_docker_insecure_registries=localhost:5000
openshift_docker_blocked_registries=all

在用户要登录到 docker registry 的项目中创建 dockercfg secret：

$ oc project <my_project>
$ oc create secret docker-registry <my_registry> --docker-server=<my.registry.example.com> --docker-username=<username> --docker-password=<my_password> --docker-email=<me@example.com>

如果存在 .dockercfg 文件，请使用 oc 命令创建 secret：

$ oc create secret generic <my_registry> --from-file=.dockercfg=<path/to/.dockercfg> --type=kubernetes.io/dockercfg

填充 $HOME/.docker/config.json 文件：

$ oc create secret generic <my_registry> --from-file=.dockerconfigjson=<path/to/.dockercfg> --type=kubernetes.io/dockerconfigjson

通过将 secret 链接到执行 pull 操作的服务帐户，使用 dockercfg secret 从经过身份验证的 registry 拉取镜像。用于拉取镜像的默认服务帐户名为 default ：

$ oc secrets link default <my_registry> --for=pull

要使用 S2I 功能推送镜像，dockercfg secret 被挂载到 S2I pod 中，因此需要链接到执行该构建的正确服务帐户。用于构建镜像的默认服务帐户名为 builder。

$ oc secrets link builder <my_registry>

在 buildconfig 中，应指定 secret 进行 push 或 pull 操作：

"type": "Source",
"sourceStrategy": {
    "from": {
        "kind": "DockerImage",
        "name": "*my.registry.example.com*/myproject/myimage:stable"
    },
    "pullSecret": {
        "name": "*mydockerregistry*"
    },
...[OUTPUT ABBREVIATED]...
"output": {
    "to": {
        "kind": "DockerImage",
        "name": "*my.registry.example.com*/myproject/myimage:latest"
    },
    "pushSecret": {
        "name": "*mydockerregistry*"
    },
...[OUTPUT ABBREVIATED]...

如果外部 Registry 将身份验证委派给外部服务，则创建 dockercfg secret：registry 使用 registry URL 和外部身份验证系统，并使用自己的 URL。这两个 secret 都应添加到服务帐户。

$ oc project <my_project>
$ oc create secret docker-registry <my_registry> --docker-server=*<my_registry_example.com> --docker-username=<username> --docker-password=<my_password> --docker-email=<me@example.com>
$ oc create secret docker-registry <my_docker_registry_ext_auth> --docker-server=<my.authsystem.example.com> --docker-username=<username> --docker-password=<my_password> --docker-email=<me@example.com>
$ oc secrets link default <my_registry> --for=pull
$ oc secrets link default <my_docker_registry_ext_auth> --for=pull
$ oc secrets link builder <my_registry>
$ oc secrets link builder <my_docker_registry_ext_auth>

如果启用了 ImagePolicy 插件，则需要修改它，以允许在每个 master 节点上修改 /etc/origin/master/master-config.yaml 文件来允许使用外部 registry：

admissionConfig:
  pluginConfig:
    openshift.io/ImagePolicy:
      configuration:
        kind: ImagePolicyConfig
        apiVersion: v1
        executionRules:
        - name: allow-images-from-other-registries
          onResources:
          - resource: pods
          - resource: builds
          matchRegistries:
          - docker.io
          - <my.registry.example.com>
          - registry.redhat.io

要在安装时启用准入插件，openshift_master_admission_plugin_config 变量可用于 json 格式的字符串，包括所有 pluginConfig 配置：

openshift_master_admission_plugin_config={"openshift.io/ImagePolicy":{"configuration":{"kind":"ImagePolicyConfig","apiVersion":"v1","executionRules":[{"name":"allow-images-from-other-registries","onResources":[{"resource":"pods"},{"resource":"builds"}],"matchRegistries":["docker.io","*my.registry.example.com*","registry.redhat.io"]}]}}}

要配置可以导入镜像的允许的 registry，请将以下内容添加到 /etc/origin/master/master-config.yaml 文件中：

imagePolicyConfig:
  allowedRegistriesForImport:
  - domainName: docker.io
  - domainName: '\*.docker.io'
  - domainName: '*.redhat.com'
  - domainName: 'my.registry.example.com'

即使不建议使用外部身份验证的 registry，或者无法信任证书，oc import-image 命令可以与 --insecure=true 选项一起使用。

如果外部经过身份验证的用户的 registry 是安全的，则 registry 证书应该在 master 主机中运行 registry 导入控制器时被信任：

在 /etc/pki/ca-trust/source/anchors/ 中复制证书：

$ sudo cp <my.registry.example.com.crt> /etc/pki/ca-trust/source/anchors/<my.registry.example.com.crt>

运行 update-ca-trust 命令：

$ sudo update-ca-trust

重启所有 master 主机上的 master 服务：

$ sudo master-restart api
$ sudo master-restart controllers

外部 registry 的证书应该在 OpenShift Container Platform registry 中信任：

$ for i in pem openssl java; do
  oc create configmap ca-trust-extracted-${i} --from-file /etc/pki/ca-trust/extracted/${i}
  oc set volume dc/docker-registry --add -m /etc/pki/ca-trust/extracted/${i} --configmap-name=ca-trust-extracted-${i} --name ca-trust-extracted-${i}
done

或者，修改 registry 镜像，以信任使用 Dockerfile 重建镜像的适当证书：

FROM registry.redhat.io/openshift3/ose-docker-registry:v3.6
ADD <my.registry.example.com.crt> /etc/pki/ca-trust/source/anchors/
USER 0
RUN update-ca-trust extract
USER 1001

重建镜像，将其推送到 docker registry，并将该镜像用作 registry deploymentconfig 中的 spec.template.spec.containers["name":"registry"].image:

$ oc patch dc docker-registry -p '{"spec":{"template":{"spec":{"containers":[{"name":"registry","image":"*myregistry.example.com/openshift3/ose-docker-registry:latest*"}]}}}}'