红帽全球峰会A.3.3. 部署融合模式
执行以下命令来部署聚合模式：
在客户端上执行以下命令，以部署 heketi 和 Red Hat Gluster Storage pod：
# cns-deploy -v -n <namespace> -g --admin-key <admin-key> --user-key <user-key> topology.json注意- 从 Container-Native Storage 3.6 中，对 Red Hat Openshift Container Storage 中的 S3 兼容对象存储的支持处于技术预览阶段。要在 Red Hat Openshift Container Storage 中部署 S3 兼容对象存储，请参阅以下子步骤。
-
在上述命令中，
admin-key的值是 heketi admin 用户的 secret 字符串。heketi 管理员有权访问所有 API 和命令。默认为不使用 secret。 cns-deploy 中的
BLOCK_HOST_SIZE参数控制自动创建的 Red Hat Gluster Storage 卷的大小（以 GB 为单位）。当需要更多空间时，此默认配置将动态创建块托管为 500GB 的卷。如果要更改此值，则在 cns-deploy 中使用 --block-host。例如：# cns-deploy -v -n storage-project -g --admin-key secret --user-key mysecret --block-host 1000 topology.json
例如：
# cns-deploy -v -n storage-project -g --admin-key secret --user-key mysecret topology.json Welcome to the deployment tool for GlusterFS on Kubernetes and OpenShift. Before getting started, this script has some requirements of the execution environment and of the container platform that you should verify. The client machine that will run this script must have: * Administrative access to an existing Kubernetes or OpenShift cluster * Access to a python interpreter 'python' Each of the nodes that will host GlusterFS must also have appropriate firewall rules for the required GlusterFS ports: * 111 - rpcbind (for glusterblock) * 2222 - sshd (if running GlusterFS in a pod) * 3260 - iSCSI targets (for glusterblock) * 24010 - glusterblockd * 24007 - GlusterFS Management * 24008 - GlusterFS RDMA * 49152 to 49251 - Each brick for every volume on the host requires its own port. For every new brick, one new port will be used starting at 49152. We recommend a default range of 49152-49251 on each host, though you can adjust this to fit your needs. The following kernel modules must be loaded: * dm_snapshot * dm_mirror * dm_thin_pool * dm_multipath * target_core_user For systems with SELinux, the following settings need to be considered: * virt_sandbox_use_fusefs should be enabled on each node to allow writing to remote GlusterFS volumes In addition, for an OpenShift deployment you must: * Have 'cluster_admin' role on the administrative account doing the deployment * Add the 'default' and 'router' Service Accounts to the 'privileged' SCC * Have a router deployed that is configured to allow apps to access services running in the cluster Do you wish to proceed with deployment? [Y]es, [N]o? [Default: Y]: Y Using OpenShift CLI. Using namespace "storage-project". Checking for pre-existing resources... GlusterFS pods ... not found. deploy-heketi pod ... not found. heketi pod ... not found. glusterblock-provisioner pod ... not found. gluster-s3 pod ... not found. Creating initial resources ... template "deploy-heketi" created serviceaccount "heketi-service-account" created template "heketi" created template "glusterfs" created role "edit" added: "system:serviceaccount:storage-project:heketi-service-account" OK node "ip-172-18-5-29.ec2.internal" labeled node "ip-172-18-8-205.ec2.internal" labeled node "ip-172-18-6-100.ec2.internal" labeled daemonset "glusterfs" created Waiting for GlusterFS pods to start ... OK secret "heketi-config-secret" created secret "heketi-config-secret" labeled service "deploy-heketi" created route "deploy-heketi" created deploymentconfig "deploy-heketi" created Waiting for deploy-heketi pod to start ... OK Creating cluster ... ID: 30cd12e60f860fce21e7e7457d07db36 Allowing file volumes on cluster. Allowing block volumes on cluster. Creating node ip-172-18-5-29.ec2.internal ... ID: 4077242c76e5f477a27c5c47247cb348 Adding device /dev/xvdc ... OK Creating node ip-172-18-8-205.ec2.internal ... ID: dda0e7d568d7b2f76a7e7491cfc26dd3 Adding device /dev/xvdc ... OK Creating node ip-172-18-6-100.ec2.internal ... ID: 30a1795ca515c85dca32b09be7a68733 Adding device /dev/xvdc ... OK heketi topology loaded. Saving /tmp/heketi-storage.json secret "heketi-storage-secret" created endpoints "heketi-storage-endpoints" created service "heketi-storage-endpoints" created job "heketi-storage-copy-job" created service "heketi-storage-endpoints" labeled deploymentconfig "deploy-heketi" deleted route "deploy-heketi" deleted service "deploy-heketi" deleted job "heketi-storage-copy-job" deleted pod "deploy-heketi-1-frjpt" deleted secret "heketi-storage-secret" deleted template "deploy-heketi" deleted service "heketi" created route "heketi" created deploymentconfig "heketi" created Waiting for heketi pod to start ... OK heketi is now running and accessible via http://heketi-storage-project.cloudapps.mystorage.com . To run administrative commands you can install 'heketi-cli' and use it as follows: # heketi-cli -s http://heketi-storage-project.cloudapps.mystorage.com --user admin --secret '<ADMIN_KEY>' cluster list You can find it at https://github.com/heketi/heketi/releases . Alternatively, use it from within the heketi pod: # /bin/oc -n storage-project exec -it <HEKETI_POD> -- heketi-cli -s http://localhost:8080 --user admin --secret '<ADMIN_KEY>' cluster list For dynamic provisioning, create a StorageClass similar to this: --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: glusterfs-storage provisioner: kubernetes.io/glusterfs parameters: resturl: "http://heketi-storage-project.cloudapps.mystorage.com" Ready to create and provide GlusterFS volumes. clusterrole "glusterblock-provisioner-runner" created serviceaccount "glusterblock-provisioner" created clusterrolebinding "glusterblock-provisioner" created deploymentconfig "glusterblock-provisioner-dc" created Waiting for glusterblock-provisioner pod to start ... OK Ready to create and provide Gluster block volumes. Deployment complete!注意For more information on the cns-deploy commands, refer to the man page of cns-deploy.+
# cns-deploy --help要部署 S3 兼容对象存储以及 Heketi 和 Red Hat Gluster Storage pod，请执行以下命令：
# cns-deploy /opt/topology.json --deploy-gluster --namespace <namespace> --yes --admin-key <admin-key> --user-key <user-key> --log-file=<path/to/logfile> --object-account <object account name> --object-user <object user name> --object-password <object user password> --verboseobject-account、object-user和object-password是部署 gluster-s3 容器所需的凭证。如果缺少其中任何一个，gluster-s3 容器部署将会被跳过。object-sc和object-capacity是可选参数。其中，object-sc用于指定已存在的 StorageClass，用于创建 Red Hat Gluster 存储卷来备份对象存储和对象容量，这是将存储对象数据的 Red Hat Gluster 存储卷的总容量。例如：
# cns-deploy /opt/topology.json --deploy-gluster --namespace storage-project --yes --admin-key secret --user-key mysecret --log-file=/var/log/cns-deploy/444-cns-deploy.log --object-account testvolume --object-user adminuser --object-password itsmine --verbose Using OpenShift CLI. Checking status of namespace matching 'storage-project': storage-project Active 56m Using namespace "storage-project". Checking for pre-existing resources... GlusterFS pods ... Checking status of pods matching '--selector=glusterfs=pod': No resources found. Timed out waiting for pods matching '--selector=glusterfs=pod'. not found. deploy-heketi pod ... Checking status of pods matching '--selector=deploy-heketi=pod': No resources found. Timed out waiting for pods matching '--selector=deploy-heketi=pod'. not found. heketi pod ... Checking status of pods matching '--selector=heketi=pod': No resources found. Timed out waiting for pods matching '--selector=heketi=pod'. not found. glusterblock-provisioner pod ... Checking status of pods matching '--selector=glusterfs=block-provisioner-pod': No resources found. Timed out waiting for pods matching '--selector=glusterfs=block-provisioner-pod'. not found. gluster-s3 pod ... Checking status of pods matching '--selector=glusterfs=s3-pod': No resources found. Timed out waiting for pods matching '--selector=glusterfs=s3-pod'. not found. Creating initial resources ... /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/deploy-heketi-template.yaml 2>&1 template "deploy-heketi" created /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/heketi-service-account.yaml 2>&1 serviceaccount "heketi-service-account" created /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/heketi-template.yaml 2>&1 template "heketi" created /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/glusterfs-template.yaml 2>&1 template "glusterfs" created /usr/bin/oc -n storage-project policy add-role-to-user edit system:serviceaccount:storage-project:heketi-service-account 2>&1 role "edit" added: "system:serviceaccount:storage-project:heketi-service-account" /usr/bin/oc -n storage-project adm policy add-scc-to-user privileged -z heketi-service-account OK Marking 'dhcp46-122.lab.eng.blr.redhat.com' as a GlusterFS node. /usr/bin/oc -n storage-project label nodes dhcp46-122.lab.eng.blr.redhat.com storagenode=glusterfs 2>&1 node "dhcp46-122.lab.eng.blr.redhat.com" labeled Marking 'dhcp46-9.lab.eng.blr.redhat.com' as a GlusterFS node. /usr/bin/oc -n storage-project label nodes dhcp46-9.lab.eng.blr.redhat.com storagenode=glusterfs 2>&1 node "dhcp46-9.lab.eng.blr.redhat.com" labeled Marking 'dhcp46-134.lab.eng.blr.redhat.com' as a GlusterFS node. /usr/bin/oc -n storage-project label nodes dhcp46-134.lab.eng.blr.redhat.com storagenode=glusterfs 2>&1 node "dhcp46-134.lab.eng.blr.redhat.com" labeled Deploying GlusterFS pods. /usr/bin/oc -n storage-project process -p NODE_LABEL=glusterfs glusterfs | /usr/bin/oc -n storage-project create -f - 2>&1 daemonset "glusterfs" created Waiting for GlusterFS pods to start ... Checking status of pods matching '--selector=glusterfs=pod': glusterfs-6fj2v 1/1 Running 0 52s glusterfs-ck40f 1/1 Running 0 52s glusterfs-kbtz4 1/1 Running 0 52s OK /usr/bin/oc -n storage-project create secret generic heketi-config-secret --from-file=private_key=/dev/null --from-file=./heketi.json --from-file=topology.json=/opt/topology.json secret "heketi-config-secret" created /usr/bin/oc -n storage-project label --overwrite secret heketi-config-secret glusterfs=heketi-config-secret heketi=config-secret secret "heketi-config-secret" labeled /usr/bin/oc -n storage-project process -p HEKETI_EXECUTOR=kubernetes -p HEKETI_FSTAB=/var/lib/heketi/fstab -p HEKETI_ADMIN_KEY= -p HEKETI_USER_KEY= deploy-heketi | /usr/bin/oc -n storage-project create -f - 2>&1 service "deploy-heketi" created route "deploy-heketi" created deploymentconfig "deploy-heketi" created Waiting for deploy-heketi pod to start ... Checking status of pods matching '--selector=deploy-heketi=pod': deploy-heketi-1-hf9rn 1/1 Running 0 2m OK Determining heketi service URL ... OK /usr/bin/oc -n storage-project exec -it deploy-heketi-1-hf9rn -- heketi-cli -s http://localhost:8080 --user admin --secret '' topology load --json=/etc/heketi/topology.json 2>&1 Creating cluster ... ID: 252509038eb8568162ec5920c12bc243 Allowing file volumes on cluster. Allowing block volumes on cluster. Creating node dhcp46-122.lab.eng.blr.redhat.com ... ID: 73ad287ae1ef231f8a0db46422367c9a Adding device /dev/sdd ... OK Adding device /dev/sde ... OK Adding device /dev/sdf ... OK Creating node dhcp46-9.lab.eng.blr.redhat.com ... ID: 0da1b20daaad2d5c57dbfc4f6ab78001 Adding device /dev/sdd ... OK Adding device /dev/sde ... OK Adding device /dev/sdf ... OK Creating node dhcp46-134.lab.eng.blr.redhat.com ... ID: 4b3b62fc0efd298dedbcdacf0b498e65 Adding device /dev/sdd ... OK Adding device /dev/sde ... OK Adding device /dev/sdf ... OK heketi topology loaded. /usr/bin/oc -n storage-project exec -it deploy-heketi-1-hf9rn -- heketi-cli -s http://localhost:8080 --user admin --secret '' setup-openshift-heketi-storage --listfile=/tmp/heketi-storage.json --image rhgs3/rhgs-volmanager-rhel7:3.3.0-17 2>&1 Saving /tmp/heketi-storage.json /usr/bin/oc -n storage-project exec -it deploy-heketi-1-hf9rn -- cat /tmp/heketi-storage.json | /usr/bin/oc -n storage-project create -f - 2>&1 secret "heketi-storage-secret" created endpoints "heketi-storage-endpoints" created service "heketi-storage-endpoints" created job "heketi-storage-copy-job" created Checking status of pods matching '--selector=job-name=heketi-storage-copy-job': heketi-storage-copy-job-87v6n 0/1 Completed 0 7s /usr/bin/oc -n storage-project label --overwrite svc heketi-storage-endpoints glusterfs=heketi-storage-endpoints heketi=storage-endpoints service "heketi-storage-endpoints" labeled /usr/bin/oc -n storage-project delete all,service,jobs,deployment,secret --selector="deploy-heketi" 2>&1 deploymentconfig "deploy-heketi" deleted route "deploy-heketi" deleted service "deploy-heketi" deleted job "heketi-storage-copy-job" deleted pod "deploy-heketi-1-hf9rn" deleted secret "heketi-storage-secret" deleted /usr/bin/oc -n storage-project delete dc,route,template --selector="deploy-heketi" 2>&1 template "deploy-heketi" deleted /usr/bin/oc -n storage-project process -p HEKETI_EXECUTOR=kubernetes -p HEKETI_FSTAB=/var/lib/heketi/fstab -p HEKETI_ADMIN_KEY= -p HEKETI_USER_KEY= heketi | /usr/bin/oc -n storage-project create -f - 2>&1 service "heketi" created route "heketi" created deploymentconfig "heketi" created Waiting for heketi pod to start ... Checking status of pods matching '--selector=heketi=pod': heketi-1-zzblp 1/1 Running 0 31s OK Determining heketi service URL ... OK heketi is now running and accessible via http://heketi-storage-project.cloudapps.mystorage.com . To run administrative commands you can install 'heketi-cli' and use it as follows: # heketi-cli -s http://heketi-storage-project.cloudapps.mystorage.com --user admin --secret '<ADMIN_KEY>' cluster list You can find it at https://github.com/heketi/heketi/releases . Alternatively, use it from within the heketi pod: # /usr/bin/oc -n storage-project exec -it <HEKETI_POD> -- heketi-cli -s http://localhost:8080 --user admin --secret '<ADMIN_KEY>' cluster list For dynamic provisioning, create a StorageClass similar to this: --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: glusterfs-storage provisioner: kubernetes.io/glusterfs parameters: resturl: "http://heketi-storage-project.cloudapps.mystorage.com" Ready to create and provide GlusterFS volumes. sed -e 's/\${NAMESPACE}/storage-project/' /usr/share/heketi/templates/glusterblock-provisioner.yaml | /usr/bin/oc -n storage-project create -f - 2>&1 clusterrole "glusterblock-provisioner-runner" created serviceaccount "glusterblock-provisioner" created clusterrolebinding "glusterblock-provisioner" created deploymentconfig "glusterblock-provisioner-dc" created Waiting for glusterblock-provisioner pod to start ... Checking status of pods matching '--selector=glusterfs=block-provisioner-pod': glusterblock-provisioner-dc-1-xm6bv 1/1 Running 0 6s OK Ready to create and provide Gluster block volumes. /usr/bin/oc -n storage-project create secret generic heketi-storage-project-admin-secret --from-literal=key= --type=kubernetes.io/glusterfs secret "heketi-storage-project-admin-secret" created /usr/bin/oc -n storage-project label --overwrite secret heketi-storage-project-admin-secret glusterfs=s3-heketi-storage-project-admin-secret gluster-s3=heketi-storage-project-admin-secret secret "heketi-storage-project-admin-secret" labeled sed -e 's/\${STORAGE_CLASS}/glusterfs-for-s3/' -e 's/\${HEKETI_URL}/heketi-storage-project.cloudapps.mystorage.com/' -e 's/\${NAMESPACE}/storage-project/' /usr/share/heketi/templates/gluster-s3-storageclass.yaml | /usr/bin/oc -n storage-project create -f - 2>&1 storageclass "glusterfs-for-s3" created sed -e 's/\${STORAGE_CLASS}/glusterfs-for-s3/' -e 's/\${VOLUME_CAPACITY}/2Gi/' /usr/share/heketi/templates/gluster-s3-pvcs.yaml | /usr/bin/oc -n storage-project create -f - 2>&1 persistentvolumeclaim "gluster-s3-claim" created persistentvolumeclaim "gluster-s3-meta-claim" created Checking status of persistentvolumeclaims matching '--selector=glusterfs in (s3-pvc, s3-meta-pvc)': gluster-s3-claim Bound pvc-35b6c1f0-9c65-11e7-9c8c-005056b3ded1 2Gi RWX glusterfs-for-s3 18s gluster-s3-meta-claim Bound pvc-35b86e7a-9c65-11e7-9c8c-005056b3ded1 1Gi RWX glusterfs-for-s3 18s /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/gluster-s3-template.yaml 2>&1 template "gluster-s3" created /usr/bin/oc -n storage-project process -p S3_ACCOUNT=testvolume -p S3_USER=adminuser -p S3_PASSWORD=itsmine gluster-s3 | /usr/bin/oc -n storage-project create -f - 2>&1 service "gluster-s3-service" created route "gluster-s3-route" created deploymentconfig "gluster-s3-dc" created Waiting for gluster-s3 pod to start ... Checking status of pods matching '--selector=glusterfs=s3-pod': gluster-s3-dc-1-x3x4q 1/1 Running 0 6s OK Ready to create and provide Gluster object volumes. Deployment complete!
执行以下命令以便客户端与容器通信：
# export HEKETI_CLI_SERVER=http://heketi-<project_name>.<sub_domain_name>例如：
# export HEKETI_CLI_SERVER=http://heketi-storage-project.cloudapps.mystorage.com要验证 Heketi 是否已使用拓扑载入，请执行以下命令：
# heketi-cli topology info
注意
The cns-deploy tool does not support scaling up of the cluster. To manually scale-up the cluster, see link:https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/3.11/html-single/operations_guide/#chap-Documentation-Red_Hat_Gluster_Storage_Container_Native_with_OpenShift_Platform-Managing_Clusters[]A.3.3.1. 部署独立模式
复制链接链接已复制到粘贴板!
执行以下命令以独立模式部署 Red Hat Openshift Container Storage：
要设置到所有 Red Hat Gluster Storage 节点的免密码 SSH，请在客户端上为每个 Red Hat Gluster Storage 节点执行以下命令：
# ssh-copy-id -i /root/.ssh/id_rsa root@<hostname>在客户端上执行以下命令，以部署 heketi pod 并创建 Red Hat Gluster Storage 节点集群：
# cns-deploy -v -n <namespace> -g --admin-key <admin-key> --user-key <user-key> topology.json注意- 对 S3 兼容对象存储的支持处于技术预览阶段。部署 S3 兼容对象存储请查看子步骤，如下所示。
-
在上述命令中，
admin-key的值是 heketi admin 用户的 secret 字符串。heketi 管理员有权访问所有 API 和命令。默认为不使用 secret。 cns-deploy 中的
BLOCK_HOST_SIZE参数控制自动创建的 Red Hat Gluster Storage 卷的大小（以 GB 为单位）。当需要更多空间时，此默认配置将动态创建块托管为 500GB 的卷。如果要更改此值，则在 cns-deploy 中使用 --block-host。例如：# cns-deploy -v -n storage-project -g --admin-key secret --user-key mysecret --block-host 1000 topology.json
例如：
# cns-deploy -v -n storage-project -g --admin-key secret -s /root/.ssh/id_rsa --user-key mysecret topology.json Welcome to the deployment tool for GlusterFS on Kubernetes and OpenShift. Before getting started, this script has some requirements of the execution environment and of the container platform that you should verify. The client machine that will run this script must have: * Administrative access to an existing Kubernetes or OpenShift cluster * Access to a python interpreter 'python' Each of the nodes that will host GlusterFS must also have appropriate firewall rules for the required GlusterFS ports: * 2222 - sshd (if running GlusterFS in a pod) * 24007 - GlusterFS Management * 24008 - GlusterFS RDMA * 49152 to 49251 - Each brick for every volume on the host requires its own port. For every new brick, one new port will be used starting at 49152. We recommend a default range of 49152-49251 on each host, though you can adjust this to fit your needs. The following kernel modules must be loaded: * dm_snapshot * dm_mirror * dm_thin_pool For systems with SELinux, the following settings need to be considered: * virt_sandbox_use_fusefs should be enabled on each node to allow writing to remote GlusterFS volumes In addition, for an OpenShift deployment you must: * Have 'cluster_admin' role on the administrative account doing the deployment * Add the 'default' and 'router' Service Accounts to the 'privileged' SCC * Have a router deployed that is configured to allow apps to access services running in the cluster Do you wish to proceed with deployment? [Y]es, [N]o? [Default: Y]: y Using OpenShift CLI. Using namespace "storage-project". Checking for pre-existing resources... GlusterFS pods ... not found. deploy-heketi pod ... not found. heketi pod ... not found. Creating initial resources ... template "deploy-heketi" created serviceaccount "heketi-service-account" created template "heketi" created role "edit" added: "system:serviceaccount:storage-project:heketi-service-account" OK secret "heketi-config-secret" created secret "heketi-config-secret" labeled service "deploy-heketi" created route "deploy-heketi" created deploymentconfig "deploy-heketi" created Waiting for deploy-heketi pod to start ... OK Creating cluster ... ID: 60bf06636eb4eb81d4e9be4b04cfce92 Allowing file volumes on cluster. Allowing block volumes on cluster. Creating node dhcp47-104.lab.eng.blr.redhat.com ... ID: eadc66f9d03563bcfc3db3fe636c34be Adding device /dev/sdd ... OK Adding device /dev/sde ... OK Adding device /dev/sdf ... OK Creating node dhcp47-83.lab.eng.blr.redhat.com ... ID: 178684b0a0425f51b8f1a032982ffe4d Adding device /dev/sdd ... OK Adding device /dev/sde ... OK Adding device /dev/sdf ... OK Creating node dhcp46-152.lab.eng.blr.redhat.com ... ID: 08cd7034ef7ac66499dc040d93cf4a93 Adding device /dev/sdd ... OK Adding device /dev/sde ... OK Adding device /dev/sdf ... OK heketi topology loaded. Saving /tmp/heketi-storage.json secret "heketi-storage-secret" created endpoints "heketi-storage-endpoints" created service "heketi-storage-endpoints" created job "heketi-storage-copy-job" created service "heketi-storage-endpoints" labeled deploymentconfig "deploy-heketi" deleted route "deploy-heketi" deleted service "deploy-heketi" deleted job "heketi-storage-copy-job" deleted pod "deploy-heketi-1-30c06" deleted secret "heketi-storage-secret" deleted template "deploy-heketi" deleted service "heketi" created route "heketi" created deploymentconfig "heketi" created Waiting for heketi pod to start ... OK heketi is now running and accessible via http://heketi-storage-project.cloudapps.mystorage.com . To run administrative commands you can install 'heketi-cli' and use it as follows: # heketi-cli -s http://heketi-storage-project.cloudapps.mystorage.com --user admin --secret '<ADMIN_KEY>' cluster list You can find it at https://github.com/heketi/heketi/releases . Alternatively, use it from within the heketi pod: # /usr/bin/oc -n storage-project exec -it <HEKETI_POD> -- heketi-cli -s http://localhost:8080 --user admin --secret '<ADMIN_KEY>' cluster list For dynamic provisioning, create a StorageClass similar to this: --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: glusterfs-storage provisioner: kubernetes.io/glusterfs parameters: resturl: "http://heketi-storage-project.cloudapps.mystorage.com" Deployment complete!注意For more information on the cns-deploy commands, refer to the man page of the cns-deploy.+
# cns-deploy --help要部署 S3 兼容对象存储以及 Heketi 和 Red Hat Gluster Storage pod，请执行以下命令：
# cns-deploy /opt/topology.json --deploy-gluster --namespace <namespace> --admin-key <admin-key> --user-key <user-key> --yes --log-file=<path/to/logfile> --object-account <object account name> --object-user <object user name> --object-password <object user password> --verboseobject-account、object-user和object-password是部署 gluster-s3 容器所需的凭证。如果缺少其中任何一个，gluster-s3 容器部署将会被跳过。object-sc和object-capacity是可选参数。其中，object-sc用于指定已存在的 StorageClass，用于创建 Red Hat Gluster 存储卷来备份对象存储和对象容量，这是将存储对象数据的 Red Hat Gluster 存储卷的总容量。例如：
# cns-deploy /opt/topology.json --deploy-gluster --namespace storage-project --admin-key secret --user-key mysecret --yes --log-file=/var/log/cns-deploy/444-cns-deploy.log --object-account testvolume --object-user adminuser --object-password itsmine --verbose Using OpenShift CLI. Checking status of namespace matching 'storage-project': storage-project Active 56m Using namespace "storage-project". Checking for pre-existing resources... GlusterFS pods ... Checking status of pods matching '--selector=glusterfs=pod': No resources found. Timed out waiting for pods matching '--selector=glusterfs=pod'. not found. deploy-heketi pod ... Checking status of pods matching '--selector=deploy-heketi=pod': No resources found. Timed out waiting for pods matching '--selector=deploy-heketi=pod'. not found. heketi pod ... Checking status of pods matching '--selector=heketi=pod': No resources found. Timed out waiting for pods matching '--selector=heketi=pod'. not found. glusterblock-provisioner pod ... Checking status of pods matching '--selector=glusterfs=block-provisioner-pod': No resources found. Timed out waiting for pods matching '--selector=glusterfs=block-provisioner-pod'. not found. gluster-s3 pod ... Checking status of pods matching '--selector=glusterfs=s3-pod': No resources found. Timed out waiting for pods matching '--selector=glusterfs=s3-pod'. not found. Creating initial resources ... /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/deploy-heketi-template.yaml 2>&1 template "deploy-heketi" created /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/heketi-service-account.yaml 2>&1 serviceaccount "heketi-service-account" created /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/heketi-template.yaml 2>&1 template "heketi" created /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/glusterfs-template.yaml 2>&1 template "glusterfs" created /usr/bin/oc -n storage-project policy add-role-to-user edit system:serviceaccount:storage-project:heketi-service-account 2>&1 role "edit" added: "system:serviceaccount:storage-project:heketi-service-account" /usr/bin/oc -n storage-project adm policy add-scc-to-user privileged -z heketi-service-account OK Marking 'dhcp46-122.lab.eng.blr.redhat.com' as a GlusterFS node. /usr/bin/oc -n storage-project label nodes dhcp46-122.lab.eng.blr.redhat.com storagenode=glusterfs 2>&1 node "dhcp46-122.lab.eng.blr.redhat.com" labeled Marking 'dhcp46-9.lab.eng.blr.redhat.com' as a GlusterFS node. /usr/bin/oc -n storage-project label nodes dhcp46-9.lab.eng.blr.redhat.com storagenode=glusterfs 2>&1 node "dhcp46-9.lab.eng.blr.redhat.com" labeled Marking 'dhcp46-134.lab.eng.blr.redhat.com' as a GlusterFS node. /usr/bin/oc -n storage-project label nodes dhcp46-134.lab.eng.blr.redhat.com storagenode=glusterfs 2>&1 node "dhcp46-134.lab.eng.blr.redhat.com" labeled Deploying GlusterFS pods. /usr/bin/oc -n storage-project process -p NODE_LABEL=glusterfs glusterfs | /usr/bin/oc -n storage-project create -f - 2>&1 daemonset "glusterfs" created Waiting for GlusterFS pods to start ... Checking status of pods matching '--selector=glusterfs=pod': glusterfs-6fj2v 1/1 Running 0 52s glusterfs-ck40f 1/1 Running 0 52s glusterfs-kbtz4 1/1 Running 0 52s OK /usr/bin/oc -n storage-project create secret generic heketi-config-secret --from-file=private_key=/dev/null --from-file=./heketi.json --from-file=topology.json=/opt/topology.json secret "heketi-config-secret" created /usr/bin/oc -n storage-project label --overwrite secret heketi-config-secret glusterfs=heketi-config-secret heketi=config-secret secret "heketi-config-secret" labeled /usr/bin/oc -n storage-project process -p HEKETI_EXECUTOR=kubernetes -p HEKETI_FSTAB=/var/lib/heketi/fstab -p HEKETI_ADMIN_KEY= -p HEKETI_USER_KEY= deploy-heketi | /usr/bin/oc -n storage-project create -f - 2>&1 service "deploy-heketi" created route "deploy-heketi" created deploymentconfig "deploy-heketi" created Waiting for deploy-heketi pod to start ... Checking status of pods matching '--selector=deploy-heketi=pod': deploy-heketi-1-hf9rn 1/1 Running 0 2m OK Determining heketi service URL ... OK /usr/bin/oc -n storage-project exec -it deploy-heketi-1-hf9rn -- heketi-cli -s http://localhost:8080 --user admin --secret '' topology load --json=/etc/heketi/topology.json 2>&1 Creating cluster ... ID: 252509038eb8568162ec5920c12bc243 Allowing file volumes on cluster. Allowing block volumes on cluster. Creating node dhcp46-122.lab.eng.blr.redhat.com ... ID: 73ad287ae1ef231f8a0db46422367c9a Adding device /dev/sdd ... OK Adding device /dev/sde ... OK Adding device /dev/sdf ... OK Creating node dhcp46-9.lab.eng.blr.redhat.com ... ID: 0da1b20daaad2d5c57dbfc4f6ab78001 Adding device /dev/sdd ... OK Adding device /dev/sde ... OK Adding device /dev/sdf ... OK Creating node dhcp46-134.lab.eng.blr.redhat.com ... ID: 4b3b62fc0efd298dedbcdacf0b498e65 Adding device /dev/sdd ... OK Adding device /dev/sde ... OK Adding device /dev/sdf ... OK heketi topology loaded. /usr/bin/oc -n storage-project exec -it deploy-heketi-1-hf9rn -- heketi-cli -s http://localhost:8080 --user admin --secret '' setup-openshift-heketi-storage --listfile=/tmp/heketi-storage.json --image rhgs3/rhgs-volmanager-rhel7:3.3.0-17 2>&1 Saving /tmp/heketi-storage.json /usr/bin/oc -n storage-project exec -it deploy-heketi-1-hf9rn -- cat /tmp/heketi-storage.json | /usr/bin/oc -n storage-project create -f - 2>&1 secret "heketi-storage-secret" created endpoints "heketi-storage-endpoints" created service "heketi-storage-endpoints" created job "heketi-storage-copy-job" created Checking status of pods matching '--selector=job-name=heketi-storage-copy-job': heketi-storage-copy-job-87v6n 0/1 Completed 0 7s /usr/bin/oc -n storage-project label --overwrite svc heketi-storage-endpoints glusterfs=heketi-storage-endpoints heketi=storage-endpoints service "heketi-storage-endpoints" labeled /usr/bin/oc -n storage-project delete all,service,jobs,deployment,secret --selector="deploy-heketi" 2>&1 deploymentconfig "deploy-heketi" deleted route "deploy-heketi" deleted service "deploy-heketi" deleted job "heketi-storage-copy-job" deleted pod "deploy-heketi-1-hf9rn" deleted secret "heketi-storage-secret" deleted /usr/bin/oc -n storage-project delete dc,route,template --selector="deploy-heketi" 2>&1 template "deploy-heketi" deleted /usr/bin/oc -n storage-project process -p HEKETI_EXECUTOR=kubernetes -p HEKETI_FSTAB=/var/lib/heketi/fstab -p HEKETI_ADMIN_KEY= -p HEKETI_USER_KEY= heketi | /usr/bin/oc -n storage-project create -f - 2>&1 service "heketi" created route "heketi" created deploymentconfig "heketi" created Waiting for heketi pod to start ... Checking status of pods matching '--selector=heketi=pod': heketi-1-zzblp 1/1 Running 0 31s OK Determining heketi service URL ... OK heketi is now running and accessible via http://heketi-storage-project.cloudapps.mystorage.com . To run administrative commands you can install 'heketi-cli' and use it as follows: # heketi-cli -s http://heketi-storage-project.cloudapps.mystorage.com --user admin --secret '<ADMIN_KEY>' cluster list You can find it at https://github.com/heketi/heketi/releases . Alternatively, use it from within the heketi pod: # /usr/bin/oc -n storage-project exec -it <HEKETI_POD> -- heketi-cli -s http://localhost:8080 --user admin --secret '<ADMIN_KEY>' cluster list For dynamic provisioning, create a StorageClass similar to this: --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: glusterfs-storage provisioner: kubernetes.io/glusterfs parameters: resturl: "http://heketi-storage-project.cloudapps.mystorage.com" Ready to create and provide GlusterFS volumes. sed -e 's/\${NAMESPACE}/storage-project/' /usr/share/heketi/templates/glusterblock-provisioner.yaml | /usr/bin/oc -n storage-project create -f - 2>&1 clusterrole "glusterblock-provisioner-runner" created serviceaccount "glusterblock-provisioner" created clusterrolebinding "glusterblock-provisioner" created deploymentconfig "glusterblock-provisioner-dc" created Waiting for glusterblock-provisioner pod to start ... Checking status of pods matching '--selector=glusterfs=block-provisioner-pod': glusterblock-provisioner-dc-1-xm6bv 1/1 Running 0 6s OK Ready to create and provide Gluster block volumes. /usr/bin/oc -n storage-project create secret generic heketi-storage-project-admin-secret --from-literal=key= --type=kubernetes.io/glusterfs secret "heketi-storage-project-admin-secret" created /usr/bin/oc -n storage-project label --overwrite secret heketi-storage-project-admin-secret glusterfs=s3-heketi-storage-project-admin-secret gluster-s3=heketi-storage-project-admin-secret secret "heketi-storage-project-admin-secret" labeled sed -e 's/\${STORAGE_CLASS}/glusterfs-for-s3/' -e 's/\${HEKETI_URL}/heketi-storage-project.cloudapps.mystorage.com/' -e 's/\${NAMESPACE}/storage-project/' /usr/share/heketi/templates/gluster-s3-storageclass.yaml | /usr/bin/oc -n storage-project create -f - 2>&1 storageclass "glusterfs-for-s3" created sed -e 's/\${STORAGE_CLASS}/glusterfs-for-s3/' -e 's/\${VOLUME_CAPACITY}/2Gi/' /usr/share/heketi/templates/gluster-s3-pvcs.yaml | /usr/bin/oc -n storage-project create -f - 2>&1 persistentvolumeclaim "gluster-s3-claim" created persistentvolumeclaim "gluster-s3-meta-claim" created Checking status of persistentvolumeclaims matching '--selector=glusterfs in (s3-pvc, s3-meta-pvc)': gluster-s3-claim Bound pvc-35b6c1f0-9c65-11e7-9c8c-005056b3ded1 2Gi RWX glusterfs-for-s3 18s gluster-s3-meta-claim Bound pvc-35b86e7a-9c65-11e7-9c8c-005056b3ded1 1Gi RWX glusterfs-for-s3 18s /usr/bin/oc -n storage-project create -f /usr/share/heketi/templates/gluster-s3-template.yaml 2>&1 template "gluster-s3" created /usr/bin/oc -n storage-project process -p S3_ACCOUNT=testvolume -p S3_USER=adminuser -p S3_PASSWORD=itsmine gluster-s3 | /usr/bin/oc -n storage-project create -f - 2>&1 service "gluster-s3-service" created route "gluster-s3-route" created deploymentconfig "gluster-s3-dc" created Waiting for gluster-s3 pod to start ... Checking status of pods matching '--selector=glusterfs=s3-pod': gluster-s3-dc-1-x3x4q 1/1 Running 0 6s OK Ready to create and provide Gluster object volumes. Deployment complete!
Brick 多路是一个功能，允许在一个进程中添加多个 brick。这可以减少资源消耗，并允许我们运行超过相同内存消耗的 brick 数。在每个集群上的其中一个 Red Hat Gluster Storage 节点上执行以下命令来启用 brick-multiplexing：
执行以下命令以启用 brick 多路功能：
# gluster vol set all cluster.brick-multiplex on例如：
# gluster vol set all cluster.brick-multiplex on Brick-multiplexing is supported only for container workloads (Independent or Converged mode). Also it is advised to make sure that either all volumes are in stopped state or no bricks are running before this option is modified.Do you still want to continue? (y/n) y volume set: success重启 heketidb 卷：
# gluster vol stop heketidbstorage Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y volume stop: heketidbstorage: success# gluster vol start heketidbstorage volume start: heketidbstorage: success
执行以下命令以便客户端与容器通信：
# export HEKETI_CLI_SERVER=http://heketi-<project_name>.<sub_domain_name>例如：
# export HEKETI_CLI_SERVER=http://heketi-storage-project.cloudapps.mystorage.com要验证 Heketi 是否已使用拓扑载入，请执行以下命令：
# heketi-cli topology info
注意
The cns-deploy tool does not support scaling up of the cluster. To manually scale-up the cluster, see link:https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/3.11/html-single/operations_guide/#chap-Documentation-Red_Hat_Gluster_Storage_Container_Native_with_OpenShift_Platform-Managing_Clusters[].
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}