Red Hat Summit 红帽全球峰会支持控制台（Console）开发人员开始试用联系人

1.3. Moving resources to infrastructure machine sets

Some of the infrastructure resources are deployed in your cluster by default. You can move them to the infrastructure machine sets that you created.

1.3.1. Moving the router
复制链接

You can deploy the router pod to a different machine set. By default, the pod is deployed to a worker node.

Prerequisites

  • Configure additional machine sets in your OpenShift Container Platform cluster.

Procedure

  1. View the IngressController custom resource for the router Operator: 

    $ oc get ingresscontroller default -n openshift-ingress-operator -o yaml
    Copy to Clipboard Toggle word wrap

    The command output resembles the following text: 

    apiVersion: operator.openshift.io/v1
kind: IngressController
metadata:
  creationTimestamp: 2019-04-18T12:35:39Z
  finalizers:
  - ingresscontroller.operator.openshift.io/finalizer-ingresscontroller
  generation: 1
  name: default
  namespace: openshift-ingress-operator
  resourceVersion: "11341"
  selfLink: /apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default
  uid: 79509e05-61d6-11e9-bc55-02ce4781844a
spec: {}
status:
  availableReplicas: 2
  conditions:
  - lastTransitionTime: 2019-04-18T12:36:15Z
    status: "True"
    type: Available
  domain: apps.<cluster>.example.com
  endpointPublishingStrategy:
    type: LoadBalancerService
  selector: ingresscontroller.operator.openshift.io/deployment-ingresscontroller=default
    Copy to Clipboard Toggle word wrap

  2. Edit the ingresscontroller resource and change the nodeSelector to use the infra label: 

    $ oc edit ingresscontroller default -n openshift-ingress-operator
    Copy to Clipboard Toggle word wrap

    Add the nodeSelector stanza that references the infra label to the spec section, as shown: 

      spec:
    nodePlacement:
      nodeSelector:
        matchLabels:
          node-role.kubernetes.io/infra: ""
    Copy to Clipboard Toggle word wrap

  3. Confirm that the router pod is running on the infra node.

    1. View the list of router pods and note the node name of the running pod: 

      $ oc get pod -n openshift-ingress -o wide
      Copy to Clipboard Toggle word wrap

      Example output

      NAME                              READY     STATUS        RESTARTS   AGE       IP           NODE                           NOMINATED NODE   READINESS GATES
router-default-86798b4b5d-bdlvd   1/1      Running       0          28s       10.130.2.4   ip-10-0-217-226.ec2.internal   <none>           <none>
router-default-955d875f4-255g8    0/1      Terminating   0          19h       10.129.2.4   ip-10-0-148-172.ec2.internal   <none>           <none>
      Copy to Clipboard Toggle word wrap

      In this example, the running pod is on the ip-10-0-217-226.ec2.internal node.

    2. View the node status of the running pod: 

      $ oc get node <node_name>
      1
      Copy to Clipboard Toggle word wrap
      1
      Specify the <node_name> that you obtained from the pod list.

      Example output

      NAME                          STATUS  ROLES         AGE   VERSION
ip-10-0-217-226.ec2.internal  Ready   infra,worker  17h   v1.18.3
      Copy to Clipboard Toggle word wrap

      Because the role list includes infra, the pod is running on the correct node.

1.3.2. Moving the default registry
复制链接

You configure the registry Operator to deploy its pods to different nodes.

Prerequisites

  • Configure additional machine sets in your OpenShift Container Platform cluster.

Procedure

  1. View the config/instance object: 

    $ oc get configs.imageregistry.operator.openshift.io/cluster -o yaml
    Copy to Clipboard Toggle word wrap

    Example output

    apiVersion: imageregistry.operator.openshift.io/v1
kind: Config
metadata:
  creationTimestamp: 2019-02-05T13:52:05Z
  finalizers:
  - imageregistry.operator.openshift.io/finalizer
  generation: 1
  name: cluster
  resourceVersion: "56174"
  selfLink: /apis/imageregistry.operator.openshift.io/v1/configs/cluster
  uid: 36fd3724-294d-11e9-a524-12ffeee2931b
spec:
  httpSecret: d9a012ccd117b1e6616ceccb2c3bb66a5fed1b5e481623
  logging: 2
  managementState: Managed
  proxy: {}
  replicas: 1
  requests:
    read: {}
    write: {}
  storage:
    s3:
      bucket: image-registry-us-east-1-c92e88cad85b48ec8b312344dff03c82-392c
      region: us-east-1
status:
...
    Copy to Clipboard Toggle word wrap

  2. Edit the config/instance object: 

    $ oc edit configs.imageregistry.operator.openshift.io/cluster
    Copy to Clipboard Toggle word wrap

  3. Add the following lines of text the spec section of the object: 

      nodeSelector:
    node-role.kubernetes.io/infra: ""
    Copy to Clipboard Toggle word wrap

  4. Verify the registry pod has been moved to the infrastructure node.

    1. Run the following command to identify the node where the registry pod is located: 

      $ oc get pods -o wide -n openshift-image-registry
      Copy to Clipboard Toggle word wrap

    2. Confirm the node has the label you specified: 

      $ oc describe node <node_name>
      Copy to Clipboard Toggle word wrap

      Review the command output and confirm that node-role.kubernetes.io/infra is in the LABELS list.

In a production deployment, deploy at least three machine sets to hold infrastructure components. Both the logging aggregation solution and the service mesh deploy Elasticsearch, and Elasticsearch requires three instances that are installed on different nodes. For high availability, install deploy these nodes to different availability zones. Since you need different machine sets for each availability zone, create at least three machine sets.

1.3.3.1. Creating a machine set
复制链接

In addition to the ones created by the installation program, you can create your own machine sets to dynamically manage the machine compute resources for specific workloads of your choice.

Prerequisites

  • Deploy an OpenShift Container Platform cluster.
  • Install the OpenShift CLI (oc).
  • Log in to oc as a user with cluster-admin permission.

Procedure

  1. Create a new YAML file that contains the machine set custom resource (CR) sample, as shown, and is named <file_name>.yaml.

    Ensure that you set the <clusterID> and <role> parameter values.

    1. If you are not sure about which value to set for a specific field, you can check an existing machine set from your cluster. 

      $ oc get machinesets -n openshift-machine-api
      Copy to Clipboard Toggle word wrap

      Example output

      NAME                                DESIRED   CURRENT   READY   AVAILABLE   AGE
agl030519-vplxk-worker-us-east-1a   1         1         1       1           55m
agl030519-vplxk-worker-us-east-1b   1         1         1       1           55m
agl030519-vplxk-worker-us-east-1c   1         1         1       1           55m
agl030519-vplxk-worker-us-east-1d   0         0                             55m
agl030519-vplxk-worker-us-east-1e   0         0                             55m
agl030519-vplxk-worker-us-east-1f   0         0                             55m
      Copy to Clipboard Toggle word wrap

    2. Check values of a specific machine set: 

      $ oc get machineset <machineset_name> -n \
     openshift-machine-api -o yaml
      Copy to Clipboard Toggle word wrap

      Example output

      ...
template:
    metadata:
      labels:
        machine.openshift.io/cluster-api-cluster: agl030519-vplxk
      1 
      
        machine.openshift.io/cluster-api-machine-role: worker
      2 
      
        machine.openshift.io/cluster-api-machine-type: worker
        machine.openshift.io/cluster-api-machineset: agl030519-vplxk-worker-us-east-1a
      Copy to Clipboard Toggle word wrap

      1
      The cluster ID.
      2
      A default node label.

  2. Create the new MachineSet CR: 

    $ oc create -f <file_name>.yaml
    Copy to Clipboard Toggle word wrap

  3. View the list of machine sets: 

    $ oc get machineset -n openshift-machine-api
    Copy to Clipboard Toggle word wrap

    Example output

    NAME                                DESIRED   CURRENT   READY   AVAILABLE   AGE
agl030519-vplxk-infra-us-east-1a    1         1         1       1           11m
agl030519-vplxk-worker-us-east-1a   1         1         1       1           55m
agl030519-vplxk-worker-us-east-1b   1         1         1       1           55m
agl030519-vplxk-worker-us-east-1c   1         1         1       1           55m
agl030519-vplxk-worker-us-east-1d   0         0                             55m
agl030519-vplxk-worker-us-east-1e   0         0                             55m
agl030519-vplxk-worker-us-east-1f   0         0                             55m
    Copy to Clipboard Toggle word wrap

    When the new machine set is available, the DESIRED and CURRENT values match. If the machine set is not available, wait a few minutes and run the command again.

  4. After the new machine set is available, check status of the machine and the node that it references: 

    $ oc describe machine <name> -n openshift-machine-api
    Copy to Clipboard Toggle word wrap

    For example: 

    $ oc describe machine agl030519-vplxk-infra-us-east-1a -n openshift-machine-api
    Copy to Clipboard Toggle word wrap

    Example output

    status:
  addresses:
  - address: 10.0.133.18
    type: InternalIP
  - address: ""
    type: ExternalDNS
  - address: ip-10-0-133-18.ec2.internal
    type: InternalDNS
  lastUpdated: "2019-05-03T10:38:17Z"
  nodeRef:
    kind: Node
    name: ip-10-0-133-18.ec2.internal
    uid: 71fb8d75-6d8f-11e9-9ff3-0e3f103c7cd8
  providerStatus:
    apiVersion: awsproviderconfig.openshift.io/v1beta1
    conditions:
    - lastProbeTime: "2019-05-03T10:34:31Z"
      lastTransitionTime: "2019-05-03T10:34:31Z"
      message: machine successfully created
      reason: MachineCreationSucceeded
      status: "True"
      type: MachineCreation
    instanceId: i-09ca0701454124294
    instanceState: running
    kind: AWSMachineProviderStatus
    Copy to Clipboard Toggle word wrap

  5. View the new node and confirm that the new node has the label that you specified: 

    $ oc get node <node_name> --show-labels
    Copy to Clipboard Toggle word wrap

    Review the command output and confirm that node-role.kubernetes.io/<your_label> is in the LABELS list.

注意

Any change to a machine set is not applied to existing machines owned by the machine set. For example, labels edited or added to an existing machine set are not propagated to existing machines and nodes associated with the machine set.

1.3.4. Creating machine sets for different clouds
复制链接

Use the sample machine set for your cloud.

This sample YAML defines a machine set that runs in the us-east-1a Amazon Web Services (AWS) zone and creates nodes that are labeled with node-role.kubernetes.io/<role>: ""

In this sample, <infrastructureID> is the infrastructure ID label that is based on the cluster ID that you set when you provisioned the cluster, and <role> is the node label to add. 

apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
  labels:
    machine.openshift.io/cluster-api-cluster: <infrastructureID>
1 

  name: <infrastructureID>-<role>-<zone>
2 

  namespace: openshift-machine-api
spec:
  replicas: 1
  selector:
    matchLabels:
      machine.openshift.io/cluster-api-cluster: <infrastructureID>
3 

      machine.openshift.io/cluster-api-machineset: <infrastructureID>-<role>-<zone>
4 

  template:
    metadata:
      labels:
        machine.openshift.io/cluster-api-cluster: <infrastructureID>
5 

        machine.openshift.io/cluster-api-machine-role: <role>
6 

        machine.openshift.io/cluster-api-machine-type: <role>
7 

        machine.openshift.io/cluster-api-machineset: <infrastructureID>-<role>-<zone>
8 

    spec:
      metadata:
        labels:
          node-role.kubernetes.io/<role>: ""
9 

      providerSpec:
        value:
          ami:
            id: ami-046fe691f52a953f9
10 

          apiVersion: awsproviderconfig.openshift.io/v1beta1
          blockDevices:
            - ebs:
                iops: 0
                volumeSize: 120
                volumeType: gp2
          credentialsSecret:
            name: aws-cloud-credentials
          deviceIndex: 0
          iamInstanceProfile:
            id: <infrastructureID>-worker-profile
11 

          instanceType: m4.large
          kind: AWSMachineProviderConfig
          placement:
            availabilityZone: us-east-1a
            region: us-east-1
          securityGroups:
            - filters:
                - name: tag:Name
                  values:
                    - <infrastructureID>-worker-sg
12 

          subnet:
            filters:
              - name: tag:Name
                values:
                  - <infrastructureID>-private-us-east-1a
13 

          tags:
            - name: kubernetes.io/cluster/<infrastructureID>
14 

              value: owned
          userDataSecret:
            name: worker-user-data
Copy to Clipboard Toggle word wrap
1 3 5 11 12 13 14
Specify the infrastructure ID that is based on the cluster ID that you set when you provisioned the cluster. If you have the OpenShift CLI installed, you can obtain the infrastructure ID by running the following command: 
$ oc get -o jsonpath='{.status.infrastructureName}{"\n"}' infrastructure cluster
Copy to Clipboard Toggle word wrap
2 4 8
Specify the infrastructure ID, node label, and zone.
6 7 9
Specify the node label to add.
10
Specify a valid Red Hat Enterprise Linux CoreOS (RHCOS) AMI for your AWS zone for your OpenShift Container Platform nodes.

Machine sets running on AWS support non-guaranteed Spot Instances. You can save on costs by using Spot Instances at a lower price compared to On-Demand Instances on AWS. Configure Spot Instances by adding spotMarketOptions to the machine set YAML file.

This sample YAML defines a machine set that runs in the 1 Microsoft Azure zone in the centralus region and creates nodes that are labeled with node-role.kubernetes.io/<role>: ""

In this sample, <infrastructureID> is the infrastructure ID label that is based on the cluster ID that you set when you provisioned the cluster, and <role> is the node label to add. 

apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
  labels:
    machine.openshift.io/cluster-api-cluster: <infrastructureID>
1 

    machine.openshift.io/cluster-api-machine-role: <role>
2 

    machine.openshift.io/cluster-api-machine-type: <role>
3 

  name: <infrastructureID>-<role>-<region>
4 

  namespace: openshift-machine-api
spec:
  replicas: 1
  selector:
    matchLabels:
      machine.openshift.io/cluster-api-cluster: <infrastructureID>
5 

      machine.openshift.io/cluster-api-machineset: <infrastructureID>-<role>-<region>
6 

  template:
    metadata:
      creationTimestamp: null
      labels:
        machine.openshift.io/cluster-api-cluster: <infrastructureID>
7 

        machine.openshift.io/cluster-api-machine-role: <role>
8 

        machine.openshift.io/cluster-api-machine-type: <role>
9 

        machine.openshift.io/cluster-api-machineset: <infrastructureID>-<role>-<region>
10 

    spec:
      metadata:
        creationTimestamp: null
        labels:
          node-role.kubernetes.io/<role>: ""
11 

      providerSpec:
        value:
          apiVersion: azureproviderconfig.openshift.io/v1beta1
          credentialsSecret:
            name: azure-cloud-credentials
            namespace: openshift-machine-api
          image:
            offer: ""
            publisher: ""
            resourceID: /resourceGroups/<infrastructureID>-rg/providers/Microsoft.Compute/images/<infrastructureID>
            sku: ""
            version: ""
          internalLoadBalancer: ""
          kind: AzureMachineProviderSpec
          location: centralus
          managedIdentity: <infrastructureID>-identity
12 

          metadata:
            creationTimestamp: null
          natRule: null
          networkResourceGroup: ""
          osDisk:
            diskSizeGB: 128
            managedDisk:
              storageAccountType: Premium_LRS
            osType: Linux
          publicIP: false
          publicLoadBalancer: ""
          resourceGroup: <infrastructureID>-rg
13 

          sshPrivateKey: ""
          sshPublicKey: ""
          subnet: <infrastructureID>-<role>-subnet
14
15 

          userDataSecret:
            name: <role>-user-data
16 

          vmSize: Standard_D2s_v3
          vnet: <infrastructureID>-vnet
17 

          zone: "1"
18
Copy to Clipboard Toggle word wrap
1 5 7 12 13 14 17
Specify the infrastructure ID that is based on the cluster ID that you set when you provisioned the cluster. If you have the OpenShift CLI installed, you can obtain the infrastructure ID by running the following command: 
$ oc get -o jsonpath='{.status.infrastructureName}{"\n"}' infrastructure cluster
Copy to Clipboard Toggle word wrap
2 3 8 9 11 15 16
Specify the node label to add.
4 6 10
Specify the infrastructure ID, node label, and region.
18
Specify the zone within your region to place Machines on. Be sure that your region supports the zone that you specify.

This sample YAML defines a machine set that runs in Google Cloud Platform (GCP) and creates nodes that are labeled with node-role.kubernetes.io/<role>: ""

In this sample, <infrastructureID> is the infrastructure ID label that is based on the cluster ID that you set when you provisioned the cluster, and <role> is the node label to add. 

apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
  labels:
    machine.openshift.io/cluster-api-cluster: <infrastructureID>
1 

  name: <infrastructureID>-w-a
2 

  namespace: openshift-machine-api
spec:
  replicas: 1
  selector:
    matchLabels:
      machine.openshift.io/cluster-api-cluster: <infrastructureID>
3 

      machine.openshift.io/cluster-api-machineset: <infrastructureID>-w-a
4 

  template:
    metadata:
      creationTimestamp: null
      labels:
        machine.openshift.io/cluster-api-cluster: <infrastructureID>
5 

        machine.openshift.io/cluster-api-machine-role: <role>
6 

        machine.openshift.io/cluster-api-machine-type: <role>
7 

        machine.openshift.io/cluster-api-machineset: <infrastructureID>-w-a
8 

    spec:
      metadata:
        labels:
          node-role.kubernetes.io/<role>: ""
9 

      providerSpec:
        value:
          apiVersion: gcpprovider.openshift.io/v1beta1
          canIPForward: false
          credentialsSecret:
            name: gcp-cloud-credentials
          deletionProtection: false
          disks:
          - autoDelete: true
            boot: true
            image: <path_to_image>
10 

            labels: null
            sizeGb: 128
            type: pd-ssd
          kind: GCPMachineProviderSpec
          machineType: n1-standard-4
          metadata:
            creationTimestamp: null
          networkInterfaces:
          - network: <infrastructureID>-network
11 

            subnetwork: <infrastructureID>-<role>-subnet
12 

          projectID: <project_name>
13 

          region: us-central1
          serviceAccounts:
          - email: <infrastructureID>-w@<project_name>.iam.gserviceaccount.com
14
15 

            scopes:
            - https://www.googleapis.com/auth/cloud-platform
          tags:
          - <infrastructureID>-<role>
16 

          userDataSecret:
            name: worker-user-data
          zone: us-central1-a
Copy to Clipboard Toggle word wrap
1 2 3 4 5 8 11 14
Specify the infrastructure ID that is based on the cluster ID that you set when you provisioned the cluster. If you have the OpenShift CLI installed, you can obtain the infrastructure ID by running the following command: 
$ oc get -o jsonpath='{.status.infrastructureName}{"\n"}' infrastructure cluster
Copy to Clipboard Toggle word wrap
12 16
Specify the infrastructure ID and node label.
6 7 9
Specify the node label to add.
13 15
Specify the name of the GCP project that you use for your cluster.
10
Specify the path to the image that is used in current machine sets. If you have the OpenShift CLI installed, you can obtain the path to the image by running the following command: 
$ oc -n openshift-machine-api \
    -o jsonpath='{.spec.template.spec.providerSpec.value.disks[0].image}{"\n"}' \
    get machineset/<infrastructureID>-worker-a
Copy to Clipboard Toggle word wrap

This sample YAML defines a machine set that runs on VMware vSphere and creates nodes that are labeled with node-role.kubernetes.io/<role>: "".

In this sample, <infrastructure_id> is the infrastructure ID label that is based on the cluster ID that you set when you provisioned the cluster, and <role> is the node label to add. 

apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
  creationTimestamp: null
  labels:
    machine.openshift.io/cluster-api-cluster: <infrastructure_id>
1 

  name: <infrastructure_id>-<role>
2 

  namespace: openshift-machine-api
spec:
  replicas: 1
  selector:
    matchLabels:
      machine.openshift.io/cluster-api-cluster: <infrastructure_id>
3 

      machine.openshift.io/cluster-api-machineset: <infrastructure_id>-<role>
4 

  template:
    metadata:
      creationTimestamp: null
      labels:
        machine.openshift.io/cluster-api-cluster: <infrastructure_id>
5 

        machine.openshift.io/cluster-api-machine-role: <role>
6 

        machine.openshift.io/cluster-api-machine-type: <role>
7 

        machine.openshift.io/cluster-api-machineset: <infrastructure_id>-<role>
8 

    spec:
      metadata:
        creationTimestamp: null
        labels:
          node-role.kubernetes.io/<role>: ""
9 

      providerSpec:
        value:
          apiVersion: vsphereprovider.openshift.io/v1beta1
          credentialsSecret:
            name: vsphere-cloud-credentials
          diskGiB: 120
          kind: VSphereMachineProviderSpec
          memoryMiB: 8192
          metadata:
            creationTimestamp: null
          network:
            devices:
            - networkName: "<vm_network_name>"
10 

          numCPUs: 4
          numCoresPerSocket: 1
          snapshot: ""
          template: <vm_template_name>
11 

          userDataSecret:
            name: worker-user-data
          workspace:
            datacenter: <vcenter_datacenter_name>
12 

            datastore: <vcenter_datastore_name>
13 

            folder: <vcenter_vm_folder_path>
14 

            resourcepool: <vsphere_resource_pool>
15 

            server: <vcenter_server_ip>
16
Copy to Clipboard Toggle word wrap
1 3 5
Specify the infrastructure ID that is based on the cluster ID that you set when you provisioned the cluster. If you have the OpenShift CLI (oc) installed, you can obtain the infrastructure ID by running the following command: 
$ oc get -o jsonpath='{.status.infrastructureName}{"\n"}' infrastructure cluster
Copy to Clipboard Toggle word wrap
2 4 8
Specify the infrastructure ID and node label.
6 7 9
Specify the node label to add.
10
Specify the vSphere VM network to deploy the machine set to.
11
Specify the vSphere VM clone of the template to use, such as user-5ddjd-rhcos.
重要

Do not specify the original VM template. The VM template must remain off and must be cloned for new RHCOS machines. Starting the VM template configures the VM template as a VM on the platform, which prevents it from being used as a template that machine sets can apply configurations to.

12
Specify the vCenter Datacenter to deploy the machine set on.
13
Specify the vCenter Datastore to deploy the machine set on.
14
Specify the path to the vSphere VM folder in vCenter, such as /dc1/vm/user-inst-5ddjd.
15
Specify the vSphere resource pool for your VMs.
16
Specify the vCenter server IP or fully qualified domain name.
返回顶部
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务，以及可以信赖的内容，帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情，请参阅红帽博客.

關於紅帽

我们提供强化的解决方案，使企业能够更轻松地跨平台和环境（从核心数据中心到网络边缘）工作。

Theme

© 2025 Red Hat