Chapter 3. Manually upgrading using the roxctl CLI

Procedure

1. Determine the roxctl architecture for the target operating system:

   $ arch="$(uname -m | sed "s/x86_64//")"; arch="${arch:+-$arch}"

2. Download the roxctl CLI:

   $ curl -L -f -o roxctl "https://mirror.openshift.com/pub/rhacs/assets/4.10.0/bin/Linux/roxctl${arch}"

3. Make the roxctl binary executable:

   $ chmod +x roxctl

4. Place the roxctl binary in a directory that is on your PATH:

   To check your PATH, execute the following command:

   $ echo $PATH

Procedure

1. Determine the roxctl architecture for the target operating system:

   $ arch="$(uname -m | sed "s/x86_64//")"; arch="${arch:+-$arch}"

2. Download the roxctl CLI:

   $ curl -L -f -o roxctl "https://mirror.openshift.com/pub/rhacs/assets/4.10.0/bin/Darwin/roxctl${arch}"

3. Remove all extended attributes from the binary:

   $ xattr -c roxctl

4. Make the roxctl binary executable:

   $ chmod +x roxctl

5. Place the roxctl binary in a directory that is on your PATH:

   To check your PATH, execute the following command:

   $ echo $PATH

Procedure

1. Use roxctl to generate a new set of resources by entering the following command:

   $ roxctl central generate k8s pvc > bundle.zip

2. Extract the CRD from the archive by entering the following command:

   $ unzip bundle.zip central/00-securitypolicy-crd.yaml

3. Apply the extracted CRD to your cluster by entering the following command:

   $ oc apply -f central/00-securitypolicy-crd.yaml

Procedure

1. To update the Central image, run the following command:

   $ oc -n stackrox set image deploy/central \
   central=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.10.0

2. To update the Central-db image, run the following command:

   $ oc -n stackrox set image deploy/central-db \
   central-db=registry.redhat.io/advanced-cluster-security/rhacs-central-db-rhel8:4.10.0 \
   init-db=registry.redhat.io/advanced-cluster-security/rhacs-central-db-rhel8:4.10.0

3. To update the config controller image, run the following command:

   $ oc -n stackrox set image deploy/config-controller \
   manager=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.10.0

Procedure

1. If you have created custom Scanner configurations, you must apply these changes before updating the Scanner configuration file:

   1. To generate Scanner, run the following command:

      $ roxctl -e "$ROX_CENTRAL_ADDRESS" scanner generate

   2. To apply the TLS secrets YAML file, run the following command:

      $ oc apply -f scanner-bundle/scanner/02-scanner-03-tls-secret.yaml

   3. To apply the Scanner configuration YAML file, run the following command:

      $ oc apply -f scanner-bundle/scanner/02-scanner-04-scanner-config.yaml

2. To update the Scanner image, run the following command:

   $ oc -n stackrox set image deploy/scanner \
   scanner=registry.redhat.io/advanced-cluster-security/rhacs-scanner-rhel8:4.10.0

3. To update the Scanner database image, run the following command:

   $ oc -n stackrox set image deploy/scanner-db \
   db=registry.redhat.io/advanced-cluster-security/rhacs-scanner-db-rhel8:4.10.0 \
   init-db=registry.redhat.io/advanced-cluster-security/rhacs-scanner-db-rhel8:4.10.0

Procedure

1. Update the Sensor image:

   $ oc -n stackrox set image deploy/sensor sensor=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.10.0

2. Update the Compliance image:

   $ oc -n stackrox set image ds/collector compliance=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.10.0

3. Update the Collector image:

   $ oc -n stackrox set image ds/collector collector=registry.redhat.io/advanced-cluster-security/rhacs-collector-rhel8:4.10.0

4. Update the admission control image:

   $ oc -n stackrox set image deploy/admission-control admission-control=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.10.0

   Important

   If you have installed RHACS on Red Hat OpenShift by using the roxctl CLI, you need to migrate the security context constraints (SCCs).

   For more information, see "Migrating SCCs during the manual upgrade" in the "Additional resources" section.

Procedure

1. Patch sensor to ensure POD_NAMESPACE is set by running the following command:

   $ [[ -z "$(oc -n stackrox get deployment sensor -o yaml | grep POD_NAMESPACE)" ]] && oc -n stackrox patch deployment sensor --type=json -p '[{"op":"add","path":"/spec/template/spec/containers/0/env/-","value":{"name":"POD_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}}}]'

2. Patch admission-control to ensure POD_NAMESPACE is set by running the following command:

   $ [[ -z "$(oc -n stackrox get deployment admission-control -o yaml | grep POD_NAMESPACE)" ]] && oc -n stackrox patch deployment admission-control --type=json -p '[{"op":"add","path":"/spec/template/spec/containers/0/env/-","value":{"name":"POD_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}}}]'

Procedure

1. List all of the RHACS services that are deployed on Central and all secured clusters:

   $ oc -n stackrox describe pods | grep 'openshift.io/scc\|^Name:'

   Example output

   Name:      admission-control-6f4dcc6b4c-2phwd
              openshift.io/scc: stackrox-admission-control
   #...
   Name:      central-575487bfcb-sjdx8
              openshift.io/scc: stackrox-central
   Name:      central-db-7c7885bb-6bgbd
              openshift.io/scc: stackrox-central-db
   Name:      collector-56nkr
              openshift.io/scc: stackrox-collector
   #...
   Name:      scanner-68fc55b599-f2wm6
              openshift.io/scc: stackrox-scanner
   Name:      scanner-68fc55b599-fztlh
   #...
   Name:      sensor-84545f86b7-xgdwf
              openshift.io/scc: stackrox-sensor
   #...

   In this example, you can see that each pod has its own custom SCC, which is specified through the openshift.io/scc field.

2. Add the required roles and role bindings to use the Red Hat OpenShift SCCs instead of the RHACS custom SCCs.

   To add the required roles and role bindings to use the Red Hat OpenShift SCCs for the Central cluster, complete the following steps:

   1. Create a file named update-central.yaml that defines the role and role binding resources by using the following content:

      Example 3.1. Example YAML file

      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role #
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: central
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: use-central-db-scc #
        namespace: stackrox #
      Rules: #
      - apiGroups:
        - security.openshift.io
        resourceNames:
        - nonroot-v2
        resources:
        - securitycontextconstraints
        verbs:
        - use
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: central
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/managed-by: Helm
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: use-central-scc
        namespace: stackrox
      rules:
      - apiGroups:
        - security.openshift.io
        resourceNames:
        - nonroot-v2
        resources:
        - securitycontextconstraints
        verbs:
        - use
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: scanner
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: use-scanner-scc
        namespace: stackrox
      rules:
      - apiGroups:
        - security.openshift.io
        resourceNames:
        - nonroot-v2
        resources:
        - securitycontextconstraints
        verbs:
        - use
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding #
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: central
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.k ubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: central-db-use-scc #
        namespace: stackrox
      roleRef: #
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: use-central-db-scc
      subjects: #
      - kind: ServiceAccount
        name: central-db
        namespace: stackrox
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: central
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: central-use-scc
        namespace: stackrox
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: use-central-scc
      subjects:
      - kind: ServiceAccount
        name: central
        namespace: stackrox
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: scanner
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: scanner-use-scc
        namespace: stackrox
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: use-scanner-scc
      subjects:
      - kind: ServiceAccount
        name: scanner
        namespace: stackrox
      - - -

      where:

      kind: Role

      Specifies the type of Kubernetes resource, in this example, Role.

      metadata.name: <rolename>

      Specifies the name of the role resource.

      metadata.namespace

      Specifies the namespace in which the role is created.

      Rules

      Specifies the permissions granted by the role resource.

      kind: RoleBinding

      Specifies the type of Kubernetes resource, in this example, RoleBinding.

      metadata.name: <rolebindingname>

      Specifies the name of the role binding resource.

      metadata.roleRef

      Specifies the role to bind in the same namespace.

      metadata.subjects

      Specifies the subjects that are bound to the role.

   2. Create the role and role binding resources specified in the update-central.yaml file by running the following command:

      $ oc -n stackrox create -f ./update-central.yaml

3. To add the required roles and role bindings to use the Red Hat OpenShift SCCs for all secured clusters, complete the following steps:

   1. Create a file named upgrade-scs.yaml that defines the role and role binding resources by using the following content:

      Example 3.2. Example YAML file

      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role  #
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: collector
           app.kubernetes.io/instance: stackrox-secured-cluster-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-secured-cluster-services
           app.kubernetes.io/version: 4.4.0
           auto-upgrade.stackrox.io/component: sensor
        name: use-privileged-scc  #
        namespace: stackrox #
      rules:  #
      - apiGroups:
        - security.openshift.io
        resourceNames:
        - privileged
        resources:
        - securitycontextconstraints
        verbs:
        - use
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding #
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: collector
           app.kubernetes.io/instance: stackrox-secured-cluster-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-secured-cluster-services
           app.kubernetes.io/version: 4.4.0
           auto-upgrade.stackrox.io/component: sensor
        name: collector-use-scc #
        namespace: stackrox
      roleRef: #
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: use-privileged-scc
      subjects: #
      - kind: ServiceAccount
        name: collector
        namespace: stackrox
      - - -

      where:

      kind: Role

      Specifies the type of Kubernetes resource, in this example, Role.

      metadata.name: <rolename>

      Specifies the name of the role resource.

      metadata.namespace

      Specifies the namespace in which the role is created.

      Rules

      Specifies the permissions granted by the role resource.

      kind: RoleBinding

      Specifies the type of Kubernetes resource, in this example, RoleBinding.

      metadata.name: <rolebindingname>

      Specifies the name of the role binding resource.

      metadata.roleRef

      Specifies the role to bind in the same namespace.

      metadata.subjects

      Specifies the subjects that are bound to the role.

   2. Create the role and role binding resources specified in the upgrade-scs.yaml file by running the following command:

      $ oc -n stackrox create -f ./update-scs.yaml

      Important

      You must run this command on each secured cluster to create the role and role bindings specified in the upgrade-scs.yaml file.

4. Delete the SCCs that are specific to RHACS:

   1. To delete the SCCs that are specific to the Central cluster, run the following command:

      $ oc delete scc/stackrox-central scc/stackrox-central-db scc/stackrox-scanner

   2. To delete the SCCs that are specific to all secured clusters, run the following command:

      $ oc delete scc/stackrox-admission-control scc/stackrox-collector scc/stackrox-sensor

      Important

      You must run this command on each secured cluster to delete the SCCs that are specific to each secured cluster.

Procedure

1. Ensure that the bundle for the secured cluster was generated with future upgrades enabled before clicking Download YAML file and keys.
2. If possible, remove that secured cluster and generate a new bundle making sure that future upgrades are enabled.

3. If you cannot re-create the cluster, you can take these actions:

   1. Ensure that the service account sensor-upgrader exists in the same namespace as Sensor.
   2. Ensure that a ClusterRoleBinding exists (default name: <namespace>:upgrade-sensors) that grants the cluster-admin ClusterRole to the sensor-upgrader service account.

Procedure

1. Ensure that the Secured Cluster can access the registry and pull the image <image_reference:tag>.
2. Ensure that the image pull secrets are configured correctly in the secured cluster.

Procedure

1. Ensure that the upgrader has enough permissions for accessing the cluster objects. For more information, see "Upgrader is missing permissions".
2. Check the upgrader logs for more insights.