Chapter 2. SiteConfig

The SiteConfig operator dynamically generates installation manifests based on user-defined templates that are instantiated from the data in the ClusterInstance custom resource.

You can source the ClusterInstance custom resource from your Git repository through ArgoCD, or you can create it directly on the hub cluster manually or through external tools and workflows.

The following is a high-level overview of the process:

You can customize the templated fields. The SiteConfig operator supports all sprig library functions.

Additionally, the ClusterInstance API provides the following function that you can use while creating your custom manifests:

See the following example of the .toYaml specification in the ClusterInstance.Spec.Proxy field:

{{ if .Spec.Proxy }}
  proxy:
{{ .Spec.Proxy | toYaml | indent 4 }}
{{ end }}

The SiteConfig operator provides the following default, validated, and immutable set of templates in the same namespace in which the operator is installed:

For more information about the ClusterInstance API, see ClusterInstance API.

The SiteConfig operator provides a set of special template variables that you can use in your templates. See the following list:

Capitalize the field name in the text template to create a custom templated field.

For example, the ClusterInstance spec field is referenced with the .Spec prefix. However, you must reference special variable fields with the .SpecialVars prefix.

Important: Instead of using the .Spec.Nodes prefix for the spec.nodes field, you must reference it with the .SpecialVars.CurrentNode special template variable.

For example, if you want to specify the name and namespace for your current node by using the CurrentNode special template variable, use the field names in the following form:

name: "{{ .SpecialVars.CurrentNode.HostName }}"
namespace: "{{ .Spec.ClusterName }}"

You can control the order in which manifests are created, updated, and deleted by using the siteconfig.open-cluster-management.io/sync-wave annotation. The annotation takes an integer as a value, and that integer constitutes as a wave.

You can add one or several manifests to a single wave. If you do not specify a value, the annotation takes the default value of 0.

The SiteConfig operator reconciles the manifests in ascending order when creating or updating resources and it deletes resources in descending order.

In the following example, if the SiteConfig operator creates or updates the manifests, the AgentClusterInstall and ClusterDeployment custom resources are reconciled in the first wave, while KlusterletAddonConfig and ManagedCluster custom resources are reconciled in the third wave.

apiVersion: v1
data:
  AgentClusterInstall: |-
    ...
        siteconfig.open-cluster-management.io/sync-wave: "1"
    ...
  ClusterDeployment: |-
    ...
        siteconfig.open-cluster-management.io/sync-wave: "1"
    ...
  InfraEnv: |-
    ...
        siteconfig.open-cluster-management.io/sync-wave: "2"
    ...
  KlusterletAddonConfig: |-
    ...
        siteconfig.open-cluster-management.io/sync-wave: "3"
    ...
  ManagedCluster: |-
    ...
        siteconfig.open-cluster-management.io/sync-wave: "3"
    ...
kind: ConfigMap
metadata:
  name: assisted-installer-templates
  namespace: example-namespace

If the SiteConfig operator deletes the resources, KlusterletAddonConfig and ManagedCluster custom resources are the first to be deleted, while the AgentClusterInstall and ClusterDeployment custom resources are the last.

You can configure additional annotations and labels to both cluster-level and node-level installation manifests by using the extraAnnotations and extraLabels fields in the ClusterInstance API. The SiteConfig operator applies your additional annotations and labels to the manifests that you specify in the ClusterInstance resource.

When creating your additional annotations and labels, you must specify a manifest type to allow the SiteConfig operator to apply them to all the matching manifests. However, the annotations and labels are arbitrary and you can set any key and value pairs that are meaningful to your applications.

Note: The additional annotations and labels are only applied to the resources that were rendered through the referenced templates.

View the following example application of extraAnnotations and extraLabels:

apiVersion: siteconfig.open-cluster-management.io/v1alpha1
kind: ClusterInstance
metadata:
  name: "example-sno"
  namespace: "example-sno"
spec:
  [...]
  clusterName: "example-sno"
  extraAnnotations: 

    ClusterDeployment:
      myClusterAnnotation: success
  extraLabels: 

    ManagedCluster:
      common: "true"
      group-du: ""
  nodes:
    - hostName: "example-sno.example.redhat.com"
      role: "master"
      extraAnnotations: 

        BareMetalHost:
          myNodeAnnotation: success
      extraLabels: 

        BareMetalHost:
          "testExtraLabel": "success"

You might want to change your cluster configuration but making changes to your cluster is not allowed during provisioning. However, after a cluster is provisioned, you can modify the following fields:

Note: <node-id> represents the updated NodeSpec object.

Patch the MultiClusterHub resource, then verify that SiteConfig operator is enabled. Complete the following procedure:

Complete the following steps to install a cluster with the SiteConfig operator:

You need a target namespace when you create the pull secret, the BMC secret, extra manifest ConfigMap objects, and the ClusterInstance custom resource.

Complete the following steps to create the target namespace:

You need a pull secret to enable your clusters to pull images from container registries. Complete the following steps to create a pull secret:

You need a secret to connect to your baseboard management controller (BMC). Complete the following steps to create a secret:

You can create extra manifests that you need to reference in the ClusterInstance custom resource. Complete the following steps to create an extra manifest:

Reference the templates and supporting manifests in the ClusterInstance custom resource. Complete the following steps to render the installation manifests by using the default cluster and node templates:

For more information about status conditions, see ClusterInstance API.

Complete the following steps to delete your clusters:

See the following example output where the (NotFound) error indicates that your cluster is deprovisioned.

Error from server (NotFound): clusterinstances.siteconfig.open-cluster-management.io "<cluster_name>" not found

Create user-defined templates that are not provided in the default set of templates.

Required access: Cluster administrator

Complete the following steps to a create custom template:

Scale in your managed cluster that was installed by the SiteConfig operator. You can scale in your cluster by removing a worker node.

Required access: Cluster administrator

{
  "baremetalhost.metal3.io/detached": "assisted-service-controller",
  "bmac.agent-install.openshift.io/hostname": "worker-node2.example.com",
  "bmac.agent-install.openshift.io/remove-agent-and-node-on-delete": "true"
  "bmac.agent-install.openshift.io/role": "master",
  "inspect.metal3.io": "disabled",
  "siteconfig.open-cluster-management.io/sync-wave": "1",
}

Scale out your managed cluster that was installed by the SiteConfig operator. You can scale out your cluster by adding a worker node.

Required access: Cluster administrator

You can deploy a cluster with the SiteConfig operator by using the Image Based Install Operator as your underlying operator. If you deploy your clusters with the Image Based Install Operator in a disconnected environment, you must supply your mirror images as extra manifests in the ClusterInstance custom resource.

Required access: Cluster administrator

Complete the following steps to mirror images for disconnected environments:

Important: Define the ConfigMap resource that contains the extra manifest in the same namespace as the ClusterInstance resource.