Chapter 1. Governance

You can find the description of a field on your managed cluster by running the oc explain --api-version=policy.open-cluster-management.io/v1 ConfigurationPolicy.<field-path> command. Replace <field-path> with the path to the field that you need.

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: policy-config
spec:
  namespaceSelector:
    include: ["default"]
    exclude: []
    matchExpressions: []
    matchLabels: {}
  remediationAction: inform 1
  customMessage:
    compliant: {}
    noncompliant: {}
  severity: low
  evaluationInterval:
    compliant: ""
    noncompliant: ""
  object-templates-raw: ""
  object-templates: 2
   - complianceType: musthave
     metadataComplianceType:
     recordDiff: ""
     recreateOption: ""
     objectSelector:
       matchLabels: {}
       matchExpressions: []
     objectDefinition:
       apiVersion: v1
       kind: Pod
       metadata:
         name: pod
       spec:
         containers:
          - image: pod-image
            name: pod-name
            ports:
             - containerPort: 80
   - complianceType: mustonlyhave
     objectDefinition:
       apiVersion: v1
       kind: ConfigMap
       metadata:
         name: myconfig
         namespace: default
         data:
           testData: hello

See the following topics for more information:

View the following example of a certificate policy and review the element in the YAML table:

apiVersion: policy.open-cluster-management.io/v1
kind: CertificatePolicy
metadata:
  name: certificate-policy-example
spec:
  namespaceSelector:
    include: ["default"]
    exclude: []
    matchExpressions: []
    matchLabels: {}
  labelSelector:
    myLabelKey: myLabelValue
  remediationAction:
  severity:
  minimumDuration:
  minimumCADuration:
  maximumDuration:
  maximumCADuration:
  allowedSANPattern:
  disallowedSANPattern:

When your certificate policy controller is created on your hub cluster, a replicated policy is created on your managed cluster. See policy-certificate.yaml to view the certificate policy sample.

Your policy set might resemble the following YAML file:

apiVersion: policy.open-cluster-management.io/v1beta1
kind: PolicySet
metadata:
  name: demo-policyset
spec:
  policies:
  - policy-demo

---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: demo-policyset-pb
placementRef:
  apiGroup: cluster.open-cluster-management.io
  kind: Placement
  name: demo-policyset-pr
subjects:
- apiGroup: policy.open-cluster-management.io
  kind: PolicySet
  name: demo-policyset
---
apiVersion: cluster.open-cluster-management.io/v1beta1
kind: Placement
metadata:
  name: demo-policyset-pr
spec:
  predicates:
  - requiredClusterSelector:
      labelSelector:
        matchExpressions:
          - key: name
            operator: In
            values:
              - local-cluster
    tolerations:
        - key: cluster.open-cluster-management.io/unavailable
          operator: Exists
        - key: cluster.open-cluster-management.io/unreachable
          operator: Exists

View the following parameter table for descriptions:

apiVersion: policy.open-cluster-management.io/v1beta1
kind: PolicySet
metadata:
  name: pci
  namespace: default
spec:
  description: Policies for PCI compliance
  policies:
  - policy-pod
  - policy-namespace
status:
  compliant: NonCompliant
  placement:
  - placementBinding: binding1
    placement: placement1
    policySet: policyset-ps

If you use the generic function, lookup, which is more advanced, familiarize yourself with the YAML structure of the resource that is being looked up. In addition to these functions, utility functions such as base64enc, base64dec, indent, autoindent, toInt, toBool, and more are available.

To conform templates with YAML syntax, you must define templates in the policy resource as strings using quotes or a block character (| or >). This causes the resolved template value to also be a string. To override this, use toInt or toBool as the final function in the template to initiate further processing that forces the value to be interpreted as an integer or boolean respectively.

Continue reading to view the descriptions and examples for some of the custom template functions that are supported:

func fromSecret (ns string, secretName string, datakey string) (dataValue string, err error)

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-fromsecret
spec:
  object-templates:
  - objectDefinition:
      apiVersion: v1
      kind: Secret 1
      metadata:
        name: demosecret
        namespace: test
      type: Opaque
      data: 2
        USER_NAME: YWRtaW4=
        PASSWORD: '{{ fromSecret "default" "localsecret" "PASSWORD" }}' 3

ca.crt: '{{ fromSecret "openshift-config" "ca-config-map-secret" "ca.crt"  | base64dec | toRawJson | toLiteral }}'

func fromConfigMap (ns string, configmapName string, datakey string) (dataValue string, err Error)

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-fromcm-lookup
spec:
  object-templates:
  - objectDefinition:
      ...
      data: 1
        app-name: sampleApp
        app-description: "this is a sample app"
        log-file: '{{ fromConfigMap "default" "logs-config" "log-file" }}' 2
        log-level: '{{ fromConfigMap "default" "logs-config" "log-level" }}' 3

func fromClusterClaim (clusterclaimName string) (dataValue string, err Error)

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-clusterclaims 1
spec:
  object-templates:
  - objectDefinition:
      ...
      data: 2
        platform: '{{ fromClusterClaim "platform.open-cluster-management.io" }}' 3
        product: '{{ fromClusterClaim "product.open-cluster-management.io" }}'
        version: '{{ fromClusterClaim "version.openshift.io" }}'

func lookup (apiversion string, kind string, namespace string, name string, labelselector ...string) (value string, err Error)

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-lookup
spec:
  object-templates:
  - objectDefinition:
      ...
      data: 1
        app-name: sampleApp
        app-description: "this is a sample app"
        metrics-url: >- 2
          http://{{ (lookup "v1" "Service" "default" "metrics").spec.clusterIP }}:8080

func base64enc (data string) (enc-data string)

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-fromsecret
spec:
  object-templates:
  - objectDefinition:
      ...
      data:
        USER_NAME: >-
          {{ fromConfigMap "default" "myconfigmap" "admin-user" | base64enc }}

func base64dec (enc-data string) (data string)

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-fromsecret
spec:
  object-templates:
  - objectDefinition:
      ...
      data:
        app-name: >-
          {{ ( lookup "v1"  "Secret" "testns" "mytestsecret") .data.appname ) | base64dec }}

func indent (spaces  int,  data string) (padded-data string)

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
spec:
  object-templates:
  - objectDefinition:
      ...
      data:
        Ca-cert: >-
          {{ ( index ( lookup "v1" "Secret" "default" "mycert-tls"  ).data  "ca.pem"  ) |  base64dec | indent 4  }}

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-fromsecret
spec:
  object-templates:
  - objectDefinition:
      ...
      data:
        Ca-cert: >-
          {{ ( index ( lookup "v1" "Secret" "default" "mycert-tls"  ).data  "ca.pem"  ) |  base64dec | autoindent }}

func toInt (input interface{}) (output int)

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-template-function
spec:
  object-templates:
  - objectDefinition:
      ...
      spec:
        vlanid: >-
          {{ (fromConfigMap "site-config" "site1" "vlan")  | toInt }}

func toBool (input string) (output bool)

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-template-function
spec:
  object-templates:
  - objectDefinition:
      ...
      spec:
        enabled: >-
          {{ (fromConfigMap "site-config" "site1" "enabled")  | toBool }}

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: demo-template-function
spec:
  object-templates:
  - objectDefinition:
    ...
    spec:
      enabled: >-
        {{hub (lookup "v1" "Secret" "default" "my-hub-secret").data.message | protect hub}}

key: '{{ "[\"10.10.10.10\", \"1.1.1.1\"]" | toLiteral }}'

key: ["10.10.10.10", "1.1.1.1"]

...
objectDefinition:
  apiVersion: v1
  kind: Secret
  metadata:
    name: my-secret-copy
  data: '{{ copySecretData "default" "my-secret" }}' 1

...
objectDefinition:
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: my-secret-copy
  data: '{{ copyConfigMapData "default" "my-configmap" }}'

...
objectDefinition:
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: my-configmap
    data:
      infraNode: |
        {{- range $i,$nd := (getNodesWithExactRoles "infra").items }}
        node{{ $i }}: {{ $nd.metadata.name }}
        {{- end }}
      replicas: {{ len ((getNodesWithExactRoles "infra").items) | toInt }}

...
objectDefinition:
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: my-configmap
  data:
    key: '{{ hasNodesWithExactRoles "infra" }}'

...
objectSelector:
  matchExpressions:
  - key: foo
    operator: In
    values:
    - bar
objectDefinition:
  apiVersion: v1
  kind: ConfigMap
  data:
    key: '{{ if (hasSuffix "-prod" .ObjectName) }}{{ skipObject }}{{ end }}{{ hasNodesWithExactRoles "infra" }}'

Hub cluster templates are resolved to the data in the referenced resources during policy creation, or when the referenced resources are updated.

If you need to manually initiate an update, use the special annotation, policy.open-cluster-management.io/trigger-update, to indicate changes for the data referenced by the templates. Any change to the special annotation value automatically initiates template processing. Additionally, the latest contents of the referenced resource are read and updated in the policy definition that is propagated for processing on managed clusters. A way to use this annotation is to increment the value by one each time.

By default, Red Hat Advanced Cluster Management processes all Go templates. You might create a policy that contains a policy in policy-templates that is not intended to be processed by Red Hat Advanced Cluster Management. To bypass template processing for a particular template, change {{ template content }} to +{{ `{{ template content }} }}+`. Then, the template returns a raw string to be processed by a subsequent templating engine.

If you want to completely bypass the template resolution for a configuration policy, add the policy.open-cluster-management.io/disable-templates: "true" annotation in the relevant ConfigurationPolicy that is contained in your Policy resource.

To iterate over many objects that are filtered by a label with ConfigurationPolicy, use the objectSelector and specify the kind in the objectDefinition. The objectSelector is similar to the namespaceSelector, except that it does not have name filtering capabilities. To filter by name, use the skipObject Go template function. If you reference {{ skipObject }} at any time, it signals to skip that particular object from those that are selected by the objectSelector for the policy.

See the following YAML sample that selects objects that have the label foo: bar, but skips objects that have a name with the suffix -prod:

...
objectSelector:
  matchExpressions:
  - key: foo
    operator: In
    values:
    - bar
objectDefinition:
  apiVersion: v1
  kind: ConfigMap
  data:
    key: '{{ if (hasSuffix "-prod" .ObjectName) }}{{ skipObject }}{{ end }}{{ hasNodesWithExactRoles "infra" }}'

The object-template-raw parameter is an optional parameter that supports advanced templating use-cases, such as the if conditional function and the range loop function. The object-templates-raw parameter accepts a string containing Go templates, and when you use these templates, they must result in an object-templates array.

For example, see the following YAML sample that adds the species-category: mammal label to any ConfigMap in the default namespace that has a name key equal to Sea Otter:

object-templates-raw: |
  {{- range (lookup "v1" "ConfigMap" "default" "").items }}
  {{- if eq .data.name "Sea Otter" }}
  - complianceType: musthave
    objectDefinition:
      kind: ConfigMap
      apiVersion: v1
      metadata:
        name: {{ .metadata.name }}
        namespace: {{ .metadata.namespace }}
        labels:
          species-category: mammal
  {{- end }}
  {{- end }}

Note: While spec.object-templates and spec.object-templates-raw are optional, exactly one of the two parameter fields must be set.

View the following policy example that uses advanced templates to create and configure infrastructure MachineSet objects for your managed clusters.

apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
  name: create-infra-machineset
spec:
  remediationAction: enforce
  severity: low
  object-templates-raw: |
    {{- /* Specify the parameters needed to create the MachineSet  */ -}}
    {{- $machineset_role := "infra" }}
    {{- $region := "ap-southeast-1" }}
    {{- $zones := list "ap-southeast-1a" "ap-southeast-1b" "ap-southeast-1c" }}
    {{- $infrastructure_id := (lookup "config.openshift.io/v1" "Infrastructure" "" "cluster").status.infrastructureName }}
    {{- $worker_ms := (index (lookup "machine.openshift.io/v1beta1" "MachineSet" "openshift-machine-api" "").items 0) }}
    {{- /* Generate the MachineSet for each zone as specified  */ -}}
    {{- range $zone := $zones }}
    - complianceType: musthave
      objectDefinition:
        apiVersion: machine.openshift.io/v1beta1
        kind: MachineSet
        metadata:
          labels:
            machine.openshift.io/cluster-api-cluster: {{ $infrastructure_id }}
          name: {{ $infrastructure_id }}-{{ $machineset_role }}-{{ $zone }}
          namespace: openshift-machine-api
        spec:
          replicas: 1
          selector:
            matchLabels:
              machine.openshift.io/cluster-api-cluster: {{ $infrastructure_id }}
              machine.openshift.io/cluster-api-machineset: {{ $infrastructure_id }}-{{ $machineset_role }}-{{ $zone }}
          template:
            metadata:
              labels:
                machine.openshift.io/cluster-api-cluster: {{ $infrastructure_id }}
                machine.openshift.io/cluster-api-machine-role: {{ $machineset_role }}
                machine.openshift.io/cluster-api-machine-type: {{ $machineset_role }}
                machine.openshift.io/cluster-api-machineset: {{ $infrastructure_id }}-{{ $machineset_role }}-{{ $zone }}
            spec:
              metadata:
                labels:
                  node-role.kubernetes.io/{{ $machineset_role }}: ""
              taints:
                - key: node-role.kubernetes.io/{{ $machineset_role }}
                  effect: NoSchedule
              providerSpec:
                value:
                  ami:
                    id: {{ $worker_ms.spec.template.spec.providerSpec.value.ami.id }}
                  apiVersion: awsproviderconfig.openshift.io/v1beta1
                  blockDevices:
                    - ebs:
                        encrypted: true
                        iops: 2000
                        kmsKey:
                          arn: ''
                        volumeSize: 500
                        volumeType: io1
                  credentialsSecret:
                    name: aws-cloud-credentials
                  deviceIndex: 0
                  instanceType: {{ $worker_ms.spec.template.spec.providerSpec.value.instanceType }}
                  iamInstanceProfile:
                    id: {{ $infrastructure_id }}-worker-profile
                  kind: AWSMachineProviderConfig
                  placement:
                    availabilityZone: {{ $zone }}
                    region: {{ $region }}
                  securityGroups:
                    - filters:
                        - name: tag:Name
                          values:
                            - {{ $infrastructure_id }}-worker-sg
                  subnet:
                    filters:
                      - name: tag:Name
                        values:
                          - {{ $infrastructure_id }}-private-{{ $zone }}
                  tags:
                    - name: kubernetes.io/cluster/{{ $infrastructure_id }}
                      value: owned
                  userDataSecret:
                    name: worker-user-data
    {{- end }}

By default, if you use the hub templates in the ConfigurationPolicy and OperatorPolicy resources, you must correspond the Policy resources that are defined on the hub cluster with these resources.

You can define configuration policies directly on the managed clusters. For example, OpenShift GitOps is one that you can define configuration policies directly on the managed cluster. To resolve hub templates on the managed cluster in that situation, you can enable the governance-standalone-hub-templating add-on.

To enable the governance-standalone-hub-templating add-on, complete the following steps:

By default, the agent on the managed cluster only has access to the ManagedCluster resources on the hub cluster. You can use the .ManagedClusterLabels template variable in the hub cluster templates inside the ConfigurationPolicies that are deployed directly to the managed cluster.

If you want the hub template to access other resources, such as the lookup or fromConfigMap function calls, you must add those specific permissions to the add-on group. You can add these permissions through resources, such as Roles, ClusterRoles, RoleBindings, and ClusterRoleBindings.

The name of the add-on group depends on the name of your managed cluster, but it has the following standard form: system:open-cluster-management:cluster:<cluster name>:addon:governance-standalone-hub-templating.

To allow access to Configmaps in your managed cluster namespace on your hub cluster, complete the following steps:

After you add these resources, the add-on can resolve the hub templates, and the state of the policy gets saved on a secret on the managed clusters. This secret prevents interruptions if the hub cluster becomes temporarily unavailable to the managed cluster.