Search

Chapter 4. Other Images

download PDF

4.1. Overview

This topic group includes information on other container images available for OpenShift Container Platform users.

4.2. Jenkins

4.2.1. Overview

OpenShift Container Platform provides a container image for running Jenkins. This image provides a Jenkins server instance, which can be used to set up a basic flow for continuous testing, integration, and delivery.

This image also includes a sample Jenkins job, which triggers a new build of a BuildConfig defined in OpenShift Container Platform, tests the output of that build, and then on successful build, retags the output to indicate the build is ready for production. For more details, see the README.

OpenShift Container Platform follows the LTS release of Jenkins. OpenShift Container Platform provides an image containing Jenkins 2.x. A separate image with Jenkins 1.x was previously made available but is now no longer maintained.

4.2.2. Images

The OpenShift Container Platform Jenkins image comes in two flavors:

RHEL 7 Based Image

The RHEL 7 image is available through the Red Hat Registry:

$ docker pull registry.redhat.io/openshift3/jenkins-2-rhel7

CentOS 7 Based Image

This image is available on Docker Hub:

$ docker pull openshift/jenkins-2-centos7

To use these images, you can either access them directly from these registries or push them into your OpenShift Container Platform container image registry. Additionally, you can create an ImageStream that points to the image, either in your container image registry or at the external location. Your OpenShift Container Platform resources can then reference the ImageStream. You can find example ImageStream definitions for all the provided OpenShift Container Platform images.

4.2.3. Configuration and Customization

4.2.3.1. Authentication

You can manage Jenkins authentication in two ways:

  • OpenShift Container Platform OAuth authentication provided by the OpenShift Login plug-in.
  • Standard authentication provided by Jenkins
4.2.3.1.1. OpenShift Container Platform OAuth authentication

OAuth authentication is activated by configuring the Configure Global Security panel in the Jenkins UI, or by setting the OPENSHIFT_ENABLE_OAUTH environment variable on the Jenkins Deployment Config to anything other than false. This activates the OpenShift Login plug-in, which retrieves the configuration information from pod data or by interacting with the OpenShift Container Platform API server.

Valid credentials are controlled by the OpenShift Container Platform identity provider. For example, if Allow All is the default identity provider, you can provide any non-empty string for both the user name and password.

Jenkins supports both browser and non-browser access.

Valid users are automatically added to the Jenkins authorization matrix at log in, where OpenShift Container Platform Roles dictate the specific Jenkins permissions the user will have.

Users with the admin role will have the traditional Jenkins administrative user permissions. Users with the edit or view role will have progressively less permissions. See the Jenkins image source repository README for the specifics on the OpenShift roles to Jenkins permissions mappings.

Note

The admin user that is pre-populated in the OpenShift Container Platform Jenkins image with administrative privileges will not be given those privileges when OpenShift Container Platform OAuth is used, unless the OpenShift Container Platform cluster administrator explicitly defines that user in the OpenShift Container Platform identity provider and assigns the admin role to the user.

Jenkins' users permissions can be changed after the users are initially established. The OpenShift Login plug-in polls the OpenShift Container Platform API server for permissions and updates the permissions stored in Jenkins for each user with the permissions retrieved from OpenShift Container Platform. If the Jenkins UI is used to update permissions for a Jenkins user, the permission changes are overwritten the next time the plug-in polls OpenShift Container Platform.

You can control how often the polling occurs with the OPENSHIFT_PERMISSIONS_POLL_INTERVAL environment variable. The default polling interval is five minutes.

The easiest way to create a new Jenkins service using OAuth authentication is to use a template as described below.

4.2.3.1.2. Jenkins Standard Authentication

Jenkins authentication is used by default if the image is run directly, without using a template.

The first time Jenkins starts, the configuration is created along with the administrator user and password. The default user credentials are admin and password. Configure the default password by setting the JENKINS_PASSWORD environment variable when using (and only when using) standard Jenkins authentication.

To create a new Jenkins application using standard Jenkins authentication:

$ oc new-app -e \
    JENKINS_PASSWORD=<password> \
    openshift/jenkins-2-centos7

4.2.3.2. Environment Variables

The Jenkins server can be configured with the following environment variables:

  • OPENSHIFT_ENABLE_OAUTH (default: false)

    Determines whether the OpenShift Login plug-in manages authentication when logging into Jenkins. To enable, set to true.

  • JENKINS_PASSWORD (default: password)

    The password for the admin user when using standard Jenkins authentication. Not applicable when OPENSHIFT_ENABLE_OAUTH is set to true.

  • OPENSHIFT_JENKINS_JVM_ARCH

    Set to x86_64 or i386 to override the JVM used to host Jenkins. For memory efficiency, by default the Jenkins image dynamically uses a 32-bit JVM if running in a container with a memory limit under 2GiB.

  • JAVA_MAX_HEAP_PARAM
    CONTAINER_HEAP_PERCENT (default: 0.5, or 50%)
    JENKINS_MAX_HEAP_UPPER_BOUND_MB

    These values control the maximum heap size of the Jenkins JVM. If JAVA_MAX_HEAP_PARAM is set (example setting: -Xmx512m), its value takes precedence. Otherwise, the maximum heap size is dynamically calculated as CONTAINER_HEAP_PERCENT% (example setting: 0.5, or 50%) of the container memory limit, optionally capped at JENKINS_MAX_HEAP_UPPER_BOUND_MB MiB (example setting: 512).

    By default, the maximum heap size of the Jenkins JVM is set to 50% of the container memory limit with no cap.

  • JAVA_INITIAL_HEAP_PARAM
    CONTAINER_INITIAL_PERCENT

    These values control the initial heap size of the Jenkins JVM. If JAVA_INITIAL_HEAP_PARAM is set (example setting: -Xms32m), its value takes precedence. Otherwise, the initial heap size may be dynamically calculated as CONTAINER_INITIAL_PERCENT% (example setting: 0.1, or 10%) of the dynamically calculated maximum heap size.

    By default, the initial heap sizing is left to the JVM.

  • CONTAINER_CORE_LIMIT

    If set, specifies an integer number of cores used for sizing numbers of internal JVM threads. Example setting: 2.

  • JAVA_TOOL_OPTIONS (default: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Dsun.zip.disableMemoryMapping=true)

    Specifies options to be heeded by all JVMs running in this container. It is not recommended to override this.

  • JAVA_GC_OPTS (default: -XX:+UseParallelGC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=10 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90)

    Specifies Jenkins JVM garbage collection parameters. It is not recommended to override this.

  • JENKINS_JAVA_OVERRIDES

    Specifies additional options for the Jenkins JVM. These options are appended to all other options, including the Java options above, and may be used to override any of them if necessary. Separate each additional option with a space; if any option contains space characters, escape them with a backslash. Example settings: -Dfoo -Dbar; -Dfoo=first\ value -Dbar=second\ value.

  • JENKINS_OPTS

    Specifies arguments to Jenkins.

  • INSTALL_PLUGINS

    Specifies additional Jenkins plug-ins to install when the container is first run or when OVERRIDE_PV_PLUGINS_WITH_IMAGE_PLUGINS is set to true (see below). Plug-ins are specified as a comma-delimited list of name:version pairs. Example setting: git:3.7.0,subversion:2.10.2.

  • OPENSHIFT_PERMISSIONS_POLL_INTERVAL (default: 300000 - 5 minutes)

    Specifies in milliseconds how often the OpenShift Login plug-in polls OpenShift Container Platform for the permissions associated with each user defined in Jenkins.

  • OVERRIDE_PV_CONFIG_WITH_IMAGE_CONFIG (default: false)

    When running this image with an OpenShift Container Platform persistent volume for the Jenkins config directory, the transfer of configuration from the image to the persistent volume is only done the first startup of the image as the persistent volume is assigned by the persistent volume claim creation. If you create a custom image that extends this image and updates configuration in the custom image after the initial startup, by default it will not be copied over, unless you set this environment variable to true.

  • OVERRIDE_PV_PLUGINS_WITH_IMAGE_PLUGINS (default: false)

    When running this image with an OpenShift Container Platform persistent volume for the Jenkins config directory, the transfer of plugins from the image to the persistent volume is only done the first startup of the image as the persistent volume is assigned by the persistent volume claim creation. If you create a custom image that extends this image and updates plugins in the custom image after the initial startup, by default they will not be copied over, unless you set this environment variable to true.

  • ENABLE_FATAL_ERROR_LOG_FILE (default: false)

    When running this image with an OpenShift Container Platform persistent claim for the Jenkins config directory, this environment variable allows the fatal error log file to persist when a fatal error occurs. The fatal error file is saved at /var/lib/jenkins/logs.

  • NODEJS_SLAVE_IMAGE

    Setting this value overrides the image used for the default NodeJS agent pod configuration. The default NodeJS agent pod uses docker.io/openshift/jenkins-agent-nodejs-8-centos7 or registry.redhat.io/openshift3/jenkins-agent-nodejs-8-rhel7 depending whether you are running the CentOS or RHEL version of the Jenkins image. This variable must be set before Jenkins starts the first time for it to have an effect.

  • MAVEN_SLAVE_IMAGE

    Setting this value overrides the image used for the default maven agent pod configuration. The default maven agent pod uses docker.io/openshift/jenkins-agent-maven-35-centos7 or registry.redhat.io/openshift3/jenkins-agent-maven-35-rhel7 depending whether you are running the CentOS or RHEL version of the Jenkins image. This variable must be set before Jenkins starts the first time for it to have an effect.

  • JENKINS_UC_INSECURE

    Determines whether Jenkins plugins downloads are allowed if the Jenkins Update Center repository uses an invalid SSL certificate. This could be the case if a self hosted repository using self-signed certificate with an unknown CA is used or if an enteprise proxy performs man-in-the-middle interceptions. This variable applies to plug-in downloads, which may occur during a Jenkins image build or if an extension of the Jenkins image is built. It is also applied when you run the Jenkins image and use one of the options to download additional plug-ins, including S2I with plugins.txt or the INSTALL_PLUGINS environment variable. Set to true to enable this variable.

4.2.3.3. Cross Project Access

If you are going to run Jenkins somewhere other than as a deployment within your same project, you will need to provide an access token to Jenkins to access your project.

  1. Identify the secret for the service account that has appropriate permissions to access the project Jenkins needs to access:

    $ oc describe serviceaccount jenkins
    Name:       default
    Labels:     <none>
    Secrets:    {  jenkins-token-uyswp    }
                {  jenkins-dockercfg-xcr3d    }
    Tokens:     jenkins-token-izv1u
                jenkins-token-uyswp

    In this case the secret is named jenkins-token-uyswp

  2. Retrieve the token from the secret:

    $ oc describe secret <secret name from above> # for example, jenkins-token-uyswp
    Name:       jenkins-token-uyswp
    Labels:     <none>
    Annotations:    kubernetes.io/service-account.name=jenkins,kubernetes.io/service-account.uid=32f5b661-2a8f-11e5-9528-3c970e3bf0b7
    Type:   kubernetes.io/service-account-token
    Data
    ====
    ca.crt: 1066 bytes
    token:  eyJhbGc..<content cut>....wRA

The token field contains the token value Jenkins needs to access the project.

4.2.3.4. Volume Mount Points

The Jenkins image can be run with mounted volumes to enable persistent storage for the configuration:

  • /var/lib/jenkins - This is the data directory where Jenkins stores configuration files including job definitions.

4.2.3.5. Customizing the Jenkins Image through Source-To-Image

To customize the official OpenShift Container Platform Jenkins image, you have two options:

  • Use Docker layering.
  • Use the image as a Source-To-Image builder, described here.

You can use S2I to copy your custom Jenkins Jobs definitions, additional plug-ins or replace the provided config.xml file with your own, custom, configuration.

In order to include your modifications in the Jenkins image, you need to have a Git repository with the following directory structure:

plugins
This directory contains those binary Jenkins plug-ins you want to copy into Jenkins.
plugins.txt
This file lists the plug-ins you want to install:
pluginId:pluginVersion
configuration/jobs
This directory contains the Jenkins job definitions.
configuration/config.xml
This file contains your custom Jenkins configuration.

The contents of the configuration/ directory will be copied into the /var/lib/jenkins/ directory, so you can also include additional files, such as credentials.xml, there.

The following is an example build configuration that customizes the Jenkins image in OpenShift Container Platform:

apiVersion: v1
kind: BuildConfig
metadata:
  name: custom-jenkins-build
spec:
  source:                       1
    git:
      uri: https://github.com/custom/repository
    type: Git
  strategy:                     2
    sourceStrategy:
      from:
        kind: ImageStreamTag
        name: jenkins:latest
        namespace: openshift
    type: Source
  output:                       3
    to:
      kind: ImageStreamTag
      name: custom-jenkins:latest
1
The source field defines the source Git repository with the layout described above.
2
The strategy field defines the original Jenkins image to use as a source image for the build.
3
The output field defines the resulting, customized Jenkins image you can use in deployment configuration instead of the official Jenkins image.

4.2.3.6. Configuring the Jenkins Kubernetes Plug-in

The OpenShift Container Platform Jenkins image includes the pre-installed Kubernetes plug-in that allows Jenkins agents to be dynamically provisioned on multiple container hosts using Kubernetes and OpenShift Container Platform.

To use the Kubernetes plug-in, OpenShift Container Platform provides five images suitable for use as Jenkins agents: the Base, Maven, and Node.js images. See Jenkins Agents for more information.

Note

the jenkins-slave-maven-* and jenkins-slave-nodejs-* images are being marked as deprecated during the v3.10 release cycle. The images will still exist in the interim so users can migrate their applications to the newer jenkins-agent-maven-* and jenkins-agent-nodejs-* images.

Both the Maven and Node.js agent images are automatically configured as Kubernetes Pod Template images within the OpenShift Container Platform Jenkins image’s configuration for the Kubernetes plug-in. That configuration includes labels for each of the images that can be applied to any of your Jenkins jobs under their "Restrict where this project can be run" setting. If the label is applied, execution of the given job will be done under an OpenShift Container Platform pod running the respective agent image.

The Jenkins image also provides auto-discovery and auto-configuration of additional agent images for the Kubernetes plug-in. With the OpenShift Sync plug-in, the Jenkins image on Jenkins start-up searches within the project that it is running, or the projects specifically listed in the plug-in’s configuration for the following:

  • Image streams that have the label role set to jenkins-slave.
  • Image stream tags that have the annotation role set to jenkins-slave.
  • ConfigMaps that have the label role set to jenkins-slave.

When it finds an image stream with the appropriate label, or image stream tag with the appropriate annotation, it generates the corresponding Kubernetes plug-in configuration so you can assign your Jenkins jobs to run in a pod running the container image provided by the image stream.

The name and image references of the image stream or image stream tag are mapped to the name and image fields in the Kubernetes plug-in pod template. You can control the label field of the Kubernetes plug-in pod template by setting an annotation on the image stream or image stream tag object with the key slave-label. Otherwise, the name is used as the label.

When it finds a ConfigMap with the appropriate label, it assumes that any values in the key-value data payload of the ConfigMap contains XML consistent with the config format for Jenkins and the Kubernetes plug-in pod templates. A key differentiator to note when using ConfigMaps, instead of image streams or image stream tags, is that you can control all the various fields of the Kubernetes plug-in pod template.

The following is an example ConfigMap:

kind: ConfigMap
apiVersion: v1
metadata:
  name: jenkins-agent
  labels:
    role: jenkins-slave
data:
  template1: |-
    <org.csanchez.jenkins.plugins.kubernetes.PodTemplate>
      <inheritFrom></inheritFrom>
      <name>template1</name>
      <instanceCap>2147483647</instanceCap>
      <idleMinutes>0</idleMinutes>
      <label>template1</label>
      <serviceAccount>jenkins</serviceAccount>
      <nodeSelector></nodeSelector>
      <volumes/>
      <containers>
        <org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate>
          <name>jnlp</name>
          <image>openshift/jenkins-agent-maven-35-centos7:v3.10</image>
          <privileged>false</privileged>
          <alwaysPullImage>true</alwaysPullImage>
          <workingDir>/tmp</workingDir>
          <command></command>
          <args>${computer.jnlpmac} ${computer.name}</args>
          <ttyEnabled>false</ttyEnabled>
          <resourceRequestCpu></resourceRequestCpu>
          <resourceRequestMemory></resourceRequestMemory>
          <resourceLimitCpu></resourceLimitCpu>
          <resourceLimitMemory></resourceLimitMemory>
          <envVars/>
        </org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate>
      </containers>
      <envVars/>
      <annotations/>
      <imagePullSecrets/>
      <nodeProperties/>
    </org.csanchez.jenkins.plugins.kubernetes.PodTemplate>

After startup, the OpenShift Sync plug-in monitors the API server of OpenShift Container Platform for updates to ImageStreams, ImageStreamTags, and ConfigMaps and adjusts the configuration of the Kubernetes plug-in.

In particular, the following rules will apply:

  • Removal of the label or annotation from the ConfigMap, ImageStream, or ImageStreamTag will result in the deletion of any existing PodTemplate from the configuration of the Kubernetes plug-in.
  • Similarly, if those objects are removed, the corresponding configuration is removed from the Kubernetes plug-in.
  • Conversely, either the creation of appropriately labeled or annotated ConfigMap, ImageStream, or ImageStreamTag objects, or the adding of labels after their initial creation, leads to the creation of a PodTemplate in the Kubernetes-plugin configuration.
  • In the case of the PodTemplate via ConfigMap form, changes to the ConfigMap data for the PodTemplate will be applied to the PodTemplate settings in the Kubernetes plug-in configuration, and will override any changes made to the PodTemplate through the Jenkins UI in the interim between changes to the ConfigMap.

To use a container image as a Jenkins agent, the image must run the slave agent as an entrypoint. For more details about this, refer to the official Jenkins documentation.

4.2.3.6.1. Permission Considerations

In the previous ConfigMap example, the <serviceAccount> element of the Pod Template XML is the OpenShift Container Platform Service Account used for the resulting Pod. The service account credentials mounted into the Pod, with permissions associated with the service account, control which operations against the OpenShift Container Platform master are allowed from the Pod.

Consider the following with service accounts used for the Pod, launched by the Kubernetes Plug-in running in the OpenShift Container Platform Jenkins image:

  • If you use the example template for Jenkins provided by OpenShift Container Platform, the jenkins service account is defined with the edit role for the project Jenkins is running in, and the master Jenkins Pod has that service account mounted.
  • The two default Maven and NodeJS Pod Templates injected into the Jenkins configuration are also set to use the same service account as the master.
  • Any Pod Templates auto-discovered by the OpenShift Sync plug-in as a result of Image streams or Image stream tags having the required label or annotations have their service account set to the master’s service account.
  • For the other ways you can provide a Pod Template definition into Jenkins and the Kubernetes plug-in, you have to explicitly specify the service account to use.
  • Those other ways include the Jenkins console, the podTemplate pipeline DSL provided by the Kubernetes plug-in, or labeling a ConfigMap whose data is the XML configuration for a Pod Template.
  • If you do not specify a value for the service account, the default service account is used.
  • You need to ensure that whatever service account is used has the necessary permissions, roles, and so on defined within OpenShift Container Platform to manipulate whatever projects you choose to manipulate from the within the Pod

4.2.4. Usage

4.2.4.1. Creating a Jenkins Service from a Template

Templates provide parameter fields to define all the environment variables (password) with predefined defaults. OpenShift Container Platform provides templates to make creating a new Jenkins service easy. The Jenkins templates should have been registered in the default openshift project by your cluster administrator during the initial cluster setup. See Loading the Default Image Streams and Templates for more details, if required.

a deployment configuration and a service.

Note

A pod may be restarted when it is moved to another node, or when an update of the deployment configuration triggers a redeployment.

  • jenkins-persistent uses a persistent volume store. Data survives a pod restart.

must instantiate the template to be able to use Jenkins:

4.2.4.2. Using the Jenkins Kubernetes Plug-in

Creating a New Jenkins Service

In the below sample, the openshift-jee-sample BuildConfig causes a Jenkins maven agent Pod to be dynamically provisioned. The Pod clones some Java source, builds a WAR file, then causes a second BuildConfig (openshift-jee-sample-docker) to run to layer the newly created WAR file into a container image.

A fuller sample which achieves a similar goal is available here.

Example 4.1. Example BuildConfig using the Jenkins Kubernetes Plug-in

kind: List
apiVersion: v1
items:
- kind: ImageStream
  apiVersion: v1
  metadata:
    name: openshift-jee-sample
- kind: BuildConfig
  apiVersion: v1
  metadata:
    name: openshift-jee-sample-docker
  spec:
    strategy:
      type: Docker
    source:
      type: Docker
      dockerfile: |-
        FROM openshift/wildfly-101-centos7:latest
        COPY ROOT.war /wildfly/standalone/deployments/ROOT.war
        CMD $STI_SCRIPTS_PATH/run
      binary:
        asFile: ROOT.war
    output:
      to:
        kind: ImageStreamTag
        name: openshift-jee-sample:latest
- kind: BuildConfig
  apiVersion: v1
  metadata:
    name: openshift-jee-sample
  spec:
    strategy:
      type: JenkinsPipeline
      jenkinsPipelineStrategy:
        jenkinsfile: |-
          node("maven") {
            sh "git clone https://github.com/openshift/openshift-jee-sample.git ."
            sh "mvn -B -Popenshift package"
            sh "oc start-build -F openshift-jee-sample-docker --from-file=target/ROOT.war"
          }
    triggers:
    - type: ConfigChange

It is also possible to override the specification of the dynamically created Jenkins agent Pod. The following is a modification to the above example which overrides the container memory and specifies an environment variable:

Example 4.2. Example BuildConfig using the Jenkins Kubernetes Plug-in, specifying memory limit and environment variable

kind: BuildConfig
apiVersion: v1
metadata:
  name: openshift-jee-sample
spec:
  strategy:
    type: JenkinsPipeline
    jenkinsPipelineStrategy:
      jenkinsfile: |-
        podTemplate(label: "mypod", 1
                    cloud: "openshift", 2
                    inheritFrom: "maven", 3
                    containers: [
            containerTemplate(name: "jnlp", 4
                              image: "openshift/jenkins-agent-maven-35-centos7:v3.10", 5
                              resourceRequestMemory: "512Mi", 6
                              resourceLimitMemory: "512Mi", 7
                              envVars: [
              envVar(key: "CONTAINER_HEAP_PERCENT", value: "0.25") 8
            ])
          ]) {
          node("mypod") { 9
            sh "git clone https://github.com/openshift/openshift-jee-sample.git ."
            sh "mvn -B -Popenshift package"
            sh "oc start-build -F openshift-jee-sample-docker --from-file=target/ROOT.war"
          }
        }
  triggers:
  - type: ConfigChange
1
A new Pod template called "mypod" is defined on-the-fly. The new Pod template name is referenced in the node stanza below.
2
The "cloud" value must be set to "openshift".
3
The new Pod template can inherit its configuration from an existing Pod template. In this case, we inherit from the "maven" Pod template which is pre-defined by OpenShift Container Platform.
4
We are overriding values in the pre-existing Container, therefore we must specify it by name. All Jenkins agent images shipped with OpenShift Container Platform use the Container name "jnlp".
5
The Container image must be re-specified. This is a known issue.
6
A memory request of 512Mi is specified.
7
A memory limit of 512Mi is specified.
8
An environment variable CONTAINER_HEAP_PERCENT, with value "0.25", is specified.
9
The node stanza references the name of the Pod template newly defined above.

By default the pod is deleted when the build completes. This behavior can be modified via the plug-in or within a pipeline Jenkinsfile - see Agent Pod Retention for further details.

For more information on Kubernetes plug-in configuration, see the Kubernetes plug-in documentation.

4.2.4.3. Memory Requirements

When deployed by the provided Jenkins Ephemeral or Jenkins Persistent templates, the default memory limit is 512MiB.

See Sizing OpenJDK on OpenShift Container Platform for background information on tuning the JVM used by Jenkins.

For memory efficiency, by default the Jenkins image dynamically uses a 32-bit JVM if running in a container with a memory limit under 2GiB. This behavior can be overridden by the OPENSHIFT_JENKINS_JVM_ARCH environment variable.

By default the Jenkins JVM uses 50% of the container memory limit for its heap. This value can be modified by the CONTAINER_HEAP_PERCENT environment variable. It can also be capped at an upper limit or overridden entirely. See Environment Variables for more details.

Consider that by default all other processes executed in the Jenkins container, such as shell scripts or oc commands run locally from pipelines, are not likely to be able to use more than the remaining 256MiB memory combined without provoking an OOM kill. It is therefore highly recommended that pipelines run external commands in a agent container wherever possible.

It is recommended to specify memory request and limit values on agent containers created by the Jenkins Kubernetes Plug-in. As admin, defaults can be set on a per-agent image basis through the Jenkins configuration. The memory request and limit can also be overridden on a per-container basis as documented above.

You can increase the amount of memory available to Jenkins by overriding the MEMORY_LIMIT paramenter when instantiating the Jenkins Ephemeral or Jenkins Persistent template.

4.2.5. Jenkins Plug-ins

The following plug-ins are provided to integrate Jenkins with OpenShift Container Platform. They are available by default in the Jenkins image.

4.2.5.1. OpenShift Container Platform Client Plug-in

The OpenShift Container Platform Client Plug-in aims to provide a readable, concise, comprehensive, and fluent Jenkins Pipeline syntax for rich interactions with OpenShift Container Platform. The plug-in leverages the oc binary, which must be available on the nodes executing the script.

This plug-in is fully supported and is included in the Jenkins image. It provides:

  • A Fluent-style syntax for use in Jenkins Pipelines.
  • Use of and exposure to any option available with oc.
  • Integration with Jenkins credentials and clusters.
  • Continued support for classic Jenkins Freestyle jobs.

See the OpenShift Pipeline Builds tutorial and the plug-in’s README for more information.

4.2.5.2. OpenShift Container Platform Pipeline Plug-in

The OpenShift Container Platform Pipeline Plug-in is a prior integration between Jenkins and OpenShift Container Platform which provides less functionality than the OpenShift Container Platform Client Plug-in. It has been deprecated but continues to work with OpenShift Container Platform versions up to v3.11. For later verions of OpenShift Container Platform, either use the oc binary directly from your Jenkins Pipelines, or use the OpenShift Container Platform Client Plug-in.

See the plug-in’s README for more information.

4.2.5.3. OpenShift Container Platform Sync Plug-in

To facilitate OpenShift Container Platform Pipeline build strategy for integration between Jenkins and OpenShift Container Platform, the OpenShift Sync Plug-in monitors the API server of OpenShift Container Platform for updates to BuildConfigs and Builds that employ the Pipeline strategy and either creates Jenkins Pipeline projects (when a BuildConfig is created) or starts jobs in the resulting projects (when a Build is started).

As noted in Configuring the Jenkins Kubernetes Plug-in, this plug-in can create PodTemplate configurations for the Kubernetes plug-in based on specifically cited ImageStream, ImageStreamTag, or ConfigMap objects defined in OpenShift Container Platform.

This plug-in can now take Secret objects with a label key of credential.sync.jenkins.openshift.io and label value of true and construct Jenkins credentials which are placed in the default global domain within the Jenkins credentials hierarchy. The ID of the credential will be composed of the namespace the Secret is defined in, a hyphen (-), followed by the name of the Secret.

Similar to the handling of ConfigMaps for PodTemplates, the Secret object defined in OpenShift Container Platform is considered the master configuration. Any subsequent updates to the object in OpenShift Container Platform will be applied to the Jenkins credential (overwriting any changes to the credential made in the interim).

Removal of the credential.sync.jenkins.openshift.io property, setting of that property to something other than true, or deletion of the Secret in OpenShift Container Platform will result in deletion of the associated credential in Jenkins.

The type of secret will be mapped to the jenkins credential type as follows:

  • With Opaque type Secret objects the plug-in looks for username and password in the data section and constructs a Jenkins UsernamePasswordCredentials credential. Remember, in OpenShift Container Platform the password field can be either an actual password or the user’s unique token. If those are not present, it will look for the ssh-privatekey field and create a Jenkins BasicSSHUserPrivateKey credential.
  • With kubernetes.io/basic-auth type `Secret`objects the plug-in creates a Jenkins UsernamePasswordCredentials credential.
  • With kubernetes.io/ssh-auth type Secret objects the plug-in creates a Jenkins BasicSSHUserPrivateKey credential.

4.2.5.4. Kubernetes Plug-in

The Kubernetes plug-in is used to run Jenkins agents as pods on your cluster. The auto-configuration of the Kubernetes plug-in is described in Using the Jenkins Kubernetes Plug-in.

4.3. Jenkins Agents

4.3.1. Overview

OpenShift Container Platform provides three images suitable for use as Jenkins agents: the Base, Maven, and Node.js images.

The first is a base image for Jenkins agents:

  • It pulls in both the required tools (headless Java, the Jenkins JNLP client) and the useful ones (including git, tar, zip, and nss among others).
  • It establishes the JNLP agent as the entrypoint.
  • It includes the oc client tooling for invoking command line operations from within Jenkins jobs.
  • It provides Dockerfiles for both CentOS and RHEL images.

Two more images that extend the base image are also provided:

The Maven and Node.js Jenkins agent images provide Dockerfiles for both CentOS and RHEL that you can reference when building new agent images. Also note the contrib and contrib/bin subdirectories. They allow for the insertion of configuration files and executable scripts for your image.

Important

Use and extend an appropriate agent image version for the version of OpenShift Container Platform that you are using. If the oc client version embedded in the agent image is not compatible with the OpenShift Container Platform version, unexpected behavior may result. See the versioning policy for more information.

4.3.2. Images

The OpenShift Container Platform Jenkins agent images come in two flavors:

RHEL 7 Based Images

RHEL 7 images are available through the Red Hat Registry:

$ docker pull registry.redhat.io/openshift3/jenkins-slave-base-rhel7
$ docker pull registry.redhat.io/openshift3/jenkins-slave-maven-rhel7
$ docker pull registry.redhat.io/openshift3/jenkins-slave-nodejs-rhel7
$ docker pull registry.redhat.io/openshift3/jenkins-agent-maven-35-rhel7
$ docker pull registry.redhat.io/openshift3/jenkins-agent-nodejs-10-rhel7
$ docker pull registry.redhat.io/openshift3/jenkins-agent-nodejs-12-rhel7

CentOS 7 Based Images

These images are available on Docker Hub:

$ docker pull openshift/jenkins-slave-base-centos7
$ docker pull openshift/jenkins-slave-maven-centos7
$ docker pull openshift/jenkins-slave-nodejs-centos7
$ docker pull openshift/jenkins-agent-maven-35-centos7
$ docker pull openshift/jenkins-agent-nodejs-10-centos7
$ docker pull openshift/jenkins-agent-nodejs-12-centos7

To use these images, you can either access them directly from these registries or push them into your OpenShift Container Platform container image registry.

4.3.3. Configuration and Customization

4.3.3.1. Environment Variables

Each Jenkins agent container can be configured with the following environment variables:

  • OPENSHIFT_JENKINS_JVM_ARCH

    Set to x86_64 or i386 to override the JVM used to host the Jenkins agent. For memory efficiency, by default the Jenkins agent images dynamically use a 32-bit JVM if running in a container with a memory limit under 2GiB.

  • JAVA_MAX_HEAP_PARAM
    CONTAINER_HEAP_PERCENT (default: 0.1, i.e. 10%)
    JNLP_MAX_HEAP_UPPER_BOUND_MB

    These values control the maximum heap size of the Jenkins agent JVM. If JAVA_MAX_HEAP_PARAM is set (example setting: -Xmx512m), its value takes precedence. Otherwise, the maximum heap size is dynamically calculated as CONTAINER_HEAP_PERCENT% (example setting: 0.5, i.e. 50%) of the container memory limit, optionally capped at JNLP_MAX_HEAP_UPPER_BOUND_MB MiB (example setting: 512).

    By default, the maximum heap size of the Jenkins agent JVM is set to 50% of the container memory limit with no cap.

  • JAVA_INITIAL_HEAP_PARAM
    CONTAINER_INITIAL_PERCENT

    These values control the initial heap size of the Jenkins agent JVM. If JAVA_INITIAL_HEAP_PARAM is set (example setting: -Xms32m), its value takes precedence. Otherwise, the initial heap size may be dynamically calculated as CONTAINER_INITIAL_PERCENT% (example setting: 0.1, i.e. 10%) of the dynamically calculated maximum heap size.

    By default, the initial heap sizing is left to the JVM.

  • CONTAINER_CORE_LIMIT

    If set, specifies an integer number of cores used for sizing numbers of internal JVM threads. Example setting: 2.

  • JAVA_TOOL_OPTIONS (default: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Dsun.zip.disableMemoryMapping=true)

    Specifies options to be heeded by all JVMs running in this container. It is not recommended to override this.

  • JAVA_GC_OPTS (default: -XX:+UseParallelGC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=10 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90)

    Specifies Jenkins agent JVM garbage collection parameters. It is not recommended to override this.

  • JNLP_JAVA_OVERRIDES

    Specifies additional options for the Jenkins agent JVM. These options are appended to all other options, including the Java options above, and may be used to override any of them if necessary. Separate each additional option with a space; if any option contains space characters, escape them with a backslash. Example settings: -Dfoo -Dbar; -Dfoo=first\ value -Dbar=second\ value.

4.3.4. Usage

4.3.4.1. Memory Requirements

A JVM is used in all Jenkins agents to host the Jenkins JNLP agent, as well as to run any Java applications (e.g. javac, Maven or Gradle). See Sizing OpenJDK on OpenShift Container Platform for background information on tuning the JVM used by Jenkins agents.

For memory efficiency, by default the Jenkins image dynamically uses a 32-bit JVM if running in a container with a memory limit under 2GiB. This behavior can be overridden by the OPENSHIFT_JENKINS_JVM_ARCH environment variable. The JVM choice applies by default both for the Jenkins JNLP agent as well as for any other Java processes within the agent container.

By default the Jenkins JNLP agent JVM uses 50% of the container memory limit for its heap. This value can be modified by the CONTAINER_HEAP_PERCENT environment variable. It can also be capped at an upper limit or overridden entirely. See Environment Variables for more details.

Consider that by default any/all other processes executed in the Jenkins agent container, e.g. shell scripts or oc commands run from pipelines, may not be able to use more than the remaining 50% memory limit without provoking an OOM kill.

By default, each further JVM process run in a Jenkins agent container will use up to 25% of the container memory limit for their heap. It may be necessary to tune this for many build workloads. See Sizing OpenJDK on OpenShift Container Platform for more information.

See the Jenkins documentation for information on specifying the memory request and limit of a Jenkins agent container.

4.3.4.1.1. Gradle builds

Hosting Gradle builds in the a Jenkins agent on OpenShift presents additional complications, not least because in addition to the Jenkins JNLP agent and Gradle JVMs, Gradle spawns a third JVM to run tests, if these are specified.

See Sizing OpenJDK on OpenShift Container Platform for background information on tuning JVMs on OpenShift.

The following settings are suggested as a starting point for running Gradle builds in a memory constrained Jenkins agent on OpenShift. Settings may be relaxed subsequently as required.

  • Ensure the long-lived gradle daemon is disabled by adding org.gradle.daemon=false to the gradle.properties file.
  • Disable parallel build execution by ensuring org.gradle.parallel=true is not set in the gradle.properties file and that --parallel is not set as a command line argument.
  • Set java { options.fork = false } in the build.gradle file to prevent Java compilations running out-of-process.
  • Disable multiple additional test processes by ensuring test { maxParallelForks = 1 } is set in the build.gradle file.
  • Override the gradle JVM memory parameters according to Sizing OpenJDK on OpenShift Container Platform by the GRADLE_OPTS, JAVA_OPTS or JAVA_TOOL_OPTIONS environment variables.
  • Set the maximum heap size and JVM arguments for any Gradle test JVM by the maxHeapSize and jvmArgs settings in build.gradle, or though the -Dorg.gradle.jvmargs command line argument.

4.3.5. Agent Pod Retention

Jenkins agent pods (also known as slave pods) are deleted by default after the build completes or is aborted. This behavior can be changed by the Kubernetes plug-in Pod Retention setting. Pod retention can be set for all Jenkins builds, with overrides for each pod template. The following behaviors are supported:

  • Always keeps the build pod regardless of build result.
  • Default uses the plug-in value (pod template only).
  • Never always deletes the pod.
  • On Failure keeps the pod if it fails during the build.

You can override pod retention in the pipeline Jenkinsfile:

podTemplate(label: "mypod",
  cloud: "openshift",
  inheritFrom: "maven",
  podRetention: onFailure(), 1
  containers: [
    ...
  ]) {
  node("mypod") {
    ...
  }
}
1
Allowed values for podRetention are never(), onFailure(), always(), and default().
Warning

Pods that are kept may continue to run and count against resource quotas.

4.4. Other Container Images

 

If you want to use container images not found in the Red Hat Container Catalog, you can use other arbitrary container images in your OpenShift Container Platform instance, for example those found on the Docker Hub.

For OpenShift Container Platform-specific guidelines on running containers using an arbitrarily assigned user ID, see Support Arbitrary User IDs in the Creating Images guide.

Important

For supportability details, see the Production Support Scope of Coverage as defined in the OpenShift Container Platform Support Policy.

See also the security warning in System and Environment Requirements.

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.