Developer Guide

When accessing the web console from a browser at <master_public_addr>:8443, you are automatically redirected to a login page.

Review the browser versions and operating systems that can be used to access the web console.

You can provide your login credentials on this page to obtain a token to make API calls. After logging in, you can navigate your projects using the web console.

You can authenticate from the command line using the CLI command oc login. You can get started with the CLI by running this command without any options:

$ oc login

The command’s interactive flow helps you establish a session to an OpenShift server with the provided credentials. If any information required to successfully log in to an OpenShift server is not provided, the command prompts for user input as required. The configuration is automatically saved and is then used for every subsequent command.

All configuration options for the oc login command, listed in the oc login --help command output, are optional. The following example shows usage with some common options:

$ oc login [-u=<username>] \
  [-p=<password>] \
  [-s=<server>] \
  [-n=<project>] \
  [--certificate-authority=</path/to/file.crt>|--insecure-skip-tls-verify]

The following table describes these common options:

$ oc login -s=<server>

$ oc login -u=<username> -p=<password>

$ oc login -u=<username> -p=<password> -n=<project>

$ oc login --certificate-authority=<path/to/file.crt>

$ oc login --insecure-skip-tls-verify

CLI configuration files allow you to easily manage multiple CLI profiles.

$ oc login -u system:admin -n default

A project allows a community of users to organize and manage their content in isolation from other communities.

If allowed by your cluster administrator, you can create a new project using the CLI or the web console.

To create a new project using the CLI:

$ oc new-project <project_name> \
    --description="<description>" --display-name="<display_name>"

For example:

$ oc new-project hello-openshift \
    --description="This is an example project to demonstrate OpenShift v3" \
    --display-name="Hello OpenShift"

When viewing projects, you are restricted to seeing only the projects you have access to view based on the authorization policy.

To view a list of projects:

$ oc get projects

You can change from the current project to a different project for CLI operations. The specified project is then used in all subsequent operations that manipulate project-scoped content:

$ oc project <project_name>

You can also use the web console to view and change between projects. After authenticating and logging in, you are presented with a list of projects that you have access to:

If you use the CLI to create a new project, you can then refresh the page in the browser to see the new project.

Selecting a project brings you to the project overview for that project.

The oc status command provides a high-level overview of the current project, with its components and their relationships. This command takes no argument:

$ oc status

You can filter the contents of a project page in the web console by using the labels of a resource. You can pick from a suggested label name and values, or type in your own. Multiple filters can be added. When multiple filters are applied, resources must match all of the filters to remain visible.

To filter by labels:

When you delete a project, the server updates the project status to Terminating from Active. The server then clears all content from a project that is Terminating before finally removing the project. While a project is in Terminating status, a user cannot add new content to the project.

To delete a project:

$ oc delete project <project_name>

When a person uses the OpenShift Enterprise CLI or web console, their API token authenticates them to the OpenShift API. However, when a regular user’s credentials are not available, it is common for components to make API calls independently. For example:

Service accounts provide a flexible way to control API access without sharing a regular user’s credentials.

Every service account has an associated username that can be granted roles, just like a regular user. The username is derived from its project and name: system:serviceaccount:<project>:<name>

For example, to add the view role to the robot service account in the top-secret project:

$ oc policy add-role-to-user view system:serviceaccount:top-secret:robot

Every service account is also a member of two groups:

For example, to allow all service accounts in all projects to view resources in the top-secret project:

$ oc policy add-role-to-group view system:serviceaccounts -n top-secret

To allow all service accounts in the managers project to edit resources in the top-secret project:

$ oc policy add-role-to-group edit system:serviceaccounts:managers -n top-secret

Three service accounts are automatically created in every project:

All service accounts in a project are given the system:image-puller role, which allows pulling images from any image stream in the project using the internal Docker registry.

$ more sa.json
{
  "apiVersion": "v1",
  "kind": "ServiceAccount",
  "metadata": {
    "name": "robot"
  }
}

$ oc create -f sa.json
serviceaccounts/robot

$ oc describe serviceaccount robot
Name:               robot
Labels:             <none>
Image pull secrets:	robot-dockercfg-624cx

Mountable secrets: 	robot-token-uzkbh
                   	robot-dockercfg-624cx

Tokens:            	robot-token-8bhpp
                   	robot-token-uzkbh

$ oc secrets new secret-plans plan1.txt plan2.txt
secret/secret-plans

$ oc secrets new-dockercfg my-pull-secret \
    --docker-username=mastermind \
    --docker-password=12345 \
    --docker-email=mastermind@example.com
secret/my-pull-secret

$ oc secrets add serviceaccount/robot secret/secret-plans --for=mount

$ oc secrets add serviceaccount/robot secret/my-pull-secret --for=pull

$ oc describe serviceaccount robot
Name:               robot
Labels:             <none>
Image pull secrets:	robot-dockercfg-624cx
                   	my-pull-secret

Mountable secrets: 	robot-token-uzkbh
                   	robot-dockercfg-624cx
                   	secret-plans

Tokens:            	robot-token-8bhpp
                   	robot-token-uzkbh

$ TOKEN="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"

$ curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
    "https://openshift.default.svc.cluster.local/oapi/v1/users/~" \
    -H "Authorization: Bearer $TOKEN"

kind: "User"
apiVersion: "v1"
metadata:
  name: "system:serviceaccount:top-secret:robot"
  selflink: "/oapi/v1/users/system:serviceaccount:top-secret:robot"
  creationTimestamp: null
identities: null
groups:
  - "system:serviceaccounts"
  - "system:serviceaccounts:top-secret"

The same token can be distributed to external applications that need to authenticate to the API.

The same token can be distributed to external applications that need to authenticate to the API.

Use oc describe secret <secret-name> to view a service account’s API token:

$ oc describe secret robot-token-uzkbh -n top-secret
Name:		robot-token-uzkbh
Labels:		<none>
Annotations:	kubernetes.io/service-account.name=robot,kubernetes.io/service-account.uid=49f19e2e-16c6-11e5-afdc-3c970e4b7ffe

Type:	kubernetes.io/service-account-token

Data
====
token:	eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...


$ oc login --token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...
Logged into "https://server:8443" as "system:serviceaccount:top-secret:robot" using the token provided.

You don't have any projects. You can try to create a new project, by running

    $ oc new-project <projectname>

$ oc whoami
system:serviceaccount:top-secret:robot

You can create a new OpenShift Enterprise application from source code, images, or templates by using either the OpenShift CLI or web console.

This topic group includes information on how to get your application up and running in OpenShift Enterprise and covers different languages and their frameworks.

The oc rsh command allows you to locally access and manage tools that are on the system. The secure shell (SSH) is the underlying technology and industry standard that provides a secure connection to the application. Access to applications with the shell environment is protected and restricted with Security-Enhanced Linux (SELinux) policies.

Open a remote shell session to a container:

$ oc rsh <pod>

While in the remote shell, you can issue commands as if you are inside the container and perform local operations like monitoring, debugging, and using CLI commands specific to what is running in the container.

For example, in a MySQL container, you can count the number of records in the database by invoking the mysql command, then using the the prompt to type in the SELECT command. You can also use use commands like ps(1) and ls(1) for validation.

BuildConfigs and DeployConfigs map out how you want things to look and pods (with containers inside) are created and dismantled as needed. Your changes are not persistent. If you make changes directly within the container and that container is destroyed and rebuilt, your changes will no longer exist.

For help with usage, options, and to see examples:

$ oc rsh -h

A template describes a set of objects that can be parameterized and processed to produce a list of objects for creation by OpenShift Enterprise. A template can be processed to create anything you have permission to create within a project, for example services, build configurations, and deployment configurations. A template may also define a set of labels to apply to every object defined in the template.

You can create a list of objects from a template using the CLI or, if a template has been uploaded to your project or the global template library, using the web console.

If you have a JSON or YAML file that defines a template, for example as seen in this example, you can upload the template to projects using the CLI. This saves the template to the project for repeated use by any user with appropriate access to that project. Instructions on writing your own templates are provided later in this topic.

To upload a template to your current project’s template library, pass the JSON or YAML file with the following command:

$ oc create -f <filename>

You can upload a template to a different project using the -n option with the name of the project:

$ oc create -f <filename> -n <project>

The template is now available for selection using the web console or the CLI.

To create the objects from an uploaded template using the web console:

You can use the CLI to process templates and use the configuration that is generated to create objects.

$ oc process -f <filename> -l name=otherLabel

$ oc process --parameters -f <filename>

$ oc process --parameters -n <project> <template_name>

$ oc process --parameters -n openshift rails-postgresql-example
NAME                         DESCRIPTION                                                                                              GENERATOR           VALUE
SOURCE_REPOSITORY_URL        The URL of the repository with your application source code                                                                  https://github.com/sclorg/rails-ex.git
SOURCE_REPOSITORY_REF        Set this to a branch name, tag or other ref of your repository if you are not using the default branch
CONTEXT_DIR                  Set this to the relative path to your project if it is not in the root of your repository
APPLICATION_DOMAIN           The exposed hostname that will route to the Rails service                                                                    rails-postgresql-example.openshiftapps.com
GITHUB_WEBHOOK_SECRET        A secret string used to configure the GitHub webhook                                                     expression          [a-zA-Z0-9]{40}
SECRET_KEY_BASE              Your secret key for verifying the integrity of signed cookies                                            expression          [a-z0-9]{127}
APPLICATION_USER             The application user that is used within the sample application to authorize access on pages                                 openshift
APPLICATION_PASSWORD         The application password that is used within the sample application to authorize access on pages                             secret
DATABASE_SERVICE_NAME        Database service name                                                                                                        postgresql
POSTGRESQL_USER              database username                                                                                        expression          user[A-Z0-9]{3}
POSTGRESQL_PASSWORD          database password                                                                                        expression          [a-zA-Z0-9]{8}
POSTGRESQL_DATABASE          database name                                                                                                                root
POSTGRESQL_MAX_CONNECTIONS   database max connections                                                                                                     10
POSTGRESQL_SHARED_BUFFERS    database shared buffers                                                                                                      12MB

$ oc process -f <filename>

$ oc process <template_name>

$ cat <first_template> <second_template> | oc process -f -

$ oc process -f <filename> | oc create -f -

$ oc process <template> | oc create -f -

$ oc process -f my-rails-postgresql \
    -v POSTGRESQL_USER=bob,POSTGRESQL_DATABASE=mydatabase

$ oc process -f my-rails-postgresql \
    -v POSTGRESQL_USER=bob,POSTGRESQL_DATABASE=mydatabase \
    | oc create -f -

You can edit a template that has already been uploaded to your project by using the following command:

$ oc edit template <template>

OpenShift Enterprise provides a number of default Instant App and Quickstart templates to make it easy to quickly get started creating a new application for different languages. Templates are provided for Rails (Ruby), Django (Python), Node.js, CakePHP (PHP), and Dancer (Perl). Your cluster administrator should have created these templates in the default, global openshift project so you have access to them. You can list the available default Instant App and Quickstart templates with:

$ oc get templates -n openshift

If they are not available, direct your cluster administrator to the Loading the Default Image Streams and Templates topic.

By default, the templates build using a public source repository on GitHub that contains the necessary application code. In order to be able to modify the source and build your own version of the application, you must:

By doing this, the build configuration created by the template will now point to your fork of the application code, and you can modify the code and rebuild the application at will. A walkthrough of this process using the web console is provided in Getting Started for Developers: Web Console.

You can define new templates to make it easy to recreate all the objects of your application. The template will define the objects it creates along with some metadata to guide the creation of those objects.

kind: "Template"
apiVersion: "v1"
metadata:
  name: "cakephp-mysql-example" 1
  annotations:
    description: "An example CakePHP application with a MySQL database" 2
    tags: "instant-app,php,cakephp,mysql" 3
    iconClass: "icon-php" 4

kind: "Template"
apiVersion: "v1"
...
labels:
  template: "cakephp-mysql-example" 1

kind: Template
apiVersion: v1
objects:
  - kind: BuildConfig
    apiVersion: v1
    metadata:
      name: cakephp-mysql-example
      annotations:
        description: Defines how to build the application
    spec:
      source:
        type: Git
        git:
          uri: "${SOURCE_REPOSITORY_URL}" 1
          ref: "${SOURCE_REPOSITORY_REF}"
        contextDir: "${CONTEXT_DIR}"
parameters:
  - name: SOURCE_REPOSITORY_URL 2
    description: The URL of the repository with your application source code 3
    value: https://github.com/sclorg/cakephp-ex.git 4
    required: true 5
  - name: GITHUB_WEBHOOK_SECRET
    description: A secret string used to configure the GitHub webhook
    generate: expression 6
    from: "[a-zA-Z0-9]{40}" 7

kind: "Template"
apiVersion: "v1"
objects:
  - kind: "Service" 1
    apiVersion: "v1"
    metadata:
      name: "cakephp-mysql-example"
      annotations:
        description: "Exposes and load balances the application pods"
    spec:
      ports:
        - name: "web"
          port: 8080
          targetPort: 8080
      selector:
        name: "cakephp-mysql-example"

$ oc export all --as-template=<template_name>

A build is the process of transforming input parameters into a resulting object. Most often, the process is used to transform source code into a runnable image.

Build configurations are characterized by a strategy and one or more sources. The strategy determines the aforementioned process, while the sources provide its input.

There are three build strategies:

And there are three types of build source:

It is up to each build strategy to consider or ignore a certain type of source, as well as to determine how it is to be used.

Binary and Git are mutually exclusive source types, while Dockerfile can be used by itself or together with Git and Binary.

A build configuration describes a single build definition and a set of triggers for when a new build should be created.

A build configuration is defined by a BuildConfig, which is a REST object that can be used in a POST to the API server to create a new instance. The following example BuildConfig results in a new build every time a Docker image tag or the source code changes:

kind: "BuildConfig"
apiVersion: "v1"
metadata:
  name: "ruby-sample-build" 1
spec:
  triggers: 2
    - type: "GitHub"
      github:
        secret: "secret101"
    - type: "Generic"
      generic:
        secret: "secret101"
    - type: "ImageChange"
  source: 3
    type: "Git"
    git:
      uri: "https://github.com/openshift/ruby-hello-world"
    dockerfile: "FROM openshift/ruby-22-centos7\nUSER example"
  strategy: 4
    type: "Source"
    sourceStrategy:
      from:
        kind: "ImageStreamTag"
        name: "ruby-20-centos7:latest"
  output: 5
    to:
      kind: "ImageStreamTag"
      name: "origin-ruby-sample:latest"

The following options are specific to the S2I build strategy.

strategy:
  type: "Source"
  sourceStrategy:
    from:
      kind: "ImageStreamTag"
      name: "builder-image:latest" 1
    forcePull: true 2

strategy:
  type: "Source"
  sourceStrategy:
    from:
      kind: "ImageStreamTag"
      name: "incremental-image:latest" 1
    incremental: true 2

strategy:
  type: "Source"
  sourceStrategy:
    from:
      kind: "ImageStreamTag"
      name: "builder-image:latest"
    scripts: "http://somehost.com/scripts_directory" 1

sourceStrategy:
...
  env:
    - name: "DISABLE_ASSET_COMPILATION"
      value: "true"

The following options are specific to the Docker build strategy.

strategy:
  type: Docker
  dockerStrategy:
    from:
      kind: "ImageStreamTag"
      name: "debian:latest"

strategy:
  type: Docker
  dockerStrategy:
    dockerfilePath: dockerfiles/app1/

strategy:
  type: "Docker"
  dockerStrategy:
    noCache: true

strategy:
  type: "Docker"
  dockerStrategy:
    forcePull: true 1

dockerStrategy:
...
  env:
    - name: "HTTP_PROXY"
      value: "http://myproxy.net:5187/"

The following options are specific to the Custom build strategy.

strategy:
  type: "Custom"
  customStrategy:
    exposeDockerSocket: true

strategy:
  type: "Custom"
  customStrategy:
    secrets:
      - secretSource: 1
          name: "secret1"
        mountPath: "/tmp/secret1" 2
      - secretSource:
          name: "secret2"
        mountPath: "/tmp/secret2"

strategy:
  type: "Custom"
  customStrategy:
    forcePull: true 1

customStrategy:
...
  env:
    - name: "HTTP_PROXY"
      value: "http://myproxy.net:5187/"

When the BuildConfig.spec.source.type is Git, a Git repository is required, and an inline Dockerfile is optional.

The source code is fetched from the location specified and, if the BuildConfig.spec.source.dockerfile field is specified, the inline Dockerfile replaces the one in the contextDir of the Git repository.

The source definition is part of the spec section in the BuildConfig:

source:
  type: "Git"
  git: 1
    uri: "https://github.com/openshift/ruby-hello-world"
    ref: "master"
  contextDir: "app/dir" 2
  dockerfile: "FROM openshift/ruby-22-centos7\nUSER example" 3

source:
  type: Git
  git:
    uri: "https://github.com/openshift/ruby-hello-world"
    httpProxy: http://proxy.example.com
    httpsProxy: https://proxy.example.com

$ ssh-keygen -t rsa -C "your_email@example.com"

When the BuildConfig.spec.source.type is Dockerfile, an inline Dockerfile is used as the build input, and no additional sources can be provided.

This source type is valid when the build strategy type is Docker or Custom.

The source definition is part of the spec section in the BuildConfig:

source:
  type: "Dockerfile"
  dockerfile: "FROM centos:7\nRUN yum install -y httpd" 1

When the BuildConfig.spec.source.type is Binary, the build will expect a binary as input, and an inline Dockerfile is optional.

The binary is generally assumed to be a tar, gzipped tar, or zip file depending on the strategy. For Docker builds, this is the build context and an optional Dockerfile may be specified to override any Dockerfile in the build context. For Source builds, this is assumed to be an archive as described above. For Source and Docker builds, if binary.asFile is set the build context will consist of a single file named by the value of binary.asFile. The contextDir field may be used when an archive is provided. Custom builds will receive this binary as input on standard input (stdin).

A binary source potentially extracts content (if asFile is not set), in which case contextDir allows changing to a subdirectory within the content before the build executes.

The source definition is part of the spec section in the BuildConfig:

source:
  git:
    uri: https://github.com/sclorg/nodejs-ex.git
  secrets:
    - secret:
        name: secret-npmrc
  type: Git

To include the secrets in a new build configuration, run the following command:

$ oc new-build openshift/nodejs-010-centos7~https://github.com/sclorg/nodejs-ex.git

During the build, the .npmrc file is copied into the directory where the source code is located. In case of the OpenShift Enterprise S2I builder images, this is the image working directory, which is set using the WORKDIR instruction in the Dockerfile. If you want to specify another directory, add a destinationDir to the secret definition:

source:
  git:
    uri: https://github.com/sclorg/nodejs-ex.git
  secrets:
    - secret:
        name: secret-npmrc
      destinationDir: /etc
  type: Git

You can also specify the destination directory when creating a new build configuration:

$ oc new-build openshift/nodejs-010-centos7~https://github.com/sclorg/nodejs-ex.git

Manually start a new build from an existing build configuration in your current project using the following command:

$ oc start-build <buildconfig_name>

Re-run a build using the --from-build flag:

$ oc start-build --from-build=<build_name>

Specify the --follow flag to stream the build’s logs in stdout:

$ oc start-build <buildconfig_name> --follow

Specify the --env flag to set any desired environment variable for the build:

$ oc start-build <buildconfig_name> --env=<key>=<value>

Rather than relying on a Git source pull or a Dockerfile for a build, you can can also start a build by directly pushing your source, which could be the contents of a Git or SVN working directory, a set of prebuilt binary artifacts you want to deploy, or a single file. This can be done by specifying one of the following options for the start-build command:

When passing any of these options directly to the build, the contents are streamed to the build and override the current build source settings.

For example, the following command sends the contents of a local Git repository as an archive from the tag v2 and starts a build:

$ oc start-build hello-world --from-repo=../hello-world --commit=v2

Manually cancel a build using the web console, or with the following CLI command:

$ oc cancel-build <build_name>

You can view build details using the web console or the following CLI command:

$ oc describe build <build_name>

The output of the describe command includes details such as the build source, strategy, and output destination. If the build uses the Docker or Source strategy, it will also include information about the source revision used for the build: commit ID, author, committer, and message.

You can access build logs using the web console or the CLI.

To stream the logs using the build directly:

$ oc logs -f build/<build_name>

To stream the logs of the latest build for a build configuration:

$ oc logs -f bc/<buildconfig_name>

To return the logs of a given version build for a build configuration:

$ oc logs --version=<number> bc/<buildconfig_name>

Log Verbosity

To enable more verbose output, pass the BUILD_LOGLEVEL environment variable as part of the sourceStrategy or dockerStrategy in a BuildConfig:

sourceStrategy:
...
  env:
    - name: "BUILD_LOGLEVEL"
      value: "2" 1

Available log levels for Source builds are as follows:

When defining a BuildConfig, you can define its maximum duration by setting the completionDeadlineSeconds field. It is specified in seconds and is not set by default. When not set, there is no maximum duration enforced.

The maximum duration is counted from the time when a build pod gets scheduled in the system, and defines how long it can be active, including the time needed to pull the builder image. After reaching the specified timeout, the build is terminated by OpenShift Enterprise.

The following example shows the part of a BuildConfig specifying completionDeadlineSeconds field for 30 minutes:

spec:
  completionDeadlineSeconds: 1800

When defining a BuildConfig, you can define triggers to control the circumstances in which the BuildConfig should be run. The following build triggers are available:

type: "GitHub"
github:
  secret: "secret101"

http://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/github

type: "Generic"
generic:
  secret: "secret101"

http://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/generic

type: "git"
git:
  uri: "<url to git repository>"
  ref: "<optional git reference>"
  commit: "<commit hash identifying a specific git commit>"
  author:
    name: "<author name>"
    email: "<author e-mail>"
  committer:
    name: "<committer name>"
    email: "<committer e-mail>"
  message: "<commit message>"

$ oc describe bc <name>

strategy:
  type: "Source"
  sourceStrategy:
    from:
      kind: "DockerImage"
      name: "172.30.17.3:5001/mynamespace/ruby-20-centos7:immutableid"

  type: "ConfigChange"

Supply the .dockercfg file with valid Docker Registry credentials in order to push the output image into a private Docker Registry or pull the builder image from the private Docker Registry that requires authentication. For the OpenShift Enterprise Docker Registry, you don’t have to do this because secrets are generated automatically for you by OpenShift Enterprise.

The .dockercfg JSON file is found in your home directory by default and has the following format:

auths:
  https://index.docker.io/v1/: 1
    auth: "YWRfbGzhcGU6R2labnRib21ifTE=" 2
    email: "user@example.com" 3

You can define multiple Docker registry entries in this file. Alternatively, you can also add authentication entries to this file by running the docker login command. The file will be created if it does not exist. Kubernetes provides secret objects, which are used to store your configuration and passwords.

Docker and Source builds result in the creation of a new Docker image. The image is then pushed to the registry specified in the output section of the Build specification.

If the output kind is ImageStreamTag, then the image will be pushed to the integrated OpenShift Enterprise registry and tagged in the specified image stream. If the output is of type DockerImage, then the name of the output reference will be used as a Docker push specification. The specification may contain a registry or will default to DockerHub if no registry is specified. If the output section of the build specification is empty, then the image will not be pushed at the end of the build.

output:
  to:
    kind: "ImageStreamTag"
    name: "sample-image:latest"

output:
  to:
    kind: "DockerImage"
    name: "my-registry.mycompany.com:5000/myimages/myimage:tag"

It is not recommended to store binary files in a source repository. Therefore, you may find it necessary to define a build which pulls additional files (such as Java .jar dependencies) during the build process. How this is done depends on the build strategy you are using.

For a Source build strategy, you must put appropriate shell commands into the assemble script:

#!/bin/sh
APP_VERSION=1.0
wget http://repository.example.com/app/app-$APP_VERSION.jar -O app.jar

#!/bin/sh
exec java -jar app.jar

For a Docker build strategy, you must modify the Dockerfile and invoke shell commands with the RUN instruction:

FROM jboss/base-jdk:8

ENV APP_VERSION 1.0
RUN wget http://repository.example.com/app/app-$APP_VERSION.jar -O app.jar

EXPOSE 8080
CMD [ "java", "-jar", "app.jar" ]

In practice, you may want to use an environment variable for the file location so that the specific file to be downloaded can be customized using an environment variable defined on the BuildConfig, rather than updating the assemble script or Dockerfile.

You can choose between different methods of defining environment variables:

By default, builds are completed by pods using unbound resources, such as memory and CPU. These resources can be limited by specifying resource limits in a project’s default container limits.

You can also limit resource use by specifying resource limits as part of the build configuration. In the following example, each of the resources, cpu, and memory parameters are optional:

apiVersion: "v1"
kind: "BuildConfig"
metadata:
  name: "sample-build"
spec:
  resources:
    limits:
      cpu: "100m" 1
      memory: "256Mi" 2

However, if a quota has been defined for your project, one of the following two items is required:

Otherwise, build pod creation will fail, citing a failure to satisfy quota.

Each container running on a node consumes compute resources.

When authoring your pod, you can optionally specify how much CPU and memory (RAM) each container needs in order to better schedule pods in the cluster and ensure satisfactory performance.

Compute resources are measurable quantities which can be requested, allocated, and consumed.

CPU is measured in units called millicores. Each node in the cluster introspects the operating system to determine the amount of CPU cores on the node and then multiples that value by 1000 to express its total capacity. For example, if a node has 2 cores, the node’s CPU capacity would be represented as 2000m. If you wanted to use a 1/10 of a single core, you would represent that as 100m.

Memory is measured in bytes. In addition, it may be used with SI suffices (E, P, T, G, M, K, m) or their power-of-two-equivalents (Ei, Pi, Ti, Gi, Mi, Ki).

Each container in a pod may specify the amount of CPU it requests on a node. CPU requests are used by the scheduler to find a node with an appropriate fit for your container. The CPU request represents a minimum amount of CPU that your container may consume, but if there is no contention for CPU, it may burst to use as much CPU as is available on the node. If there is CPU contention on the node, CPU requests provide a relative weight across all containers on the system for how much CPU time the container may use. On the node, CPU requests map to Kernel CFS shares to enforce this behavior.

Each container in a pod may specify the amount of CPU it is limited to use on a node. CPU limits are used to control the maximum amount of CPU that your container may use independent of contention on the node. If a container attempts to use more than the specified limit, the system will throttle the container. This allows your container to have a consistent level of service independent of the number of pods scheduled to the node.

By default, a container is able to consume as much memory on the node as possible. In order to improve placement of pods in the cluster, it is recommended to specify the amount of memory required for a container to run. The scheduler will then take available node memory capacity into account prior to binding your pod to a node. A container is still able to consume as much memory on the node as possible even when specifying a request.

If you specify a memory limit, you can constrain the amount of memory your container can use. For example, if you specify a limit of 200Mi, your container will be limited to using that amount of memory on the node, and if it exceeds the specified memory limit, it will be terminated and potentially restarted dependent upon the container restart policy.

A compute resource is classified with a quality of service based on the specified request and limit value.

A BestEffort quality of service is provided when a request and limit are not specified.

A Burstable quality of service is provided when a request is specified that is less than an optionally specified limit.

A Guaranteed quality of service is provided when a limit is specified that is equal to an optionally specified request.

A container may have a different quality of service for each compute resource. For example, a container can have Burstable CPU and Guaranteed memory qualities of service.

The quality of service has an impact on what happens if the resource is compressible or not.

CPU is a compressible resource. A BestEffort CPU container will be able to consume as much CPU as is available on a node with the lowest priority. A Burstable CPU container is guaranteed to get the minimum amount of CPU requested, but it may or may not get additional CPU time. Excess CPU resources are distributed based on the amount requested across all containers on the node. A Guaranteed CPU container is guaranteed to get the amount requested and no more even if there are additional CPU cycles available. This provides a consistent level of performance independent of other activity on the node.

Memory is an incompressible resource. A BestEffort memory container will be able to consume as much memory as is available on the node, but the scheduler is subject to placing that container on a node with too small of an amount of memory to meet a need. In addition, a BestEffort memory container has the greatest chance of being killed if there is an out of memory event on the node. A Burstable memory container will be scheduled on the node to get the amount of memory requested, but it may consume more. If there is an out of memory event on the node, containers of this type will be killed after BestEffort containers when attempting to recover memory. A Guaranteed memory container will get the amount of memory requested, but no more. In the event of a system out of memory event, it will only be killed if there are no more BestEffort or Burstable memory containers on the system.

pod.yaml

apiVersion: v1
kind: Pod
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        cpu: 100m 1
        memory: 200Mi 2
      limits:
        cpu: 200m 3
        memory: 400Mi 4

To specify compute resources via the CLI:

$ oc run nginx --image=nginx --limits=cpu=200m,memory=400Mi --requests=cpu=100m,memory=200Mi

To view compute resources for a pod:

$ oc describe pod nginx-tfjxt
Name:       nginx-tfjxt
Namespace:      default
Image(s):     nginx
Node:       /
Labels:       run=nginx
Status:       Pending
Reason:
Message:
IP:
Replication Controllers:  nginx (1/1 replicas created)
Containers:
  nginx:
    Container ID:
    Image:    nginx
    Image ID:
    QoS Tier:
      cpu:  Burstable
      memory: Burstable
    Limits:
      cpu:  200m
      memory: 400Mi
    Requests:
      cpu:    100m
      memory:   200Mi
    State:    Waiting
    Ready:    False
    Restart Count:  0
    Environment Variables:

A deployment in OpenShift Enterprise is a replication controller based on a user defined template called a deployment configuration. Deployments are created manually or in response to triggered events.

The deployment system provides:

The deployment configuration contains a version number that is incremented each time a new deployment is created from that configuration. In addition, the cause of the last deployment is added to the configuration.

A deployment configuration consists of the following key parts:

Deployment configurations are deploymentConfig OpenShift Enterprise API resources which can be managed with the oc command like any other resource. The following is an example of a deploymentConfig resource:

kind: "DeploymentConfig"
apiVersion: "v1"
metadata:
  name: "frontend"
spec:
  template: 1
    metadata:
      labels:
        name: "frontend"
    spec:
      containers:
        - name: "helloworld"
          image: "openshift/origin-ruby-sample"
          ports:
            - containerPort: 8080
              protocol: "TCP"
  replicas: 5 2
  selector:
    name: "frontend"
  triggers:
    - type: "ConfigChange" 3
    - type: "ImageChange" 4
      imageChangeParams:
        automatic: true
        containerNames:
          - "helloworld"
        from:
          kind: "ImageStreamTag"
          name: "origin-ruby-sample:latest"
  strategy: 5
    type: "Rolling"

You can start a new deployment manually using the web console, or from the CLI:

$ oc deploy <deployment_config> --latest

To get basic information about recent deployments:

$ oc deploy <deployment_config>

This will show details about the latest and recent deployments, including any currently running deployment.

For more detailed information about a deployment configuration and the latest deployment:

$ oc describe dc <deployment_config>

To cancel a running or stuck deployment:

$ oc deploy <deployment_config> --cancel

To retry the last failed deployment:

$ oc deploy <deployment_config> --retry

If the last deployment didn’t fail, the command will display a message and the deployment will not be retried.

Rollbacks revert an application back to a previous deployment and can be performed using the REST API, the CLI, or the web console.

To rollback to the last successful deployment:

$ oc rollback <deployment_config>

The deployment configuration’s template will be reverted to match the deployment specified in the rollback command, and a new deployment will be started.

Image change triggers on the deployment configuration are disabled as part of the rollback to prevent unwanted deployments soon after the rollback is complete. To re-enable the image change triggers:

$ oc deploy <deployment_config> --enable-triggers

To roll back to a specific version:

$ oc rollback <deployment_config> --to-version=1

To see what the rollback would look like without performing the rollback:

$ oc rollback <deployment_config> --dry-run

To view the logs of the latest deployment for a given deployment configuration:

$ oc logs dc <deployment_config> [--follow]

Logs can be retrieved either while the deployment is running or if it has failed. If the deployment was successful, there will be no logs to view.

You can also view logs from older deployments:

$ oc logs --version=1 dc <deployment_config>

This command returns the logs from the first deployment of the provided deployment configuration, if and only if that deployment exists (i.e., it has failed and has not been manually deleted or pruned).

A deployment configuration can contain triggers, which drive the creation of new deployments in response to events, only inside OpenShift Enterprise at the moment.

triggers:
  - type: "ConfigChange"

triggers:
  - type: "ImageChange"
    imageChangeParams:
      automatic: true 1
      from:
        kind: "ImageStreamTag"
        name: "origin-ruby-sample:latest"
      containerNames:
        - "helloworld"

A deployment strategy determines the deployment process, and is defined by the deployment configuration. Each application has different requirements for availability (and other considerations) during deployments. OpenShift Enterprise provides strategies to support a variety of deployment scenarios.

A deployment strategy uses readiness checks to determine if a new pod is ready for use. If a readiness check fails, the deployment is stopped.

The Rolling strategy is the default strategy used if no strategy is specified on a deployment configuration.

strategy:
  type: Rolling
  rollingParams:
    timeoutSeconds: 120 1
    maxSurge: "20%" 2
    maxUnavailable: "10%" 3
    pre: {} 4
    post: {}

strategy:
  type: Recreate
  recreateParams: 1
    pre: {} 2
    post: {}

strategy:
  type: Custom
  customParams:
    image: organization/strategy
    command: [ "command", "arg1" ]
    environment:
      - name: ENV_1
        value: VALUE_1

The Recreate and Rolling strategies support lifecycle hooks, which allow behavior to be injected into the deployment process at predefined points within the strategy:

The following is an example of a "pre" lifecycle hook:

pre:
  failurePolicy: Abort
  execNewPod: {} 1

Every hook has a failurePolicy, which defines the action the strategy should take when a hook failure is encountered:

Hooks have a type-specific field that describes how to execute the hook. Currently pod-based hooks are the only supported hook type, specified by the execNewPod field.

kind: DeploymentConfig
apiVersion: v1
metadata:
  name: frontend
spec:
  template:
    metadata:
      labels:
        name: frontend
    spec:
      containers:
        - name: helloworld
          image: openshift/origin-ruby-sample
  replicas: 5
  selector:
    name: frontend
  strategy:
    type: Rolling
    rollingParams:
      pre:
        failurePolicy: Abort
        execNewPod:
          containerName: helloworld 1
          command: [ "/usr/bin/command", "arg1", "arg2" ] 2
          env: 3
            - name: CUSTOM_VAR1
              value: custom_value1
          volumes:
            - data 4

A deployment is completed by a pod that consumes resources (memory and CPU) on a node. By default, pods consume unbounded node resources. However, if a project specifies default container limits, then pods consume resources up to those limits.

You can also limit resource use by specifying resource limits as part of the deployment strategy. Deployment resources can be used with the Recreate, Rolling, or Custom deployment strategies.

In the following example, each of resources, cpu, and memory is optional:

type: "Recreate"
resources:
  limits:
    cpu: "100m" 1
    memory: "256Mi" 2

However, if a quota has been defined for your project, one of the following two items is required:

Otherwise, deploy pod creation will fail, citing a failure to satisfy quota.

In addition to rollbacks, you can exercise fine-grained control over the number of replicas from the web console, or by using the oc scale command. For example, the following command sets the replicas in the deployment configuration frontend to 3.

$ oc scale dc frontend --replicas=3

The number of replicas eventually propagates to the desired and current state of the deployment configured by the deployment configuration frontend.

You can use node selectors in conjunction with labeled nodes to control pod placement.

Cluster administrators can set the default node selector for your project in order to restrict pod placement to specific nodes. As an OpenShift developer, you can set a node selector on a pod configuration to restrict nodes even further.

To add a node selector when creating a pod, edit the pod configuration, and add the nodeSelector value. This can be added to a single pod configuration, or in a pod template:

apiVersion: v1
kind: Pod
spec:
  nodeSelector:
    disktype: ssd
...

Pods created when the node selector is in place are assigned to nodes with the specified labels.

The labels specified here are used in conjunction with the labels added by a cluster administrator. For example, if a project has the type=user-node and region=east labels added to a project by the cluster administrator, and you add the above disktype: ssd label to a pod, the pod will only ever be scheduled on nodes that have all three labels.

An OpenShift Enterprise route exposes a service at a host name, like www.example.com, so that external clients can reach it by name.

DNS resolution for a host name is handled separately from routing; your administrator may have configured a cloud domain that will always correctly resolve to the OpenShift Enterprise router, or if using an unrelated host name you may need to modify its DNS records independently to resolve to the router.

Tooling for creating routes is developing. In the web console, they are displayed but there is not yet a method to create them. Using the CLI, you currently can create only an unsecured route:

$ oc expose service/<name> --hostname=<www.example.com>

The new route inherits the name from the service unless you specify one.

apiVersion: v1
kind: Route
metadata:
  name: route-name
spec:
  host: www.example.com
  to:
    kind: Service
    name: service-name

Unsecured routes are the default configuration, and are therefore the simplest to set up. However, secured routes offer security for connections to remain private. To create a secured HTTPS route encrypted with a key and certificate (PEM-format files which you must generate and sign separately), you must edit the unsecured route to add TLS termination.

$ oc edit route/<name>

apiVersion: v1
kind: Route
metadata:
  name: route-name
spec:
  host: www.example.com
  to:
    kind: Service
    name: service-name
  tls:
    termination: edge
    key: |-
      -----BEGIN PRIVATE KEY-----
      [...]
      -----END PRIVATE KEY-----
    certificate: |-
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
    caCertificate: |-
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----

You can specify a secured route without a key and certificate, in which case the router’s default certificate will be used for TLS termination.

Further information on all types of TLS termination as well as path-based routing are available in the Architecture section.

Many OpenShift Enterprise applications use external resources, such as external databases, or an external SaaS endpoint. These external resources can be modeled as native OpenShift Enterprise services, so that applications can work with them as they would any other internal service.

One of the most common types of external services is an external database. To support an external database, an application needs:

The solution for integrating with an external database includes:

The following steps outline a scenario for integrating with an external MySQL database:

External Database Environment Variables

Using an external service in your application is similar to using an internal service. Your application will be assigned environment variables for the service and the additional environment variables with the credentials described in the previous step. For example, a MySQL container receives the following environment variables:

The application is responsible for reading the coordinates and credentials for the service from the environment and establishing a connection with the database via the service.

A common type of external service is an external SaaS endpoint. To support an external SaaS provider, an application needs:

The following steps outline a scenario for integrating with an external SaaS provider:

External SaaS Provider Environment Variables

Similarly, when using an internal service, your application is assigned environment variables for the service and the additional environment variables with the credentials described in the above steps. In the above example, the container receives the following environment variables:

The application reads the coordinates and credentials for the service from the environment and establishes a connection with the service.

The Secret object type provides a mechanism to hold sensitive information such as passwords, OpenShift Enterprise client config files, dockercfg files, private source repository credentials, etc. Secrets decouple sensitive content from the pods that use it and can be mounted into containers using a volume plug-in or used by the system to perform actions on behalf of a pod. This topic discusses important properties of secrets and provides an overview on how developers can use them.

apiVersion: "v1"
kind: "Secret"
metadata:
  name: "mysecret"
namespace: "myns"
data: 1
  username: "dmFsdWUtMQ0K"
  password: "dmFsdWUtMg0KDQo="

Key properties include:

When creating secrets:

$ oc create -f secret.json

To use a secret, a pod needs to reference the secret. A secret can be used with a pod in two ways: either as files in a volume mounted on one or more of its containers, or used by kubelet when pulling images for the pod.

Volume type secrets write data into the container as a file using the volume mechanism. imagePullSecrets use service accounts for the automatic injection of the secret into all pods in a namespaces.

When a template contains a secret definition, the only way for the template to use the provided secret is to ensure that the secret volume sources are validated and that the specified object reference actually points to an object of type Secret. Therefore, a secret needs to be created before any pods that depend on it. The most effective way to ensure this is to have it get injected automatically through the use of a service account.

Secret API objects reside in a namespace. They can only be referenced by pods in that same namespace.

Individual secrets are limited to 1MB in size. This is to discourage the creation of large secrets that would exhaust apiserver and kubelet memory. However, creation of a number of smaller secrets could also exhaust memory.

Docker registries can be secured to prevent unauthorized parties from accessing certain images. If you are using OpenShift’s integrated Docker registry and are pulling from image streams located in the same project, then your pod’s service account should already have permissions and no additional action should be required. If this is not the case, then additional configuration steps are required.

OpenShift’s integrated Docker registry authenticates using the same tokens as the OpenShift API. To perform a docker login against the integrated registry, you can choose any user name and email, but the password must be a valid OpenShift token. In order to pull an image, the authenticated user must have get rights on the requested imagestreams/layers. In order to push an image, the authenticated user must have update rights on the requested imagestreams/layers.

By default, all service accounts in a project have rights to pull any image in the same project, and the builder service account has rights to push any image in the same project.

$ oc policy add-role-to-user \
    system:image-puller system:serviceaccount:project-a:default \
    --namespace=project-b

$ oc policy add-role-to-group \
    system:image-puller system:serviceaccounts:project-a \
    --namespace=project-b

To pull a secured Docker image that is not from OpenShift’s integrated registry, you must create a secret and add it to your service account.

If you already have a file for the secured registry, you can create a secret from that file by running the following for .docker/config.json:

# oc secrets new <pull_secret_name> \
     .dockerconfigjson=path/to/.docker/config.json

Alternatively, run the following for .dockercfg:

$ oc secrets new <pull_secret_name> \
    .dockercfg=<path/to/.dockercfg>

If you do not already have a file for the secured registry, you can create a secret by running:

$ oc secrets new-dockercfg <pull_secret_name> \
    --docker-server=<registry_server> --docker-username=<user_name> \
    --docker-password=<password> --docker-email=<email>

To use a secret for pulling images for pods, you must add the secret to your service account. The name of the service account in this example should match the name of the service account the pod will use; default is the default service account:

$ oc secrets add serviceaccount/default secrets/<pull_secret_name> --for=pull

To use a secret for pushing and pulling build images, the secret must be mountable inside of a pod. You can do this by running:

$ oc secrets add serviceaccount/builder secrets/<pull_secret_name>

A LimitRange object enumerates compute resource constraints in a project at the pod and container level. This allows you to specify the amount of resources that a pod or container can consume.

All resource create and modification requests are evaluated against each LimitRange object in the project. If the resource violates any of the enumerated constraints, then the resource is rejected. If the resource does not set an explicit value, and if the constraint supports a default value, then the default value is applied to the resource.

apiVersion: "v1"
kind: "LimitRange"
metadata:
  name: "limits" 1
spec:
  limits:
    -
      type: "Pod"
      max:
        cpu: "2" 2
        memory: "1Gi" 3
      min:
        cpu: "200m" 4
        memory: "6Mi" 5
    -
      type: "Container"
      max:
        cpu: "2" 6
        memory: "1Gi" 7
      min:
        cpu: "100m" 8
        memory: "4Mi" 9
      default:
        cpu: "300m" 10
        memory: "200Mi" 11
      defaultRequest:
        cpu: "200m" 12
        memory: "100Mi" 13
      maxLimitRequestRatio:
        cpu: "10" 14

To apply a limit range to a project, create a limit range object definition on your file system to your desired specifications, then run:

$ oc create -f <limit_range_file>

To view limits enforced in a project:

$ oc get limits
NAME
limits

$ oc describe limits limits
Name:        limits
Namespace:   default
Type         Resource  Min Max Request Limit Limit/Request
----         --------  --- --- ------- ----- -------------
Pod          memory    6Mi 1Gi -       -     -
Pod          cpu       200m  2 -       -     -
Container    cpu       100m  2 200m    300m  10
Container    memory    4Mi 1Gi 100Mi   200Mi -

Remove any active limit range to no longer enforce the limits of a project:

$ oc delete limits <limit_name>

A ResourceQuota object enumerates hard resource usage limits per project. It limits the total number of a particular type of object that may be created in a project, and the total amount of compute resources that may be consumed by resources in that project.

The following describes the set of limits that may be enforced by a ResourceQuota.

After a project quota is first created, the project restricts the ability to create any new resources that may violate a quota constraint until it has calculated updated usage statistics.

Once a quota is created and usage statistics are up-to-date, the project accepts the creation of new content. When you create or modify resources, your quota usage is incremented immediately upon the request to create or modify the resource. When you delete a resource, your quota use is decremented during the next full recalculation of quota statistics for the project. When you delete resources, a configurable amount of time determines how long it takes to reduce quota usage statistics to their current observed system value.

If project modifications exceed a quota usage limit, the server denies the action, and an appropriate error message is returned to the end-user explaining the quota constraint violated, and what their currently observed usage stats are in the system.

resource-quota.json

{
  "apiVersion": "v1",
  "kind": "ResourceQuota",
  "metadata": {
    "name": "quota" 1
  },
  "spec": {
    "hard": {
      "memory": "1Gi", 2
      "cpu": "20", 3
      "pods": "10", 4
      "services": "5", 5
      "replicationcontrollers":"5", 6
      "resourcequotas":"1" 7
    }
  }
}

To apply a quota to a project:

$ oc create -f resource-quota.json

To view usage statistics related to any hard limits defined in your quota:

$ oc get quota
NAME
quota
$ oc describe quota quota
Name:                   quota
Resource                Used    Hard
--------                ----    ----
cpu                     5       20
memory                  500Mi   1Gi
pods                    5       10
replicationcontrollers  5       5
resourcequotas          1       1
services                3       5

When a set of resources are deleted, the synchronization timeframe of resources is determined by the resource-quota-sync-period setting in the /etc/origin/master/master-config.yaml file. Before your quota usage is restored, you may encounter problems when attempting to reuse the resources. Change the resource-quota-sync-period setting to have the set of resources regenerate at the desired amount of time (in seconds) and for the resources to be available again:

kubernetesMasterConfig:
  apiLevels:
  - v1beta3
  - v1
  apiServerArguments: null
  controllerArguments:
    resource-quota-sync-period:
      - "10s"

Adjusting the regeneration time can be helpful for creating resources and determining resource usage when automation is used.

If a quota has been defined for your project, see Deployment Resources for considerations on any deployment configurations.

A horizontal pod autoscaler, defined by a HorizontalPodAutoscaler object, specifies how the system should automatically increase or decrease the scale of a replication controller or deployment configuration, based on metrics collected from the pods that belong to that replication controller or deployment configuration.

In order to use horizontal pod autoscalers, your cluster administrator must have properly configured cluster metrics.

The following metrics are supported by horizontal pod autoscalers:

After a horizontal pod autoscaler is created, it begins attempting to query Heapster for metrics on the pods. It may take one to two minutes before Heapster obtains the initial metrics.

After metrics are available in Heapster, the horizontal pod autoscaler computes the ratio of the current metric utilization with the desired metric utilization, and scales up or down accordingly. The scaling will occur at a regular interval, but it may take one to two minutes before metrics make their way into Heapster.

For replication controllers, this scaling corresponds directly to the replicas of the replication controller. For deployment configurations, scaling corresponds directly to the replica count of the deployment configuration. Note that autoscaling applies only to the latest deployment in the Complete phase.

To create a horizontal pod autoscaler, first define it in a file. For example:

apiVersion: extensions/v1beta1
kind: HorizontalPodAutoscaler
metadata:
  name: frontend-scaler 1
spec:
  scaleRef:
    kind: DeploymentConfig 2
    name: frontend 3
    apiVersion: v1 4
    subresource: scale
  minReplicas: 1 5
  maxReplicas: 10 6
  cpuUtilization:
    targetPercentage: 80 7

Save your definition to a file, such as scaler.yaml, then use the CLI to create the object:

$ oc create -f scaler.yaml

To view the status of a horizontal pod autoscaler:

$ oc get hpa
NAME              REFERENCE                                 TARGET    CURRENT   MINPODS        MAXPODS   AGE
frontend-scaler   DeploymentConfig/default/frontend/scale   80%       79%       1              10        8d

$ oc describe hpa frontend-scaler
Name:                           frontend-scaler
Namespace:                      default
Labels:                         <none>
CreationTimestamp:              Mon, 26 Oct 2015 21:13:47 -0400
Reference:                      DeploymentConfig/default/frontend/scale
Target CPU utilization:         80%
Current CPU utilization:        79%
Min pods:                       1
Max pods:                       10

Containers are not persistent by default; on restart, their contents are cleared. Volumes are mounted file systems available to pods and their containers which may be backed by a number of host-local or network attached storage endpoints.

The simplest volume type is EmptyDir, which is a temporary directory on a single machine. Administrators may also allow you to request a persistent volume that is automatically attached to your pods.

You can use the CLI command oc volume to add, update, or remove volumes and volume mounts for any object that has a pod template like replication controllers or deployment configurations. You can also list volumes in pods or any object that has a pod template.

The oc volume command uses the following general syntax:

$ oc volume <object_selection> <operation> <mandatory_parameters> <optional_parameters>

This topic uses the form <object_type>/<name> for <object_selection> in later examples, however you can choose one of the following options:

The <operation> can be one of --add, --remove, or --list.

Any <mandatory_parameters> or <optional_parameters> are specific to the selected operation and are discussed in later sections.

To add a volume, a volume mount, or both to pod templates:

$ oc volume <object_type>/<name> --add [options]

Examples

Add a new volume source emptyDir to deployment configuration registry:

$ oc volume dc/registry --add

Add volume v1 with secret $ecret for replication controller r1 and mount inside the containers at /data:

$ oc volume rc/r1 --add --name=v1 --type=secret --secret-name='$ecret' --mount-path=/data

Add existing persistent volume v1 with claim name pvc1 to deployment configuration dc.json on disk, mount the volume on container c1 at /data, and update the deployment configuration on the server:

$ oc volume -f dc.json --add --name=v1 --type=persistentVolumeClaim \
  --claim-name=pvc1 --mount-path=/data --containers=c1

Add volume v1 based on Git repository https://github.com/namespace1/project1 with revision 5125c45f9f563 for all replication controllers:

$ oc volume rc --all --add --name=v1 \
  --source='{"gitRepo": {
                "repository": "https://github.com/namespace1/project1",
                "revision": "5125c45f9f563"
            }}'

Updating existing volumes or volume mounts is the same as adding volumes, but with the --overwrite option:

$ oc volume <object_type>/<name> --add --overwrite [options]

Examples

Replace existing volume v1 for replication controller r1 with existing persistent volume claim pvc1:

$ oc volume rc/r1 --add --overwrite --name=v1 --type=persistentVolumeClaim --claim-name=pvc1

Change deployment configuration d1 mount point to /opt for volume v1:

$ oc volume dc/d1 --add --overwrite --name=v1 --mount-path=/opt

To remove a volume or volume mount from pod templates:

$ oc volume <object_type>/<name> --remove [options]

Some examples:

Remove a volume v1 from deployment config d1:

$ oc volume dc/d1 --remove --name=v1

Unmount volume v1 from container c1 for deployment configuration d1 and remove the volume v1 if it is not referenced by any containers on d1:

$ oc volume dc/d1 --remove --name=v1 --containers=c1

Remove all volumes for replication controller r1:

$ oc volume rc/r1 --remove --confirm

To list volumes or volume mounts for pods or pod templates:

$ oc volume <object_type>/<name> --list [options]

List volume supported options:

Examples

List all volumes for pod p1:

$ oc volume pod/p1 --list

List volume v1 defined on all deployment configurations:

$ oc volume dc --all --name=v1

A PersistentVolume object is a storage resource in an OpenShift Enterprise cluster. Storage is provisioned by your cluster administrator by creating PersistentVolume objects from sources such as GCE Persistent Disk, AWS Elastic Block Store (EBS), and NFS mounts.

Storage can be made available to you by laying claims to the resource. You can make a request for storage resources using a PersistentVolumeClaim object; the claim is paired with a volume that generally matches your request.

You can request storage by creating PersistentVolumeClaim objects in your projects:

apiVersion: "v1"
kind: "PersistentVolumeClaim"
metadata:
  name: "claim1"
spec:
  accessModes:
    - "ReadWriteOnce"
  resources:
    requests:
      storage: "5Gi"
  volumeName: "pv0001"

A PersistentVolume is a specific resource. A PersistentVolumeClaim is a request for a resource with specific attributes, such as storage size. In between the two is a process that matches a claim to an available volume and binds them together. This allows the claim to be used as a volume in a pod. OpenShift Enterprise finds the volume backing the claim and mounts it into the pod.

You can tell whether a claim or volume is bound by querying using the CLI:

$ oc get pvc
NAME        LABELS    STATUS    VOLUME
claim1      map[]     Bound     pv0001

$ oc get pv
NAME                LABELS              CAPACITY            ACCESSMODES         STATUS    CLAIM
pv0001              map[]               5368709120          RWO                 Bound     yournamespace / claim1

A PersistentVolumeClaim is used by a pod as a volume. OpenShift Enterprise finds the claim with the given name in the same namespace as the pod, then uses the claim to find the corresponding volume to mount.

apiVersion: "v1"
kind: "Pod"
metadata:
  name: "mypod"
  labels:
    name: "frontendhttp"
spec:
  containers:
    -
      name: "myfrontend"
      image: "nginx"
      ports:
        -
          containerPort: 80
          name: "http-server"
      volumeMounts:
        -
          mountPath: "/var/www/html"
          name: "pvol"
  volumes:
    -
      name: "pvol"
      persistentVolumeClaim:
        claimName: "claim1"

You can use the CLI to execute remote commands in a container. This allows you to run general Linux commands for routine operations in the container.

Support for remote container command execution is built into the CLI:

$ oc exec <pod> [-c <container>] <command> [<arg_1> ... <arg_n>]

For example:

$ oc exec mypod date
Thu Apr  9 02:21:53 UTC 2015

Clients initiate the execution of a remote command in a container by issuing a request to the Kubernetes API server:

/proxy/minions/<node_name>/exec/<namespace>/<pod>/<container>?command=<command>

In the above URL:

For example:

/proxy/minions/node123.openshift.com/exec/myns/mypod/mycontainer?command=date

Additionally, the client can add parameters to the request to indicate if:

After sending an exec request to the API server, the client upgrades the connection to one that supports multiplexed streams; the current implementation uses SPDY.

The client creates one stream each for stdin, stdout, and stderr. To distinguish among the streams, the client sets the streamType header on the stream to one of stdin, stdout, or stderr.

The client closes all streams, the upgraded connection, and the underlying connection when it is finished with the remote command execution request.

You can use the CLI to copy local files to or from a remote directory in a container.

You can use the CLI to copy local files to or from a remote directory in a container. This is a useful tool for copying database archives to and from your pods for backup and restore purposes. It can also be used to copy source code changes into a running pod for development debugging, when the running pod supports hot reload of source files.

Support for copying local files to or from a container is built into the CLI:

$ oc rsync <source> <destination> [-c <container>]

For example:

# Copy local directory to a pod directory
$ oc rsync /home/user/source devpod1234:/src

# Copy pod directory to a local directory
$ oc rsync devpod1234:/src /home/user/source

The oc rsync command uses the local rsync command if present on the client’s machine. This requires that the remote container also have the rsync command. If rsync is not found locally or in the remote container, then a tar archive will be created locally and sent to the container where tar will be used to extract the files. If tar is not available in the remote container, then the copy will fail.

The source argument of the oc rsync command must point to either a local directory or a pod directory. Individual files are not currently supported. When specifying a pod directory the directory name must be prefixed with the pod name: <pod name>:<dir>. Just as with UNIX rsync, if the directory name ends in a path separator ('/'), only the contents of the directory are copied to the destination. Otherwise, the directory itself is copied to the destination with all its contents.

The destination argument of the oc rsync command must point to a directory. If the directory does not exist, but rsync is used for copy, the directory is created for you.

The --delete flag may be used to delete any files in the remote directory that are not in the local directory.

You can use the CLI to forward one or more local ports to a pod. This allows you to listen on a given or random port locally, and have data forwarded to and from given ports in the pod.

Support for port forwarding is built into the CLI:

$ oc port-forward <pod> [<local_port>:]<remote_port> [...[<local_port_n>:]<remote_port_n>]

The CLI listens on each local port specified by the user, forwarding via the protocol described below.

Ports may be specified using the following formats:

For example, to listen on ports 5000 and 6000 locally and forward data to and from ports 5000 and 6000 in the pod, run:

$ oc port-forward <pod> 5000 6000

To listen on port 8888 locally and forward to 5000 in the pod, run:

$ oc port-forward <pod> 8888:5000

To listen on a free port locally and forward to 5000 in the pod, run:

$ oc port-forward <pod> :5000

Or, alternatively:

$ oc port-forward <pod> 0:5000

Clients initiate port forwarding to a pod by issuing a request to the Kubernetes API server:

/proxy/minions/<node_name>/portForward/<namespace>/<pod>

In the above URL:

For example:

/proxy/minions/node123.openshift.com/portForward/myns/mypod

After sending a port forward request to the API server, the client upgrades the connection to one that supports multiplexed streams; the current implementation uses SPDY.

The client creates a stream with the port header containing the target port in the pod. All data written to the stream is delivered via the Kubelet to the target pod and port. Similarly, all data sent from the pod for that forwarded connection is delivered back to the same stream in the client.

The client closes all streams, the upgraded connection, and the underlying connection when it is finished with the port forwarding request.

There are two types of shared memory objects in Linux: System V and POSIX. The containers in a pod share the IPC namespace of the pod infrastructure container and so are able to share the System V shared memory objects. This document describes how they can also share POSIX shared memory objects.

POSIX shared memory requires that a tmpfs be mounted at /dev/shm. The containers in a pod do not share their mount namespaces so we use volumes to provide the same /dev/shm into each container in a pod. The following example shows how to set up POSIX shared memory between two containers.

---
apiVersion: v1
id: hello-openshift
kind: Pod
metadata:
  name: hello-openshift
  labels:
    name: hello-openshift
spec:
  volumes:                          1
    - name: dshm
      emptyDir:
        medium: Memory
  containers:
    - image: kubernetes/pause
      name: hello-container1
      ports:
        - containerPort: 8080
          hostPort: 6061
      volumeMounts:                 2
        - mountPath: /dev/shm
          name: dshm
    - image: kubernetes/pause
      name: hello-container2
      ports:
        - containerPort: 8081
          hostPort: 6062
      volumeMounts:                 3
        - mountPath: /dev/shm
          name: dshm

Create the pod using the shared-memory.yaml file:

$ oc create -f shared-memory.yaml

In software systems, components can become unhealthy due to transient issues (such as temporary connectivity loss), configuration errors, or problems with external dependencies. OpenShift Enterprise applications have a number of options to detect and handle unhealthy containers.

A probe is a Kuberbetes action that periodically performs diagnostics on a running container. Currently, two types of probes exist, each serving a different purpose:

The exact timing of a probe is controlled by two fields, both expressed in units of seconds:

Both probes can be configured in three ways:

HTTP Checks

The kubelet uses a web hook to determine the healthiness of the container. The check is deemed successful if the hook returns with 200 or 399. The following is an example of a readiness check using the HTTP checks method:

...
readinessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 15
  timeoutSeconds: 1
...

A HTTP check is ideal for complex applications that can return with a 200 status when completely initialized.

Container Execution Checks

The kubelet executes a command inside the container. Exiting the check with status 0 is considered a success. The following is an example of a liveness check using the container execution method:

...
livenessProbe:
  exec:
    command:
    - cat
    - /tmp/health
  initialDelaySeconds: 15
  timeoutSeconds: 1
...

TCP Socket Checks

The kubelet attempts to open a socket to the container. The container is only considered healthy if the check can establish a connection. The following is an example of a liveness check using the TCP socket check method:

...
livenessProbe:
  tcpSocket:
    port: 8080
  initialDelaySeconds: 15
  timeoutSeconds: 1
...

A TCP socket check is ideal for applications that do not start listening until initialization is complete.

For more information on health checks, see the Kubernetes documentation.

OpenShift events are modeled based on events that happen to API objects in an OpenShift cluster. Events allow OpenShift to record information about real-world events in a resource-agnostic manner. They also allow developers and administrators to consume information about system components in a unified way.

This section describes the events of OpenShift Enterprise.

It is common for containers to consume information about API objects. The downward API is a mechanism that allows containers to do this without coupling to OpenShift Enterprise. Containers can consume information from the downward API using environment variables or a volume plug-in.

Fields within the pod are selected using the FieldRef API type. FieldRef has two fields:

Currently, there are four valid selectors in the v1 API:

The apiVersion field, if not specified, defaults to the API version of the enclosing pod template.

In the future, more information, such as resource limits for pods and information about services, will be available using the downward API.

One mechanism for consuming the downward API is using a container’s environment variables. The EnvVar type’s valueFrom field (of type EnvVarSource) is used to specify that the variable’s value should come from a FieldRef source instead of the literal value specified by the value field. In the future, additional sources may be supported; currently the source’s fieldRef field is used to select a field from the downward API.

Only constant attributes of the pod can be consumed this way, as environment variables cannot be updated once a process is started in a way that allows the process to be notified that the value of a variable has changed. The fields supported using environment variables are:

For example:

Another mechanism for consuming the downward API is using a volume plug-in. The downward API volume plug-in creates a volume with configured fields projected into files. The metadata field of the VolumeSource API object is used to configure this volume. The plug-in supports the following fields:

spec:
  volumes:
    - name: podinfo
      metadata: 1
        items:  2
          - name: "labels" 3
            fieldRef:
              fieldPath: metadata.labels 4

For example:

You can set, unset or list environment variables in pods or pod templates using the oc env command.

OpenShift Enterprise provides the oc env command to set or unset environment variables for objects that have a pod template, such as replication controllers or deployment configurations. It can also list environment variables in pods or any object that has a pod template.

The oc env command uses the following general syntax:

$ oc env <object-selection> <environment-variables> [options]

There are several ways to express <object-selection>.

Supported common options for set, unset or list environment variables:

To set environment variables in the pod templates:

$ oc env <object-selection> KEY_1=VAL_1 ... KEY_N=VAL_N [<set-env-options>] [<common-options>]

Set environment options:

In the following example, both commands modify environment variable STORAGE in the deployment config registry. The first adds, with value /data. The second updates, with value /opt.

$ oc env dc/registry STORAGE=/data
$ oc env dc/registry --overwrite STORAGE=/opt

The following example finds environment variables in the current shell whose names begin with RAILS_ and adds them to the replication controller r1 on the server:

$ env | grep RAILS_ | oc env rc/r1 -e -

The following example does not modify the replication controller defined in file rc.json. Instead, it writes a YAML object with updated environment STORAGE=/local to new file rc.yaml.

$ oc env -f rc.json STORAGE=/opt -o yaml > rc.yaml

KUBERNETES_SERVICE_PORT=53
MYSQL_DATABASE=root
KUBERNETES_PORT_53_TCP=tcp://10.0.0.11:53
KUBERNETES_SERVICE_HOST=10.0.0.11

To unset environment variables in the pod templates:

$ oc env <object-selection> KEY_1- ... KEY_N- [<common-options>]

This example removes environment variables ENV1 and ENV2 from deployment config d1:

$ oc env dc/d1 ENV1- ENV2-

This removes environment variable ENV from all replication controllers:

$ oc env rc --all ENV-

This removes environment variable ENV from container c1 for replication controller r1:

$ oc env rc r1 --containers='c1' ENV-

To list environment variables in pods or pod templates:

$ oc env <object-selection> --list [<common-options>]

This example lists all environment variables for pod p1:

$ oc env pod/p1 --list

A job, in contrast to a replication controller, runs any number of pods to completion. A job tracks the overall progress of a task and updates its status with information about active, succeeded, and failed pods. Deleting a job will clean up any pods it created. Jobs are part of the extensions Kubernetes API, which can be managed with oc commands like other object types.

See the Kubernetes documentation for more information about jobs.

A job configuration consists of the following key parts:

The following is an example of a job resource:

apiVersion: extensions/v1beta1
kind: Job
metadata:
  name: pi
spec:
  selector:         1
    matchLabels:
      app: pi
  parallelism: 1    2
  completions: 1    3
  template:         4
    metadata:
      name: pi
      labels:
        app: pi
    spec:
      containers:
      - name: pi
        image: perl
        command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never

A job can be scaled up or down by changing the parallelism parameter accordingly. You can also use the oc scale command with the --replicas option, which, in the case of jobs, modifies the parallelism parameter.

The following command uses the example job above, and sets the parallelism parameter to three:

$ oc scale job pi --replicas=3