Chapter 3. Service Mesh installation
3.1. Preparing to install Red Hat OpenShift Service Mesh
Before you can install Red Hat OpenShift Service Mesh, review the installation activities, ensure that you meet the prerequisites:
Prerequisites
- Possess an active OpenShift Container Platform subscription on your Red Hat account. If you do not have a subscription, contact your sales representative for more information.
- Review the OpenShift Container Platform 4.2 overview.
Install OpenShift Container Platform 4.2.
Install the version of the OpenShift Container Platform command line utility (the
oc
client tool) that matches your OpenShift Container Platform version and add it to your path.- If you are using OpenShift Container Platform 4.2, see About the CLI.
3.1.1. Red Hat OpenShift Service Mesh supported configurations
The following are the only supported configurations for the Red Hat OpenShift Service Mesh:
- Red Hat OpenShift Container Platform version 4.x.
OpenShift Online and OpenShift Dedicated are not supported for Red Hat OpenShift Service Mesh 1.1.4.
- The deployment must be contained to a single OpenShift Container Platform cluster that is not federated.
- This release of Red Hat OpenShift Service Mesh is only available on OpenShift Container Platform x86_64.
- This release only supports configurations where all Service Mesh components are contained in the OpenShift cluster in which it operates. It does not support management of microservices that reside outside of the cluster, or in a multi-cluster scenario.
- This release only supports configurations that do not integrate external services such as virtual machines.
3.1.1.1. Supported configurations for Kiali on Red Hat OpenShift Service Mesh
- The Kiali observability console is only supported on the two most recent releases of the Chrome, Edge, Firefox, or Safari browsers.
3.1.1.2. Supported Mixer adapters
This release only supports the following Mixer adapter:
- 3scale Istio Adapter
3.1.2. Red Hat OpenShift Service Mesh installation activities
To install the Red Hat OpenShift Service Mesh Operator, you must first install these Operators:
- Elasticsearch - Based on the open source Elasticsearch project that enables you to configure and manage an Elasticsearch cluster for tracing and logging with Jaeger.
- Jaeger - based on the open source Jaeger project, lets you perform tracing to monitor and troubleshoot transactions in complex distributed systems.
- Kiali - based on the open source Kiali project, provides observability for your service mesh. By using Kiali you can view configurations, monitor traffic, and view and analyze traces in a single console.
After you install the Elasticsearch, Jaeger, and Kiali Operators, then you install the Red Hat OpenShift Service Mesh Operator. The Service Mesh Operator defines and monitors the ServiceMeshControlPlane
resources that manage the deployment, updating, and deletion of the Service Mesh components.
- Red Hat OpenShift Service Mesh - based on the open source Istio project, lets you connect, secure, control, and observe the microservices that make up your applications.
Please see configuring Elasticsearch for details on configuring the default Jaeger parameters for Elasticsearch in a production environment.
Next steps
- Install Red Hat OpenShift Service Mesh in your OpenShift Container Platform environment.
3.2. Installing Red Hat OpenShift Service Mesh
Installing the Service Mesh involves installing the Elasticsearch, Jaeger, Kiali and Service Mesh Operators, creating and managing a ServiceMeshControlPlane
resource to deploy the control plane, and creating a ServiceMeshMemberRoll
resource to specify the namespaces associated with the Service Mesh.
Mixer’s policy enforcement is disabled by default. You must enable it to run policy tasks. See Update Mixer policy enforcement for instructions on enabling Mixer policy enforcement.
Multi-tenant control plane installations are the default configuration starting with Red Hat OpenShift Service Mesh 1.0.
The Service Mesh documentation uses istio-system
as the example project, but you may deploy the service mesh to any project.
Prerequisites
- Follow the Preparing to install Red Hat OpenShift Service Mesh process.
-
An account with the
cluster-admin
role.
3.2.1. Installing the Operators from OperatorHub
The Service Mesh installation process uses the OperatorHub to install the ServiceMeshControlPlane
custom resource definition within the openshift-operators
project. The Red Hat OpenShift Service Mesh defines and monitors the ServiceMeshControlPlane
related to the deployment, update, and deletion of the control plane.
Starting with Red Hat OpenShift Service Mesh 1.1.4, you must install the Elasticsearch Operator, the Jaeger Operator, and the Kiali Operator before the Red Hat OpenShift Service Mesh Operator can install the control plane.
3.2.1.1. Installing the Elasticsearch Operator
You must install the Elasticsearch Operator for the Red Hat OpenShift Service Mesh Operator to install the control plane.
Do not install Community versions of the Operators. Community Operators are not supported.
Prerequisites
- Access to the OpenShift Container Platform web console.
Procedure
- Log in to the OpenShift Container Platform web console.
-
Navigate to Operators
OperatorHub. - Type Elasticsearch into the filter box to locate the Elasticsearch Operator.
- Click the Elasticsearch Operator to display information about the Operator.
- Click Install.
-
On the Create Operator Subscription page, select All namespaces on the cluster (default). This installs the Operator in the default
openshift-operators
project and makes the Operator available to all projects in the cluster. - In the Update Channel, select the most current version.
Select the Automatic Approval Strategy.
NoteThe Manual approval strategy requires a user with appropriate credentials to approve the Operator install and subscription process.
- Click Subscribe.
- The Installed Operators page displays the Elasticsearch Operator’s installation progress.
3.2.1.2. Installing the Jaeger Operator
You must install the Jaeger Operator for the Red Hat OpenShift Service Mesh Operator to install the control plane.
Do not install Community versions of the Operators. Community Operators are not supported.
Prerequisites
- Access to the OpenShift Container Platform web console.
- The Elasticsearch Operator must be installed.
Procedure
- Log in to the OpenShift Container Platform web console.
-
Navigate to Operators
OperatorHub. - Type Jaeger into the filter box to locate the Jaeger Operator.
- Click the Jaeger Operator provided by Red Hat to display information about the Operator.
- Click Install.
-
On the Create Operator Subscription page, select All namespaces on the cluster (default). This installs the Operator in the default
openshift-operators
project and makes the Operator available to all projects in the cluster. - Select the stable Update Channel.
Select the Automatic Approval Strategy.
NoteThe Manual approval strategy requires a user with appropriate credentials to approve the Operator install and subscription process.
- Click Subscribe.
- The Installed Operators page displays the Jaeger Operator’s installation progress.
3.2.1.3. Installing the Kiali Operator
You must install the Kiali Operator for the Red Hat OpenShift Service Mesh Operator to install the control plane.
Do not install Community versions of the Operators. Community Operators are not supported.
Prerequisites
- Access to the OpenShift Container Platform web console.
Procedure
- Log in to the OpenShift Container Platform web console.
-
Navigate to Operators
OperatorHub. - Type Kiali into the filter box to find the Kiali Operator.
- Click the Kiali Operator provided by Red Hat to display information about the Operator.
- Click Install.
-
On the Create Operator Subscription page, select All namespaces on the cluster (default). This installs the Operator in the default
openshift-operators
project and makes the Operator available to all projects in the cluster. - Select the stable Update Channel.
Select the Automatic Approval Strategy.
NoteThe Manual approval strategy requires a user with appropriate credentials to approve the Operator install and subscription process.
- Click Subscribe.
- The Installed Operators page displays the Kiali Operator’s installation progress.
3.2.1.4. Installing the Red Hat OpenShift Service Mesh Operator
Prerequisites
- Access to the OpenShift Container Platform web console.
- The Elasticsearch Operator must be installed.
- The Jaeger Operator must be installed.
- The Kiali Operator must be installed.
Procedure
- Log in to the OpenShift Container Platform web console.
-
Navigate to Operators
OperatorHub. - Type Red Hat OpenShift Service Mesh into the filter box to find the Red Hat OpenShift Service Mesh Operator.
- Click the Red Hat OpenShift Service Mesh Operator to display information about the Operator.
-
On the Create Operator Subscription page, select All namespaces on the cluster (default). This installs the Operator in the default
openshift-operators
project and makes the Operator available to all projects in the cluster. - Click Install.
- Select the stable Update Channel.
Select the Automatic Approval Strategy.
NoteThe Manual approval strategy requires a user with appropriate credentials to approve the Operator install and subscription process.
- Click Subscribe.
- The Installed Operators page displays the Red Hat OpenShift Service Mesh Operator’s installation progress.
3.2.1.5. Deploying the Red Hat OpenShift Service Mesh control plane
The ServiceMeshControlPlane
resource defines the configuration to be used during installation. You can deploy the default configuration provided by Red Hat or customize the ServiceMeshControlPlane
file to fit your business needs.
You can deploy the Service Mesh control plane by using the OpenShift Container Platform web console or from the command line using the oc
client tool.
3.2.1.5.1. Deploying the control plane from the web console
Follow this procedure to deploy the Red Hat OpenShift Service Mesh control plane by using the web console.
Prerequisites
- The Red Hat OpenShift Service Mesh Operator must be installed.
- Review the instructions for how to customize the Red Hat OpenShift Service Mesh installation.
-
An account with the
cluster-admin
role.
Procedure
-
Log in to the OpenShift Container Platform web console as a user with the
cluster-admin
role. Create a project named
istio-system
.-
Navigate to Home
Projects. - Click Create Project.
-
Enter
istio-system
in the Name field. - Click Create.
-
Navigate to Home
-
Navigate to Operators
Installed Operators. -
If necessary, select
istio-system
from the Project menu. You may have to wait a few moments for the Operators to be copied to the new project. Click the Red Hat OpenShift Service Mesh Operator. Under Provided APIs, the Operator provides links to create two resource types:
-
A
ServiceMeshControlPlane
resource -
A
ServiceMeshMemberRoll
resource
-
A
- Under Istio Service Mesh Control Plane click Create ServiceMeshControlPlane.
On the Create Service Mesh Control Plane page, modify the YAML for the default
ServiceMeshControlPlane
template as needed.NoteFor additional information about customizing the control plane, see customizing the Red Hat OpenShift Service Mesh installation. For production, you must change the default Jaeger template.
- Click Create to create the control plane. The Operator creates Pods, services, and Service Mesh control plane components based on your configuration parameters.
- Click the Istio Service Mesh Control Plane tab.
- Click the name of the new control plane.
- Click the Resources tab to see the Red Hat OpenShift Service Mesh control plane resources the Operator created and configured.
3.2.1.5.2. Deploying the control plane from the CLI
Follow this procedure to deploy the Red Hat OpenShift Service Mesh control plane the command line.
Prerequisites
- The Red Hat OpenShift Service Mesh Operator must be installed.
- Review the instructions for how to customize the Red Hat OpenShift Service Mesh installation.
-
An account with the
cluster-admin
role. -
Access to the OpenShift Container Platform Command-line Interface (CLI), commonly known as
oc
.
Procedure
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role.$ oc login https://{HOSTNAME}:6443
Create a project named
istio-system
.$ oc new-project istio-system
-
Create a
ServiceMeshControlPlane
file namedistio-installation.yaml
using the example found in "Customize the Red Hat OpenShift Service Mesh installation". You can customize the values as needed to match your use case. For production deployments you must change the default Jaeger template. Run the following command to deploy the control plane:
$ oc create -n istio-system -f istio-installation.yaml
Execute the following command to see the status of the control plane installation.
$ oc get smcp -n istio-system
The installation has finished successfully when the READY column is true.
NAME READY basic-install True
Run the following command to watch the progress of the Pods during the installation process:
$ oc get pods -n istio-system -w
You should see output similar to the following:
NAME READY STATUS RESTARTS AGE grafana-7bf5764d9d-2b2f6 2/2 Running 0 28h istio-citadel-576b9c5bbd-z84z4 1/1 Running 0 28h istio-egressgateway-5476bc4656-r4zdv 1/1 Running 0 28h istio-galley-7d57b47bb7-lqdxv 1/1 Running 0 28h istio-ingressgateway-dbb8f7f46-ct6n5 1/1 Running 0 28h istio-pilot-546bf69578-ccg5x 2/2 Running 0 28h istio-policy-77fd498655-7pvjw 2/2 Running 0 28h istio-sidecar-injector-df45bd899-ctxdt 1/1 Running 0 28h istio-telemetry-66f697d6d5-cj28l 2/2 Running 0 28h jaeger-896945cbc-7lqrr 2/2 Running 0 11h kiali-78d9c5b87c-snjzh 1/1 Running 0 22h prometheus-6dff867c97-gr2n5 2/2 Running 0 28h
For a multitenant installation, Red Hat OpenShift Service Mesh supports multiple independent control planes within the cluster. You can create reusable configurations with ServiceMeshControlPlane
templates. For more information, see Creating control plane templates.
3.2.1.6. Creating the Red Hat OpenShift Service Mesh member roll
The ServiceMeshMemberRoll
lists the projects belonging to the control plane. Only projects listed in the ServiceMeshMemberRoll
are affected by the control plane. A project does not belong to a service mesh until you add it to the member roll for a particular control plane deployment.
You must create a ServiceMeshMemberRoll
resource named default
in the same project as the ServiceMeshControlPlane
.
The member projects are only updated if the Service Mesh control plane installation succeeds.
3.2.1.6.1. Creating the member roll from the web console
Follow this procedure to add one or more projects to the Service Mesh member roll by using the web console.
Prerequisites
- An installed, verified Red Hat OpenShift Service Mesh Operator.
-
Location of the installed
ServiceMeshControlPlane
. - List of existing projects to add to the service mesh.
Procedure
If you don’t already have projects for your mesh, or you are starting from scratch, create a project. It must be different from
istio-system
.-
Navigate to Home
Projects. - Enter a name in the Name field.
- Click Create.
-
Navigate to Home
- Log in to the OpenShift Container Platform web console.
-
Navigate to Operators
Installed Operators. -
Click the Project menu and choose the project where your
ServiceMeshControlPlane
is deployed from the list, for exampleistio-system
. - Click the Red Hat OpenShift Service Mesh Operator.
- Click the All Instances tab.
Click Create New, and then select Create Istio Service Mesh Member Roll.
NoteIt can take a short time for the Operator to finish copying the resources, therefore you may need to refresh the screen to see the Create Istio Service Mesh Member Roll button.
-
On the Create Service Mesh Member Roll page, modify the YAML to add your projects as members. You can add any number of projects, but a project can only belong to one
ServiceMeshMemberRoll
resource. - Click Create to save the Service Mesh Member Roll.
3.2.1.6.2. Creating the member roll from the CLI
Follow this procedure to add a project to the ServiceMeshMemberRoll
from the command line.
Prerequisites
- An installed, verified Red Hat OpenShift Service Mesh Operator.
-
Location of the installed
ServiceMeshControlPlane
. - List of projects to add to the service mesh.
-
Access to the OpenShift Container Platform Command-line Interface (CLI) commonly known as
oc
.
Procedure
Log in to the OpenShift Container Platform CLI.
$ oc login
Create a
ServiceMeshMemberRoll
resource in the same project as theServiceMeshControlPlane
resource, in our example that isistio-system
. The resource must be nameddefault
.$ oc create -n istio-system -f servicemeshmemberroll-default.yaml
Example servicemeshmemberroll-default.yaml
apiVersion: maistra.io/v1 kind: ServiceMeshMemberRoll metadata: name: default namespace: istio-system spec: members: # a list of projects joined into the service mesh - your-project-name - another-project-name
-
Modify the default YAML to add your projects as
members
. You can add any number of projects, but a project can only belong to oneServiceMeshMemberRoll
resource.
3.2.1.6.3. Creating the Red Hat OpenShift Service Mesh members
ServiceMeshMember
resources can be created by service mesh users who don’t have privileges to add members to the ServiceMeshMemberRoll
directly. While project administrators are automatically given permission to create the ServiceMeshMember
resource in their project, they cannot point it to any ServiceMeshControlPlane
until the service mesh administrator explicitly grants access to the service mesh. Administrators can grant users permissions to access the mesh by granting them the mesh-user
user role, for example:
$ oc policy add-role-to-user -n <control-plane-namespace> --role-namespace <control-plane-namespace> mesh-user <user-name>.
Administrators can modify the mesh user
role binding in the control plane project to specify the users and groups that are granted access. The ServiceMeshMember
adds the project to the ServiceMeshMemberRoll
within the control plane project it references.
apiVersion: maistra.io/v1 kind: ServiceMeshMember metadata: name: default spec: controlPlaneRef: namespace: control-plane-namespace name: minimal-install
The mesh-users role binding is created automatically after the administrator creates the ServiceMeshControlPlane
resource. An administrator can use the following command to add a role to a user.
$ oc policy add-role-to-user
The administrator can also create the mesh-user
role binding before the administrator creates the ServiceMeshControlPlane
resource. For example, the administrator can create it in the same oc apply
operation as the ServiceMeshControlPlane
resource.
This example adds a role binding for alice
:
apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: namespace: control-plane-namespace name: mesh-users roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: mesh-user subjects: - apiGroup: rbac.authorization.k8s.io kind: User name: alice
3.2.1.7. Adding or removing projects from the service mesh
Follow this procedure to modify an existing Service Mesh ServiceMeshMemberRoll
resource using the web console.
-
You can add any number of projects, but a project can only belong to one
ServiceMeshMemberRoll
resource. -
The
ServiceMeshMemberRoll
resource is deleted when its correspondingServiceMeshControlPlane
resource is deleted.
3.2.1.7.1. Modifying the member roll from the web console
Prerequisites
- An installed, verified Red Hat OpenShift Service Mesh Operator.
-
An existing
ServiceMeshMemberRoll
resource. -
Name of the project with the
ServiceMeshMemberRoll
resource. - Names of the projects you want to add or remove from the mesh.
Procedure
- Log in to the OpenShift Container Platform web console.
-
Navigate to Operators
Installed Operators. -
Click the Project menu and choose the project where your
ServiceMeshControlPlane
is deployed from the list, for exampleistio-system
. - Click the Red Hat OpenShift Service Mesh Operator.
- Click the Istio Service Mesh Member Roll tab.
-
Click the
default
link. - Click the YAML tab.
-
Modify the YAML to add or remove projects as members. You can add any number of projects, but a project can only belong to one
ServiceMeshMemberRoll
resource. - Click Save.
- Click Reload.
3.2.1.7.2. Modifying the member roll from the CLI
Follow this procedure to modify an existing Service Mesh member roll using the command line.
Prerequisites
- An installed, verified Red Hat OpenShift Service Mesh Operator.
-
An existing
ServiceMeshMemberRoll
resource. -
Name of the project with the
ServiceMeshMemberRoll
resource. - Names of the projects you want to add or remove from the mesh.
-
Access to the OpenShift Container Platform Command-line Interface (CLI) commonly known as
oc
.
Procedure
- Log in to the OpenShift Container Platform CLI.
Edit the
ServiceMeshMemberRoll
resource.$ oc edit smmr -n <controlplane-namespace>
Modify the YAML to add or remove projects as members. You can add any number of projects, but a project can only belong to one
ServiceMeshMemberRoll
resource.Example servicemeshmemberroll-default.yaml
apiVersion: maistra.io/v1 kind: ServiceMeshMemberRoll metadata: name: default namespace: istio-system spec: members: # a list of projects joined into the service mesh - your-project-name - another-project-name
3.2.1.8. Deleting the Red Hat OpenShift Service Mesh member roll
The ServiceMeshMemberRoll
resource is automatically deleted when you delete the ServiceMeshControlPlane
resource it is associated with.
3.2.2. Updating your application pods
If you selected the Automatic Approval Strategy when you were installing your Operators, then the Operators update the control plane automatically, but not your applications. Existing applications continue to be part of the mesh and function accordingly. The application administrator must restart applications to upgrade the sidecar.
If your deployment uses Automatic sidecar injection, you can update the pod template in the deployment by adding or modifying an annotation. Run the following command to redeploy the pods:
$ oc patch deployment/<deployment> -p '{"spec":{"template":{"metadata":{"annotations":{"kubectl.kubernetes.io/restartedAt": "'`date -Iseconds`'"}}}}}'
If your deployment does not use automatic sidecar injection, you must manually update the sidecars by modifying the sidecar container image specified in the deployment or pod.
Next steps
- Customize the Red Hat OpenShift Service Mesh installation.
- Prepare to deploy applications on Red Hat OpenShift Service Mesh.
3.3. Customizing the Red Hat OpenShift Service Mesh installation
You can customize your Red Hat OpenShift Service Mesh by modifying the default Service Mesh custom resource or by creating a new custom resource.
Prerequisites
-
An account with the
cluster-admin
role. - Completed the Preparing to install Red Hat OpenShift Service Mesh process.
- Have installed the operators.
3.3.1. Red Hat OpenShift Service Mesh custom resources
The istio-system
project is used as an example throughout the Service Mesh documentation, but you can use other projects as necessary.
A custom resource allows you to extend the API in an Red Hat OpenShift Service Mesh project or cluster. When you deploy Service Mesh it creates a default ServiceMeshControlPlane
that you can modify to change the project parameters.
The Service Mesh operator extends the API by adding the ServiceMeshControlPlane
resource type, which enables you to create ServiceMeshControlPlane
objects within projects. By creating a ServiceMeshControlPlane
object, you instruct the Operator to install a Service Mesh control plane into the project, configured with the parameters you set in the ServiceMeshControlPlane
object.
This example ServiceMeshControlPlane
definition contains all of the supported parameters and deploys Red Hat OpenShift Service Mesh 1.1.4 images based on Red Hat Enterprise Linux (RHEL).
The 3scale Istio Adapter is deployed and configured in the custom resource file. It also requires a working 3scale account (SaaS or On-Premises).
Example istio-installation.yaml
apiVersion: maistra.io/v1 kind: ServiceMeshControlPlane metadata: name: basic-install spec: istio: global: proxy: resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 128Mi gateways: istio-egressgateway: autoscaleEnabled: false istio-ingressgateway: autoscaleEnabled: false ior_enabled: false mixer: policy: autoscaleEnabled: false telemetry: autoscaleEnabled: false resources: requests: cpu: 100m memory: 1G limits: cpu: 500m memory: 4G pilot: autoscaleEnabled: false traceSampling: 100 kiali: enabled: true grafana: enabled: true tracing: enabled: true jaeger: template: all-in-one
3.3.2. ServiceMeshControlPlane
parameters
The following examples illustrate use of the ServiceMeshControlPlane
parameters and the tables provide additional information about supported parameters.
The resources you configure for Red Hat OpenShift Service Mesh with these parameters, including CPUs, memory, and the number of pods, are based on the configuration of your OpenShift cluster. Configure these parameters based on the available resources in your current cluster configuration.
3.3.2.1. Istio global example
Here is an example that illustrates the Istio global parameters for the ServiceMeshControlPlane
and a description of the available parameters with appropriate values.
In order for the 3scale Istio Adapter to work, disablePolicyChecks
must be false
.
Example global parameters
istio: global: tag: 1.1.0 hub: registry.redhat.io/openshift-service-mesh/ proxy: resources: requests: cpu: 10m memory: 128Mi limits: mtls: enabled: false disablePolicyChecks: true policyCheckFailOpen: false imagePullSecrets: - MyPullSecret
Parameter | Description | Values | Default value |
---|---|---|---|
| This parameter enables/disables policy checks. |
|
|
| This parameter indicates whether traffic is allowed to pass through to the Envoy sidecar when the Mixer policy service cannot be reached. |
|
|
| The tag that the Operator uses to pull the Istio images. | A valid container image tag. |
|
| The hub that the Operator uses to pull Istio images. | A valid image repository. |
|
| This parameter controls whether to enable/disable Mutual Transport Layer Security (mTLS) between services by default. |
|
|
| If access to the registry providing the Istio images is secure, list an imagePullSecret here. | redhat-registry-pullsecret OR quay-pullsecret | None |
These parameters are specific to the proxy subset of global parameters.
Type | Parameter | Description | Values | Default value |
---|---|---|---|---|
Resources |
| The amount of CPU resources requested for Envoy proxy. | CPU resources, specified in cores or millicores (for example, 200m, 0.5, 1) based on your environment’s configuration. |
|
| The amount of memory requested for Envoy proxy | Available memory in bytes(for example, 200Ki, 50Mi, 5Gi) based on your environment’s configuration. |
| |
Limits |
| The maximum amount of CPU resources requested for Envoy proxy. | CPU resources, specified in cores or millicores (for example, 200m, 0.5, 1) based on your environment’s configuration. |
|
| The maximum amount of memory Envoy proxy is permitted to use. | Available memory in bytes (for example, 200Ki, 50Mi, 5Gi) based on your environment’s configuration. |
|
3.3.2.2. Istio gateway configuration
Here is an example that illustrates the Istio gateway parameters for the ServiceMeshControlPlane
and a description of the available parameters with appropriate values.
Example gateway parameters
gateways: istio-egressgateway: autoscaleEnabled: false autoscaleMin: 1 autoscaleMax: 5 istio-ingressgateway: autoscaleEnabled: false autoscaleMin: 1 autoscaleMax: 5
Type | Parameter | Description | Values | Default value |
---|---|---|---|---|
|
| This parameter enables/disables autoscaling. |
|
|
|
The minimum number of pods to deploy for the egress gateway based on the | A valid number of allocatable pods based on your environment’s configuration. |
| |
|
The maximum number of pods to deploy for the egress gateway based on the | A valid number of allocatable pods based on your environment’s configuration. |
| |
|
| This parameter enables/disables autoscaling. |
|
|
|
The minimum number of pods to deploy for the ingress gateway based on the | A valid number of allocatable pods based on your environment’s configuration. |
| |
|
The maximum number of pods to deploy for the ingress gateway based on the | A valid number of allocatable pods based on your environment’s configuration. |
|
3.3.2.3. Istio Mixer configuration
Here is an example that illustrates the Mixer parameters for the ServiceMeshControlPlane
and a description of the available parameters with appropriate values.
Example mixer parameters
mixer: enabled: true policy: autoscaleEnabled: false telemetry: autoscaleEnabled: false resources: requests: cpu: 10m memory: 128Mi limits:
Parameter | Description | Values | Default value |
---|---|---|---|
| This parameter enables/disables Mixer. |
|
|
| This parameter enables/disables autoscaling. Disable this for small environments. |
|
|
|
The minimum number of pods to deploy based on the | A valid number of allocatable pods based on your environment’s configuration. |
|
|
The maximum number of pods to deploy based on the | A valid number of allocatable pods based on your environment’s configuration. |
|
Type | Parameter | Description | Values | Default |
---|---|---|---|---|
Resources |
| The percentage of CPU resources requested for Mixer telemetry. | CPU resources in millicores based on your environment’s configuration. |
|
| The amount of memory requested for Mixer telemetry. | Available memory in bytes (for example, 200Ki, 50Mi, 5Gi) based on your environment’s configuration. |
| |
Limits |
| The maximum percentage of CPU resources Mixer telemetry is permitted to use. | CPU resources in millicores based on your environment’s configuration. |
|
| The maximum amount of memory Mixer telemetry is permitted to use. | Available memory in bytes (for example, 200Ki, 50Mi, 5Gi) based on your environment’s configuration. |
|
3.3.2.4. Istio Pilot configuration
Here is an example that illustrates the Istio Pilot parameters for the ServiceMeshControlPlane
and a description of the available parameters with appropriate values.
Example pilot parameters
pilot: resources: requests: cpu: 100m memory: 128Mi autoscaleEnabled: false traceSampling: 100
Parameter | Description | Values | Default value |
---|---|---|---|
| The percentage of CPU resources requested for Pilot. | CPU resources in millicores based on your environment’s configuration. |
|
| The amount of memory requested for Pilot. | Available memory in bytes (for example, 200Ki, 50Mi, 5Gi) based on your environment’s configuration. |
|
| This parameter enables/disables autoscaling. Disable this for small environments. |
|
|
| This value controls how often random sampling occurs. Note: Increase for development or testing. | A valid percentage. |
|
3.3.3. Configuring Kiali
When the Service Mesh Operator creates the ServiceMeshControlPlane
it also processes the Kiali resource. The Kiali Operator then uses this object when creating Kiali instances.
The default Kiali parameters specified in the ServiceMeshControlPlane
are as follows:
Example Kiali parameters
apiVersion: maistra.io/v1 kind: ServiceMeshControlPlane spec: kiali: enabled: true dashboard: viewOnlyMode: false ingress: enabled: true
Parameter | Description | Values | Default value |
---|---|---|---|
enabled | This parameter enables/disables Kiali. Kiali is enabled by default. |
|
|
dashboard viewOnlyMode | This parameter enables/disables view-only mode for the Kiali console. When view-only mode is enabled, users cannot use the console to make changes to the Service Mesh. |
|
|
ingress enabled | This parameter enables/disables ingress for Kiali. |
|
|
3.3.3.1. Configuring Kiali for Grafana
When you install Kiali and Grafana as part of Red Hat OpenShift Service Mesh the Operator configures the following by default:
- Grafana is enabled as an external service for Kiali
- Grafana authorization for the Kiali console
- Grafana URL for the Kiali console
Kiali can automatically detect the Grafana URL. However if you have a custom Grafana installation that is not easily auto-detectable by Kiali, you must update the URL value in the ServiceMeshControlPlane
resource.
Additional Grafana parameters
spec: kiali: enabled: true dashboard: viewOnlyMode: false grafanaURL: "https://grafana-istio-system.127.0.0.1.nip.io" ingress: enabled: true
3.3.3.2. Configuring Kiali for Jaeger
When you install Kiali and Jaeger as part of Red Hat OpenShift Service Mesh the Operator configures the following by default:
- Jaeger is enabled as an external service for Kiali
- Jaeger authorization for the Kiali console
- Jaeger URL for the Kiali console
Kiali can automatically detect the Jaeger URL. However if you have a custom Jaeger installation that is not easily auto-detectable by Kiali, you must update the URL value in the ServiceMeshControlPlane
resource.
Additional Jaeger parameters
spec: kiali: enabled: true dashboard: viewOnlyMode: false jaegerURL: "http://jaeger-query-istio-system.127.0.0.1.nip.io" ingress: enabled: true
3.3.4. Configuring Jaeger
When the Service Mesh Operator creates the ServiceMeshControlPlane
resource it also creates the Jaeger resource. The Jaeger Operator then uses this object when creating Jaeger instances.
The default Jaeger parameters specified in the ServiceMeshControlPlane
are as follows:
Default all-in-one
Jaeger parameters
apiVersion: maistra.io/v1 kind: ServiceMeshControlPlane spec: istio: tracing: enabled: true jaeger: template: all-in-one
Parameter | Description | Values | Default value |
---|---|---|---|
tracing enabled | This parameter enables/disables tracing in Service Mesh. Jaeger is installed by default. |
|
|
jaeger template | This parameter specifies which Jaeger deployment strategy to use. |
|
|
The default template in the ServiceMeshControlPlane
resource is the all-in-one
deployment strategy which uses in-memory storage. For production, the only supported storage option is Elasticsearch, therefore you must configure the ServiceMeshControlPlane
to request the production-elasticsearch
template when you deploy Service Mesh within a production environment.
3.3.4.1. Configuring Elasticsearch
The default Jaeger deployment strategy uses the all-in-one
template so that the installation can be completed using minimal resources. However, because the all-in-one
template uses in-memory storage, it is only recommended for development, demo, or testing purposes and should NOT be used for production environments.
If you are deploying Service Mesh and Jaeger in a production environment you must change the template to the production-elasticsearch
template, which uses Elasticsearch for Jaeger’s storage needs.
Elasticsearch is a memory intensive application. The initial set of nodes specified in the default OpenShift Container Platform installation may not be large enough to support the Elasticsearch cluster. You should modify the default Elasticsearch configuration to match your use case and the resources you have requested for your OpenShift Container Platform installation. You can adjust both the CPU and memory limits for each component by modifying the resources block with valid CPU and memory values. Additional nodes must be added to the cluster if you want to run with the recommended amount (or more) of memory. Ensure that you do not exceed the resources requested for your OpenShift Container Platform installation.
Default "production" Jaeger parameters with Elasticsearch
apiVersion: maistra.io/v1 kind: ServiceMeshControlPlane spec: istio: tracing: enabled: true ingress: enabled: true jaeger: template: production-elasticsearch elasticsearch: nodeCount: 3 redundancyPolicy: resources: requests: cpu: "1" memory: "16Gi" limits: cpu: "1" memory: "16Gi"
Parameter | Description | Values | Default Value | Examples |
---|---|---|---|---|
tracing: enabled | This parameter enables/disables tracing in Service Mesh. Jaeger is installed by default. |
|
| |
ingress: enabled | This parameter enables/disables ingress for Jaeger. |
|
| |
jaeger template | This parameter specifies which Jaeger deployment strategy to use. |
|
| |
elasticsearch: nodeCount | Number of Elasticsearch nodes to create. | Integer value. | 1 | Proof of concept = 1, Minimum deployment =3 |
requests: cpu | Number of central processing units for requests, based on your environment’s configuration. | Specified in cores or millicores (for example, 200m, 0.5, 1). | 1Gi | Proof of concept = 500m, Minimum deployment =1 |
requests: memory | Available memory for requests, based on your environment’s configuration. | Specified in bytes (for example, 200Ki, 50Mi, 5Gi). | 500m | Proof of concept = 1Gi, Minimum deployment = 16Gi* |
limits: cpu | Limit on number of central processing units, based on your environment’s configuration. | Specified in cores or millicores (for example, 200m, 0.5, 1). | Proof of concept = 500m, Minimum deployment =1 | |
limits: memory | Available memory limit based on your environment’s configuration. | Specified in bytes (for example, 200Ki, 50Mi, 5Gi). | Proof of concept = 1Gi, Minimum deployment = 16Gi* | |
* Each Elasticsearch node can operate with a lower memory setting though this is not recommended for production deployments. For production use, you should have no less than 16Gi allocated to each Pod by default, but preferably allocate as much as you can, up to 64Gi per Pod. |
Procedure
-
Log in to the OpenShift Container Platform web console as a user with the
cluster-admin
role. -
Navigate to Operators
Installed Operators. - Click the Red Hat OpenShift Service Mesh Operator.
- Click the Istio Service Mesh Control Plane tab.
-
Click the name of your control plane file, for example,
basic-install
. - Click the YAML tab.
-
Edit the Jaeger parameters, replacing the default
all-in-one
template with parameters for theproduction-elasticsearch
template, modified for your use case. Ensure that the indentation is correct. - Click Save.
- Click Reload. OpenShift Container Platform redeploys Jaeger and creates the Elasticsearch resources based on the specified parameters.
For more information about configuring Elasticsearch with OpenShift Container Platform, see Configuring Elasticsearch.
3.3.5. 3scale configuration
Here is an example that illustrates the 3scale Istio Adapter parameters for the Red Hat OpenShift Service Mesh custom resource and a description of the available parameters with appropriate values.
Example 3scale parameters
threeScale: enabled: false PARAM_THREESCALE_LISTEN_ADDR: 3333 PARAM_THREESCALE_LOG_LEVEL: info PARAM_THREESCALE_LOG_JSON: true PARAM_THREESCALE_LOG_GRPC: false PARAM_THREESCALE_REPORT_METRICS: true PARAM_THREESCALE_METRICS_PORT: 8080 PARAM_THREESCALE_CACHE_TTL_SECONDS: 300 PARAM_THREESCALE_CACHE_REFRESH_SECONDS: 180 PARAM_THREESCALE_CACHE_ENTRIES_MAX: 1000 PARAM_THREESCALE_CACHE_REFRESH_RETRIES: 1 PARAM_THREESCALE_ALLOW_INSECURE_CONN: false PARAM_THREESCALE_CLIENT_TIMEOUT_SECONDS: 10 PARAM_THREESCALE_GRPC_CONN_MAX_SECONDS: 60
Parameter | Description | Values | Default value |
---|---|---|---|
| Whether to use the 3scale adapter |
|
|
| Sets the listen address for the gRPC server | Valid port number |
|
| Sets the minimum log output level. |
|
|
| Controls whether the log is formatted as JSON |
|
|
| Controls whether the log contains gRPC info |
|
|
| Controls whether 3scale system and backend metrics are collected and reported to Prometheus |
|
|
|
Sets the port that the 3scale | Valid port number |
|
| Time period, in seconds, to wait before purging expired items from the cache | Time period in seconds |
|
| Time period before expiry when cache elements are attempted to be refreshed | Time period in seconds |
|
|
Max number of items that can be stored in the cache at any time. Set to | Valid number |
|
| The number of times unreachable hosts are retried during a cache update loop | Valid number |
|
|
Allow to skip certificate verification when calling |
|
|
| Sets the number of seconds to wait before terminating requests to 3scale System and Backend | Time period in seconds |
|
| Sets the maximum amount of seconds (+/-10% jitter) a connection may exist before it is closed | Time period in seconds | 60 |
Next steps
- Prepare to deploy applications on Red Hat OpenShift Service Mesh.
3.4. Updating Red Hat OpenShift Service Mesh
3.4.1. Manual updates required by CVE-2020-8663
The fix for CVE-2020-8663: envoy: Resource exhaustion when accepting too many connections
added a configurable limit on downstream connections. The configuration option for this limit must be configured to mitigate this vulnerability.
These manual steps are required to mitigate this CVE whether you are using the 1.1 version or the 1.0 version of Red Hat OpenShift Service Mesh.
This new configuration option is called overload.global_downstream_max_connections
, and it is configurable as a proxy runtime
setting. Perform the following steps to configure limits at the Ingress Gateway.
Procedure
Create a file named
bootstrap-override.json
with the following text to force the proxy to override the bootstrap template and load runtime configuration from disk:{ "runtime": { "symlink_root": "/var/lib/istio/envoy/runtime" } }
Create a secret from the
bootstrap-override.json
file, replacing <SMCPnamespace> with the namespace where you created the service mesh control plane (SMCP):$ oc create secret generic -n <SMCPnamespace> gateway-bootstrap --from-file=bootstrap-override.json
Update the SMCP configuration to activate the override.
Updated SMCP configuration example #1
apiVersion: maistra.io/v1 kind: ServiceMeshControlPlane spec: istio: gateways: istio-ingressgateway: env: ISTIO_BOOTSTRAP_OVERRIDE: /var/lib/istio/envoy/custom-bootstrap/bootstrap-override.json secretVolumes: - mountPath: /var/lib/istio/envoy/custom-bootstrap name: custom-bootstrap secretName: gateway-bootstrap
To set the new configuration option, create a secret that has the desired value for the
overload.global_downstream_max_connections
setting. The following example uses a value of10000
:$ oc create secret generic -n <SMCPnamespace> gateway-settings --from-literal=overload.global_downstream_max_connections=10000
- Update the SMCP again to mount the secret in the location where Envoy is looking for runtime configuration:
Updated SMCP configuration example #2
apiVersion: maistra.io/v1 kind: ServiceMeshControlPlane spec: template: default #Change the version to "v1.0" if you are on the 1.0 stream. version: v1.1 istio: gateways: istio-ingressgateway: env: ISTIO_BOOTSTRAP_OVERRIDE: /var/lib/istio/envoy/custom-bootstrap/bootstrap-override.json secretVolumes: - mountPath: /var/lib/istio/envoy/custom-bootstrap name: custom-bootstrap secretName: gateway-bootstrap # below is the new secret mount - mountPath: /var/lib/istio/envoy/runtime name: gateway-settings secretName: gateway-settings
3.4.2. Manual updates from 1.0 to 1.1
If you are updating from Red Hat OpenShift Service Mesh 1.0 to 1.1, you must update the ServiceMeshControlPlane
resource to update the control plane components to the new version.
- In the web console, click the Red Hat OpenShift Service Mesh Operator.
-
Click the Project menu and choose the project where your
ServiceMeshControlPlane
is deployed from the list, for exampleistio-system
. -
Click the name of your control plane, for example
basic-install
. -
Click YAML and add a version field to the
spec:
of yourServiceMeshControlPlane
resource. For example, to update to Red Hat OpenShift Service Mesh 1.1.0, addversion: v1.1
.
spec: version: v1.1 ...
The version field specifies the version of ServiceMesh to install and defaults to the latest available version.
3.4.3. Manual updates
If you choose to update manually, the Operator Lifecycle Manager (OLM) controls the installation, upgrade, and role-based access control (RBAC) of Operators in a cluster. OLM runs by default in OpenShift Container Platform. OLM uses CatalogSources, which use the Operator Registry API, to query for available Operators as well as upgrades for installed Operators.
- For more information about how OpenShift Container Platform handled upgrades, refer to the Operator Lifecycle Manager documentation.
3.5. Removing Red Hat OpenShift Service Mesh
This process allows you to remove Red Hat OpenShift Service Mesh from an existing OpenShift Container Platform instance. Remove the control plane before removing the operators.
3.5.1. Removing the Red Hat OpenShift Service Mesh control plane
You can remove the Service Mesh control plane by using the OpenShift Container Platform web console or the CLI.
3.5.1.1. Removing the control plane with the web console
Follow this procedure to remove the Red Hat OpenShift Service Mesh control plane by using the web console.
Prerequisites
- The Red Hat OpenShift Service Mesh control plane must be deployed.
Procedure
- Log in to the OpenShift Container Platform web console.
-
Click the Project menu and choose the
istio-system
project from the list. -
Navigate to Operators
Installed Operators. - Click on Service Mesh Control Plane under Provided APIs.
-
Click the
ServiceMeshControlPlane
menu . - Click Delete Service Mesh Control Plane.
-
Click Delete on the confirmation dialog window to remove the
ServiceMeshControlPlane
.
3.5.1.2. Removing the control plane from the CLI
Follow this procedure to remove the Red Hat OpenShift Service Mesh control plane by using the CLI.
Prerequisites
- The Red Hat OpenShift Service Mesh control plane must be deployed.
-
Access to the OpenShift Container Platform Command-line Interface (CLI) also known as
oc
.
When you remove the ServiceMeshControlPlane
, Service Mesh tells the Operator to begin uninstalling everything it installed.
You can use the shortened smcp
alias in place of servicemeshcontrolplane
.
- Log in to the OpenShift Container Platform CLI.
Run this command to retrieve the name of the installed
ServiceMeshControlPlane
:$ oc get servicemeshcontrolplanes -n istio-system
Replace
<name_of_custom_resource>
with the output from the previous command, and run this command to remove the custom resource:$ oc delete servicemeshcontrolplanes -n istio-system <name_of_custom_resource>
3.5.2. Removing the installed Operators
You must remove the Operators to successfully remove Red Hat OpenShift Service Mesh. Once you remove the Red Hat OpenShift Service Mesh Operator, you must remove the Jaeger Operator, Kiali Operator, and the Elasticsearch Operator.
3.5.2.1. Removing the Red Hat OpenShift Service Mesh Operator
Follow this procedure to remove the Red Hat OpenShift Service Mesh Operator.
Prerequisites
- Access to the OpenShift Container Platform web console.
- The Red Hat OpenShift Service Mesh Operator must be installed.
Procedure
- Log in to the OpenShift Container Platform web console.
-
From the Operators
Installed Operators page, scroll or type a keyword into the Filter by name to find the Red Hat OpenShift Service Mesh Operator. Then, click on it. - On the right-hand side of the Operator Details page, select Uninstall Operator from the Actions drop-down menu.
- When prompted by the Remove Operator Subscription window, optionally select the Also completely remove the Operator from the selected namespace check box if you want all components related to the installation to be removed. This removes the CSV, which in turn removes the Pods, Deployments, CRDs, and CRs associated with the Operator.
3.5.2.2. Removing the Jaeger Operator
Follow this procedure to remove the Jaeger Operator.
Prerequisites
- Access to the OpenShift Container Platform web console.
- The Jaeger Operator must be installed.
Procedure
- Log in to the OpenShift Container Platform web console.
-
From the Operators
Installed Operators page, scroll or type a keyword into the Filter by name to find the Jaeger Operator. Then, click on it. - On the right-hand side of the Operator Details page, select Uninstall Operator from the Actions drop-down menu.
- When prompted by the Remove Operator Subscription window, optionally select the Also completely remove the Operator from the selected namespace check box if you want all components related to the installation to be removed. This removes the CSV, which in turn removes the Pods, Deployments, CRDs, and CRs associated with the Operator.
3.5.2.3. Removing the Kiali Operator
Follow this procedure to remove the Kiali Operator.
Prerequisites
- Access to the OpenShift Container Platform web console.
- The Kiali Operator must be installed.
Procedure
- Log in to the OpenShift Container Platform web console.
-
From the Operators
Installed Operators page, scroll or type a keyword into the Filter by name to find the Kiali Operator. Then, click on it. - On the right-hand side of the Operator Details page, select Uninstall Operator from the Actions drop-down menu.
- When prompted by the Remove Operator Subscription window, optionally select the Also completely remove the Operator from the selected namespace check box if you want all components related to the installation to be removed. This removes the CSV, which in turn removes the Pods, Deployments, CRDs, and CRs associated with the Operator.
3.5.2.4. Removing the Elasticsearch Operator
Follow this procedure to remove the Elasticsearch Operator.
Prerequisites
- Access to the OpenShift Container Platform web console.
- The Elasticsearch Operator must be installed.
Procedure
- Log in to the OpenShift Container Platform web console.
-
From the Operators
Installed Operators page, scroll or type a keyword into the Filter by name to find the Elasticsearch Operator. Then, click on it. - On the right-hand side of the Operator Details page, select Uninstall Operator from the Actions drop-down menu.
- When prompted by the Remove Operator Subscription window, optionally select the Also completely remove the Operator from the selected namespace check box if you want all components related to the installation to be removed. This removes the CSV, which in turn removes the Pods, Deployments, CRDs, and CRs associated with the Operator.
3.5.2.5. Clean up Operator resources
Follow this procedure to manually remove resources left behind after removing the Red Hat OpenShift Service Mesh Operator by using the OperatorHub interface.
Prerequisites
- An account with cluster administration access.
-
Access to the OpenShift Container Platform Command-line Interface (CLI) also known as
oc
.
Procedure
- Log in to the OpenShift Container Platform CLI as a cluster administrator.
Run the following commands to clean up resources after uninstalling the Operators:
NoteReplace
<operator-project>
with the name of the project where the Red Hat OpenShift Service Mesh Operator was installed. This is typicallyopenshift-operators
.$ oc delete validatingwebhookconfiguration/<operator-project>.servicemesh-resources.maistra.io $ oc delete mutatingwebhoookconfigurations/<operator-project>.servicemesh-resources.maistra.io $ oc delete -n <operator-project> daemonset/istio-node $ oc delete clusterrole/istio-admin clusterrole/istio-cni clusterrolebinding/istio-cni $ oc get crds -o name | grep '.*\.istio\.io' | xargs -r -n 1 oc delete $ oc get crds -o name | grep '.*\.maistra\.io' | xargs -r -n 1 oc delete