Startseite
Produkte
OpenShift Container Platform
4.14
Network Observability
Chapter 4. Installing the Network Observability Operator

Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.

Chapter 4. Installing the Network Observability Operator

Installing the Loki Operator is recommended before using the Network Observability Operator. You can use network observability without Loki, but special considerations apply if you only need metrics or external exporters.

The Loki Operator integrates a gateway that implements multi-tenancy and authentication with Loki for data flow storage. The

LokiStack

resource manages Loki, which is a scalable, highly-available, multi-tenant log aggregation system, and a web proxy with OpenShift Container Platform authentication. The

LokiStack

proxy uses OpenShift Container Platform authentication to enforce multi-tenancy and facilitate the saving and indexing of data in Loki log stores.

Note

The Loki Operator can also be used for configuring the LokiStack log store. The Network Observability Operator requires a dedicated LokiStack separate from the logging.

4.1. Network observability without Loki
Link kopieren

You can use network observability without Loki by not performing the Loki installation steps and skipping directly to "Installing the Network Observability Operator". If you only want to export flows to a Kafka consumer or IPFIX collector, or you only need dashboard metrics, then you do not need to install Loki or provide storage for Loki. The following table compares available features with and without Loki.

Expand

Table 4.1. Comparison of feature availability with and without Loki
	With Loki	Without Loki
Exporters	X	X
Multi-tenancy	X	X
Complete filtering and aggregations capabilities ^[1]	X
Partial filtering and aggregations capabilities ^[2]	X	X
Flow-based metrics and dashboards	X	X
Traffic flows view overview ^[3]	X	X
Traffic flows view table	X
Topology view	X	X
OpenShift Container Platform console Network Traffic tab integration	X	X

Such as per pod.
Such as per workload or namespace.
Statistics on packet drops are only available with Loki.

4.2. Installing the Loki Operator
Link kopieren

The Loki Operator versions 6.0+ are the supported Loki Operator versions for Network Observability; these versions provide the ability to create a

LokiStack

instance using the

openshift-network

tenant configuration mode and provide fully-automatic, in-cluster authentication and authorization support for Network Observability. There are several ways you can install Loki. One way is by using the OpenShift Container Platform web console Operator Hub.

Prerequisites

You have administrator permissions.
You have access to the OpenShift Container Platform web console.
You have access to a supported object store. For example: AWS S3, Google Cloud Storage, Azure, Swift, Minio, or OpenShift Data Foundation.

Procedure

In the OpenShift Container Platform web console, click Operators OperatorHub.
Choose Loki Operator from the list of available Operators, and click Install.
Under Installation Mode, select All namespaces on the cluster.

Verification

Verify that you installed the Loki Operator. Visit the Operators Installed Operators page and look for Loki Operator.
Verify that Loki Operator is listed with Status as Succeeded in all the projects.

Important

To uninstall Loki, refer to the uninstallation process that corresponds with the method you used to install Loki. You might have remaining

ClusterRoles

and

ClusterRoleBindings

, data stored in object store, and persistent volume that must be removed.

4.2.1. Creating a secret for Loki storage
Link kopieren

The Loki Operator supports a few log storage options, such as AWS S3, Google Cloud Storage, Azure, Swift, Minio, OpenShift Data Foundation. The following example shows how to create a secret for AWS S3 storage. The secret created in this example,

loki-s3

, is referenced in "Creating a LokiStack custom resource". You can create this secret in the web console or CLI.

Using the web console, navigate to the Project All Projects dropdown and select Create Project.
Name the project
```
netobserv
```
and click Create.

Navigate to the Import icon, +, in the top right corner. Paste your YAML file into the editor.

The following shows an example secret YAML file for S3 storage:

apiVersion: v1
kind: Secret
metadata:
  name: loki-s3
  namespace: netobserv


stringData:
  access_key_id: QUtJQUlPU0ZPRE5ON0VYQU1QTEUK
  access_key_secret: d0phbHJYVXRuRkVNSS9LN01ERU5HL2JQeFJmaUNZRVhBTVBMRUtFWQo=
  bucketnames: s3-bucket-name
  endpoint: https://s3.eu-central-1.amazonaws.com
  region: eu-central-1

1: The installation examples in this documentation use the same namespace, netobserv, across all components. You can optionally use a different namespace for the different components

Verification

After you create the secret, you view the secret listed under Workloads Secrets in the web console.

4.2.2. Creating a LokiStack custom resource
Link kopieren

You can deploy a

LokiStack

custom resource (CR) by using the web console or OpenShift CLI (

oc

) to create a namespace, or new project.

Procedure

Navigate to Operators Installed Operators, viewing All projects from the Project dropdown.
Look for Loki Operator. In the details, under Provided APIs, select LokiStack.
Click Create LokiStack.
Ensure the following fields are specified in either Form View or YAML view:
```
apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
  name: loki
  namespace: netobserv 
```
1
```
spec:
  size: 1x.small 
```
2
```
  storage:
    schemas:
    - version: v13
      effectiveDate: '2022-06-01'
    secret:
      name: loki-s3
      type: s3
  storageClassName: gp3 
```
3
```
  tenants:
    mode: openshift-network
```
1
The installation examples in this documentation use the same namespace, netobserv, across all components. You can optionally use a different namespace.
2
Specify the deployment size. In the Loki Operator 5.8 and later versions, the supported size options for production instances of Loki are 1x.extra-small, 1x.small, or 1x.medium.
Important
It is not possible to change the number

1x

for the deployment size.
3
Use a storage class name that is available on the cluster for ReadWriteOnce access mode. For best performance, specify a storage class that allocates block storage. You can use oc get storageclasses to see what is available on your cluster.
Important
You must not reuse the same

LokiStack

CR that is used for logging.
Click Create.

4.2.3. Creating a new group for the cluster-admin user role
Link kopieren

Important

Querying application logs for multiple namespaces as a

cluster-admin

user, where the sum total of characters of all of the namespaces in the cluster is greater than 5120, results in the error

Parse error: input size too long (XXXX > 5120)

. For better control over access to logs in LokiStack, make the

cluster-admin

user a member of the

cluster-admin

group. If the

cluster-admin

group does not exist, create it and add the desired users to it.

Use the following procedure to create a new group for users with

cluster-admin

permissions.

Procedure

Enter the following command to create a new group:
```
$ oc adm groups new cluster-admin
```

Enter the following command to add the desired user to the

cluster-admin

group:

$ oc adm groups add-users cluster-admin <username>

Enter the following command to add

cluster-admin

user role to the group:

$ oc adm policy add-cluster-role-to-group cluster-admin cluster-admin

4.2.4. Custom admin group access
Link kopieren

If you need to see cluster-wide logs without necessarily being an administrator, or if you already have any group defined that you want to use here, you can specify a custom group using the

adminGroup

field. Users who are members of any group specified in the

adminGroups

field of the

LokiStack

custom resource (CR) have the same read access to logs as administrators.

Administrator users have access to all network logs across the cluster.

Example LokiStack CR

apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
  name: loki
  namespace: netobserv
spec:
  tenants:
    mode: openshift-network


    openshift:
      adminGroups:


      - cluster-admin
      - custom-admin-group

1: Custom admin groups are only available in this mode.
2: Entering an empty list [] value for this field disables admin groups.
3: Overrides the default groups (system:cluster-admins, cluster-admin, dedicated-admin)

4.2.5. Loki deployment sizing
Link kopieren

Sizing for Loki follows the format of

1x.<size>

where the value

1x

is number of instances and

<size>

specifies performance capabilities.

Important

It is not possible to change the number

1x

for the deployment size.

Expand

Table 4.2. Loki sizing
	1x.demo	1x.extra-small	1x.small	1x.medium
Data transfer	Demo use only	100GB/day	500GB/day	2TB/day
Queries per second (QPS)	Demo use only	1-25 QPS at 200ms	25-50 QPS at 200ms	25-75 QPS at 200ms
Replication factor	None	2	2	2
Total CPU requests	None	14 vCPUs	34 vCPUs	54 vCPUs
Total memory requests	None	31Gi	67Gi	139Gi
Total disk requests	40Gi	430Gi	430Gi	590Gi

4.2.6. LokiStack ingestion limits and health alerts
Link kopieren

The LokiStack instance comes with default settings according to the configured size. It is possible to override some of these settings, such as the ingestion and query limits. An automatic alert in the web console notifies you when these limits are reached.

Note

You might want to update the ingestion and query limits if you get Loki errors showing up in the Console plugin, or in

flowlogs-pipeline

logs.

Here is an example of configured limits:

spec:
  limits:
    global:
      ingestion:
        ingestionBurstSize: 40
        ingestionRate: 20
        maxGlobalStreamsPerTenant: 25000
      queries:
        maxChunksPerQuery: 2000000
        maxEntriesLimitPerQuery: 10000
        maxQuerySeries: 3000

For more information about these settings, see the LokiStack API reference.

4.3. Installing the Network Observability Operator
Link kopieren

You can install the Network Observability Operator using the OpenShift Container Platform web console Operator Hub. When you install the Operator, it provides the

FlowCollector

custom resource definition (CRD). You can set specifications in the web console when you create the

FlowCollector

Important

The actual memory consumption of the Operator depends on your cluster size and the number of resources deployed. Memory consumption might need to be adjusted accordingly. For more information refer to "Network Observability controller manager pod runs out of memory" in the "Important Flow Collector configuration considerations" section.

Prerequisites

If you choose to use Loki, install the Loki Operator version 5.7+.
You must have
```
cluster-admin
```
privileges.
One of the following supported architectures is required:
```
amd64
```
,
```
ppc64le
```
,
```
arm64
```
, or
```
s390x
```
.
Any CPU supported by Red Hat Enterprise Linux (RHEL) 9.
Must be configured with OVN-Kubernetes or OpenShift SDN as the main network plugin, and optionally using secondary interfaces with Multus and SR-IOV.

Note

Additionally, this installation example uses the

netobserv

namespace, which is used across all components. You can optionally use a different namespace.

Procedure

In the OpenShift Container Platform web console, click Operators OperatorHub.
Choose Network Observability Operator from the list of available Operators in the OperatorHub, and click Install.

Select the checkbox

Enable Operator recommended cluster monitoring on this Namespace

Navigate to Operators Installed Operators. Under Provided APIs for Network Observability, select the Flow Collector link.
Follow the Network Observability FlowCollector setup wizard.
Click Create.

Verification

To confirm this was successful, when you navigate to Observe you should see Network Traffic listed in the options.

In the absence of Application Traffic within the OpenShift Container Platform cluster, default filters might show that there are "No results", which results in no visual flow. Beside the filter selections, select Clear all filters to see the flow.

4.4. Enabling multi-tenancy in network observability
Link kopieren

Multi-tenancy in the Network Observability Operator allows and restricts individual user access, or group access, to the flows stored in Loki and or Prometheus. Access is enabled for project administrators. Project administrators who have limited access to some namespaces can access flows for only those namespaces.

For Developers, multi-tenancy is available for both Loki and Prometheus but requires different access rights.

Prerequisite

If you are using Loki, you have installed at least Loki Operator version 5.7.
You must be logged in as a project administrator.

Procedure

For per-tenant access, you must have the

netobserv-loki-reader

cluster role and the

netobserv-metrics-reader

namespace role to use the developer perspective. Run the following commands for this level of access:

$ oc adm policy add-cluster-role-to-user netobserv-loki-reader <user_group_or_name>

$ oc adm policy add-role-to-user netobserv-metrics-reader <user_group_or_name> -n <namespace>

For cluster-wide access, non-cluster-administrators must have the

netobserv-loki-reader

cluster-monitoring-view

, and

netobserv-metrics-reader

cluster roles. In this scenario, you can use either the admin perspective or the developer perspective. Run the following commands for this level of access:

$ oc adm policy add-cluster-role-to-user netobserv-loki-reader <user_group_or_name>

$ oc adm policy add-cluster-role-to-user cluster-monitoring-view <user_group_or_name>

$ oc adm policy add-cluster-role-to-user netobserv-metrics-reader <user_group_or_name>

4.5. Important Flow Collector configuration considerations
Link kopieren

Once you create the

FlowCollector

instance, you can reconfigure it, but the pods are terminated and recreated again, which can be disruptive. Therefore, you can consider configuring the following options when creating the

FlowCollector

for the first time:

Additional resources

4.5.1. Migrating removed stored versions of the FlowCollector CRD
Link kopieren

Network Observability Operator version 1.6 removes the old and deprecated

v1alpha1

version of the

FlowCollector

API. If you previously installed this version on your cluster, it might still be referenced in the

storedVersion

of the

FlowCollector

CRD, even if it is removed from the etcd store, which blocks the upgrade process. These references need to be manually removed.

There are two options to remove stored versions:

Use the Storage Version Migrator Operator.
Uninstall and reinstall the Network Observability Operator, ensuring that the installation is in a clean state.

Prerequisites

You have an older version of the Operator installed, and you want to prepare your cluster to install the latest version of the Operator. Or you have attempted to install the Network Observability Operator 1.6 and run into the error:
```
Failed risk of data loss updating "flowcollectors.flows.netobserv.io": new CRD removes version v1alpha1 that is listed as a stored version on the existing CRD
```
.

Procedure

Verify that the old

FlowCollector

CRD version is still referenced in the

storedVersion

$ oc get crd flowcollectors.flows.netobserv.io -ojsonpath='{.status.storedVersions}'

If
```
v1alpha1
```
appears in the list of results, proceed with Step a to use the Kubernetes Storage Version Migrator or Step b to uninstall and reinstall the CRD and the Operator.
1. Option 1: Kubernetes Storage Version Migrator: Create a YAML to define the
  StorageVersionMigration
  object, for example
  migrate-flowcollector-v1alpha1.yaml
  :
  apiVersion: migration.k8s.io/v1alpha1 kind: StorageVersionMigration metadata: name: migrate-flowcollector-v1alpha1 spec: resource: group: flows.netobserv.io resource: flowcollectors version: v1alpha1
  1. Save the file.
  2. Apply the
    
    StorageVersionMigration
    
    by running the following command:
    
    $ oc apply -f migrate-flowcollector-v1alpha1.yaml
  3. Update the
    
    FlowCollector
    
    CRD to manually remove
    
    v1alpha1
    
    from the
    
    storedVersion
    
    :
    
    $ oc edit crd flowcollectors.flows.netobserv.io
2. Option 2: Reinstall: Save the Network Observability Operator 1.5 version of the
  FlowCollector
  CR to a file, for example
  flowcollector-1.5.yaml
  .
  $ oc get flowcollector cluster -o yaml > flowcollector-1.5.yaml
  1. Follow the steps in "Uninstalling the Network Observability Operator", which uninstalls the Operator and removes the existing
    
    FlowCollector
    
    CRD.
  2. Install the Network Observability Operator latest version, 1.6.0.
  3. Create the
    
    FlowCollector
    
    using backup that was saved in Step b.

Verification

Run the following command:

$ oc get crd flowcollectors.flows.netobserv.io -ojsonpath='{.status.storedVersions}'

The list of results should no longer show

v1alpha1

and only show the latest version,

v1beta1

Additional resources

Kubernetes Storage Version Migrator Operator

4.6. Installing Kafka (optional)
Link kopieren

The Kafka Operator is supported for large scale environments. Kafka provides high-throughput and low-latency data feeds for forwarding network flow data in a more resilient, scalable way. You can install the Kafka Operator as Red Hat AMQ Streams from the Operator Hub, just as the Loki Operator and Network Observability Operator were installed. Refer to "Configuring the FlowCollector resource with Kafka" to configure Kafka as a storage option.

Note

To uninstall Kafka, refer to the uninstallation process that corresponds with the method you used to install.

4.7. Uninstalling the Network Observability Operator
Link kopieren

You can uninstall the Network Observability Operator using the OpenShift Container Platform web console Operator Hub, working in the Operators Installed Operators area.

Procedure

Remove the
```
FlowCollector
```
custom resource.
1. Click Flow Collector, which is next to the Network Observability Operator in the Provided APIs column.
2. Click the options menu for the cluster and select Delete FlowCollector.
Uninstall the Network Observability Operator.
1. Navigate back to the Operators Installed Operators area.
2. Click the options menu next to the Network Observability Operator and select Uninstall Operator.
3. Home Projects and select
  openshift-netobserv-operator
4. Navigate to Actions and select Delete Project
Remove the
```
FlowCollector
```
custom resource definition (CRD).
1. Navigate to Administration CustomResourceDefinitions.
2. Look for FlowCollector and click the options menu .
3. Select Delete CustomResourceDefinition.
  Important
  The Loki Operator and Kafka remain if they were installed and must be removed separately. Additionally, you might have remaining data stored in an object store, and a persistent volume that must be removed.

Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.

Chapter 4. Installing the Network Observability Operator

4.1. Network observability without Loki
Link kopieren

4.2. Installing the Loki Operator
Link kopieren

4.2.1. Creating a secret for Loki storage
Link kopieren

4.2.2. Creating a LokiStack custom resource
Link kopieren

4.2.3. Creating a new group for the cluster-admin user role
Link kopieren

4.2.4. Custom admin group access
Link kopieren

4.2.5. Loki deployment sizing
Link kopieren

4.2.6. LokiStack ingestion limits and health alerts
Link kopieren

4.3. Installing the Network Observability Operator
Link kopieren

4.4. Enabling multi-tenancy in network observability
Link kopieren

4.6. Installing Kafka (optional)
Link kopieren

4.7. Uninstalling the Network Observability Operator
Link kopieren

Lernen

Testen, kaufen und verkaufen

Communitys

Über Red Hat Dokumentation

Mehr Inklusion in Open Source

Über Red Hat

Theme

Red Hat legal and privacy links

Red Hat legal and privacy links

Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.

Chapter 4. Installing the Network Observability Operator

4.1. Network observability without LokiLink kopierenLink in die Zwischenablage kopiert!

4.2. Installing the Loki OperatorLink kopierenLink in die Zwischenablage kopiert!

4.2.1. Creating a secret for Loki storageLink kopierenLink in die Zwischenablage kopiert!

4.2.2. Creating a LokiStack custom resourceLink kopierenLink in die Zwischenablage kopiert!

4.2.3. Creating a new group for the cluster-admin user roleLink kopierenLink in die Zwischenablage kopiert!

4.2.4. Custom admin group accessLink kopierenLink in die Zwischenablage kopiert!

4.2.5. Loki deployment sizingLink kopierenLink in die Zwischenablage kopiert!

4.2.6. LokiStack ingestion limits and health alertsLink kopierenLink in die Zwischenablage kopiert!

4.3. Installing the Network Observability OperatorLink kopierenLink in die Zwischenablage kopiert!

4.4. Enabling multi-tenancy in network observabilityLink kopierenLink in die Zwischenablage kopiert!

4.6. Installing Kafka (optional)Link kopierenLink in die Zwischenablage kopiert!

4.7. Uninstalling the Network Observability OperatorLink kopierenLink in die Zwischenablage kopiert!

Lernen

Testen, kaufen und verkaufen

Communitys

Über Red Hat Dokumentation

Mehr Inklusion in Open Source

Über Red Hat

Theme

Red Hat legal and privacy links

Red Hat legal and privacy links

4.1. Network observability without Loki
Link kopieren

4.2. Installing the Loki Operator
Link kopieren

4.2.1. Creating a secret for Loki storage
Link kopieren

4.2.2. Creating a LokiStack custom resource
Link kopieren

4.2.3. Creating a new group for the cluster-admin user role
Link kopieren

4.2.4. Custom admin group access
Link kopieren

4.2.5. Loki deployment sizing
Link kopieren

4.2.6. LokiStack ingestion limits and health alerts
Link kopieren

4.3. Installing the Network Observability Operator
Link kopieren

4.4. Enabling multi-tenancy in network observability
Link kopieren

4.6. Installing Kafka (optional)
Link kopieren

4.7. Uninstalling the Network Observability Operator
Link kopieren