Chapter 2. Configuring the logging collector

Procedure

1. Modify the ClusterLogForwarder CR to add configuration for the http receiver input:

   Example ClusterLogForwarder CR

   apiVersion: observability.openshift.io/v1
   kind: ClusterLogForwarder
   metadata:
     name: <clusterlogforwarder_name> 

   1

   
     namespace: <namespace>
   # ...
   spec:
     serviceAccount:
       name: <service_account_name>
     inputs:
     - name: http-receiver 

   2

   
       type: receiver
       receiver:
         type: http 

   3

   
         port: 8443 

   4

   
         http:
           format: kubeAPIAudit 

   5

   
     outputs:
     - name: <output_name>
       type: http
       http:
         url: <url>
     pipelines: 

   6

   
       - name: http-pipeline
         inputRefs:
           - http-receiver
         outputRefs:
           - <output_name>
   # ...

   Copy to Clipboard Toggle word wrap

   1

   Specify a name for the ClusterLogForwarder CR.

   2

   Specify a name for your input receiver.

   3

   Specify the input receiver type as http.

   4

   Optional: Specify the port that the input receiver listens on. This must be a value between 1024 and 65535. The default value is 8443 if this is not specified.

   5

   Currently, only the kube-apiserver webhook format is supported for http input receivers.

   6

   Configure a pipeline for your input receiver.

2. Apply the changes to the ClusterLogForwarder CR by running the following command:

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

3. Verify that the collector is listening on the service that has a name in the <clusterlogforwarder_resource_name>-<input_name> format by running the following command:

   $ oc get svc

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)            AGE
   collector                 ClusterIP   172.30.85.239    <none>        24231/TCP          3m6s
   collector-http-receiver   ClusterIP   172.30.205.160   <none>        8443/TCP           3m6s

   Copy to Clipboard Toggle word wrap

   In the example, the service name is collector-http-receiver.

Verification

1. Extract the certificate authority (CA) certificate file by running the following command:

   $ oc extract cm/openshift-service-ca.crt -n <namespace>

   Copy to Clipboard Toggle word wrap
   Note

   If the CA in the cluster where the collectors are running changes, you must extract the CA certificate file again.

2. As an example, use the curl command to send logs by running the following command:

   $ curl --cacert <openshift_service_ca.crt> https://collector-http-receiver.<namespace>.svc:8443 -XPOST -d '{"<prefix>":"<message>"}'

   Copy to Clipboard Toggle word wrap

   Replace <openshift_service_ca.crt> with the extracted CA certificate file.

Procedure

1. Grant the collect-infrastructure-logs cluster role to the service account by running the following command:

   Example binding command

   $ oc adm policy add-cluster-role-to-user collect-infrastructure-logs -z logcollector

   Copy to Clipboard Toggle word wrap

2. Modify the ClusterLogForwarder CR to add configuration for the syslog receiver input:

   Example ClusterLogForwarder CR

   apiVersion: observability.openshift.io/v1
   kind: ClusterLogForwarder
   metadata:
     name: <clusterlogforwarder_name> 

   1

   
     namespace: <namespace>
   # ...
   spec:
     serviceAccount:
       name: <service_account_name> 

   2

   
     inputs:
       - name: syslog-receiver 

   3

   
         type: receiver
         receiver:
           type: syslog 

   4

   
           port: 10514 

   5

   
     outputs:
     - name: <output_name>
       lokiStack:
         authentication:
           token:
             from: serviceAccount
         target:
           name: logging-loki
           namespace: openshift-logging
       tls: 

   6

   
         ca:
           key: service-ca.crt
           configMapName: openshift-service-ca.crt
       type: lokiStack
   # ...
     pipelines: 

   7

   
       - name: syslog-pipeline
         inputRefs:
           - syslog-receiver
         outputRefs:
           - <output_name>
   # ...

   Copy to Clipboard Toggle word wrap

   1 2

   Use the service account that you granted the collect-infrastructure-logs permission in the previous step.

   3

   Specify a name for your input receiver.

   4

   Specify the input receiver type as syslog.

   5

   Optional: Specify the port that the input receiver listens on. This must be a value between 1024 and 65535.

   6

   If TLS configuration is not set, the default certificates will be used. For more information, run the command oc explain clusterlogforwarders.spec.inputs.receiver.tls.

   7

   Configure a pipeline for your input receiver.

3. Apply the changes to the ClusterLogForwarder CR by running the following command:

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

4. Verify that the collector is listening on the service that has a name in the <clusterlogforwarder_resource_name>-<input_name> format by running the following command:

   $ oc get svc

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)            AGE
   collector                   ClusterIP   172.30.85.239    <none>        24231/TCP          33m
   collector-syslog-receiver   ClusterIP   172.30.216.142   <none>        10514/TCP          2m20s

   Copy to Clipboard Toggle word wrap

   In this example output, the service name is collector-syslog-receiver.

Verification

1. Extract the certificate authority (CA) certificate file by running the following command:

   $ oc extract cm/openshift-service-ca.crt -n <namespace>

   Copy to Clipboard Toggle word wrap
   Note

   If the CA in the cluster where the collectors are running changes, you must extract the CA certificate file again.

2. As an example, use the curl command to send logs by running the following command:

   $ curl --cacert <openshift_service_ca.crt> collector-syslog-receiver.<namespace>.svc:10514  “test message”

   Copy to Clipboard Toggle word wrap

   Replace <openshift_service_ca.crt> with the extracted CA certificate file.

Procedure

1. Update the ClusterLogForwarder CR:

   apiVersion: observability.openshift.io/v1
   kind: ClusterLogForwarder
   metadata:
     name: my-forwarder
     namespace: openshift-logging
   spec:
     collector:
       networkPolicy:
         ruleSet: <RestrictIngressEgress_or_AllowAllIngressEgress>
     # ...

   Copy to Clipboard Toggle word wrap

   Where the value for the ruleSet field can be either RestrictIngressEgress or AllowAllIngressEgress.

2. Apply the configuration by running the following command:

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Update the LogFileMetricExporter CR:

   apiVersion: logging.openshift.io/v1alpha1
   kind: LogFileMetricExporter
   metadata:
     name: my-lfme
     namespace: openshift-logging
   spec:
     networkPolicy:
       ruleSet: <AllowAllIngressEgress_or_AllowIngressMetrics>
   #...

   Copy to Clipboard Toggle word wrap

   Where the value for the ruleSet field can be either AllowAllIngressEgress or AllowIngressMetrics.

2. Apply the configuration by running the following command:

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create an AdminNetworkPolicy rule:

   apiVersion: policy.networking.k8s.io/v1alpha1
   kind: AdminNetworkPolicy
   metadata:
     name: allow-logging-collector-delegation
   spec:
     priority: 50 # Adjust based on your cluster's ANP priority scheme. Lower number means higher priority
     subject:
       pods: # Target the pods in openshift-logging namespace
         namespaceSelector:
           matchLabels:
             kubernetes.io/metadata.name: openshift-logging
         podSelector:
           matchLabels:
             app.kubernetes.io/name: <name>
             app.kubernetes.io/instance: <instance>
             app.kubernetes.io/managed-by: cluster-logging-operator
             app.kubernetes.io/part-of: cluster-logging
             app.kubernetes.io/component: <component>
     ingress:
       - name: "delegate-to-collector-ingress"
         action: "Pass"
         from:
           # Select all pods in all namespaces as the source.
           - pods:
               namespaceSelector: {}
               podSelector: {}
     egress:
       - name: "delegate-to-collector-egress"
         action: "Pass"
         to:
           # Select all pods in all namespaces for internal traffic.
           - pods:
               namespaceSelector: {}
               podSelector: {}
           # Select all external destinations for egress traffic.
           - networks:
             - "0.0.0.0/0"
             - "::/0"

   Copy to Clipboard Toggle word wrap

   To apply this network policy to all the resources in a namespace, remove app.kubernetes.io/name, app.kubernetes.io/instance, and app.kubernetes.io/component fields. To apply the admin network policy to specific resources, define the fields as follows:

   • Replace <name> with the name of the collector or the LogFileMetricExporter resource.
   • Replace <instance> with the instance name of the collector or the LogFileMetricExporter resource.
   • Replace <component> with the value collector or logfilesmetricexporter.

2. Apply the AdminNetworkPolicy rule by running the following command:

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap