Chapter 5. Setting Up Clair Security Scanning
Once you have created the necessary key and pem files from the Red Hat Quay config UI, you are ready to start up the Clair container and associated database. Once that is done, you an restart your Red Hat Quay cluster to have those changes take effect.
Procedures for running the Clair container and associated database are different on OpenShift than they are for running those containers directly on a host.
5.1. Run Clair on a Red Hat Quay OpenShift deployment
To run the Clair image scanning container and its associated database on an OpenShift environment with your Red Hat Quay cluster, see Add Clair image scanning to Red Hat Quay.
5.2. Run Clair on a Red Hat Quay Basic or HA deployment
To run Clair and its associated database on non-OpenShift environments (directly on a host), you need to:
- Start up a database
- Configure and start Clair
5.2.1. Get Postgres and Clair
In order to run Clair, a database is required. For production deployments, MySQL is not supported. For production, we recommend you use PostgreSQL or other supported database:
- Running on machines other than those running Red Hat Quay
- Ideally with automatic replication and failover
For testing purposes, a single PostgreSQL instance can be started locally:
To start Postgres locally, do the following:
# docker run --name postgres -p 5432:5432 -d postgres # sleep 5 # docker run --rm --link postgres:postgres postgres \ sh -c 'echo "create database clairtest" | psql -h \ "$POSTGRES_PORT_5432_TCP_ADDR" -p \ "$POSTGRES_PORT_5432_TCP_PORT" -U postgres'
The configuration string for this test database is:
postgresql://postgres@{DOCKER HOST GOES HERE}:5432/clairtest?sslmode=disable
Pull the security-enabled Clair image:
docker pull quay.io/redhat/clair-jwt:v3.2.2
Make a configuration directory for Clair
# mkdir clair-config # cd clair-config
5.2.2. Configure Clair
Clair can run either as a single instance or in high-availability mode. It is recommended to run more than a single instance of Clair, ideally in an auto-scaling group with automatic healing.
-
Create a
config.yaml
file to be used in the Clair config directory (/clair/config
) from one of the two Clair configuration files shown here. - If you are doing a high-availability installation, go through the procedure in Authentication for high-availability scanners to create a Key ID and Private Key (PEM).
- Save the Private Key (PEM) to a file (such as, $HOME/config/security_scanner.pem).
Replace the value of key_id (CLAIR_SERVICE_KEY_ID) with the Key ID you generated and the value of private_key_path with the location of the PEM file (for example, /config/security_scanner.pem).
For example, those two value might now appear as:
key_id: { 4fb9063a7cac00b567ee921065ed16fed7227afd806b4d67cc82de67d8c781b1 } private_key_path: /clair/config/security_scanner.pem
- Change other values in the configuration file as needed.
5.2.2.1. Clair configuration: High availability
clair: database: type: pgsql options: # A PostgreSQL Connection string pointing to the Clair Postgres database. # Documentation on the format can be found at: http://www.postgresql.org/docs/9.4/static/libpq-connect.html source: { POSTGRES_CONNECTION_STRING } cachesize: 16384 api: # The port at which Clair will report its health status. For example, if Clair is running at # https://clair.mycompany.com, the health will be reported at # http://clair.mycompany.com:6061/health. healthport: 6061 port: 6062 timeout: 900s # paginationkey can be any random set of characters. *Must be the same across all Clair instances*. paginationkey: "XxoPtCUzrUv4JV5dS+yQ+MdW7yLEJnRMwigVY/bpgtQ=" updater: # interval defines how often Clair will check for updates from its upstream vulnerability databases. interval: 6h notifier: attempts: 3 renotifyinterval: 1h http: # QUAY_ENDPOINT defines the endpoint at which Quay is running. # For example: https://myregistry.mycompany.com endpoint: { QUAY_ENDPOINT }/secscan/notify proxy: http://localhost:6063 jwtproxy: signer_proxy: enabled: true listen_addr: :6063 ca_key_file: /certificates/mitm.key # Generated internally, do not change. ca_crt_file: /certificates/mitm.crt # Generated internally, do not change. signer: issuer: security_scanner expiration_time: 5m max_skew: 1m nonce_length: 32 private_key: type: preshared options: # The ID of the service key generated for Clair. The ID is returned when setting up # the key in [Quay Setup](security-scanning.md) key_id: { CLAIR_SERVICE_KEY_ID } private_key_path: /clair/config/security_scanner.pem verifier_proxies: - enabled: true # The port at which Clair will listen. listen_addr: :6060 # If Clair is to be served via TLS, uncomment these lines. See the "Running Clair under TLS" # section below for more information. # key_file: /clair/config/clair.key # crt_file: /clair/config/clair.crt verifier: # CLAIR_ENDPOINT is the endpoint at which this Clair will be accessible. Note that the port # specified here must match the listen_addr port a few lines above this. # Example: https://myclair.mycompany.com:6060 audience: { CLAIR_ENDPOINT } upstream: http://localhost:6062 key_server: type: keyregistry options: # QUAY_ENDPOINT defines the endpoint at which Quay is running. # Example: https://myregistry.mycompany.com registry: { QUAY_ENDPOINT }/keys/
5.2.2.2. Clair configuration: Single instance
clair: database: type: pgsql options: # A PostgreSQL Connection string pointing to the Clair Postgres database. # Documentation on the format can be found at: http://www.postgresql.org/docs/9.4/static/libpq-connect.html source: { POSTGRES_CONNECTION_STRING } cachesize: 16384 api: # The port at which Clair will report its health status. For example, if Clair is running at # https://clair.mycompany.com, the health will be reported at # http://clair.mycompany.com:6061/health. healthport: 6061 port: 6062 timeout: 900s # paginationkey can be any random set of characters. *Must be the same across all Clair instances*. paginationkey: updater: # interval defines how often Clair will check for updates from its upstream vulnerability databases. interval: 6h notifier: attempts: 3 renotifyinterval: 1h http: # QUAY_ENDPOINT defines the endpoint at which Quay is running. # For example: https://myregistry.mycompany.com endpoint: { QUAY_ENDPOINT }/secscan/notify proxy: http://localhost:6063 jwtproxy: signer_proxy: enabled: true listen_addr: :6063 ca_key_file: /certificates/mitm.key # Generated internally, do not change. ca_crt_file: /certificates/mitm.crt # Generated internally, do not change. signer: issuer: security_scanner expiration_time: 5m max_skew: 1m nonce_length: 32 private_key: type: autogenerated options: rotate_every: 12h key_folder: /clair/config/ key_server: type: keyregistry options: # QUAY_ENDPOINT defines the endpoint at which Quay is running. # For example: https://myregistry.mycompany.com registry: { QUAY_ENDPOINT }/keys/ verifier_proxies: - enabled: true # The port at which Clair will listen. listen_addr: :6060 # If Clair is to be served via TLS, uncomment these lines. See the "Running Clair under TLS" # section below for more information. # key_file: /clair/config/clair.key # crt_file: /clair/config/clair.crt verifier: # CLAIR_ENDPOINT is the endpoint at which this Clair will be accessible. Note that the port # specified here must match the listen_addr port a few lines above this. # Example: https://myclair.mycompany.com:6060 audience: { CLAIR_ENDPOINT } upstream: http://localhost:6062 key_server: type: keyregistry options: # QUAY_ENDPOINT defines the endpoint at which Quay is running. # Example: https://myregistry.mycompany.com registry: { QUAY_ENDPOINT }/keys/
5.2.3. Configuring Clair for TLS
To configure Clair to run with TLS, a few additional steps are required.
5.2.3.1. Using certificates from a public CA
For certificates that come from a public certificate authority, follow these steps:
- Generate a TLS certificate and key pair for the DNS name at which Clair will be accessed
-
Place these files as
clair.crt
andclair.key
in your Clair configuration directory -
Uncomment the
key_file
andcrt_file
lines underverifier_proxies
in your Clairconfig.yaml
If your certificates use a public CA, you are now ready to run Clair. If you are using your own certificate authority, configure Clair to trust it below.
5.2.3.2. Configuring trust of self-signed SSL
Similar to the process for setting up Docker to trust your self-signed certificates, Clair must also be configured to trust your certificates. Using the same CA certificate bundle used to configure Docker, complete the following steps:
-
Rename the same CA certificate bundle used to set up Quay Registry to
ca.crt
Make sure the
ca.crt
file is mounted inside the Clair container under/etc/pki/ca-trust/source/anchors/
as in the example below:NoteAdd
--loglevel=debug
to thedocker run
command line for the clair container to enable debug level logging.# docker run --restart=always -p 6060:6060 -p 6061:6061 \ -v /path/to/clair/config/directory:/clair/config \ -v /path/to/quay/cert/ca.crt:/etc/pki/ca-trust/source/anchors/ca.crt \ quay.io/redhat/clair-jwt:v3.2.2
Now Clair will be able to trust the source of your TLS certificates and use them to secure communication between Clair and Quay.
5.2.4. Using Clair data sources
Before scanning container images, Clair tries to figure out the operating system on which the container was built. It does this by looking for specific filenames inside that image (see Table 1). Once Clair knows the operating system, it uses specific security databases to check for vulnerabilities (see Table 2).
Operating system | Files identifying OS type |
---|---|
Redhat/CentOS/Oracle | etc/oracle-release etc/centos-release etc/redhat-release etc/system-release |
Alpine | etc/alpine-release |
Debian/Ubuntu: | etc/os-release usr/lib/os-release etc/apt/sources.list |
Ubuntu | etc/lsb-release |
The data sources that Clair uses to scan containers are shown in Table 2.
You must be sure that Clair has access to all listed data sources by whitelisting access to each data source’s location. You might need to add a wild-card character (*) at the end of some URLS that may not be fully complete because they are dynamically built by code.
Data source | Data collected | Whitelist links | Format | License |
---|---|---|---|---|
Debian 6, 7, 8, unstable namespaces | ||||
Ubuntu 12.04, 12.10, 13.04, 14.04, 14.10, 15.04, 15.10, 16.04 namespaces | ||||
CentOS 5, 6, 7 namespace | ||||
Oracle Linux 5, 6, 7 namespaces | ||||
Alpine 3.3, 3.4, 3.5 namespaces | ||||
Generic vulnerability metadata | N/A |
5.2.5. Run Clair
Execute the following command to run Clair:
# docker run --restart=always -p 6060:6060 -p 6061:6061 \ -v /path/to/clair/config/directory:/clair/config \ quay.io/redhat/clair-jwt:v3.2.2
Output similar to the following will be seen on success:
2016-05-04 20:01:05,658 CRIT Supervisor running as root (no user in config file) 2016-05-04 20:01:05,662 INFO supervisord started with pid 1 2016-05-04 20:01:06,664 INFO spawned: 'jwtproxy' with pid 8 2016-05-04 20:01:06,666 INFO spawned: 'clair' with pid 9 2016-05-04 20:01:06,669 INFO spawned: 'generate_mitm_ca' with pid 10 time="2016-05-04T20:01:06Z" level=info msg="No claims verifiers specified, upstream should be configured to verify authorization" time="2016-05-04T20:01:06Z" level=info msg="Starting reverse proxy (Listening on ':6060')" 2016-05-04 20:01:06.715037 I | pgsql: running database migrations time="2016-05-04T20:01:06Z" level=error msg="Failed to create forward proxy: open /certificates/mitm.crt: no such file or directory" goose: no migrations to run. current version: 20151222113213 2016-05-04 20:01:06.730291 I | pgsql: database migration ran successfully 2016-05-04 20:01:06.730657 I | notifier: notifier service is disabled 2016-05-04 20:01:06.731110 I | api: starting main API on port 6062. 2016-05-04 20:01:06.736558 I | api: starting health API on port 6061. 2016-05-04 20:01:06.736649 I | updater: updater service is disabled. 2016-05-04 20:01:06,740 INFO exited: jwtproxy (exit status 0; not expected) 2016-05-04 20:01:08,004 INFO spawned: 'jwtproxy' with pid 1278 2016-05-04 20:01:08,004 INFO success: clair entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2016-05-04 20:01:08,004 INFO success: generate_mitm_ca entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) time="2016-05-04T20:01:08Z" level=info msg="No claims verifiers specified, upstream should be configured to verify authorization" time="2016-05-04T20:01:08Z" level=info msg="Starting reverse proxy (Listening on ':6060')" time="2016-05-04T20:01:08Z" level=info msg="Starting forward proxy (Listening on ':6063')" 2016-05-04 20:01:08,541 INFO exited: generate_mitm_ca (exit status 0; expected) 2016-05-04 20:01:09,543 INFO success: jwtproxy entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
To verify Clair is running, execute the following command:
curl -X GET -I http://path/to/clair/here:6061/health
If a 200 OK
code is returned, Clair is running:
HTTP/1.1 200 OK Server: clair Date: Wed, 04 May 2016 20:02:16 GMT Content-Length: 0 Content-Type: text/plain; charset=utf-8
Once Clair and its associated database are running, you man need to restart your quay application for the changes to take effect.