Chapter 8. Troubleshooting Builds

The DEBUG configuration flag in Red Hat Quay prevents builder instances from shutting down automatically after builds complete or fail. Set this flag to True to debug builder node issues.

  EXECUTORS:
    - EXECUTOR: ec2
      DEBUG: true
      ...
    - EXECUTOR: kubernetes
      DEBUG: true
      ...

When set to True, the debug feature prevents the build nodes from shutting down after the quay-builder service is done or fails. It also prevents the build manager from cleaning up the instances by terminating EC2 instances or deleting Kubernetes jobs. This allows debugging builder node issues.

Debugging should not be set in a production cycle. The lifetime service still exists; for example, the instance still shuts down after approximately two hours. When this happens, EC2 instances are terminated and Kubernetes jobs are completed.

Enabling debug also affects the ALLOWED_WORKER_COUNT because the unterminated instances and jobs still count toward the total number of running workers. As a result, the existing builder workers must be manually deleted if ALLOWED_WORKER_COUNT is reached to be able to schedule new builds.

To troubleshoot OpenShift Container Platform and Kubernetes builds in Red Hat Quay, you can create a port forwarding tunnel to the builder pod and access it via SSH. Check the quay-builder service logs to diagnose build issues.