Chapter 8. Redis high availability (HA) support for 3scale
There are known issues with Redis high availability (HA) support for 3scale. For more information, see the Red Hat 3scale API Management 2.5 release notes, Chapter 8. Known issues in the release notes.
8.1. Introduction
High availability (HA) is provided for most components by the OpenShift Container Platform (OCP). For more information see OpenShift Container Platform 3.11 Chapter 30. High Availability.
The database components for HA in 3scale include:
-
system-redis: provides temporary storage for background jobs for 3scale and it is also used as a message bus for Ruby processes ofsystem-apppods. -
backend-redis: used for statistics storage and temporary job storage.
Both system-redis and backend-redis can be replaced by the Redis Cluster (open-source or Redis Labs).
The following env vars can be set into system-(app,sidekiq,sphinx) deployment configurations, though it is only a requirement for Redis Enterprise:
-
MESSAGE_BUS_REDIS_URL(a redis URL) -
REDIS_NAMESPACE(a short string to namespace Sidekiq’s Redis keys) -
MESSAGE_BUS_REDIS_NAMESPACE(a short string to namespace System message bus’s Redis keys)
A new pod is created automatically when the Redis pod dies or is killed by OCP and the data is restored from the persistent storage, so the pod continues to work. In the scenario described, there would be a small amount of downtime while the new pod is being started. This is due to the limitation that Redis does not support a multi-master setup. Downtime can be reduced by preloading the Redis images onto all nodes that have Redis deployed to them, which will speed up the pod restart.
8.2. Setting up Redis for zero downtime
If zero downtime is required, Redis would need to be set up outside of OCP. There are several ways to set it up using the configuration options of 3scale pods:
- Set up your own self-managed Redis
- Use Redis Sentinel: Reference Redis Sentinel Documentation
Redis provided as a service:
For example by:
- Amazon ElastiCache
- Redis Labs
Red Hat does not provide support for the above mentioned services. The mention of any such services does not imply endorsement by Red Hat of the products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) any external content.
8.3. Configurating backend components for 3scale
There are configuration settings in 3scale 2.5 to configure Redis HA (failover) for the backend component. They can be set as environment variables in the following deployment configurations: backend-cron, backend-listener, and backend-worker:
CONFIG_REDIS_SENTINEL_HOSTSandCONFIG_QUEUES_SENTINEL_HOSTS:A comma-separated list of Sentinel hosts for the main statistics database and the Resque background job database.
NoteValues should be in the format: name:value
<host>:<port>For example:host1:26379,host2:26379, orhost3:26379CONFIG_REDIS_SENTINEL_ROLEandCONFIG_QUEUES_SENTINEL_ROLE:The role of each Sentinels group, either master or slave. Currently only master (default) is supported.
This makes the value of CONFIG_REDIS_PROXY and CONFIG_QUEUES_MASTER_NAME take the meaning of Sentinel group name instead of a specific server.
When no Sentinel hosts are configured, the environment variables
CONFIG_REDIS_PROXYandCONFIG_QUEUES_MASTER_NAMEcan use URLs and support password-protected databases, for example:CONFIG_REDIS_PROXY=redis://user:password@server:port/database- The connection is then established with the password-protected Redis instance.
-
When Sentinel hosts are configured, the password must be set in the Sentinel configuration and use the Sentinel group name instead:
CONFIG_REDIS_PROXY=master_group
