Chapter 2. 3scale template-based upgrade guide: from 2.8 to 2.9.1
This section contains information about upgrading Red Hat 3scale API Management from version 2.8 to 2.9.1, in a template-based deployment.
Important considerations
- In order to understand the required conditions and procedure, read the entire upgrade guide before applying the listed steps. The upgrade process disrupts the provision of the service until the procedure finishes. Due to this disruption, make sure to have a maintenance window.
- If your 3scale template-based installation runs with Oracle Database, refer to the Support to Oracle Databases in the 3scale 2.9.1 Release Notes.
2.1. Prerequisites to perform the upgrade
This section describes the required configurations, tasks and tools to upgrade 3scale from 2.8 to 2.9.1 in a template-based installation.
2.1.1. Configurations
- 3scale supports upgrade paths from 2.8 to 2.9.1 with templates on OpenShift 3.11.
2.1.2. Preliminary tasks
- Ensure your OpenShift CLI tool is configured in the same project where 3scale is deployed.
- Perform a backup of the database you are using with 3scale. The procedure of the backup is specific to each database type and setup.
2.1.3. Tools
You need these tools to perform the upgrade:
- 3scale 2.8 deployed with templates in an OpenShift 3.11 project.
- Bash shell: To run the commands detailed in the upgrade procedure.
- base64: To encode and decode secret information.
- jq: For JSON transformation purposes.
2.2. Upgrading from 2.8 to 2.9.1 in a template-based installation
Follow the procedure described in this section to upgrade 3scale 2.8 to 2.9.1 in a template-based installation.
To start with the upgrade, go to the project where 3scale is deployed.
$ oc project <3scale-project>
Then, follow these steps in this order:
2.2.1. Creating a backup of the 3scale project
Previous step
None.
Current step
This step lists the actions necessary to create a backup of the 3scale project.
Depending on the database used with 3scale, set ${SYSTEM_DB} with one of the following values:
-
If the database is MySQL,
SYSTEM_DB=system-mysql
. -
If the database is PostgreSQL,
SYSTEM_DB=system-postgresql
.
-
If the database is MySQL,
Create a back-up file with the existing DeploymentConfigs:
THREESCALE_DC_NAMES="apicast-production apicast-staging backend-cron backend-listener backend-redis backend-worker system-app system-memcache ${SYSTEM_DB} system-redis system-sidekiq system-sphinx zync zync-database zync-que" for component in ${THREESCALE_DC_NAMES}; do oc get --export -o yaml dc ${component} > ${component}_dc.yml ; done
Backup all existing OpenShift resources in the project that are exported through the
export all
command:oc get -o yaml --export all > threescale-project-elements.yaml
Create a back-up file with the additional elements that are not exported with the
export all
command:for object in rolebindings serviceaccounts secrets imagestreamtags cm rolebindingrestrictions limitranges resourcequotas pvc templates cronjobs statefulsets hpa deployments replicasets poddisruptionbudget endpoints do oc get -o yaml --export $object > $object.yaml done
- Verify that all of the generated files are not empty, and that all of them have the expected content.
2.2.2. Removing the BASE_URL field
Previous step
Current step
This section explains how to remove the BASE_URL field from the system-master-apicast
secret. This field is removed because no component uses it in 3scale 2.9
Run the following command:
oc patch secret system-master-apicast --type=json -p='[{"op": "remove", "path": "/data/BASE_URL"}]'
Confirm that the BASE_URL field has been successfully removed:
oc get secret system-master-apicast -o json | jq .data
2.2.3. Updating 3scale version number
Previous step
Current step
This step updates the 3scale release version number from 2.8 to 2.9 in the system-environment
ConfigMap. AMP_RELEASE is a ConfigMap entry referenced in some DeploymentConfig container environments.
To patch AMP_RELEASE, run this command:
oc patch cm system-environment --patch '{"data": {"AMP_RELEASE": "2.9"}}'
Confirm that the AMP_RELEASE key in the system-environment ConfigMap has the
2.9
value:oc get cm system-environment -o json | jq '.data["AMP_RELEASE"]'
Next step
2.2.4. Upgrading 3scale images
Previous step
Current step
This step updates the 3scale images required for the upgrade process.
2.2.4.1. Patch the system
image
Create the new image stream tag:
oc patch imagestream/amp-system --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "AMP system 2.9"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/3scale-amp2/system-rhel7:3scale2.9"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
To continue the procedure, consider the database used with your 3scale deployment:
- If the database is Oracle DB, follow the steps listed in Section 2.2.4.1.1, “Patching the system image: 3scale with Oracle Database”
- If the database is different from Oracle DB, follow the steps listed in Section 2.2.4.1.2, “Patching the system image: 3scale with other databases”
2.2.4.1.1. Patching the system image: 3scale with Oracle Database
To start patching the system image of 3scale with an Oracle Database, perform one of the procedures below according to your database version:
Patch the system-app ImageChangeTrigger:
Remove the latest trigger:
oc set triggers dc/system-app --from-image=amp-system:latest --containers=system-master,system-developer,system-provider --remove
Add the new version-specific trigger:
oc set triggers dc/system-app --from-image=amp-system:2.9-oracle --containers=system-master,system-developer,system-provider
This triggers a redeployment of
system-app
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.Remove the
:latest
tag from theImageStream
:oc tag -d amp-system:latest
Patch the
system-sidekiq
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/system-sidekiq --from-image=amp-system:latest --containers=system-sidekiq,check-svc --remove
Add the new version-specific trigger:
oc set triggers dc/system-sidekiq --from-image=amp-system:2.9-oracle --containers=system-sidekiq,check-svc
This triggers a redeployment of
system-sidekiq
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.
Patch the
system-sphinx
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/system-sphinx --from-image=amp-system:latest --containers=system-sphinx,system-master-svc --remove
Add the new version-specific trigger:
oc set triggers dc/system-sphinx --from-image=amp-system:2.9-oracle --containers=system-sphinx,system-master-svc
This triggers a redeployment of
system-sphinx
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.
- Scale 3scale back if you scaled it down.
2.2.4.1.2. Patching the system image: 3scale with other databases
Patch the
system-app
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/system-app --from-image=amp-system:latest --containers=system-master,system-developer,system-provider --remove
Add the new version-specific trigger:
oc set triggers dc/system-app --from-image=amp-system:2.9 --containers=system-master,system-developer,system-provider
This triggers a redeployment of
system-app
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.
Patch the
system-sidekiq
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/system-sidekiq --from-image=amp-system:latest --containers=system-sidekiq,check-svc --remove
Add the new version-specific trigger:
oc set triggers dc/system-sidekiq --from-image=amp-system:2.9 --containers=system-sidekiq,check-svc
This triggers a redeployment of
system-sidekiq
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.
Patch the
system-sphinx
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/system-sphinx --from-image=amp-system:latest --containers=system-sphinx,system-master-svc --remove
Add the new version-specific trigger:
oc set triggers dc/system-sphinx --from-image=amp-system:2.9 --containers=system-sphinx,system-master-svc
This triggers a redeployment of
system-sphinx
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.
2.2.4.2. Patch the apicast
image
Patch the
amp-apicast
image stream:oc patch imagestream/amp-apicast --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "AMP APIcast 2.9"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/3scale-amp2/apicast-gateway-rhel8:3scale2.9"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
Patch the
apicast-staging
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/apicast-staging --from-image=amp-apicast:latest --containers=apicast-staging --remove
Add the new version-specific trigger:
oc set triggers dc/apicast-staging --from-image=amp-apicast:2.9 --containers=apicast-staging
This triggers a redeployment of
apicast-staging
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.
Patch the
apicast-production
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/apicast-production --from-image=amp-apicast:latest --containers=apicast-production,system-master-svc --remove
Add the new version-specific trigger:
oc set triggers dc/apicast-production --from-image=amp-apicast:2.9 --containers=apicast-production,system-master-svc
This triggers a redeployment of
apicast-production
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.Remove the
:latest
tag from theImageStream
:oc tag -d amp-apicast:latest
2.2.4.3. Patch the backend
image
Patch the
amp-backend
image stream:oc patch imagestream/amp-backend --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "AMP Backend 2.9"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/3scale-amp2/backend-rhel7:3scale2.9"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
Patch the
backend-listener
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/backend-listener --from-image=amp-backend:latest --containers=backend-listener --remove
Add the new version-specific trigger:
oc set triggers dc/backend-listener --from-image=amp-backend:2.9 --containers=backend-listener
This triggers a redeployment of
backend-listener
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.
Patch the
backend-worker
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/backend-worker --from-image=amp-backend:latest --containers=backend-worker,backend-redis-svc --remove
Add the new version-specific trigger:
oc set triggers dc/backend-worker --from-image=amp-backend:2.9 --containers=backend-worker,backend-redis-svc
This triggers a redeployment of
backend-worker
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.Remove the
:latest
tag from theImageStream
:oc tag -d amp-backend:latest
Patch the
backend-cron
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/backend-cron --from-image=amp-backend:latest --containers=backend-cron,backend-redis-svc --remove
Add the new version-specific trigger:
oc set triggers dc/backend-cron --from-image=amp-backend:2.9 --containers=backend-cron,backend-redis-svc
This triggers a redeployment of
backend-cron
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.
2.2.4.4. Patch the zync
image
Patch the
amp-zync
image stream:oc patch imagestream/amp-zync --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "AMP Zync 2.9"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/3scale-amp2/zync-rhel7:3scale2.9"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
Patch the
zync
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/zync --from-image=amp-zync:latest --containers=zync,zync-db-svc --remove
Add the new version-specific trigger:
oc set triggers dc/zync --from-image=amp-zync:2.9 --containers=zync,zync-db-svc
This triggers a redeployment of
zync
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.
Patch the
zync-que
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/zync-que --from-image=amp-zync:latest --containers=que --remove
Add the new version-specific trigger:
oc set triggers dc/zync-que --from-image=amp-zync:2.9 --containers=que
This triggers a redeployment of
zync-que
. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.Remove the
:latest
tag from theImageStream
:oc tag -d amp-zync:latest
2.2.4.5. Patch the system-memcached
image
Patch the
system-memcached
ImageStream:oc patch imagestream/system-memcached --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "System 2.9 Memcached"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/3scale-amp2/memcached-rhel7:3scale2.9"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
Patch the
system-memcache
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/system-memcache --from-image=system-memcached:latest --containers=memcache --remove
Add the new version-specific trigger:
oc set triggers dc/system-memcache --from-image=system-memcached:2.9 --containers=memcache
This triggers a redeployment of the
system-memcache
DeploymentConfig. Wait until it is redeployed, its corresponding new pods are ready, and the old ones terminated.Remove the
:latest
tag from theImageStream
:oc tag -d system-memcached:latest
2.2.4.6. Patch the zync-database-postgresql
image
Patch the
zync-database-postgresql
image stream:oc patch imagestream/zync-database-postgresql --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "Zync 2.9 PostgreSQL"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/rhscl/postgresql-10-rhel7"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
This patch command updates the
zync-database-postgresql
image stream to contain the 2.9 tag. You can verify that the 2.9 tag has been created with these steps:Run this command:
oc get is zync-database-postgresql
- Check that the Tags column shows the 2.9 tag.
Patch the
zync-database
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/zync-database --from-image=zync-database-postgresql:latest --containers=postgresql --remove
Add the new version-specific trigger:
oc set triggers dc/zync-database --from-image=zync-database-postgresql:2.9 --containers=postgresql
In case there are new updates on the image, this patch might also trigger a redeployment of the
zync-database
DeploymentConfig. If this happens, wait until the new pods are redeployed and ready, and the old pods are terminated.Remove the
:latest
tag from theImageStream
:oc tag -d zync-database-postgresql:latest
2.2.4.7. Additional image changes
If one or more of the following DeploymentConfigs are available in your 3scale 2.8 installation, click the links that apply to obtain more information on how to proceed:
2.2.4.7.1. backend-redis
DeploymentConfig
If the backend-redis
DeploymentConfig exists in your current 3scale installation, patch the redis
image for backend-redis
:
Patch the
backend-redis
image stream:oc patch imagestream/backend-redis --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "Backend 2.9 Redis"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/rhscl/redis-32-rhel7:3.2"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
This patch updates the backend-redis image stream to contain the 2.9 tag. With the command below, you can confirm that the tag has been created if the Tags column shows 2.9:
oc get is backend-redis
Patch the
backend-redis
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/backend-redis --from-image=backend-redis:latest --containers=backend-redis --remove
Add the new version-specific trigger:
oc set triggers dc/backend-redis --from-image=backend-redis:2.9 --containers=backend-redis
In case there are new updates on the image, this patch might also trigger a redeployment of the
backend-redis
DeploymentConfig. If this happens, wait until the new pods are redeployed and ready, and the old pods are terminated.Remove the
:latest
tag from theImageStream
:oc tag -d backend-redis:latest
2.2.4.7.2. system-redis
DeploymentConfig
If the system-redis
DeploymentConfig exists in your current 3scale installation, patch the redis
image for system-redis
.
Patch the
system-redis
image stream:oc patch imagestream/system-redis --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "System 2.9 Redis"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/rhscl/redis-32-rhel7:3.2"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
This patch updates the
system-redis
image stream to contain the 2.9 tag. With the command below, you can confirm that the tag has been created if the Tags column shows 2.9:oc get is system-redis
Patch the
system-redis
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/system-redis --from-image=system-redis:latest --containers=system-redis --remove
Add the new version-specific trigger:
oc set triggers dc/system-redis --from-image=system-redis:2.9 --containers=system-redis
In case there are new updates on the image, this patch might also trigger a redeployment of the
system-redis
DeploymentConfig. If this happens, wait until the new pods are redeployed and ready, and the old pods are terminated.Remove the
:latest
tag from theImageStream
:oc tag -d system-redis:latest
2.2.4.7.3. system-mysql
DeploymentConfig
If the system-mysql
DeploymentConfig exists in your current 3scale installation, patch the MySQL image for system-mysql
.
Patch the
system-mysql
image stream:oc patch imagestream/system-mysql --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "System 2.9 MySQL"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/rhscl/mysql-57-rhel7:5.7"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
This patch updates the
system-mysql
image stream to contain the 2.9 tag. With the command below, you can confirm that the tag has been created if the Tags column shows 2.9:oc get is system-mysql
Patch the
system-mysql
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/system-mysql --from-image=system-mysql:latest --containers=system-mysql --remove
Add the new version-specific trigger:
oc set triggers dc/system-mysql --from-image=system-mysql:2.9 --containers=system-mysql
In case there are new updates on the image, this patch might also trigger a redeployment of the
system-mysql
DeploymentConfig. If this happens, wait until the new pods are redeployed and ready, and the old pods are terminated.Remove the
:latest
tag from theImageStream
:oc tag -d system-mysql:latest
2.2.4.7.4. system-postgresql
DeploymentConfig
If the system-postgresql
DeploymentConfig exists in your current 3scale installation, patch the PostgreSQL image for system-postgresql
.
Patch the
system-postgresql
image stream:oc patch imagestream/system-postgresql --type=json -p '[{"op": "add", "path": "/spec/tags/-", "value": {"annotations": {"openshift.io/display-name": "System 2.9 PostgreSQL"}, "from": { "kind": "DockerImage", "name": "registry.redhat.io/rhscl/postgresql-10-rhel7"}, "name": "2.9", "referencePolicy": {"type": "Source"}}}]'
This patch updates the
system-postgresql
image stream to contain the 2.9 tag. With the command below, you can confirm that the tag has been created if the Tags column shows 2.9:oc get is system-postgresql
Patch the
system-postgresql
ImageChange trigger:Remove the latest trigger:
oc set triggers dc/system-postgresql --from-image=system-postgresql:latest --containers=system-postgresql --remove
Add the new version-specific trigger:
oc set triggers dc/system-postgresql --from-image=system-postgresql:2.9 --containers=system-postgresql
In case there are new updates on the image, this patch might also trigger a redeployment of the
system-postgresql
DeploymentConfig. If this happens, wait until the new pods are redeployed and ready, and the old pods are terminated.Remove the
:latest
tag from theImageStream
:oc tag -d system-postgresql:latest
2.2.4.8. Confirm image URLs
Previous step
Confirm that all the image URLs of the DeploymentConfigs contain the new image registry URLs with a hash added at the end of each URL address:
THREESCALE_DC_NAMES="apicast-production apicast-staging backend-cron backend-listener backend-redis backend-worker system-app system-memcache system-mysql system-redis system-sidekiq system-sphinx zync zync-database zync-que" for component in ${THREESCALE_DC_NAMES}; do echo -n "${component} image: " && oc get dc $component -o json | jq .spec.template.spec.containers[0].image ; done
Next step
None. After you have performed all the listed steps, 3scale upgrade from 2.8 to 2.9.1 in a template-based deployment is now complete.
2.3. Upgrading 3scale with an Oracle Database in a template-based installation
This section explains how to upgrade Red Hat 3scale API Management when you are using a 3scale system image with an Oracle Database, in a template-based installation with OpenShift 3.11.
Prerequisites
A 3scale installation with the Oracle Database. See Setting up your 3scale system image with an Oracle Database.
To upgrade your 3scale system image with an Oracle Database in a template-based installation, perform one of the procedures below according to your database version:
2.3.1. Upgrading 3scale with Oracle 19c
This procedure guides you through changing from Oracle Database 12c to Oracle Database 19c, for 3scale 2.9.1 from an existing 3scale 2.8 installation.
IMPORTANT: Loss of connection to the database can potentially corrupt 3scale. Make a backup before proceeding to perform the upgrade. See the official Oracle Database documentation: Oracle Database Backup and Recovery User’s Guide.
Prerequisites
- A 3scale 2.8 installation
An Oracle Database 12c installation.
- For more details about configuring 3scale with Oracle, see Preparing the Oracle Database
Procedure
Scale 3scale down to 0 (zero), with the following considerations:
- The 3scale connection to the database fails because it is disconnected from Oracle 12c.
Upgrade Oracle 12c to Oracle 19c.
- To perform the upgrade, follow the official Oracle Database Upgrade Guide.
Clone the OpenShift Templates for 3scale 2.9.1
$ git clone --branch 2.9.1.GA https://github.com/3scale/3scale-amp-openshift-templates.git
-
Place your Oracle Database Instant Client Package files into the
3scale-amp-openshift-templates/amp/system-oracle/oracle-client-files
directory. Run the
oc process
command with the-f
option and specify thebuild.yml
OpenShift template:$ oc process -f build.yml | oc apply -f -
Enter the
oc start-build
command to build the new system image:$ oc start-build 3scale-amp-system-oracle --from-dir=.
Wait until the build completes. To see the state of the build, run the following command:
$ oc get build <build-name> -o jsonpath="{.status.phase}"
- Wait until the build is in a Complete state.
2.3.2. Upgrading 3scale with Oracle 12c
This procedure guides you through an Oracle Database 12c update for 3scale 2.9.1 from an existing 3scale 2.8 installation.
IMPORTANT: Loss of connection to the database can potentially corrupt 3scale. Make a backup before proceeding to perform the upgrade. See the official Oracle Database documentation: Oracle Database Backup and Recovery User’s Guide.
Prerequisites
- A 3scale 2.8 installation.
An Oracle Database 12c installation.
- For more details about configuring 3scale with Oracle, see Preparing the Oracle Database
Procedure
Clone the OpenShift Templates for 3scale 2.9.1
$ git clone --branch 2.9.1.GA https://github.com/3scale/3scale-amp-openshift-templates.git
-
Place your Oracle Database Instant Client Package files into the
3scale-amp-openshift-templates/amp/system-oracle/oracle-client-files
directory. Run the
oc process
command with the-f
option and specify thebuild.yml
OpenShift template:$ oc process -f build.yml | oc apply -f -
Enter the
oc start-build
command to build the new system image:$ oc start-build 3scale-amp-system-oracle --from-dir=.
Wait until the build completes. To see the state of the build, run the following command:
$ oc get build <build-name> -o jsonpath="{.status.phase}"
- Wait until the build is in a Complete state.
Additional resources
For more information about 3scale and Oracle Database support, see Red Hat 3scale API Management Supported Configurations.