Chapter 5. Non-Director Environments: Upgrading Individual OpenStack Services (Live Compute) in a Standard Environment
This section describes the steps you should follow to upgrade your cloud deployment by updating one service at a time with live compute in a non High Availability (HA) environment. This scenario upgrades from Red Hat OpenStack Platform 7 to Red Hat OpenStack Platform 8 in environments that do not use the director.
A live Compute upgrade minimizes interruptions to your Compute service, with only a few minutes for the smaller services, and a longer migration interval for the workloads moving to newly-upgraded Compute hosts. Existing workloads can run indefinitely, and you do not need to wait for a database migration.
Due to certain package dependencies, upgrading the packages for one OpenStack service might cause Python libraries to upgrade before other OpenStack services upgrade. This might cause certain services to fail prematurely. In this situation, continue upgrading the remaining services. All services should be operational upon completion of this scenario.
This method may require additional hardware resources to bring up the Compute nodes.
The procedures in this chapter follow the architectural naming convention followed by all Red Hat OpenStack Platform documentation. If you are unfamiliar with this convention, refer to the Architecture Guide available in the Red Hat OpenStack Platform Documentation Suite before proceeding.
5.1. Pre-Upgrade Tasks Copy linkLink copied to clipboard!
On each node, change to the Red Hat OpenStack Platform 8 repository using the subscription-manager command.
subscription-manager repos --disable=rhel-7-server-openstack-7.0-rpms subscription-manager repos --enable=rhel-7-server-openstack-8-rpms
# subscription-manager repos --disable=rhel-7-server-openstack-7.0-rpms
# subscription-manager repos --enable=rhel-7-server-openstack-8-rpms
Upgrade the openstack-selinux package:
yum upgrade openstack-selinux
# yum upgrade openstack-selinux
This is necessary to ensure that the upgraded services will run correctly on a system with SELinux enabled.
5.2. Upgrading Identity (keystone) and Dashboard (horizon) Copy linkLink copied to clipboard!
Disable the Identity service and the Dashboard service. Depending on your configuration, this involves either:
Disabling
httpdif both the Dashboard and the Identity service are running as WSGI applets:systemctl stop httpd
# systemctl stop httpdCopy to Clipboard Copied! Toggle word wrap Toggle overflow Disabling
openstack-keystonefor the Identity service if it is running as a separate service, then disablinghttpdfor the Dashboard:openstack-service stop keystone systemctl stop httpd
# openstack-service stop keystone # systemctl stop httpdCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Update the packages for both services:
yum -d1 -y upgrade \*keystone\* yum -y upgrade \*horizon\* \*openstack-dashboard\* yum -d1 -y upgrade \*horizon\* \*python-django\*
# yum -d1 -y upgrade \*keystone\*
# yum -y upgrade \*horizon\* \*openstack-dashboard\*
# yum -d1 -y upgrade \*horizon\* \*python-django\*
It is possible that the Identity service’s token table has a large number of expired entries. This can dramatically increase the time it takes to complete the database schema upgrade. To flush expired tokens from the database and alleviate the problem, the keystone-manage command can be used before running the Identity database upgrade.
keystone-manage token_flush openstack-db --service keystone --update
# keystone-manage token_flush
# openstack-db --service keystone --update
This flushes expired tokens from the database. You can arrange to run this command periodically using cron.
Restart the services. Depending on your configuration, this involves either:
Enabling
httpdif both the Dashboard and the Identity service are running as WSGI applets:systemctl start httpd
# systemctl start httpdCopy to Clipboard Copied! Toggle word wrap Toggle overflow Enabling
openstack-keystonefor the Identity service if it is running as a separate service, then disablinghttpdfor the Dashboard:openstack-service start keystone systemctl start httpd
# openstack-service start keystone # systemctl start httpdCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.3. Upgrading Object Storage (swift) Copy linkLink copied to clipboard!
On your Object Storage hosts, run:
openstack-service stop swift yum -d1 -y upgrade \*swift\* openstack-service start swift
# openstack-service stop swift
# yum -d1 -y upgrade \*swift\*
# openstack-service start swift
5.4. Upgrading Image Service (glance) Copy linkLink copied to clipboard!
On your Image Service host, run:
openstack-service stop glance yum -d1 -y upgrade \*glance\* openstack-db --service glance --update openstack-service start glance
# openstack-service stop glance
# yum -d1 -y upgrade \*glance\*
# openstack-db --service glance --update
# openstack-service start glance
5.5. Upgrading Block Storage (cinder) Copy linkLink copied to clipboard!
On your Block Storage host, run:
openstack-service stop cinder yum -d1 -y upgrade \*cinder\* openstack-db --service cinder --update openstack-service start cinder
# openstack-service stop cinder
# yum -d1 -y upgrade \*cinder\*
# openstack-db --service cinder --update
# openstack-service start cinder
5.6. Upgrading Orchestration (heat) Copy linkLink copied to clipboard!
On your Orchestration host, run:
openstack-service stop heat yum -d1 -y upgrade \*heat\* openstack-db --service heat --update openstack-service start heat
# openstack-service stop heat
# yum -d1 -y upgrade \*heat\*
# openstack-db --service heat --update
# openstack-service start heat
5.7. Upgrading Telemetry (ceilometer) Copy linkLink copied to clipboard!
On all nodes hosting Telemetry component services, run:
openstack-service stop ceilometer yum -d1 -y upgrade \*ceilometer\*
# openstack-service stop ceilometer # yum -d1 -y upgrade \*ceilometer\*Copy to Clipboard Copied! Toggle word wrap Toggle overflow On the controller node, where database is installed, run:
ceilometer-dbsync
# ceilometer-dbsyncCopy to Clipboard Copied! Toggle word wrap Toggle overflow After completing the package upgrade, restart the Telemetry service by running the following command on all nodes hosting Telemetry component services:
openstack-service start ceilometer
# openstack-service start ceilometerCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.8. Upgrading Compute (nova) Copy linkLink copied to clipboard!
If you are performing a rolling upgrade of your compute hosts you need to set explicit API version limits to ensure compatibility in your environment.
Before starting Compute services on Controller or Compute nodes, set the
computeoption in the[upgrade_levels]section ofnova.confto the previous Red Hat OpenStack Platform version (kilo):crudini --set /etc/nova/nova.conf upgrade_levels compute kilo
# crudini --set /etc/nova/nova.conf upgrade_levels compute kiloCopy to Clipboard Copied! Toggle word wrap Toggle overflow You need to make this change on your Controller and Compute nodes.
You should undo this operation after upgrading all of your Compute nodes.
On your Compute host, run:
openstack-service stop nova yum -d1 -y upgrade \*nova\* openstack-db --service nova --update
# openstack-service stop nova # yum -d1 -y upgrade \*nova\* # openstack-db --service nova --updateCopy to Clipboard Copied! Toggle word wrap Toggle overflow After you have upgraded all of your hosts, you will want to remove the API limits configured in the previous step. On all of your hosts:
crudini --del /etc/nova/nova.conf upgrade_levels compute
# crudini --del /etc/nova/nova.conf upgrade_levels computeCopy to Clipboard Copied! Toggle word wrap Toggle overflow Restart the Compute service on all the Controller and Compute nodes:
openstack-service start nova
# openstack-service start novaCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.9. Upgrading OpenStack Networking (neutron) Copy linkLink copied to clipboard!
On your OpenStack Networking host, run:
openstack-service stop neutron yum -d1 -y upgrade \*neutron\* openstack-db --service neutron --update
# openstack-service stop neutron # yum -d1 -y upgrade \*neutron\* # openstack-db --service neutron --updateCopy to Clipboard Copied! Toggle word wrap Toggle overflow Restart the OpenStack Networking service:
openstack-service start neutron
# openstack-service start neutronCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.10. Post-Upgrade Tasks Copy linkLink copied to clipboard!
After completing all of your individual service upgrades, you should perform a complete package upgrade on all of your systems:
yum upgrade
# yum upgrade
This will ensure that all packages are up-to-date. You may want to schedule a restart of your OpenStack hosts at a future date in order to ensure that all running processes are using updated versions of the underlying binaries.
Review the resulting configuration files. The upgraded packages will have installed .rpmnew files appropriate to the Red Hat OpenStack Platform 8 version of the service.
New versions of OpenStack services may deprecate certain configuration options. You should also review your OpenStack logs for any deprecation warnings, because these may cause problems during a future upgrade. For more information on the new, updated and deprecated configuration options for each service , see Configuration Reference available from: Red Hat OpenStack Platform Documentation Suite.