Instances and Images Guide

1. In the dashboard, select Project > Compute > Images.
2. Click Create Image.
3. Fill out the values, and click Create Image when finished.

1. In the dashboard, select Project > Compute > Images.

2. Click Edit Image from the dropdown list.

   Note

   The Edit Image option is available only when you log in as an admin user. When you log in as a demo user, you have the option to Launch an instance or Create Volume.

3. Update the fields and click Update Image when finished. You can update the following values - name, description, kernel ID, ramdisk ID, architecture, format, minimum disk, minimum RAM, public, protected.
4. Click the drop-down menu and select Update Metadata option.
5. Specify metadata by adding items from the left column to the right one. In the left column, there are metadata definitions from the Image Service Metadata Catalog. Select Other to add metadata with the key of your choice and click Save when finished.

1. In the dashboard, select Project > Compute > Images.
2. Select the image you want to delete and click Delete Images.

1. In the dashboard, select Project.
2. Select Network > Networks, and ensure there is a private network to which you can attach the new instance (to create a network, see Create a Network section in the Networking Guide).
3. Select Compute > Access & Security > Key Pairs, and ensure there is a key pair (to create a key pair, see Section 4.2.1.1, “Create a Key Pair”).

4. Ensure that you have either an image or a volume that can be used as a boot source:

   • To view boot-source images, select the Images tab (to create an image, see Section 1.2.1, “Create an Image”).
   • To view boot-source volumes, select the Volumes tab (to create a volume, see Create a Volume in the Storage Guide).
5. Select Compute > Access & Security > Security Groups, and ensure you have created a security group rule (to create a security group, see Project Security Management in the Users and Identity Management Guide).

1. In the dashboard, select Project > Compute > Instances.
2. Click Launch Instance.
3. Fill out the fields (those marked with '* ' are required), and click Launch.

1. Ensure communication between hosts by setting up each host with SSH key authentication so that Compute can use SSH to move disks to other hosts (for example, compute nodes can share the same SSH key).
2. Enable resizing on the original host by setting the allow_resize_to_same_host parameter to "True" in your Compute environment file.

1. In the dashboard, select Project > Compute > Instances.
2. Click the instance’s Actions arrow, and select Resize Instance.
3. Select a new flavor in the New Flavor field.

4. If you want to manually partition the instance when it launches (results in a faster build time):

   1. Select Advanced Options.
   2. In the Disk Partition field, select Manual.
5. Click Resize.

1. In the dashboard, select Project > Compute > Instances, and select your instance.
2. Click Terminate Instance.

1. Click the check boxes in the first column for the instances that you want to start. If you want to select all of the instances, click the check box in the first row in the table.
2. Click More Actions above the table and select Start Instances.

1. In the dashboard, select the Project tab, and click Compute > Access & Security.
2. On the Security Groups tab, click + Create Security Group.
3. Provide a name and description for the group, and click Create Security Group.

1. Change the permissions of the generated key-pair certificate.

   $ chmod 600 os-key.pem

2. Check whether ssh-agent is already running:

   # ps -ef | grep ssh-agent

3. If not already running, start it up with:

   # eval `ssh-agent`

4. On your local machine, load the key-pair certificate into SSH. For example:

   $ ssh-add ~/.ssh/os-key.pem

5. You can now SSH into the file with the user supplied by the image.

1. In the /etc/openstack-dashboard/local_settings file, set the change_set_password parameter value to True.

   can_set_password: True

2. Set the inject_password parameter to "True" in your Compute environment file.

   inject_password=true

3. Restart the Compute service.

   # service nova-compute restart

1. As an admin user in the dashboard, select Admin > System > Flavors.

2. Click Create Flavor, and specify the following fields:

   Table 4.4. Flavor Options

   Tab	Field	Description
   Flavor Information	Name	Unique name.
   	ID	Unique ID. The default value, auto, generates a UUID4 value, but you can also manually specify an integer or UUID4 value.
   	VCPUs	Number of virtual CPUs.
   	RAM (MB)	Memory (in megabytes).
   	Root Disk (GB)	Ephemeral disk size (in gigabytes); to use the native image size, specify 0. This disk is not used if Instance Boot Source=Boot from Volume.
   	Epehemeral Disk (GB)	Secondary ephemeral disk size (in gigabytes) available to an instance. This disk is destroyed when an instance is deleted. The default value is 0, which implies that no ephemeral disk is created.
   	Swap Disk (MB)	Swap disk size (in megabytes).
   Flavor Access	Selected Projects	Projects which can use the flavor. If no projects are selected, all projects have access (Public=Yes).

3. Click Create Flavor.

1. As an admin user in the dashboard, select Admin > System > Flavors.
2. Click the flavor’s Edit Flavor button.
3. Update the values, and click Save.

1. Open your Compute environment file.

2. Add the following values to the NovaSchedulerDefaultFilters parameter, if they are not already present:

   • AggregateInstanceExtraSpecsFilter for host aggregate metadata.

     Note

     Scoped specifications must be used for setting flavor extra_specs when specifying both AggregateInstanceExtraSpecsFilter and ComputeCapabilitiesFilter filters as values of the same NovaSchedulerDefaultFilters parameter, otherwise the ComputeCapabilitiesFilter will fail to select a suitable host. For details on the namespaces to use to scope the flavor extra_specs keys for these filters, see Table 4.7, “Scheduling Filters”.

   • AvailabilityZoneFilter for availability zone host specification when launching an instance.
3. Save the configuration file.
4. Deploy the overcloud.

1. As an admin user in the dashboard, select Admin > System > Host Aggregates. All currently defined aggregates are listed in the Host Aggregates section.
2. Click Create Host Aggregate.
3. Add a name for the aggregate in the Name field, and a name by which the end user should see it in the Availability Zone field.
4. Click Manage Hosts within Aggregate.
5. Select a host for use by clicking its + icon.
6. Click Create Host Aggregate.

1. As an admin user in the dashboard, select Admin > System > Host Aggregates. All currently defined aggregates are listed in the Host Aggregates section.

2. To update the instance’s Name or Availability zone:

   • Click the aggregate’s Edit Host Aggregate button.
   • Update the Name or Availability Zone field, and click Save.

3. To update the instance’s Assigned hosts:

   • Click the aggregate’s arrow icon under Actions.
   • Click Manage Hosts.
   • Change a host’s assignment by clicking its + or - icon.
   • When finished, click Save.

4. To update the instance’s Metadata:

   • Click the aggregate’s arrow icon under Actions.
   • Click the Update Metadata button. All current values are listed on the right-hand side under Existing Metadata.
   • Under Available Metadata, click on the Other field, and specify the key you want to add. Use predefined keys (see Table 4.6, “Host Aggregate Metadata”) or add your own (which will only be valid if exactly the same key is set in an instance’s flavor).

   • Click the + button; you can now view the new key under Existing Metadata.

     Note

     Remove a key by clicking its - icon.

   • Click Save.

     Table 4.6. Host Aggregate Metadata

     Key	Description
     filter_tenant_id	If specified, the aggregate only hosts this tenant (project). Depends on the AggregateMultiTenancyIsolation filter being set for the Compute scheduler.

1. As an admin user in the dashboard, select Admin > System > Host Aggregates. All currently defined aggregates are listed in the Host Aggregates section.

2. Remove all assigned hosts from the aggregate:

   1. Click the aggregate’s arrow icon under Actions.
   2. Click Manage Hosts.
   3. Remove all hosts by clicking their - icon.
   4. When finished, click Save.
3. Click the aggregate’s arrow icon under Actions.
4. Click Delete Host Aggregate in this and the next dialog screen.

1. Evacuate an instance using:

   # nova evacuate [--password pass] instance_name [target_host]

   Where:

   • --password - Admin password to set for the evacuated instance. If a password is not specified, a random password is generated and output when evacuation is complete.
   • instance_name - Name of the instance to be evacuated.

   • target_host - Host to which the instance is evacuated; if you do not specify the host, the Compute scheduler selects one for you. You can find possible hosts using:

     # nova host-list | grep compute

     For example:

     # nova evacuate myDemoInstance Compute2_OnEL7.myDomain

1. On the controller host:

   1. Ensure the /var/lib/nova/instances directory has read-write access by the Compute service user (this user must be the same across controller and nodes). For example:

      drwxr-xr-x.  9 nova nova 4096 Nov  5 20:37 instances

   2. Add the following lines to the /etc/exports file; switch out node1_IP and node2_IP for the IP addresses of the two compute nodes:

      /var/lib/nova/instances (rw,sync,fsid=0,no_root_squash)
      /var/lib/nova/instances (rw,sync,fsid=0,no_root_squash)

   3. Export the /var/lib/nova/instances directory to the compute nodes.

      # exportfs -avr

   4. Restart the NFS server:

      # systemctl restart nfs-server

2. On each compute node:

   1. Ensure the /var/lib/nova/instances directory exists locally.

   2. Add the following line to the /etc/fstab file:

      :/var/lib/nova/instances /var/lib/nova/instances nfs4 defaults 0 0

   3. Mount the controller’s instance directory (all devices listed in /etc/fstab ):

      # mount -a -v

   4. Ensure qemu can access the directory’s images:

      # ls -ld /var/lib/nova/instances
      drwxr-xr-x. 9 nova nova 4096 Nov  5 20:37 /var/lib/nova/instances

   5. Ensure that the node can see the instances directory with:

      drwxr-xr-x. 9 nova nova 4096 Nov  5 20:37 /var/lib/nova/instances

1. In the dashboard, select Project > Compute > Instances.
2. Select the instance from which you want to create a snapshot.
3. In the Actions column, click Create Snapshot.

4. In the Create Snapshot dialog, enter a name for the snapshot and click Create Snapshot.

   The Images category now shows the instance snapshot.

1. In the dashboard, select Project > Images.
2. All snapshots you created, appear under the Project option.

3. For every snapshot you create, you can perform the following functions, using the dropdown list:

   1. Use the Create Volume option to create a volume and entering the values for volume name, description, image source, volume type, size and availability zone. For more information, see Create a Volume in the Storage Guide.
   2. Use the Edit Image option to update the snapshot image by updating the values for name, description, Kernel ID, Ramdisk ID, Architecture, Format, Minimum Disk (GB), Minimum RAM (MB), public or private. For more information, see Section 1.2.3, “Update an Image”.
   3. Use the Delete Image option to delete the snapshot.

1. In the dashboard, select Project > Compute > Images.
2. Select the snapshot from which you want to restore the instance.
3. In the Actions column, click Launch Instance.
4. In the Launch Instance dialog, enter a name and the other details for the instance and click Launch.

1. Add the rescue image to the Image service:

   # glance image-create --name IMAGE_NAME --disk-format qcow2 \
     --container-format bare --is-public True --file IMAGE_PATH

   Here IMAGE_NAME is the name of the image, IMAGE_PATH is the location of the image.

2. Use the image-list command to obtain the IMAGE_ID required for launching an instace in the rescue mode.

   # glance image-list

1. Since you need to rescue an instance with a specific image, rather than the default one, use the --image parameter:

   # nova rescue --image IMAGE_ID VIRTUAL_MACHINE_ID

   Here IMAGE_ID is the ID of the image you want to use and VIRTUAL_MACHINE_ID is ID of a virtual machine that you want to rescue.

   Note

   The nova rescue command allows an instance to perform a soft shut down. This allows the guest operating system to perform a controlled shutdown before the instance is powered off. The shut down behavior is configured using shutdown_timeout in your Compute configuration file. The value stands for the overall period (in seconds) a guest operation system is allowed to complete the shutdown. The default timeout is 60 seconds.

   The timeout value can be overridden on a per image basis by means of os_shutdown_timeout that is an image metadata setting allowing different types of operating systems to specify how much time they need to shut down cleanly.

2. Reboot the virtual machine.
3. Confirm the status of the virtual machine is RESCUE on the controller node by using nova list command or by using dashboard.
4. Log in to the new virtual machine dashboard by using the password for rescue mode.

1. Execute the following commands on the controller node.

   # nova unrescue VIRTUAL_MACHINE_ID

   Here VIRTUAL_MACHINE_ID is ID of a virtual machine that you want to unrescue.

1. Copy the overcloud image and add the gpu suffix to the copied image.

   $ cp overcloud-full.qcow2 overcloud-full-gpu.qcow2

2. Install an ISO image generator tool from YUM.

   $ sudo yum install genisoimage -y

3. Download the NVIDIA GRID host driver RPM package that corresponds to your GPU device from the NVIDIA website. To determine which driver you need, see the NVIDIA Driver Downloads Portal.

   Note

   You must be a registered NVIDIA customer to download the drivers from the portal.

4. Create an ISO image from the driver RPM package and save the image in the nvidia-guest directory. You will use this ISO image to install the driver on your Compute nodes in subsequent steps.

   $ genisoimage -o nvidia-guest.iso -R -J -V NVIDIA nvidia-guest/
   I: -input-charset not specified, using utf-8 (detected in locale settings)
     9.06% done, estimate finish Wed Oct 31 11:24:46 2018
    18.08% done, estimate finish Wed Oct 31 11:24:46 2018
    27.14% done, estimate finish Wed Oct 31 11:24:46 2018
    36.17% done, estimate finish Wed Oct 31 11:24:46 2018
    45.22% done, estimate finish Wed Oct 31 11:24:46 2018
    54.25% done, estimate finish Wed Oct 31 11:24:46 2018
    63.31% done, estimate finish Wed Oct 31 11:24:46 2018
    72.34% done, estimate finish Wed Oct 31 11:24:46 2018
    81.39% done, estimate finish Wed Oct 31 11:24:46 2018
    90.42% done, estimate finish Wed Oct 31 11:24:46 2018
    99.48% done, estimate finish Wed Oct 31 11:24:46 2018
   Total translation table size: 0
   Total rockridge attributes bytes: 358
   Total directory bytes: 0
   Path table size(bytes): 10
   Max brk space used 0
   55297 extents written (108 MB)

5. Create a driver installation script for your Compute nodes. This script installs the NVIDIA GRID host driver on each Compute node that you run it on. In this example the script is named install_nvidia.sh.

   #/bin/bash
   
   # NVIDIA GRID package
   mkdir /tmp/mount
   mount LABEL=NVIDIA /tmp/mount
   rpm -ivh /tmp/mount/NVIDIA-vGPU-rhel-7.6-410.91.x86_64.rpm

6. Customize the overcloud image by attaching the ISO image that you generated and running the driver installation script that you created.

   $ virt-customize --attach nvidia-packages.iso -a overcloud-full-gpu.qcow2  -v --run install_nvidia.sh
   [   0.0] Examining the guest ...
   libguestfs: launch: program=virt-customize
   libguestfs: launch: version=1.36.10rhel=7,release=6.el7_5.2,libvirt
   libguestfs: launch: backend registered: unix
   libguestfs: launch: backend registered: uml
   libguestfs: launch: backend registered: libvirt

7. Relabel the customized image with SELinux.

   $ virt-customize -a overcloud-full-gpu.qcow2 --selinux-relabel
   [   0.0] Examining the guest ...
   [   2.2] Setting a random seed
   [   2.2] SELinux relabelling
   [  27.4] Finishing off

8. Prepare the custom image files for a Glance upload.

   $ mkdir /var/image/x86_64/image
   $ guestmount -a overcloud-full-gpu.qcow2 -i --ro image
   $ cp image/boot/vmlinuz-3.10.0-862.14.4.el7.x86_64 ./overcloud-full-gpu.vmlinuz
   $ cp image/boot/initramfs-3.10.0-862.14.4.el7.x86_64.img ./overcloud-full-gpu.initrd

9. From the undercloud, upload the custom image to Glance.

   (undercloud) $ openstack overcloud image upload --update-existing --os-image-name overcloud-full-gpu.qcow2

1. Create the new ComputeGPU role file by copying the file /home/stack/templates/roles/Compute.yaml to /home/stack/templates/roles/ComputeGPU.yaml and editing the following file sections:

   Table 6.1. ComputeGPU role file edits

   Section	Current value	New value
   Role comment	Role: Compute	Role: ComputeGpu
   Role name	name: Compute	name: ComputeGpu
   Description	Basic Compute Node role	GPU role
   CountDefault	1	0
   ImageDefault	overcloud-full	overcloud-gpu
   HostnameFormatDefault	-compute-	-computegpu-
   deprecated_nic_config_name	compute.yaml	compute-gpu.yaml

2. Generate a new roles data file named gpu_roles_data.yaml that includes the Controller, Compute, and ComputeGpu roles.

   (undercloud) [stack@director templates]$ openstack overcloud roles generate -o /home/stack/templates/gpu_roles_data.yaml Controller Compute ComputeGpu

   The following example shows the ComputeGpu role details:

   #####################################################################
   # Role: ComputeGpu                                                  #
   #####################################################################
   - name: ComputeGpu
     description: |
       GPU Compute Node role
     CountDefault: 1
     ImageDefault: overcloud-gpu
     networks:
       - InternalApi
       - Tenant
       - Storage
     HostnameFormatDefault: '%stackname%-computegpu-%index%'
     RoleParametersDefault:
       TunedProfileName: "virtual-host"
     # Deprecated & backward-compatible values (FIXME: Make parameters consistent)
     # Set uses_deprecated_params to True if any deprecated params are used.
     uses_deprecated_params: True
     deprecated_param_image: 'NovaImage'
     deprecated_param_extraconfig: 'NovaComputeExtraConfig'
     deprecated_param_metadata: 'NovaComputeServerMetadata'
     deprecated_param_scheduler_hints: 'NovaComputeSchedulerHints'
     deprecated_param_ips: 'NovaComputeIPs'
     deprecated_server_resource_name: 'NovaCompute'
     deprecated_nic_config_name: 'compute-gpu.yaml'
     ServicesDefault:
       - OS::TripleO::Services::Aide
       - OS::TripleO::Services::AuditD
       - OS::TripleO::Services::CACerts
       - OS::TripleO::Services::CephClient
       - OS::TripleO::Services::CephExternal
       - OS::TripleO::Services::CertmongerUser
       - OS::TripleO::Services::Collectd
       - OS::TripleO::Services::ComputeCeilometerAgent
       - OS::TripleO::Services::ComputeNeutronCorePlugin
       - OS::TripleO::Services::ComputeNeutronL3Agent
       - OS::TripleO::Services::ComputeNeutronMetadataAgent
       - OS::TripleO::Services::ComputeNeutronOvsAgent
       - OS::TripleO::Services::Docker
       - OS::TripleO::Services::Fluentd
       - OS::TripleO::Services::Ipsec
       - OS::TripleO::Services::Iscsid
       - OS::TripleO::Services::Kernel
       - OS::TripleO::Services::LoginDefs
       - OS::TripleO::Services::MetricsQdr
       - OS::TripleO::Services::MySQLClient
       - OS::TripleO::Services::NeutronBgpVpnBagpipe
       - OS::TripleO::Services::NeutronLinuxbridgeAgent
       - OS::TripleO::Services::NeutronVppAgent
       - OS::TripleO::Services::NovaCompute
       - OS::TripleO::Services::NovaLibvirt
       - OS::TripleO::Services::NovaLibvirtGuests
       - OS::TripleO::Services::NovaMigrationTarget
       - OS::TripleO::Services::Ntp
       - OS::TripleO::Services::ContainersLogrotateCrond
       - OS::TripleO::Services::OpenDaylightOvs
       - OS::TripleO::Services::Rhsm
       - OS::TripleO::Services::RsyslogSidecar
       - OS::TripleO::Services::Securetty
       - OS::TripleO::Services::SensuClient
       - OS::TripleO::Services::SkydiveAgent
       - OS::TripleO::Services::Snmp
       - OS::TripleO::Services::Sshd
       - OS::TripleO::Services::Timezone
       - OS::TripleO::Services::TripleoFirewall
       - OS::TripleO::Services::TripleoPackages
       - OS::TripleO::Services::Tuned
       - OS::TripleO::Services::Vpp
       - OS::TripleO::Services::OVNController
       - OS::TripleO::Services::OVNMetadataAgent
       - OS::TripleO::Services::Ptp

3. Create the compute-vgpu-nvidia flavor to tag nodes that you want to designate for vGPU workloads.

   (undercloud) [stack@director templates]$ openstack flavor create --id auto --ram 6144 --disk 40 --vcpus 4 compute-vgpu-nvidia
   +----------------------------+--------------------------------------+
   | Field                      | Value                                |
   +----------------------------+--------------------------------------+
   | OS-FLV-DISABLED:disabled   | False                                |
   | OS-FLV-EXT-DATA:ephemeral  | 0                                    |
   | disk                       | 40                                   |
   | id                         | 9cb47954-be00-47c6-a57f-44db35be3e69 |
   | name                       | compute-vgpu-nvidia                  |
   | os-flavor-access:is_public | True                                 |
   | properties                 |                                      |
   | ram                        | 6144                                 |
   | rxtx_factor                | 1.0                                  |
   | swap                       |                                      |
   | vcpus                      | 4                                    |
   +----------------------------+--------------------------------------+

4. Tag each node that you want to designate for GPU workloads with the compute-vgpu-nvidia profile.

   (undercloud) [stack@director templates]$ openstack baremetal node set --property capabilities='profile:compute-vgpu-nvidia,boot_option:local' 9d07a673-b6bf-4a20-a538-3b05e8fa2c13

5. Register the overcloud and run the standard hardware introspection on your nodes.

1. Install Red Hat Enterprise Linux and the NVIDIA GRID driver on one Compute node and launch the node. For information on installing the NVIDIA GRID driver, see Section 6.2.1, “Build a custom GPU overcloud image”.

2. On the Compute node, locate the vGPU type of the physical GPU device that you want to enable. For libvirt, virtual GPUs are seen as mediated devices, or mdev type devices. To discover the supported mdev devices, run the following command:

   [root@overcloud-computegpu-0 ~]# ls /sys/class/mdev_bus/0000\:06\:00.0/mdev_supported_types/
   nvidia-11  nvidia-12  nvidia-13  nvidia-14  nvidia-15  nvidia-16  nvidia-17  nvidia-18  nvidia-19  nvidia-20  nvidia-21  nvidia-210  nvidia-22
   
   [root@overcloud-computegpu-0 ~]# cat /sys/class/mdev_bus/0000\:06\:00.0/mdev_supported_types/nvidia-18/description
   num_heads=4, frl_config=60, framebuffer=2048M, max_resolution=4096x2160, max_instance=4

Prepare the configuration templates

1. Add the compute-gpu.yaml file to the network-environment.yaml file. For example:

   resource_registry:
     OS::TripleO::Compute::Net::SoftwareConfig: /home/stack/templates/nic-configs/compute.yaml
     OS::TripleO::ComputeGpu::Net::SoftwareConfig: /home/stack/templates/nic-configs/compute-gpu.yaml
     OS::TripleO::Controller::Net::SoftwareConfig: /home/stack/templates/nic-configs/controller.yaml
     #OS::TripleO::AllNodes::Validation: OS::Heat::None

2. Add the OvercloudComputeGpuFlavor flavor to the node-info.yaml file. For example:

   parameter_defaults:
     OvercloudControllerFlavor: control
     OvercloudComputeFlavor: compute
     OvercloudComputeGpuFlavor: compute-vgpu-nvidia
     ControllerCount: 1
     ComputeCount: 0
     ComputeGpuCount: 1
     NtpServer: `NTP_SERVER_URL`
     NeutronNetworkType: vxlan,vlan
     NeutronTunnelTypes: vxlan

   Replace the NTP_SERVER_URL variable with the address of your NTP server.

3. Create a gpu.yaml file with the vGPU type that you retrieved for your GPU device. For example:

   parameter_defaults:
     ComputeGpuExtraConfig:
       nova::compute::vgpu::enabled_vgpu_types:
         - nvidia-18

   Note

   Only one virtual GPU type is supported per physical GPU. If you specify multiple vGPU types in this property, only the first type is used.

Prepare the guest image and the NVIDIA GRID guest driver

1. Download the NVIDIA GRID guest driver RPM package that corresponds to your GPU device from the NVIDIA website. To determine which driver you need, see the NVIDIA Driver Downloads Portal.

   Note

   You must be a registered NVIDIA customer to download the drivers from the portal.

2. Create an ISO image from the driver RPM package. You will use this ISO image to install the driver on your Compute nodes in subsequent steps.

   [root@virtlab607 guest]# genisoimage -o nvidia-guest.iso -R -J -V NVIDIA nvidia-guest/
   I: -input-charset not specified, using utf-8 (detected in locale settings)
     9.06% done, estimate finish Wed Oct 31 10:59:50 2018
    18.08% done, estimate finish Wed Oct 31 10:59:50 2018
    27.14% done, estimate finish Wed Oct 31 10:59:50 2018
    36.17% done, estimate finish Wed Oct 31 10:59:50 2018
    45.22% done, estimate finish Wed Oct 31 10:59:50 2018
    54.25% done, estimate finish Wed Oct 31 10:59:50 2018
    63.31% done, estimate finish Wed Oct 31 10:59:50 2018
    72.34% done, estimate finish Wed Oct 31 10:59:50 2018
    81.39% done, estimate finish Wed Oct 31 10:59:50 2018
    90.42% done, estimate finish Wed Oct 31 10:59:50 2018
    99.48% done, estimate finish Wed Oct 31 10:59:50 2018
   Total translation table size: 0
   Total rockridge attributes bytes: 358
   Total directory bytes: 0
   Path table size(bytes): 10
   Max brk space used 0
   55297 extents written (108 MB)

3. Copy the guest image that you want to customize for GPU instances. For example:

   [root@virtlab607 guest]# cp rhel-server-7.6-update-4-x86_64-kvm.qcow2 rhel-server-7.6-update-4-x86_64-kvm-gpu.qcow2

1. Create a script named nvidia-prepare-guest.sh to enable the required repositories, update the instance to the latest kernel, install the NVIDIA GRID guest driver, and attach the gridd.conf license file to the instance.

   #/bin/bash
   
   # Add build tooling
   subscription-manager register --username [USERNAME] --password [PASSWORD]
   subscription-manager attach --pool=8a85f98c651a88990165399d8eea03e7
   subscription-manager repos --disable=*
   subscription-manager repos --enable=rhel-7-server-rpms
   yum upgrade -y
   yum install -y gcc make kernel-devel cpp glibc-devel glibc-headers kernel-headers libmpc mpfr
   
   # NVIDIA GRID guest script
   mkdir /tmp/mount
   mount LABEL=NVIDIA /tmp/mount
   /bin/sh /tmp/mount/NVIDIA-Linux-x86_64-410.71-grid.run
   
   mkdir -p /etc/nvidia
   cp /tmp/mount/gridd.conf /etc/nvidia

2. Run the script on the guest image that you copied previously. For example:

   $ virt-customize --attach nvidia-guest.iso -a rhel-server-7.6-update-4-x86_64-kvm-gpu.qcow2 -v --run nvidia-prepare-guest.sh

3. Upload the custom guest image to Glance.

   (overcloud) [stack@director ~]$ openstack image create rhelgpu --file /var/images/x86_64/rhel-server-7.6-update-4-x86_64-kvm-gpu.qcow2 --disk-format qcow2 --container-format bare --public

1. Create an NVIDIA GPU flavor to tag each instance that you want to designate for GPU workloads. For example:

   (overcloud) [stack@virtlab-director2 ~]$ openstack flavor create --vcpus 6 --ram 8192 --disk 100 m1.small-gpu
   +----------------------------+--------------------------------------+
   | Field                      | Value                                |
   +----------------------------+--------------------------------------+
   | OS-FLV-DISABLED:disabled   | False                                |
   | OS-FLV-EXT-DATA:ephemeral  | 0                                    |
   | disk                       | 100                                  |
   | id                         | a27b14dd-c42d-4084-9b6a-225555876f68 |
   | name                       | m1.small-gpu                         |
   | os-flavor-access:is_public | True                                 |
   | properties                 |                                      |
   | ram                        | 8192                                 |
   | rxtx_factor                | 1.0                                  |
   | swap                       |                                      |
   | vcpus                      | 6                                    |
   +----------------------------+--------------------------------------+

2. Assign a vGPU resource to the flavor that you created. Currently you can assign only one vGPU for each instance.

   (overcloud) [stack@virtlab-director2 ~]$ openstack flavor set m1.small-gpu --property "resources:VGPU=1"
   
   (overcloud) [stack@virtlab-director2 ~]$ openstack flavor show m1.small-gpu
   +----------------------------+--------------------------------------+
   | Field                      | Value                                |
   +----------------------------+--------------------------------------+
   | OS-FLV-DISABLED:disabled   | False                                |
   | OS-FLV-EXT-DATA:ephemeral  | 0                                    |
   | access_project_ids         | None                                 |
   | disk                       | 100                                  |
   | id                         | a27b14dd-c42d-4084-9b6a-225555876f68 |
   | name                       | m1.small-gpu                         |
   | os-flavor-access:is_public | True                                 |
   | properties                 | resources:VGPU='1'                   |
   | ram                        | 8192                                 |
   | rxtx_factor                | 1.0                                  |
   | swap                       |                                      |
   | vcpus                      | 6                                    |
   +----------------------------+--------------------------------------+

1. Launch a new instance with the GPU flavor that you created in Section 6.2.5, “Create a vGPU profile for instances”. For example:

   (overcloud) [stack@virtlab-director2 ~]$ openstack server create --flavor m1.small-gpu --image rhelgpu --security-group web --nic net-id=internal0 --key-name lambda instance0

2. Log in to the instance and install the NVIDIA GRID driver. The exact installer name is available from the files that you attached to the guest image. For example:

   [root@instance0 tmp]# sh NVIDIA-Linux-x86_64-410.71-grid.run

3. Check the status of the NVIDIA GRID daemon.

   [root@instance0 nvidia]# systemctl status nvidia-gridd.service
   ● nvidia-gridd.service - NVIDIA Grid Daemon
      Loaded: loaded (/usr/lib/systemd/system/nvidia-gridd.service; enabled; vendor preset: disabled)
      Active: active (running) since Wed 2018-10-31 20:00:41 EDT; 15s ago
     Process: 18143 ExecStopPost=/bin/rm -rf /var/run/nvidia-gridd (code=exited, status=0/SUCCESS)
     Process: 18145 ExecStart=/usr/bin/nvidia-gridd (code=exited, status=0/SUCCESS)
    Main PID: 18146 (nvidia-gridd)
      CGroup: /system.slice/nvidia-gridd.service
              └─18146 /usr/bin/nvidia-gridd
   
   Oct 31 20:00:41 instance0 systemd[1]: Stopped NVIDIA Grid Daemon.
   Oct 31 20:00:41 instance0 systemd[1]: Starting NVIDIA Grid Daemon...
   Oct 31 20:00:41 instance0 systemd[1]: Started NVIDIA Grid Daemon.
   Oct 31 20:00:41 instance0 nvidia-gridd[18146]: Started (18146)
   Oct 31 20:00:41 instance0 nvidia-gridd[18146]: Ignore Service Provider Licensing.
   Oct 31 20:00:41 instance0 nvidia-gridd[18146]: Calling load_byte_array(tra)
   Oct 31 20:00:42 instance0 nvidia-gridd[18146]: Acquiring license for GRID vGPU Edition.
   Oct 31 20:00:42 instance0 nvidia-gridd[18146]: Calling load_byte_array(tra)
   Oct 31 20:00:45 instance0 nvidia-gridd[18146]: License acquired successfully. (Info: http://dhcp158-15.virt.lab.eng.bos.redhat.com:7070/request; GRID-Virtual-WS,2.0)