Chapter 6. Configure Node Group Templates
Data processing through Hadoop involves running jobs on clusters. Each cluster is made up of specially-configured nodes (or instances). OpenStack Data Processing allows you to configure templates defining the different types of nodes required for each cluster. These templates are called node group templates.
A node group templates defines the settings required for each node in a Hadoop cluster. The following procedure describes how to configure a node group template:
- In the dashboard, select Project > Data Processing > Node Group Templates.
- Click Create Template.
- Use the Plugin Name and Version drop-down menus to select the name and version of the Hadoop plug-in that the node will use.
- Click Create.
- Enter a name for your template in the Template Name field.
- Use the Description field to describe the node group template you are creating (optional).
- Select a flavor that the nodes should use from the OpenStack Flavor drop-down menu. The flavor should be appropriate to the computing, memory, and storage needs of the node. For more information about flavors, see Manage Flavors.
Select a storage location for your node from the Storage location drop-down menu:
- Ephemeral Drive: the OpenStack Data Processing service will generate the necessary ephemeral storage for the node.
Cinder Volume: with this option, you can configure the Block Storage service to create a set number of volumes for each node. Use the Volumes per node field to set the number of volumes, and the Volumes size (GB) field to specify the size of each volume.
For most deployments, we recommend the Cinder Volume option. Data on ephemeral drives are dependent on the integrity of the host, and therefore vulnerable to host failures.
- Use the Floating IP pool drop-down menu to set whether the node should use floating IPs; and if so, which one. For more information about floating IPs, see Configure IP Addressing.
Next, configure the node’s security. To create a security group for the node group, select the Auto Security Group check box. You can also launch the node in existing security groups by selecting their corresponding check boxes from the Security Groups list.
See Project Security Management for more details about security groups.
- Finally, choose which Hadoop processes should be launched in the node group. To do so, select each chosen processes’ check box from the Processes list. Consult your chosen Hadoop plug-in’s documentation for more information about each process.
- Click Create. The template should now appear in the Node Group Template table.
The Node Group Template table lists all available node group templates, their respective Hadoop plug-ins, and versions. This table also lists which Hadoop processes will run on each template’s nodes.
