4.16. Appendix: Configuration files for Red Hat Gluster Storage Deployment

Copyright 2015 Google Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
glusterfs-config.yaml
The Gluster FS deployment consists of a primary pool and a secondary pool
  of resources, each on a separate zone.

# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#
# glusterfs-config.yaml
#
# The Gluster FS deployment consists of a primary pool and a secondary pool
#   of resources, each on a separate zone.
#

imports:
  - path: gluster_instance.jinja
  - path: path_utils.jinja
resources:
- name: gluster_instance
  type: gluster_instance.jinja
  properties:
    namePrefix: rhgs
    numPrimaryReplicas: 10
    primaryZone: us-central1-a
    secondaryZone: us-central1-b
    numSecondaryReplicas: 10
    backupZone: europe-west1-b
    sourceImage: global/images/rhgs-image01
    dataSourceImage: global/images/rhgs-data-image01
    machineType: n1-highmem-4
    network: default
    bootDiskType: pd-standard
    dataDiskType: pd-standard
    dataDiskSizeGb: 10230

Copyright 2015 Google Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
GlusterFs configuration variables
Required Cloud resource input parameters:
 * numPrimaryReplicas - number of instances to create in the primary zone
 * numSecondaryReplicas - number of instances to create in the secondary zone
 * namePrefix - hostname prefix
   The instance number (0 based) will be appended ("-n<#><#>")
 * primaryZone - Compute Engine zone for the instance (short name)
 * secondaryZone - Compute Engine zone for the instance (short name)
 * network - Compute Engine network for the instance (full URI)
 * image - Compute Engine image for the instance (full URI)
 * machineType - Compute Engine machine type for the instance (full URI)
 * bootDiskType - Compute Engine boot disk type for the instance (full URI)
 * dataDiskType: Compute Engine data disk type for the instance (full URI)
 * dataDiskSizeGb: Data disk size in Gigabytes
Grab the config properties
Macros and variables dealing with naming
Expand resource input parameters into full URLs
Add clone instances in the local Zone
Setup in-region replicas
Add clone instances in the remote Zone
   sourceImage: {{ properties["dataSourceImage"] }}

# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# GlusterFs configuration variables
#
# Required Cloud resource input parameters:
#  * numPrimaryReplicas - number of instances to create in the primary zone
#  * numSecondaryReplicas - number of instances to create in the secondary zone
#  * namePrefix - hostname prefix
#    The instance number (0 based) will be appended ("-n<#><#>")
#  * primaryZone - Compute Engine zone for the instance (short name)
#  * secondaryZone - Compute Engine zone for the instance (short name)
#  * network - Compute Engine network for the instance (full URI)
#  * image - Compute Engine image for the instance (full URI)
#  * machineType - Compute Engine machine type for the instance (full URI)
#  * bootDiskType - Compute Engine boot disk type for the instance (full URI)
#  * dataDiskType: Compute Engine data disk type for the instance (full URI)
#  * dataDiskSizeGb: Data disk size in Gigabytes

{% import 'path_utils.jinja' as path_utils with context %}

# Grab the config properties
{% set numPrimaryReplicas = properties["numPrimaryReplicas"] + 1%}
{% set numSecondaryReplicas = properties["numSecondaryReplicas"] + 1 %}
{% set image = properties["image"] %}

# Macros and variables dealing with naming
{% set prefix = properties["namePrefix"] %}

{% macro hostname(prefix, id) -%}
{{ "%s-n%02d"|format(prefix, id) }}
{%- endmacro %}

{% macro diskname(prefix, id) -%}
{{ "%s-data-disk-n%02d"|format(prefix, id) }}
{%- endmacro %}

# Expand resource input parameters into full URLs
{% set network = path_utils.networkPath(properties["network"]) %}
{% set primaryZone = properties["primaryZone"] %}
{% set bootDiskType = path_utils.diskTypePath(
    primaryZone, properties["bootDiskType"]) %}
{% set dataDiskType = path_utils.diskTypePath(
    primaryZone, properties["dataDiskType"]) %}
{% set machineType = path_utils.machineTypePath(
    primaryZone, properties["machineType"]) %}

resources:
# Add clone instances in the local Zone
{% for n_suffix in range(1, numPrimaryReplicas) %}

  {% set namePrefix = prefix + '-primary' %}

- type: compute.v1.disk
  name: {{ diskname(namePrefix, n_suffix) }}
  properties:

    zone: {{ primaryZone }}
    type: {{ dataDiskType }}
    sizeGb: {{ properties["dataDiskSizeGb"] }}
    sourceImage: {{ properties["dataSourceImage"] }}

- type: compute.v1.instance
  name: {{ hostname(namePrefix, n_suffix) }}
  properties:
    zone: {{ primaryZone }}
    machineType: {{ machineType }}

    disks:
    # Request boot disk creation (mark for autodelete)
    - deviceName: boot
      type: PERSISTENT
      boot: true
      autoDelete: true
      initializeParams:
        sourceImage: {{ properties["sourceImage"] }}
        diskType: {{ bootDiskType }}
        diskSizeGb: 10

    # Attach the existing data disk (mark for autodelete)
    - deviceName: {{ diskname(namePrefix, n_suffix) }}
      source: $(ref.{{ diskname(namePrefix, n_suffix) }}.selfLink)
      autoDelete: true
      type: PERSISTENT

    networkInterfaces:
    - network: {{ network }}
      accessConfigs:
      - name: External NAT
        type: ONE_TO_ONE_NAT

    tags:
      items:
      - "glusterfs-deployed-from-google-developer-console"

{% endfor %}

# Setup in-region replicas
{% set network = path_utils.networkPath(properties["network"]) %}
{% set secondaryZone = properties["secondaryZone"] %}
{% set bootDiskType = path_utils.diskTypePath(
    secondaryZone, properties["bootDiskType"]) %}
{% set dataDiskType = path_utils.diskTypePath(
    secondaryZone, properties["dataDiskType"]) %}
{% set machineType = path_utils.machineTypePath(
    secondaryZone, properties["machineType"]) %}
{% for n_suffix in range(1, numPrimaryReplicas) %}

  {% set namePrefix = prefix + '-secondary' %}

- type: compute.v1.disk
  name: {{ diskname(namePrefix, n_suffix) }}
  properties:
    zone: {{ secondaryZone }}
    type: {{ dataDiskType }}
    sizeGb: {{ properties["dataDiskSizeGb"] }}
    sourceImage: {{ properties["dataSourceImage"] }}

- type: compute.v1.instance
  name: {{ hostname(namePrefix, n_suffix) }}
  properties:
    zone: {{ secondaryZone }}
    machineType: {{ machineType }}

    disks:
    # Request boot disk creation (mark for autodelete)
    - deviceName: boot
      type: PERSISTENT
      boot: true
      autoDelete: true
      initializeParams:
        sourceImage: {{ properties["sourceImage"] }}
        diskType: {{ bootDiskType }}
        diskSizeGb: 10

    # Attach the existing data disk (mark for autodelete)
    - deviceName: {{ diskname(namePrefix, n_suffix) }}
      source: $(ref.{{ diskname(namePrefix, n_suffix) }}.selfLink)
      autoDelete: true
      type: PERSISTENT

    networkInterfaces:
    - network: {{ network }}
      accessConfigs:
      - name: External NAT
        type: ONE_TO_ONE_NAT

    tags:
      items:
      - "glusterfs-deployed-from-google-developer-console"

{% endfor %}


# Add clone instances in the remote Zone
{% set backupZone = properties["backupZone"] %}
{% set bootDiskType = path_utils.diskTypePath(
    backupZone, properties["bootDiskType"]) %}
{% set dataDiskType = path_utils.diskTypePath(
    backupZone, properties["dataDiskType"]) %}
{% set machineType = path_utils.machineTypePath(
    backupZone, properties["machineType"]) %}
{% for n_suffix in range(1, numSecondaryReplicas) %}
  {% set namePrefix = prefix + '-backup' %}

- type: compute.v1.disk
  name: {{ diskname(namePrefix, n_suffix) }}
  properties:
    zone: {{ backupZone }}
    type: {{ dataDiskType }}
    sizeGb: {{ properties["dataDiskSizeGb"] }}
#    sourceImage: {{ properties["dataSourceImage"] }}

- type: compute.v1.instance
  name: {{ hostname(namePrefix, n_suffix) }}
  properties:
    zone: {{ backupZone }}
    machineType: {{ machineType }}

    disks:
    # Request boot disk creation (mark for autodelete)
    - deviceName: boot
      type: PERSISTENT
      boot: true
      autoDelete: true
      initializeParams:
        sourceImage: {{ properties["sourceImage"] }}
        diskType: {{ bootDiskType }}
        diskSizeGb: 10

    # Attach the existing data disk (mark for autodelete)
    - deviceName: {{ diskname(namePrefix, n_suffix) }}
      source: $(ref.{{ diskname(namePrefix, n_suffix) }}.selfLink)
      autoDelete: true
      type: PERSISTENT

    networkInterfaces:
    - network: {{ network }}
      accessConfigs:
      - name: External NAT
        type: ONE_TO_ONE_NAT

    tags:
      items:
      - "glusterfs-deployed-from-google-developer-console"

{% endfor %}

Copyright 2015 Google Inc. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
path_utils.jinja
Jinja macros for expanding short resource names into full paths
Must have reference to the global env object, so when including this file,
use the jinja import "with context" option.

# Copyright 2015 Google Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# path_utils.jinja
#
# Jinja macros for expanding short resource names into full paths
# Must have reference to the global env object, so when including this file,
# use the jinja import "with context" option.

{% macro projectPrefix() -%}
{{ "https://www.googleapis.com/compute/v1/projects/%s"|format(env["project"]) }}
{%- endmacro %}

{% macro imagePath(image) -%}
{% if image.startswith("https://") -%}
{{ image }}
{% elif image.startswith("debian-") -%}
{{ "https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/" + image }}
{% elif image.startswith("windows-") -%}
{{ "https://www.googleapis.com/compute/v1/projects/windows-cloud/global/images/" + image }}
{% endif -%}
{%- endmacro %}

{% macro machineTypePath(zone, machineType) -%}
{% if machineType.startswith("https://") -%}
{{ machineType }}
{% else -%}
{{ "%s/zones/%s/machineTypes/%s"|format(projectPrefix(), zone, machineType) }}
{% endif -%}
{%- endmacro %}

{% macro networkPath(network) -%}
{% if network.startswith("https://") -%}
{{ network }}
{% else -%}
{{ "%s/global/networks/%s"|format(projectPrefix(), network) }}
{% endif -%}
{%- endmacro %}

{% macro diskTypePath(zone, diskType) -%}
{% if diskType.startswith("https://") -%}
{{ diskType }}
{% else -%}
{{ "%s/zones/%s/diskTypes/%s"|format(projectPrefix(), zone, diskType) }}
{% endif -%}
{%- endmacro %}