Chapter 4. Automating API lifecycle with 3scale toolbox

Procedure

1. Add the following to the beginning of your Jenkins pipeline to reference the 3scale shared library from your pipeline:

   #!groovy
   
   library identifier: '3scale-toolbox-jenkins@master',
      retriever: modernSCM([$class: 'GitSCMSource',
        remote: 'https://github.com/rh-integration/3scale-toolbox-jenkins.git'])

2. Declare a global variable to hold the ThreescaleService object so that you can use it from the different stages of your pipeline.

   def service = null

3. Create the ThreescaleService with all the relevant information:

   stage("Prepare") {
     service = toolbox.prepareThreescaleService(
        openapi: [ filename: "swagger.json" ],
        environment: [ baseSystemName: "my_service" ],
        toolbox: [ openshiftProject: "toolbox",
                      destination: "3scale-tenant",
                      secretName: "3scale-toolbox" ],
        service: [:],
        applications: [
           [ name: "my-test-app", description: "This is used for tests", plan: "test", account: "<CHANGE_ME>" ]
           ],
        applicationPlans: [
          [ systemName: "test", name: "Test", defaultPlan: true, published: true ],
          [ systemName: "silver", name: "Silver" ],
          [ artefactFile: "https://raw.githubusercontent.com/my_username/API-Lifecycle-Mockup/master/testcase-01/plan.yaml"],
        ]
       )
   
     echo "toolbox version = " + service.toolbox.getToolboxVersion()
    }

   • openapi.filename is the path to the file containing the OpenAPI specification.
   • environment.baseSystemName is used to compute the final system_name, based on environment.environmentName and the API major version from the OpenAPI specification info.version.
   • toolbox.openshiftProject is the OpenShift project in which Kubernetes jobs will be created.
   • toolbox.secretName is the name of the Kubernetes secret containing the 3scale toolbox configuration file, as shown in Installing the 3scale toolbox and enabling access.
   • toolbox.destination is the name of the 3scale toolbox remote instance.
   • applicationPlans is a list of application plans to create by using a .yaml file or by providing application plan property details.

4. Add a pipeline stage to provision the service in 3scale:

   stage("Import OpenAPI") {
     service.importOpenAPI()
     echo "Service with system_name ${service.environment.targetSystemName} created !"
   }

5. Add a stage to create the application plans:

   stage("Create an Application Plan") {
     service.applyApplicationPlans()
   }

6. Add a global variable and a stage to create the test application:

   stage("Create an Application") {
     service.applyApplication()
   }

7. Add a stage to run your integration tests. When using APIcast Hosted instances, you must fetch the proxy definition to extract the staging public URL:

   stage("Run integration tests") {
     def proxy = service.readProxy("sandbox")
     sh """set -e +x
     curl -f -w "ListBeers: %{http_code}\n" -o /dev/null -s ${proxy.sandbox_endpoint}/api/beer -H 'api-key: ${service.applications[0].userkey}'
     curl -f -w "GetBeer: %{http_code}\n" -o /dev/null -s ${proxy.sandbox_endpoint}/api/beer/Weissbier -H 'api-key: ${service.applications[0].userkey}'
     curl -f -w "FindBeersByStatus: %{http_code}\n" -o /dev/null -s ${proxy.sandbox_endpoint}/api/beer/findByStatus/   available -H 'api-key: ${service.applications[0].userkey}'
     """
   }

8. Add a stage to promote your API to production:

   stage("Promote to production") {
     service.promoteToProduction()
   }

Procedure

1. Write a utility function to call the 3scale toolbox. The following creates a Kubernetes job that runs the 3scale toolbox:

   #!groovy
   
   def runToolbox(args) {
     def kubernetesJob = [
       "apiVersion": "batch/v1",
       "kind": "Job",
       "metadata": [
         "name": "toolbox"
       ],
       "spec": [
         "backoffLimit": 0,
         "activeDeadlineSeconds": 300,
         "template": [
           "spec": [
             "restartPolicy": "Never",
             "containers": [
               [
                "name": "job",
                "image": "registry.redhat.io/3scale-amp2/toolbox-rhel7:3scale2.9",
                "imagePullPolicy": "Always",
                "args": [ "3scale", "version" ],
                "env": [
                  [ "name": "HOME", "value": "/config" ]
                ],
                "volumeMounts": [
                  [ "mountPath": "/config", "name": "toolbox-config" ],
                  [ "mountPath": "/artifacts", "name": "artifacts" ]
                 ]
               ]
             ],
             "volumes": [
               [ "name": "toolbox-config", "secret": [ "secretName": "3scale-toolbox" ] ],
               [ "name": "artifacts", "configMap": [ "name": "openapi" ] ]
             ]
           ]
         ]
       ]
     ]
   
     kubernetesJob.spec.template.spec.containers[0].args = args
   
     sh "rm -f -- job.yaml"
     writeYaml file: "job.yaml", data: kubernetesJob
   
     sh """set -e
     oc delete job toolbox --ignore-not-found
     sleep 2
     oc create -f job.yaml
     sleep 20 # Adjust the sleep duration to your server velocity
     """
   
     def logs = sh(script: "set -e; oc logs -f job/toolbox", returnStdout: true)
     echo logs
     return logs
   }

   Kubernetes object template

   This function uses a Kubernetes object template to run the 3scale toolbox, which you can adjust to your needs. It sets the 3scale toolbox CLI arguments and writes the resulting Kubernetes job definition to a YAML file, cleans up any previous run of the toolbox, creates the Kubernetes job, and waits:

   • You can adjust the wait duration to your server velocity to match the time that a pod needs to transition between the Created and the Running state. You can refine this step using a polling loop.
   • The OpenAPI specification file is fetched from a ConfigMap named openapi.
   • The 3scale Admin Portal hostname and access token are fetched from a secret named 3scale-toolbox, as shown in Installing the 3scale toolbox and enabling access.
   • The ConfigMap will be created by the pipeline in step 3. However, the secret was already provisioned outside the pipeline and is subject to Role-Based Access Control (RBAC) for enhanced security.

2. Define the global environment variables to use with 3scale toolbox in your Jenkins pipeline stages. For example:

   3scale Hosted

   def targetSystemName = "saas-apikey-usecase"
   def targetInstance = "3scale-saas"
   def privateBaseURL = "http://echo-api.3scale.net"
   def testUserKey = "abcdef1234567890"
   def developerAccountId = "john"

   3scale On-premises

   When using self-managed APIcast or an on-premises installation of 3scale, you must declare two more variables:

   def publicStagingBaseURL = "http://my-staging-api.example.test"
   def publicProductionBaseURL = "http://my-production-api.example.test"

   The variables are described as follows:

   • targetSystemName: The name of the service to be created.
   • targetInstance: This matches the name of the 3scale remote instance created in Installing the 3scale toolbox and enabling access.
   • privateBaseURL: The endpoint host of your API backend.
   • testUserKey: The user API key used to run the integration tests. It can be hardcoded as shown or generated from an HMAC function.
   • developerAccountId: The ID of the target account in which the test application will be created.
   • publicStagingBaseURL: The public staging base URL of the service to be created.
   • publicProductionBaseURL: The public production base URL of the service to be created.

3. Add a pipeline stage to fetch the OpenAPI specification file and provision it as a ConfigMap on OpenShift as follows:

   node() {
    stage("Fetch OpenAPI") {
      sh """set -e
      curl -sfk -o swagger.json https://raw.githubusercontent.com/microcks/api-lifecycle/master/beer-catalog-demo/api-contracts/beer-catalog-api-swagger.json
      oc delete configmap openapi --ignore-not-found
      oc create configmap openapi --from-file="swagger.json"
      """
    }

4. Add a pipeline stage that uses the 3scale toolbox to import the API into 3scale:

   3scale Hosted

   stage("Import OpenAPI") {
     runToolbox([ "3scale", "import", "openapi", "-d", targetInstance, "/artifacts/swagger.json", "--override-private-base-url=${privateBaseURL}", "-t", targetSystemName ])
   }

   3scale On-premises

   When using self-managed APIcast or an on-premises installation of 3scale, you must also specify the options for the public staging and production base URLs:

   stage("Import OpenAPI") {
     runToolbox([ "3scale", "import", "openapi", "-d", targetInstance, "/artifacts/swagger.json", "--override-private-base-url=${privateBaseURL}", "-t", targetSystemName, "--production-public-base-url=${publicProductionBaseURL}", "--staging-public-base-url=${publicStagingBaseURL}" ])
   }

5. Add pipeline stages that use the toolbox to create a 3scale application plan and an application:

   stage("Create an Application Plan") {
     runToolbox([ "3scale", "application-plan", "apply", targetInstance, targetSystemName, "test", "-n", "Test Plan", "--default" ])
   }
   
   stage("Create an Application") {
     runToolbox([ "3scale", "application", "apply", targetInstance, testUserKey, "--account=${developerAccountId}", "--name=Test Application", "--description=Created by Jenkins", "--plan=test", "--service=${targetSystemName}" ])
   }

   stage("Run integration tests") {
     def proxyDefinition = runToolbox([ "3scale", "proxy", "show", targetInstance, targetSystemName, "sandbox" ])
     def proxy = readJSON text: proxyDefinition
     proxy = proxy.content.proxy
   
     sh """set -e
     echo "Public Staging Base URL is ${proxy.sandbox_endpoint}"
     echo "userkey is ${testUserKey}"
     curl -vfk ${proxy.sandbox_endpoint}/beer -H 'api-key: ${testUserKey}'
     curl -vfk ${proxy.sandbox_endpoint}/beer/Weissbier -H 'api-key: ${testUserKey}'
     curl -vfk ${proxy.sandbox_endpoint}/beer/findByStatus/available -H 'api-key: ${testUserKey}'
     """
   }

6. Add a stage that uses the toolbox to promote the API to your production environment.

   stage("Promote to production") {
     runToolbox([ "3scale", "proxy", "promote", targetInstance,  targetSystemName ])
   }