Red Hat Summit Red Hat SummitSupportKonsoleEntwicklerDemo startenKontakt

Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.

Chapter 8. Bulk importing in Red Hat Developer Hub

Important

These features are for Technology Preview only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs), might not be functionally complete, and Red Hat does not recommend using them for production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information on Red Hat Technology Preview features, see Technology Preview Features Scope.

Red Hat Developer Hub can automate the onboarding of GitHub repositories and GitLab projects, and track their import status.

You can enable the Bulk Import feature for users and give them the necessary permissions to access it. This feature is available for GitHub repositories and GitLab projects.

Prerequisites

Procedure

  1. The Bulk Import plugins are installed but disabled by default. To enable the ./dynamic-plugins/dist/red-hat-developer-hub-backstage-plugin-bulk-import-backend-dynamic and ./dynamic-plugins/dist/red-hat-developer-hub-backstage-plugin-bulk-import plugins, edit your dynamic-plugins.yaml with the following content: 

    plugins:
  - package: ./dynamic-plugins/dist/red-hat-developer-hub-backstage-plugin-bulk-import-backend-dynamic
    disabled: false
  - package: ./dynamic-plugins/dist/red-hat-developer-hub-backstage-plugin-bulk-import
    disabled: false
    Copy to Clipboard Toggle word wrap

    See Installing and viewing plugins in Red Hat Developer Hub.

  2. Configure the required bulk.import RBAC permission for the users who are not administrators as shown in the following code:

    rbac-policy.csv fragment

    p, role:default/bulk-import, bulk.import, use, allow
g, user:default/<your_user>, role:default/bulk-import
    Copy to Clipboard Toggle word wrap

    Note that only Developer Hub administrators or users with the bulk.import permission can use the Bulk Import feature. See Permission policies in Red Hat Developer Hub.

Verification

  • The sidebar displays a Bulk Import option.
  • The Bulk Import page shows a list of added GitHub repositories and GitLab projects.

8.2. Importing multiple GitHub repositories
Link kopieren

In Red Hat Developer Hub, you can select your GitHub repositories and automate their onboarding to the Developer Hub catalog.

Prerequisites

Procedure

  1. Click Bulk Import in Developer Hub left sidebar.
  2. If your RHDH instance has multiple source control tools configured, select GitHub from the Source control tool list.

  3. Select the repositories to import, and validate.

    Developer Hub creates a pull request in each selected repository to add the required catalog-info.yaml file.

  4. For each repository to import, click on the PR link to review and merge the changes in GitHub.

Verification

  1. Click Bulk Import in Developer Hub left sidebar.
  2. Verify that each imported GitHub repository in the Selected repositories list has the status Waiting for approval or Imported.
  3. For each Waiting for approval repository, click the pull request link to review and merge the catalog-info.yaml file in the corresponding repository.

8.3. Importing multiple GitLab repositories
Link kopieren

In Red Hat Developer Hub, you can select your GitLab projects and automate their onboarding to the Developer Hub catalog. This feature is a Technology Preview.

Important

Technology Preview features provide early access to upcoming product innovations, enabling you to test functionality and provide feedback during the development process. However, these features are not fully supported under Red Hat Subscription Level Agreements, may not be functionally complete, and are not intended for production use. As Red Hat considers making future iterations of Technology Preview features generally available, we will attempt to resolve any issues that customers experience when using these features. See: Technology Preview support scope.

Prerequisites

Procedure

  1. Click Bulk Import in Developer Hub left sidebar.
  2. If your RHDH instance has multiple source control tools configured, select GitLab as your Source control tool option.

  3. Select the projects to import, and validate.

    Developer Hub creates a merge request in each selected project to add the required catalog-info.yaml file.

  4. For each project to import, click on the PR link to review and merge the changes in Gitlab.

Verification

  1. Click Bulk Import in Developer Hub left sidebar.
  2. Verify that each imported GitLab project in the Selected projects list has the status Waiting for approval or Imported.
  3. For projects with the Waiting for approval status, click the merge request link to add the catalog-info.yaml file to the project repository.

8.4. Monitoring Bulk Import actions using audit logs
Link kopieren

The Bulk Import backend plugin adds the following events to the Developer Hub audit logs. See Audit logs in Red Hat Developer Hub for more information on how to configure and view audit logs.

Bulk Import Events:

BulkImportUnknownEndpoint
Tracks requests to unknown endpoints.
BulkImportPing
Tracks GET requests to the /ping endpoint, which allows us to make sure the bulk import backend is up and running.
BulkImportFindAllOrganizations
Tracks GET requests to the /organizations endpoint, which returns the list of organizations accessible from all configured GitHub Integrations.
BulkImportFindRepositoriesByOrganization
Tracks GET requests to the /organizations/:orgName/repositories endpoint, which returns the list of repositories for the specified organization (accessible from any of the configured GitHub Integrations).
BulkImportFindAllRepositories
Tracks GET requests to the /repositories endpoint, which returns the list of repositories accessible from all configured GitHub Integrations.
BulkImportFindAllImports
Tracks GET requests to the /imports endpoint, which returns the list of existing import jobs along with their statuses.
BulkImportCreateImportJobs
Tracks POST requests to the /imports endpoint, which allows to submit requests to bulk-import one or many repositories into the Developer Hub catalog, by eventually creating import pull requests in the target repositories.
BulkImportFindImportStatusByRepo
Tracks GET requests to the /import/by-repo endpoint, which fetches details about the import job for the specified repository.
BulkImportDeleteImportByRepo
Tracks DELETE requests to the /import/by-repo endpoint, which deletes any existing import job for the specified repository, by closing any open import pull request that could have been created.

Example bulk import audit logs

{
  "actor": {
    "actorId": "user:default/myuser",
    "hostname": "localhost",
    "ip": "::1",
    "userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36"
  },
  "eventName": "BulkImportFindAllOrganizations",
  "isAuditLog": true,
  "level": "info",
  "message": "'get /organizations' endpoint hit by user:default/myuser",
  "meta": {},
  "plugin": "bulk-import",
  "request": {
    "body": {},
    "method": "GET",
    "params": {},
    "query": {
      "pagePerIntegration": "1",
      "sizePerIntegration": "5"
    },
    "url": "/api/bulk-import/organizations?pagePerIntegration=1&sizePerIntegration=5"
  },
  "response": {
    "status": 200
  },
  "service": "backstage",
  "stage": "completion",
  "status": "succeeded",
  "timestamp": "2024-08-26 16:41:02"
}
Copy to Clipboard Toggle word wrap

8.5. Input parameters for Bulk Import Scaffolder template
Link kopieren

As an administrator, you can use the Bulk Import plugin to run a Scaffolder template task with specified parameters, which you must define within the template.

The Bulk Import plugin analyzes Git repository information and provides the following parameters for the Scaffolder template task:

repoUrl

Normalized repository URL in the following format: 

  ${gitProviderHost}?owner=${owner}&repo=${repository-name}
Copy to Clipboard Toggle word wrap
name
The repository name.
organization
The repository owner, which can be a user nickname or organization name.
branchName
The proposed repository branch. By default, the proposed repository branch is bulk-import-catalog-entity.
targetBranchName
The default branch of the Git repository.
gitProviderHost
The Git provider host parsed from the repository URL. You can use this parameter to write Git-provider-agnostic templates.

Example of a Scaffolder template: 

parameters:
  - title: Repository details
    required:
      - repoUrl
      - branchName
      - targetBranchName
      - name
      - organization
    properties:
      repoUrl:
        type: string
        title: Repository URL (Backstage format)
        description: github.com?owner=Org&repo=repoName
      organization:
        type: string
        title: Owner of the repository
      name:
        type: string
        title: Name of the repository
      branchName:
        type: string
        title: Branch to add the catalog entity to
      targetBranchName:
        type: string
        title: Branch to target the PR/MR to
      gitProviderHost:
        type: string
        title: Git provider host
Copy to Clipboard Toggle word wrap

8.6. Setting up a custom Scaffolder workflow for Bulk Import
Link kopieren

As an administrator, you can create a custom Scaffolder template in line with the repository conventions of your organization and add the template into the Red Hat Developer Hub catalog for use by the Bulk Import plugin on multiple selected repositories.

You can define various custom tasks, including, but not limited to the following:

  • Importing existing catalog entities from a repository
  • Creating pull requests for cleanup
  • Calling webhooks for external system integration

Prerequisites

  • You created a custom Scaffolder template for the Bulk Import plugin.

  • You have run your RHDH instance with the following environment variable enabled to allow the use of the Scaffolder functionality: 

    export NODE_OPTIONS=--no-node-snapshot
    Copy to Clipboard Toggle word wrap

Procedure

  • Configure your app-config.yaml configuration to instruct the Bulk Import plugin to use your custom template as shown in the following example: 

    bulkImport:
  importTemplate: <your_template_entity_reference_or_template_name>
  importAPI: `open-pull-requests` | `scaffolder`;
    Copy to Clipboard Toggle word wrap

    where:

    importTemplate:
    Enter your Scaffolder template entity reference.
    importAPI
    Set the API to 'scaffolder' to trigger the defined workflow for high-fidelity automation. This field defines the import workflow and currently supports two following options:
    open-pull-requests
    This is the default import workflow, which includes the logic for creating pull requests for every selected repository.
    scaffolder

    This workflow uses an import scenario defined in the Scaffolder template to create import jobs. Select this option to use the custom import scenario defined in your Scaffolder template.

    Optional: You can direct the Bulk Import plugin to hand off the entire list of selected repositories to a custom Orchestrator workflow.

    Important

    The Scaffolder template must be generic and not specific to a single repository if you want your custom Scaffolder template to run successfully for every repository in the bulk list.

Verification

  • The Bulk Import plugin runs the custom Scaffolder template for the list of repositories using the /task-imports API endpoint.

8.7. Running Orchestrator workflows for bulk imports
Link kopieren

As a platform engineer, you can configure the Bulk Import plugin to run Orchestrator workflows for bulk import operations. This mode uses the Orchestrator engine to provide advanced capabilities, such as creating pull requests or publishing configurations across multiple repositories.

Prerequisites

  • You have installed and configured the Orchestrator plugin in your Backstage instance.
  • You have registered a generic custom workflow (for example, universal-pr) in the Orchestrator plugin.
  • You have Role-Based Access Control (RBAC) permissions to configure the Bulk Import plugin.

Procedure

  1. Configure the Bulk Import plugin by editing your app-config.yaml file to enable Orchestrator mode. 

    bulkImport:
  orchestratorWorkflow: your_workflow_id
  importAPI: 'orchestrator'
    Copy to Clipboard Toggle word wrap

    where:

    orchestratorWorkflow
    The ID of the workflow to run for each repository.
    importAPI
    The execution mode for the workflow. Enter orchestrator to enable workflow execution.

  2. Verify that the Orchestrator workflow receives the following input: 

    {
  "inputData": {
    "owner": "redhat-developer",
    "repo": "rhdh-plugins",
    "baseBranch": "main",
    "targetBranch": "bulk-import-orchestrator"
  },
  "authTokens": [
    {
      "token": "<github_token>",
      "provider": "github"
    }
  ]
}
    Copy to Clipboard Toggle word wrap

    where:

    owner
    Specifies the repository owner (organization or user name).
    repo
    Specifies the repository name.
    baseBranch
    Specifies the default branch of the Git repository (for example, main).
    targetBranch
    Specifies the target branch for the import operation. By default, this is set to bulk-import-orchestrator.
    authTokens

    Specifies the authentication tokens for the Git provider:

    • For GitHub: { token: <github_token>, provider: github }
    • For GitLab: { token: <gitlab_token>, provider: gitlab }

  3. Navigate to the Bulk Import page in the sidebar and complete the following steps:

    1. Select your Git provider (for example, GitHub or GitLab).
    2. Select the projects you want to import.
  4. Click import to run the workflow.

Verification

  • Locate your repository and confirm status is COMPLETED.

8.8. Data handoff and custom workflow design
Link kopieren

When you configure the Bulk Import plugin by setting the importAPI field to scaffolder, the Bulk Import Backend passes all necessary context directly to the Scaffolder API.

As an administrator, you can define the Scaffolder template workflow and structure the workflow to do the following:

Define template parameters to consume input
Structure the Scaffolder template to receive the repository data as template parameters for the current workflow run. The template must be generic, and not specific to a single repository, so that it can successfully run for every repository in the bulk list.
Automate processing for each repository
Implement the custom logic needed for a single repository within the template. The Orchestrator iterates through the repository list, launching the template once for each repository and passes only the data for that single repository to the template run. This allows you to automate tasks such as creating the catalog-info.yaml, running compliance checks, or registering the entity with the catalog.
Red Hat logoGithubredditYoutubeTwitter

Lernen

Testen, kaufen und verkaufen

Communitys

Über Red Hat Dokumentation

Wir helfen Red Hat Benutzern, mit unseren Produkten und Diensten innovativ zu sein und ihre Ziele zu erreichen – mit Inhalten, denen sie vertrauen können. Entdecken Sie unsere neuesten Updates.

Mehr Inklusion in Open Source

Red Hat hat sich verpflichtet, problematische Sprache in unserem Code, unserer Dokumentation und unseren Web-Eigenschaften zu ersetzen. Weitere Einzelheiten finden Sie in Red Hat Blog.

Über Red Hat

Wir liefern gehärtete Lösungen, die es Unternehmen leichter machen, plattform- und umgebungsübergreifend zu arbeiten, vom zentralen Rechenzentrum bis zum Netzwerkrand.

Theme

© 2026 Red HatNach oben