16.3. Using an External Git Repository
Overview
Figure 16.2, “External Git Repository Architecture” shows an overview of the Fabric architecture when the fabric is configured to use an external Git repository.
Figure 16.2. External Git Repository Architecture
External git repository architecture
When you configure a fabric with an external Git repository (which must be done at fabric creation time), the external Git repository becomes the primary Git repository for all of the containers in the Fabric. All of the Fabric servers in the ensemble maintain their own copy of the Git repository (under their respective
data/ directories), but this local copy is kept up-to-date by regularly polling the external Git repository for updates. If a change is detected in the external Git repository, every Fabric server will do a git pull to update it's local copy of the Git repository.
It is also possible for an administrator to clone a local copy of the external Git repository. Using standard
git commands, the administrator can now edit the configuration files in the local copy and push the changes to the external Git repository. As soon as those changes are received by the external Git repository, the Fabric servers will detect that an update has occurred and pull the latest configuration.
Preparing an external Git repository
When setting up this type of Fabric architecture, the first step is to prepare an external Git repository. When setting up this repository, you should pay attention to the following points:
- The Git repository must be initialized. For example, if you were creating a new Git repository on your local file system, you would initialize it using the command
git init. If you are using a Git server to host your repository (for example, Gerrit, GitLab, or GitHub), the Git repository is usually initialized automatically, after you create it. - You must ensure that all of your Fabric servers are able to access the external Git repository. For example, if your Git server uses a HTTP based protocol to access the repository, you are generally required to have username/password credentials for the HTTP BASIC authentication protocol.
Authentication
In this architecture, authentication is handled by the external Git repository (and the Git server that hosts it). The most common cases are:
- HTTP URL—in this case, the Git server is likely to use HTTP with TLS (HTTPS), to verify the server identity, and HTTP BASIC authentication, to verify the client identity. When creating the fabric (with the
fabric:createcommand), you need to specify the following additional options in this case:--external-git-url ExternalGitHttpUrl--external-git-user ExternalGitUser--external-git-password ExternalGitPass
- File URL—in this case, no authentication is required. You can specify the Git URL either in the form
/path/to/repo(recommended) orfile:///path/to/repo(slower). If the Fabric servers are deployed on separate hosts, you must make sure that they all have access to the specified directory (for example, through a Network File Server). When creating the fabric (with thefabric:createcommand), you need to specify the following additional options in this case:--external-git-url ExternalGitFileUrl
Creating a fabric with an external Git repository
Typically, to create a fabric with an external Git repository, you would enter a console command like the following:
fabric:create --new-user admin --new-user-password AdminPass --new-user-role Administrator \ --zookeeper-password ZooPass --global-resolver manualip \ --resolver manualip --manual-ip StaticIPAddress --wait-for-provisioning \ --external-git-url ExternalGitHttpUrl \ --external-git-user ExternalGitUser --external-git-password ExternalGitPass
Note the following points:
- A new user is created with username,
admin, password,AdminPass, and role,Administrator. You can use these JAAS credentials to log on to any of the containers in the fabric. - The Zookeeper password is set to
ZooPass(the only time you are prompted to enter the Zookeeper password is when joining a container to the fabric). - The resolver policy for the root container is set to
manualip(using the--resolveroption) and the global resolver policy (which becomes the default resolver policy for containers created in this fabric) is also set tomanualip. This enables you to specify the root container's IP address,StaticIPAddress, explicitly. It is essential that you assign a static IP address to the Fabric server host (for demonstrations and tests on a single machine, you can use the loopback address,127.0.0.1). - The Git URL,
ExternalGitHttpUrl, is specified through the--external-git-urloption. - Assuming that you use a HTTP Git URL with BASIC authentication enabled, you will also need to specify credentials, using the
--external-git-userand--external-git-passwordoptions.
What happens if the external Git repository fails?
Because the external Git repository is the primary Git repository, which is used to synchronize configuration data with the other Fabric servers, it is technically a single point of failure. The effect of a failure of the external Git repository is not as serious as you might think, however. It does not lead to a failure of the Fabric servers. In case of an external Git repository failure (or a loss of connectivity) the Fabric servers continue to operate with the configuration data they have already cached in their local copies of the Git repository. As soon as the external Git repository comes back on line, they will re-synchronize their configuration data.
External Git repository tutorial
The following tutorial explains how to create a fabric, which synchronizes its configuration with an external Git repository:
- Create a new (empty) Git repository, which you can use as the external Git repository. Typically, you would create the Git repository in a hosting service, such as GitLab, Gerrit, or GitHub. Make a note of the new repository's HTTP URL,
ExternalGitHttpUrl, and make sure that it is possible to access the external Git repository from the hosts where you will be deploying your Fabric servers. - (Optional) Prepare the container for a cold start. Delete the following directories:
InstallDir/data InstallDir/instances
WarningPerforming a cold start completely wipes the current state of the root container, including all of the deployed bundles, and features, and most of the stored data. Do not perform this operation on a production system. - Start up the container, by entering the following command:
./bin/amq
- Create a new fabric. At the container prompt, enter the following console command:
fabric:create --new-user admin --new-user-password AdminPass --new-user-role Administrator \ --zookeeper-password ZooPass --global-resolver manualip \ --resolver manualip --manual-ip 127.0.0.1 --wait-for-provisioning \ --external-git-url ExternalGitHttpUrl \ --external-git-user ExternalGitUser --external-git-password ExternalGitPass
You need to substitute your own values forAdminPassandZooPass. TheExternalGitHttpUrlis the HTTP URL of the external Git repository you created earlier and theExternalGitUservalue and theExternalGitPassvalue are the username/password credentials required to access the external Git repository (using HTTP BASIC authentication).This sample command uses the--manual-ipoption to assign the loopback address,127.0.0.1, to the root container. If your host has a static IP address and hostname assigned to it, however, it would be better to use the assigned hostname here instead.You need to wait a minute or two for this command to complete. - After your fabric has been created, navigate to the contents of the external Git repository in your browser (assuming that your Git server supports this functionality). The external repository should now be populated with the default configuration of your fabric, with two branches available:
masterand1.0. The1.0branch is the branch that is initially used by your fabric. - Create a local clone of the external Git repository, which you can then use to push or pull profile configurations. Open a new command prompt and, in a convenient location on the file system, enter the following command:
git clone -b 1.0
ExternalGitHttpUrlThisgitcommand will prompt you to enter the username and password credentials for the external Git repository.This command clones the Fabric Git repository and checks out the1.0branch. You should now be able to see the profile configuration files under thefabric/profilessubdirectory. - You can now use regular
gitcommands to configure your Fabric profiles. Simply edit the files in your local Git repository, add the changes, commit, and then push the changes to the external Git repository (working in the1.0branch). Shortly after the changes are pushed to the external Git repository, the containers in your Fabric ensemble (the Fabric servers) will poll the repository, pull the changes, and redeploy any changed profiles..
