Chapter 6. Designing the replication process

When you consider implementing replication, answer the following fundamental questions:

Learn about the concepts that provide understanding of how Directory Server implements replication:

Data consistency refers to how closely the contents of replicated databases match each other at a given time. A supplier determines when consumers must be updated, and initiates replication. Replication can start only after consumers have been initialized.

Directory Server can always keep replicas synchronized or schedule updates for a particular time of day or day in a week.

Use constantly synchronized replicas when:

Use scheduled updates when:

The main reasons for the loose consistency are the following:

In the single-supplier replication scenario, a supplier server maintains the main copy of the directory data (read-write replica) and sends updates of this data to one or more consumer servers. All directory modifications occur on the read-write replica on the supplier server, and the consumer servers contain read-only replicas of the data.

The supplier server maintains a changelog that records all the changes made to the supplier replica.

The following diagram shows the single-supplier replication scenario:

Figure 6.1. Single-supplier replication

The total number of consumer servers that a single supplier server can manage depends on the speed of the networks and the total number of entries that are modified on a daily basis.

In a multi-supplier replication environment, main copies of the same information can exist on multiple servers, and the directory data can be updated simultaneously in different locations. The changes that occur on each server are replicated to the other servers meaning that each server functions as both a supplier and a consumer.

When the same data is modified on multiple servers, replication conflicts occur. Using a conflict resolution procedure, Directory Server uses the most recent change as the valid one.

In a multi-supplier environment, each supplier needs to have replication agreements that point to consumers and other suppliers. For example, you configure replication with two suppliers, Supplier A and Supplier B, and two consumers, Consumer C and Consumer D. In addition, you decide that one supplier updates only one consumer. On Supplier A, you create a replication agreement that points to Supplier B and Consumer C. On Supplier B, you create a replication agreement that points to Supplier A and Consumer D. The following diagram illustrates the replication agreements:

Figure 6.2. Multi-supplier replication with two suppliers

Using many suppliers requires creating a range of replication agreements. In addition, each supplier can be configured in different topologies meaning that your Directory Server environment can have 20 different directory trees and even schema differences. Many other variables may have a direct impact on the topology selection.

Suppliers can send updates to all other suppliers or to some subset of suppliers. When updates are sent to all suppliers, changes are propagated faster and the overall scenario has better failure tolerance. However, it increases the complexity of supplier configuration and introduces high network and high server demand. Sending updates to a subset of suppliers is much simpler to configure and reduces the network and server loads, but increases the risk of data loss if multiple server failures occur.

If you have 20 suppliers, then you need to create 380 replication agreements in total (20 suppliers with 19 agreements each).

If the possibility of two or more servers failing at the same time is small or connection between certain suppliers is better, consider using a partly connected topology.

In a cascading replication scenario, a hub server receives updates from a supplier server and sends those updates to consumer servers. The hub server is a hybrid, because it holds a read-only replica, like a typical consumer server, and it also maintains a changelog like a typical supplier server.

Hub server forward the supplier data to consumers and refer update requests from directory clients to suppliers.

The following diagram shows the cascading replication scenario:

Figure 6.3. Cascading replication scenario

The following diagram shows how replicas are configured on each server (read-write or read-only) and which servers maintain the changelog.

Figure 6.4. Replication traffic and changelogs in cascading replication

Cascading replication is useful in the following cases:

Any of the replication scenarios can be combined to suit the needs of the network and directory environment. One common combination is to use a multi-supplier configuration with a cascading configuration.

The following diagram shows an example topology for a mixed scenario:

Figure 6.5. Combined multi-supplier and cascading replication

Gather information about the network quality and usage to help define the replication strategy:

Replication requires resources. Consider the following resource requirements when defining the replication strategy:

In multi-supplier topologies, suppliers maintain additional logs required for replication, including the changelog of the directory edits, state information for updated entries, and tombstone entries for deleted entries. Because these log files can become very large, you must periodically clean up these files to avoid unnecessary usage of the disk space.

On each server, you can use the following attributes to configure the replication logs maintenance in a replicated environment:

Use replication to prevent directory unavailability when a single server fails. At a minimum, replicate the local directory tree to at least one backup server. How often you replicate for fault tolerance depends on your requirements. However, base this decision on the quality of the hardware and networks used by your directory. Unreliable hardware requires more backup servers.

You can choose the following strategies to prevent directory unavailability:

LDAP client applications are usually configured to search only one LDAP server. If you do not have a custom client application to rotate through LDAP servers located at different DNS hostnames, you can only configure your LDAP client application to look at a single DNS hostname for Directory Server. Therefore, you may need to use either DNS round-robins or network sorts to provide failover to your backup Directory Server.

You may need to use replication for local availability depending on the quality of your network and if your data is mission-critical.

Use replication for local availability for the following reasons:

One of the main reasons to replicate directory data is to balance the workload of your network and to improve the directory performance.

As directory entries are usually 1 KB in size, every directory search adds approximately 1 KB to your network load. If your directory users perform ten directory searches per day, then the increased network load for every directory user is around 10 KB per day. If you have a slow, heavily loaded, or unreliable WAN, you may need to replicate your directory tree to a local server.

However, determine if locally available data is worth the cost of the increased network load caused by replication. If you replicate an entire directory tree to a remote site, you potentially add a larger load on your network in comparison to the traffic that results from users' searches. This is especially true if your directory changes frequently, yet you have only a few users at the remote site who perform a few directory searches per day.

The following table compares the load impact of replicating a directory with one million entries, where 100,000 of the entries undergo daily change, with the load impact of having a small remote site of 100 employees that perform 10 searches per day each.

A compromise between making data available to local sites without overloading the network is to use scheduled replication. For more information on data consistency and replication schedules, see Data consistency.

6.3.6.1. Example of network load balancing

This example describes an enterprise that has offices in New York (NY) and Los Angeles (LA), and each office manages a separate sub-tree.

The following diagram show how the enterprise manages the sub-trees:

Figure 6.6. The enterprise NY and LA sub-trees

Each office contains a high-speed network, but the connection between the two cities is unreliable. To balance the network load, use the following strategy:

• Select one server in each office to be the supplier server for the locally managed data.

  Replicate locally managed data from that supplier to the corresponding supplier server in the remote office. When you have main copy of the data in each location, users do not perform update and search operations over the unreliable connection. As a result, performance is optimized.

• Replicate the directory tree on each supplier server (including data supplied from the remote office) to at least one local Directory Server to ensure availability of the directory data.

• Configure cascading replication in each location with an increased number of consumers dedicated to search on the local data to provide further load balancing.

  The NY office generates more NY specific searches than LA specific searches. The example shows the NY office with three NY data consumers and one LA consumer. The LA office has three LA data consumers and one NY data consumer.

Figure 6.7. Example of load balancing for the enterprise

Additional resources

• About suffixes
• Cascading-replication
• Multi-supplier replication

With fractional replication, you can choose a set of attributes that Directory Server does not replicate from a supplier to a consumer or another supplier. Therefore, a database can be replicated without replicating all the information that the database contains.

Fractional replication is enabled and configured per replication agreement. Directory Server applies the exclusion of attributes equally to all entries. The excluded attributes always have no value on consumers. Therefore, a client performing a search against the consumer server never sees the excluded attributes, even if search filters explicitly specify these attributes.

Use fractional replication in the following situations:

Wide area networks (WAN) typically have higher latency, a higher bandwidth-delay product, and lower speeds than local area networks. Directory Server supports efficient replication when a supplier and consumer are connected using a wide-area network.

Previously, the replication protocols that Directory Server used were highly latency-sensitive, because the supplier sent only one update operation and then waited for a response from the consumer. This led to reduced throughput with higher latencies.

Currently, a supplier sends many updates and entries to the consumer without waiting for a response, and the replication throughput is similar to throughput of a local area network.

Consider the following performance and security issues when using a WAN:

The directory stores access control instructions (ACIs) as attributes of entries and Directory Server replicates these ACIs together with other directory content. For example, to restrict access to the directory from a certain host, use only host-specific settings in the ACI. Otherwise, when the ACI is replicated to other servers, the access to the directory will be denied on all servers because Directory Server evaluates ACIs locally.

For more information about designing access control for the directory, see Designing access control.

Replication works with most of the plug-ins delivered with Directory Server. However, the following plug-ins have limitations and exceptions in multi-supplier environments:

When you use chaining to distribute entries across the directory, the server containing the database link refers to a remote server that contains the actual data. In this environment, you cannot replicate the database link. However, you might replicate the database that contains the actual data on the remote server.

In a replicated environment, the schema must be consistent across all of the servers that participate in replication. To ensure schema consistency, make schema modifications only on a single supplier server.

If you configured replication between servers, schema replication occurs by default.

Directory Server replicates changes to the schema made by using dsconf command, the web console, LDAP modify operations, or made directly to the 99user.ldif file.

If you make schema modifications on two supplier servers, consumers receive the data from the two suppliers, each with a different schema. The consumer applies the modifications of the supplier that has the more recent schema version. In this situation, the schema of the consumers always differs from one of the suppliers. To avoid this, always make sure you make schema modifications on one supplier only.

You do not need to create special replication agreements to replicate a schema. However, the same Directory Server can hold supplier and consumer replicas. Therefore, always identify the server that functions as a supplier for the schema, and then set up replication agreements between this supplier and all other servers in the replication environment that will function as consumers for the schema information.

For more information on standard schema files, see Standard schema.

If you use other custom files, you must copy these files to all servers in your topology manually after making changes on the supplier.