15.11. Managing Attributes Within Fractional Replication
As Section 15.1.7, “Replicating a Subset of Attributes with Fractional Replication” describes, fractional replication allows administrators to set attributes that are excluded from replication updates. Administrators can do this for a variety of performance reasons — to limit the number of large attributes that are sent over a network or to reduce the number of times that fixup tasks (like
memberOf calculations) are run.
The list of attributes to exclude from replication are defined in the
nsDS5ReplicatedAttributeList attribute. This attribute is part of the replication agreement and it can be configured in the replication agreement wizard in the web console or through the command line when the replication agreement is created.
nsDS5ReplicatedAttributeList: (objectclass=*) $ EXCLUDE memberof authorityRevocationList accountUnlockTime
Important
Directory Server requires the
(objectclass=*) $ EXCLUDE part in the value of the nsDS5ReplicatedAttributeList attribute. If you edit the attribute directly, for example using the ldapmodify utility, you must specify this part together with the list of attributes as displayed in the example above. However, both the dsconf and web console automatically add the (objectclass=*) $ EXCLUDE part, and you must only specify the attributes.
15.11.1. Setting Different Fractional Replication Attributes for Total and Incremental Updates
When fractional replication is first configured, the list of excluded attributes applies to every update operation. Meaning, this list of attributes is excluded for a total update as well as regular incremental updates. However, there can be times when attributes should be excluded from incremental updates for performance but should be included in a total update to ensure the directory data sets are complete. In this case, it is possible to add a second attribute that defines a separate list of attributes to exclude from total updates,
nsDS5ReplicatedAttributeListTotal.
Note
nsDS5ReplicatedAttributeList is the primary fractional replication attribute. If only nsDS5ReplicatedAttributeList is set, then it applies to both incremental updates and total updates. If both nsDS5ReplicatedAttributeList and nsDS5ReplicatedAttributeListTotal are set, then nsDS5ReplicatedAttributeList only applies to incremental updates.
For example, every time a
memberOf attribute is added to an entry, a memberOf fixup task is run to resolve the group membership. This can cause overhead on the server if that task is run every time replication occurs. Since a total update only occurs for a database which is newly-added to replication or that has been offline for a long time, running a memberOf fixup task after a total update is an acceptable option. In this case, the nsDS5ReplicatedAttributeList attribute lists memberOf so it is excluded from incremental updates, but nsDS5ReplicatedAttributeListTotal does not list memberOf so that it is included in total updates.
The exclusion list for incremental updates is set in the
nsDS5ReplicatedAttributeList attribute for the replication agreement. For example:
nsds5replicatedattributelist: (objectclass=*) $ EXCLUDE authorityRevocationList accountUnlockTime memberof
To set the
nsDS5ReplicatedAttributeList attribute, use the dsconf repl-agmt set command. For example:
# dsconf -D "cn=Directory Manager" ldap://supplier.example.com repl-agmt set \
--suffix="suffix" --frac-list="authorityRevocationList accountUnlockTime memberof" \
agreement_name
If
nsDS5ReplicatedAttributeList is the only attribute set, then that list applies to both incremental and total updates. To set a separate list for total updates, add the nsDS5ReplicatedAttributeListTotal attribute to the replication agreement:
# dsconf -D "cn=Directory Manager" ldap://supplier.example.com repl-agmt set \
--suffix="suffix" --frac-list-total="accountUnlockTime" \
agreement_nameNote
The
nsDS5ReplicatedAttributeList attribute must be set for incremental updates before nsDS5ReplicatedAttributeListTotal can be set for total updates.
15.11.2. The Replication Keep-alive Entry
When you update an attribute on a supplier, the changelog change sequence number (CSN) is increased on the supplier. In a replication topology, this server now connects to the first consumer and compares the local CSN with the CSN on the consumer. If it is lower, the update is retrieved from the local changelog and replicated to the consumer. In a replication topology with fractional replication enabled, this can cause problems: For example, if only attributes are updated on the supplier that are excluded from replication, no update to replicate is found, and therefore the CSN is not updated on the consumer. In certain scenarios, such as when only attributes are updated on a supplier that are excluded from replication, unnecessary searching for updates on the supplier can cause other servers to receive the data later than needed . To work around this problem, Directory Server uses keep-alive entries.
If all updated attributes on the supplier are excluded from replication and the number of skipped updates exceeds 100, the
keepalivetimestamp attribute is updated on the supplier and replicated to the consumer. Because the keepalivetimestamp attribute is not excluded from replication, the update of the keep-alive entry is replicated, the CSN on the consumer is updated, and then equal to the one on the supplier. The next time the supplier connects to the consumer, only updates that are newer than the CSN on the consumer are searched. This reduces the amount of time spent by a supplier to search for new updates to send.
Directory Server automatically creates the replication keep-alive entry on demand on a supplier. It contains the replica ID of the supplier in the distinguished name (DN). Each keep-alive entry is specific to a given supplier. For example, to display the hidden keep-alive entry:
# ldapsearch -D "cn=Directory Manager" -b "dc=example,dc=com" -W -p 389 -h server.example.com -x 'objectClass=ldapsubentry' dn: cn=repl keep alive 1,dc=example,dc=com objectclass: top objectclass: ldapsubentry objectclass: extensibleObject cn: repl keep alive 1 keepalivetimestamp: 20181112150654Z
The keep-alive entry is updated in the following situations (if it does not exist before the update, it is created first):
- When a fractional replication agreement skips more than 100 updates and does not send any updates before ending the replication session.
- When a supplier initializes a consumer, initially it creates its own keep-alive entry. A consumer that is also a supplier does not create its own keep-alive entry unless it also initializes another consumer.
15.11.3. Preventing "Empty" Updates from Fractional Replication
Fractional replication allows a list of attributes which are removed from replication updates (
nsDS5ReplicatedAttributeList). However, a changed to an excluded attribute still triggers a modify event and generates an empty replication update.
The
nsds5ReplicaStripAttrs attribute adds a list of attributes which cannot be sent in an empty replication event and are stripped from the update sequence. This logically includes operational attribtes like modifiersName.
For example, let's say that the
accountUnlockTime attribute is excluded. John Smith's user account is locked and then the time period expires and it is automatically unlocked. Only the accountUnlockTime attribute has changed, and that attribute is excluded from replication. However, the operational attribute internalmodifytimestamp also changed. A replication event is triggered because John Smith's user account was modified — but the only data to send is the new modify time stamp and the update is otherwise emtpy. If there are a large number of attributes related to login times or password expiration times (for example), this could create a flood of empty replication updates that negatively affect server performance or that interfere with associated applications.
To prevent this, add the
nsds5ReplicaStripAttrs attribute to the replication agreement to help tune the fractional replication behavior:
# dsconf -D "cn=Directory Manager" ldap://supplier.example.com repl-agmt set \
--suffix="suffix" \
--strip-list="modifiersname modifytimestamp internalmodifiersname" \
agreement_name
If a replication event is not empty, the stripped attributes are still replicated with the other changes. These attributes are removed from updates only if the event would otherwise be emtpy.
