Chapter 3. Setting an index scan limit to improve the performance when loading long lists of IDs
In large directories, the search results list can be huge. For example, a directory with one million entries with inetorgperson
attributes would return all these entries in a search with a filter, such as (objectclass=inetorgperson)
.
Loading a long ID list from the database significantly reduces search performance. An ID list scan limit sets a limit on the number of IDs Directory Server reads before a key is considered to match the entire primary index. This means that Directory Server treats the search as an unindexed search with a different set of resource limits.
For large indexes, it is actually more efficient to treat any search which matches the index as an unindexed search. The search operation only has to look in one place, the entire directory, to process results rather than searching through an index that is nearly the size of a directory, plus the directory itself.
You can set an index scan limit globally or for specific databases.
3.1. Setting a global index scan limit using the command line
By default, the ID list scan limit in Directory Server is 4000
. In most scenarios, this value provides good performance for a common range of database sizes and access patterns, and you do not need to change the default value. If the database index is slightly larger than 4000 entries, but still significantly smaller than the overall directory, raising the ID list scan limit improves searches.
On the other hand, lowering the limit can significantly speed up searches that would otherwise hit the 4000 entry limit, but where it is not necessary to scan every entry.
Procedure
Update the ID list scan limit:
# dsconf -D "cn=Directory Manager" ldap://server.example.com backend config set --idlistscanlimit=8000
This command sets the limit to
8000
entries.Restart the instance:
# dsctl instance_name restart
3.2. Setting a global index scan limit using the web console
By default, the ID list scan limit in Directory Server is 4000
. In most scenarios, this value provides good performance for a common range of database sizes and access patterns, and you do not need to change the default value. If the database index is slightly larger than 4000 entries, but still significantly smaller than the overall directory, raising the ID list scan limit improves searches.
On the other hand, lowering the limit can significantly speed up searches that would otherwise hit the 4000 entry limit, but where it is not necessary to scan every entry.
Procedure
-
Navigate to
. -
Update the
ID List Scan Limit
field. - Click .
-
Click
Restart Instance
. in the top right corner, and select
3.3. Setting an index scan limit to a database using the command line
In some cases, it is useful to define a limit for certain indexes, or to not use an ID list at all. You can configure individual settings for ID list scan limits for different types of search filters.
For example, in a large database with 10 million entries that contain the object class inetOrgPerson
, the (&(objectClass=inetOrgPerson)(uid=user))
filter creates first an ID list containing all 10 million IDs matching objectClass=inetOrgPerson
. When the database applies the second part of the filter, it searches the result list for objects matching uid=user
. In this case, it is useful to define a limit for certain indexes, or to not use an ID list at all.
This procedure demonstrates how to configure Directory Server to note create an ID list for objectClass=inetOrgPerson
conditions in AND
clauses.
Procedure
Set the
nsIndexIDListScanLimit
parameter:# ldapmodify -D "cn=Directory Manager" -W -H ldap://server.example.com -x dn: cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config changetype: modify replace: nsIndexIDListScanLimit nsIndexIDListScanLimit: limit=0 type=eq flags=AND values=inetOrgPerson
With these settings, Directory Server does not create any ID list for
objectClass=inetOrgPerson
conditions inAND
clauses. In all other situations, Directory Server applies the global ID list scan limit value.The
nsIndexIDListScanLimit
parameter uses the following syntax:nsIndexIDListScanLimit: limit=NNN [type=eq[,sub,...]] [flags=AND[,XXX,...]] [values=val[,val,...]]
limit
: Sets the maximum size of the ID list. Valid values are:-
-1
: Unlimited -
0
: Do not use the index -
1
to the maximum of the 32-bit integer (2147483647
): Maximum number of IDs
-
type
: Optional: Sets flags that alter the scan limit’s behavior. Valid values are:-
AND
: Apply the scan limit only to searches in which the attribute appears in anAND
clause. -
OR
: Apply the scan limit only to searches in which the attribute appears in anOR
clause.
-
values
: Optional: A comma-separated list of values which must match the search filter in order for the limit to be applied. Since the matches are done one at a time, the values will match if any of the values matches.Use the values only with one type at a time. The values must correspond to the index type and to the syntax of the attribute to which the index is applied. For example, if you specified the integer-based attribute
uidNumber
and it is indexed for theeq
type, you cannot usetype=eq values=abc
.If the value contains spaces, commas, NULL, or other values which require escaping, use the LDAP filter escape syntax: A backslash (\) followed by the 2 hex digit code of the character. In the following example, the commas in the DN value are escaped with
\2C
:nsIndexIDListScanLimit: limit=0 type=eq values=uid=user\2Cou=People\2Cdc=example\2Cdc=com