4.3. Setting Index Scan Limits

In large directories, the search results list can get huge. A directory with a million inetorgperson entries would have a million entries that were returned with a filter like (objectclass=inetorgperson), and an index for the sn attribute would have at least a million entries in it.

Loading a long ID list from the database significantly reduces search performance. The configuration parameter, nsslapd-idlistscanlimit, sets a limit on the number of IDs that are read before a key is considered to match the entire primary index (meaning the search is treated as an unindexed search with a different set of resource limits).

For large indexes, it is actually more efficient to treat any search which matches the index as an unindexed search. The search operation only has to look in one place to process results (the entire directory) rather than searching through an index that is nearly the size of a directory, plus the directory itself.

The default value of the nsslapd-idlistscanlimit attribute is 4000, which is gives good performance for a common range of database sizes and access patterns. It's usually not necessary to change this value. If the database index is slightly larger than the 4000 entries, but still significantly smaller than the overall directory, then raising the scan limit improves searches which would otherwise hit the default limit of 4000.

On the other hand, lowering the limit can significantly speed up searches that would otherwise hit the 4000 entry limit, but where it is not necessary to scan every entry.