ITS#9613, ITS#9614 - Document multival, idlexp, and maxentrysize options to slapd-mdb

multival, idlexp, and maxentrysize are all new options as of OpenLDAP 2.5 that were missing from the admin guide.  Add these options and additional information beyond what is found in the man pages as appropriate.

Also fixed some discrepencies between the slapd.conf and slapd-config admin guide sections on the Database backends as well as missing information in that section.

diff --git a/doc/guide/admin/slapdconf2.sdf b/doc/guide/admin/slapdconf2.sdf
index 282502441cd6641efd38f005311e07b27480641b..08b84219b901b5ad764e75f05ab1263f30b1f848 100644 (file)
--- a/doc/guide/admin/slapdconf2.sdf
+++ b/doc/guide/admin/slapdconf2.sdf
@@ -361,27 +361,31 @@ supported backend types listed in Table 5.2.
 !block table; align=Center; coltags="EX,N"; \
        title="Table 5.2: Database Backends"
 Types  Description
+asyncmeta      Asynchronous Metadirectory backend
 config Slapd configuration backend
 dnssrv DNS SRV backend
 ldap   Lightweight Directory Access Protocol (Proxy) backend
 ldif   Lightweight Data Interchange Format backend
 mdb    Memory-Mapped DB backend
-meta   Meta Directory backend
+meta   Metadirectory backend
 monitor        Monitor backend
+ndb    MySQL NDB backend
+null   Null backend
 passwd Provides read-only access to {{passwd}}(5)
 perl   Perl Programmable backend
-shell  Shell (extern program) backend
+relay  Relay backend
+sock   Socket backend
 sql    SQL Programmable backend
+wt     WiredTiger backend
 !endblock
 
 \Example:
 
 >      olcBackend: mdb
 
-There are no other directives defined for this entry.  Specific backend
-types may define additional attributes for their particular use but so
-far none have ever been defined.  As such, these directives usually do
-not appear in any actual configurations.
+This marks the beginning of a new {{TERM:MDB}} backend
+definition. At present, only back-mdb implements any options
+of this type, so this setting is not needed for any other backends.
 
 
 H4: Sample Entry
@@ -389,6 +393,7 @@ H4: Sample Entry
 > dn: olcBackend=mdb,cn=config
 > objectClass: olcBackendConfig
 > olcBackend: mdb
+> olcBkMdbIdlExp: 16
 
 
 H3: Database-specific Directives
@@ -724,6 +729,33 @@ H4: Sample Entries
 >olcRootDN: cn=Manager,dc=example,dc=com
 
 
+H3: MDB Backend Directives
+
+Directives in this category only apply to the {{TERM:MDB}}
+database backend. They will apply to all "database mdb"
+instances in the configuration. For a complete reference
+of MDB backend configuration directives, see {{slapd-mdb}}(5).
+
+H4: olcBkMdbIdlExp <exponent>
+
+Specify a power of 2 for the maximum size of an index slot.
+The default is 16, yielding a maximum slot size of 2^16 or 65536.
+The specified value must be in the range of 16-31.
+
+This setting helps with the case where certain search filters are
+slow to return results due to an index slot having collapsed to a
+range value.  This occurs when the number of candidate entries that
+match the filter for the index slot exceed the configured slot size.
+
+If this setting is decreased on a server with existing {{TERM:MDB}}
+databases, each db will immediately need its indicies to be rebuilt
+while slapd is offline with the "slapindex -q -t" command.
+
+If this setting is increased on a server with existing {{TERM:MDB}}
+databases, each db will need its indices rebuilt to take advantage
+of the change for indices that have already been converted to ranges.
+
+
 H3: MDB Database Directives
 
 Directives in this category apply to the {{TERM:MDB}}
@@ -849,6 +881,13 @@ stopped before the index task completes, indexing will have to be
 manually completed using the slapindex tool.
 
 
+H4: olcDbMaxEntrySize: <bytes>
+
+Specify the maximum size of an entry in bytes. Attempts to store
+an entry larger than this size will be rejected with the error
+LDAP_ADMINLIMIT_EXCEEDED. The default is 0, which is unlimited.
+
+
 H4: olcDbMaxReaders: <integer>
 
 This directive specifies the maximum number of threads that may have
@@ -880,6 +919,28 @@ created database index files should have. This can be in the form
 >      olcDbMode: 0600
 
 
+H4: olcDbMultival: { <attrlist> | default } <integer> hi,<integer> lo
+
+Specify the number of values for which a multivalued attribute is
+stored in a separate table. Normally entries are stored as a single
+blob inside the database. When an entry gets very large or contains
+attributes with a very large number of values, modifications on that
+entry may get very slow. Splitting the large attributes out to a separate
+table can improve the performance of modification operations.
+The threshold is specified as a pair of integers. If the number of
+values exceeds the hi threshold the values will be split out. If
+a modification deletes enough values to bring an attribute below
+the lo threshold the values will be removed from the separate
+table and merged back into the main entry blob.
+The threshold can be set for a specific list of attributes, or
+the default can be configured for all other attributes.
+The default value for both hi and lo thresholds is UINT_MAX, which keeps
+all attributes in the main blob.
+
+In addition to increasing write performance of operations the use of
+multival can also decrease fragmentation of the primary {{TERM:MDB}} database.
+
+
 H4: olcDbRtxnsize: <entries>
 
 This directive specifies the maximum number of entries to process in a single

diff --git a/doc/guide/admin/slapdconfig.sdf b/doc/guide/admin/slapdconfig.sdf
index dc96fd5ce8fab88f4638221cc637566fedc18481..52ade130338f7bbb6e7e3ce1cac2d9d8f663e38d 100644 (file)
--- a/doc/guide/admin/slapdconfig.sdf
+++ b/doc/guide/admin/slapdconfig.sdf
@@ -261,23 +261,32 @@ supported backend types listed in Table 6.2.
 !block table; align=Center; coltags="EX,N"; \
        title="Table 6.2: Database Backends"
 Types  Description
+asyncmeta      Asynchronous Metadirectory backend
+config Slapd configuration backend
 dnssrv DNS SRV backend
 ldap   Lightweight Directory Access Protocol (Proxy) backend
+ldif   Lightweight Data Interchange Format backend
 mdb    Memory-Mapped DB backend
-meta   Meta Directory backend
+meta   Metadirectory backend
 monitor        Monitor backend
+ndb    MySQL NDB backend
+null   Null backend
 passwd Provides read-only access to {{passwd}}(5)
 perl   Perl Programmable backend
-shell  Shell (extern program) backend
+relay  Relay backend
+sock   Socket backend
 sql    SQL Programmable backend
+wt     WiredTiger backend
 !endblock
 
 \Example:
 
 >      backend mdb
+>      idlexp 16
 
 This marks the beginning of a new {{TERM:MDB}} backend
-definition.
+definition. At present, only back-mdb implements any options
+of this type, so this setting is not needed for any other backends.
 
 
 H3: General Database Directives
@@ -575,6 +584,33 @@ If specified multiple times, each {{TERM:URL}} is provided.
 >      updateref       ldap://provider.example.net
 
 
+H3: MDB Backend Directives
+
+Directives in this category only apply to the {{TERM:MDB}}
+database backend. They will apply to all "database mdb"
+instances in the configuration. For a complete reference
+of MDB backend configuration directives, see {{slapd-mdb}}(5).
+
+H4: idlexp <exponent>
+
+Specify a power of 2 for the maximum size of an index slot.
+The default is 16, yielding a maximum slot size of 2^16 or 65536.
+The specified value must be in the range of 16-31.
+
+This setting helps with the case where certain search filters are
+slow to return results due to an index slot having collapsed to a
+range value.  This occurs when the number of candidate entries that
+match the filter for the index slot exceed the configured slot size.
+
+If this setting is decreased on a server with existing {{TERM:MDB}}
+databases, each db will immediately need its indicies to be rebuilt
+while slapd is offline with the "slapindex -q -t" command.
+
+If this setting is increased on a server with existing {{TERM:MDB}}
+databases, each db will need its indices rebuilt to take advantage
+of the change for indices that have already been converted to ranges.
+
+
 H3: MDB Database Directives
 
 Directives in this category only apply to the {{TERM:MDB}}
@@ -583,7 +619,6 @@ That is, they must follow a "database mdb" line
 and come before any subsequent "backend" or "database" lines.
 For a complete reference of MDB configuration directives, see {{slapd-mdb}}(5).
 
-
 H4: directory <directory>
 
 This directive specifies the directory where the MDB files
@@ -697,6 +732,13 @@ normal use of the directory. Most applications don't use presence
 searches, so usually presence indexing is not very useful.
 
 
+H4: maxentrysize: <bytes>
+
+Specify the maximum size of an entry in bytes. Attempts to store
+an entry larger than this size will be rejected with the error
+LDAP_ADMINLIMIT_EXCEEDED. The default is 0, which is unlimited.
+
+
 H4: maxreaders: <integer>
 
 This directive specifies the maximum number of threads that may have
@@ -728,6 +770,28 @@ created database index files should have. This can be in the form
 >   mode: 0600
 
 
+H4: multival: { <attrlist> | default } <integer> hi,<integer> lo
+
+Specify the number of values for which a multivalued attribute is
+stored in a separate table. Normally entries are stored as a single
+blob inside the database. When an entry gets very large or contains
+attributes with a very large number of values, modifications on that
+entry may get very slow. Splitting the large attributes out to a separate
+table can improve the performance of modification operations.
+The threshold is specified as a pair of integers. If the number of
+values exceeds the hi threshold the values will be split out. If
+a modification deletes enough values to bring an attribute below
+the lo threshold the values will be removed from the separate
+table and merged back into the main entry blob.
+The threshold can be set for a specific list of attributes, or
+the default can be configured for all other attributes.
+The default value for both hi and lo thresholds is UINT_MAX, which keeps
+all attributes in the main blob.
+
+In addition to increasing write performance of operations the use of
+multival can also decrease fragmentation of the primary {{TERM:MDB}} database.
+
+
 H4: rtxnsize: <entries>
 
 This directive specifies the maximum number of entries to process in a single