searchidx: split shards at 100000 docs by default

Testing on a busy btrfs system with indexlevel=medium reveals
another ~15% speedup compared to the previous 450000 value since
shards are smaller and less prone to slowdown.  The smaller
splits should also work better with indexlevel=full (the
default) since full indexing with positions takes up the bulk of
the space.

diff --git a/Documentation/public-inbox-index.pod b/Documentation/public-inbox-index.pod
index 35ec8cb7bd5c6842bbb07a3f0bf5886917cd9883..9d6c6a2ffb6a9475d02aa1146caec2a7d1113d17 100644 (file)
--- a/Documentation/public-inbox-index.pod
+++ b/Documentation/public-inbox-index.pod
@@ -233,7 +233,7 @@ Implies and used for tuning C<--split-shards>.  The default is
 likely fine.  Tiny values may overflow system file/command-line
 limits while giant values negate the performance benefit.
 
-Default: 450000
+Default: 100000
 
 =item --multipass
 

diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm
index a9a0e505dffc60a92da0aa1f8190c0ca96a98e1b..13bf4544c5710bed52aeb05ab35cc20ab9856d20 100644 (file)
--- a/lib/PublicInbox/SearchIdx.pm
+++ b/lib/PublicInbox/SearchIdx.pm
@@ -41,7 +41,7 @@ our $DB_NO_SYNC = 0;
 our $DB_DANGEROUS = 0;
 our $CHECKPOINT_INTVL = 15; # seconds
 our $DEFRAG_NR = 100000; # document count
-our $SHARD_SPLIT_AT = 450000; # document count
+our $SHARD_SPLIT_AT = 100000; # document count
 our $BATCH_BYTES = $ENV{XAPIAN_FLUSH_THRESHOLD} ? 0x7fffffff :
        # assume a typical 64-bit system has 8x more RAM than a
        # typical 32-bit system: