init|*index|convert: --wal enables Write-Ahead Log in SQLite
SQLite WAL <https://sqlite.org/wal.html> ought to improve
concurrency for readers when writers are active. While users
always had the option of enabling it with the sqlite3(1) tool,
having a `--wal' switch may encourage its use.
It's already on by default for lei/store and mail_sync.sqlite3
where readers and writers are the same user, but enabling it by
default for public-facing daemons would cause permissions
problems so it remains optional.
The key downside of WAL is readers (-(netd|httpd|imapd|nntpd|pop3d)
require write access to the directories containing over.sqlite3
and msgmap.sqlite3). In the past, public-facing read-only
daemons were not expected to ever have write permissions to
inbox and extindex directories, but WAL requires writability
since SQLite||DBD::SQLite currently doesn't provide a way to
guarantee the persistence of -wal and -shm files, especially
when accessed by sqlite3(1) or 3rd-party scripts.
The main advantage of supporting WAL is so the writers won't
block readers, avoiding busy timeouts on the read side and
reducing the need for writers to commit transactions
periodically to keep readers happy (currently a ridiculously
short 5s). We can investigate higher commit intervals (or
eliminating them entirely) for users of WAL.
WAL should also reduce fragmentation in btrfs and similar CoW
filesystems by reducing fsync(2) calls, this remains to be
proven as measuring such changes takes a while and I don't
want to put more wear on my SSD.