From 5e2bbdf267f4ba263489fade9bd428604925a5f0 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Sun, 12 Oct 2025 23:44:56 +0000 Subject: [PATCH] doc: more updates ahead of 2.0.0 release Hopefully I didn't miss much, it's been a long while :x --- Documentation/RelNotes/v2.0.0.wip | 51 ++++++++++++++++++++++++- Documentation/public-inbox-extindex.pod | 2 + Documentation/public-inbox-index.pod | 10 +++++ 3 files changed, 61 insertions(+), 2 deletions(-) diff --git a/Documentation/RelNotes/v2.0.0.wip b/Documentation/RelNotes/v2.0.0.wip index 534cddf10..be7cdba94 100644 --- a/Documentation/RelNotes/v2.0.0.wip +++ b/Documentation/RelNotes/v2.0.0.wip @@ -26,6 +26,9 @@ grokmirror-compatible manifests. Internal process and object management data structures are vastly simplified throughout and error handling made more robust. +Pathologically slow Xapian queries no longer block non-Xapian +requests when using the new --xapian-helpers switch. + git SHA-256 support remains a work-in-progress for inboxes and extindex due to the need to interoperate with SHA-1 epochs. @@ -97,10 +100,16 @@ PublicInbox::WWW * linkify peer public-inbox addresses in To/Cc headers + * publicinbox.nameIsUrl allows admins to avoid specifying + a publicinbox.*.url entry for every single inbox + public-inbox-watch: * watching MH folders is now supported + * publicinbox.*.watch=false can prevent attempted writes due to + publicinboxwatch.watchspam + lei * use http.proxy / http..proxy from system-wide git-config if @@ -115,9 +124,15 @@ lei * fix `lei q -tt' on locally indexed messages (still broken for remotes: https://public-inbox.org/meta/20230226170931.M947721@dcvr/ ) + * `lei q --thread-id=$MSGID' is now supported to search for messages + within certain thread + * `lei import' now sets labels+keywords consistently on all already imported messages + * `lei import' is less noisy by default, with a new --noisy switch + to restore previous behavior. + * fix `lei up' on saved local queries which previously used -t/--threads * `lei convert' output to v2 public-inboxes is now idempotent @@ -138,6 +153,8 @@ solver (used by lei (rediff|blob), and PublicInbox::WWW) * improve ambiguous OID debug messages + * can use coderepo indices generated by public-inbox-cindex + portability * SIGWINCH is handled properly on less common architectures and OSes @@ -158,6 +175,15 @@ portability * inotify support no longer requires Linux::Inotify2 XS package for most architectures +public-inbox-learn: + + * support --keep-going/-k switch to continue after unwritable inboxes + +public-inbox-mda / public-inbox-learn / public-inbox-watch: + + * support publicinboxImport.dropUniqueUnsubscribe config to + remove unique identifiers in List-Unsubscribe headers + public-inbox-pop3d * support `limit=NUM' and `initial_limit=NUM' query parameters @@ -209,17 +235,38 @@ Search improvements (lei and PublicInbox::WWW) frontends where SQLite WAL isn't possible, use `--commit=SECONDS' + * --commit=SECONDS adjusts commit interval to increase or decrease + the rate at which SQLite and Xapian data is made available to + read-only processes + * --wal switch added to public-inbox-init(1), public-inbox-index(1), - public-inbox-extindex(1), public-inbox-convert(1) + public-inbox-extindex(1), public-inbox-convert(1) to enable WAL + for SQLite, which allows higher commit intervals to be used without + causing readers to timeout. * publicinbox.$NAME.altid is now supported with public-inbox-extindex * publicinbox.$NAME.indexheader added to support custom header indexing, see public-inbox-config(5) for details. + * --defrag=MSGCOUNT defragments SQLite and Xapian DBs on btrfs(5) + + * --split-shards speeds up initial Xapian indexing and reduces + fragmentation + + * --no-multi-pack-index disables writing git-multi-pack-index(1) to + avoid OOM errors on memory constrained systems + + * --cow / --no-cow may be used with public-inbox-init(1), + public-inbox-compact(1), public-inbox-convert(1), + public-inbox-cindex(1), public-inbox-xcpdb(1), + public-inbox-index(1), public-inbox-extindex(1) + to enable or disable (the default) No_COW attribute on + Linux CoW filesystems (e.g. btrfs). + public-inbox-daemon (-netd, -imapd, -httpd, -nntpd, -pop3d) - * per-listener servername=, serverport=, and multi-accept= + * per-listener backlog=, servername=, serverport=, and multi-accept= parameters, --multi-accept= is also supported globally for all listeners. diff --git a/Documentation/public-inbox-extindex.pod b/Documentation/public-inbox-extindex.pod index f083678c3..2755064bd 100644 --- a/Documentation/public-inbox-extindex.pod +++ b/Documentation/public-inbox-extindex.pod @@ -26,6 +26,8 @@ along with L and L Perl modules. =item --no-fsync +=item --defrag=MSGCOUNT + =item --dangerous =item --rethread diff --git a/Documentation/public-inbox-index.pod b/Documentation/public-inbox-index.pod index 81134301f..35ec8cb7b 100644 --- a/Documentation/public-inbox-index.pod +++ b/Documentation/public-inbox-index.pod @@ -150,6 +150,16 @@ data to accumulate, resulting on latency spikes from writeback. Available in public-inbox 1.6.0+. +=item --defrag=MSGCOUNT + +Defragment SQLite and Xapian files after C messages indexed. +Currently only affects Linux L, but other FSes may be supported +in the future. + +Default: 100000 messages + +Available in public-inbox 2.0.0+ + =item --dangerous Speed up initial index by using in-place updates and denying support for -- 2.47.3