stephan [Sat, 7 Mar 2026 06:10:29 +0000 (06:10 +0000)]
Extend the JS pre-bootstrapping configuration to include an option to disable inclusion of any given extension VFS (not the default VFSes, like memdb). The primary motivation for this is to give people who don't use OPFS to a way to keep the OPFS VFSes from loading their proxy workers, which are expensive.
stephan [Sat, 7 Mar 2026 04:19:08 +0000 (04:19 +0000)]
Fix the mangling of the opfs async proxy script's URI caused by both OPFS VFSes modifying it to accound for sqlite3.dir. Add a link to speedtest1 with opfw-wl to index.html.
stephan [Sat, 7 Mar 2026 03:32:17 +0000 (03:32 +0000)]
Cleanups and docs in the opfs vfs and its concurrency tester. Experimentally add sqlite3.capi.sqlite3_js_retry_busy(), which runs a callback repeatedly until it _stops_ returning (or throwing) an SQLITE_BUSY error.
stephan [Sat, 7 Mar 2026 01:01:13 +0000 (01:01 +0000)]
For backwards compatibility, ensure that the "opfs" VFS does not specifically require Atomics.waitAsync() (a new requirement of "opfs-wl"), but make use of it if available. Only apply jitter to the concurrency test runs at random intervals.
stephan [Fri, 6 Mar 2026 23:50:44 +0000 (23:50 +0000)]
Add some jitter and duration info to the opfs concurrency tester. Ensure that Atomics.waitAsync() is available for opfs-wl. Further testing has shown that Web opfs-wl is consistently fairer about doling out contested locks but it's dog slow compared to the original VFS under moderate concurrency. In single-connection use they're effectively on par.
stephan [Fri, 6 Mar 2026 16:04:21 +0000 (16:04 +0000)]
This Web Lock impl can reliably run a single OPFS connection but rather unreliably 'loses' workers with higher counts, presumably due to deadlock or deadly embrace (how _all_ of them can deadlock at once is unclear, but clearly a bug).
stephan [Thu, 5 Mar 2026 11:51:27 +0000 (11:51 +0000)]
Strip the opfs-wl "back to formula", removing the current false starts so that this can be tried again without tripping over any cruft. The current impl is subject, with no obvious way out of it, to starvation in the async proxy.
stephan [Thu, 5 Mar 2026 06:30:57 +0000 (06:30 +0000)]
The opfs-wl lock hang has been traced to starvation between the WebLock and the tight wait-on-VFS-calls Atomics.wait() loop. This can reportedly be resolved with another level of indirection in which the WebLock takes over the wait-on-VFS-calls part until it's unlocked, returning to the global loop when it's done. That exceeds this morning's ambitions but is next to try out.
stephan [Wed, 4 Mar 2026 21:16:36 +0000 (21:16 +0000)]
Add some debugging output to opfs and track down the breakage to the initial WebLock request, which is never reaching its callback. Still broken, but this is progress.
stephan [Wed, 4 Mar 2026 20:33:21 +0000 (20:33 +0000)]
Get opfs-wl plugged in to the concurrency tester. Somewhat ironically, all competing workers fail with locking errors while worker 1 is busy running off the rails somewhere. Stashing for closer investigation later.
stephan [Tue, 3 Mar 2026 23:43:40 +0000 (23:43 +0000)]
An initial attempt to plugging opfs-wl into the build but its initial handshake with the async half collides with the opfs VFS's handshake, causing bootstrapping to fail miserably. We'll need to either devise a handshake which can differentiate between the two instances or we'll need to preprocess sqlite3-opfs-async-proxy into two copies. Move the (now) three copies of some common code shared by the opfs pieces into a preprocessor #include.
dan [Mon, 2 Mar 2026 11:22:28 +0000 (11:22 +0000)]
Fix a possible 9-byte buffer overread in the zipfile extension hit when processing a corrupt zip file. Forum post [forum:/forumpost/721a05d2c5 | 721a05d2c5].
stephan [Fri, 27 Feb 2026 03:11:40 +0000 (03:11 +0000)]
An initial copy/paste stub for experimenting with using WebLocks for a new copy of the OPFS vfs. This is not yet functional, or even loading, but needs stashing to avoid potential loss.
dan [Thu, 26 Feb 2026 11:03:22 +0000 (11:03 +0000)]
Fix a problem where the wrong collation could be used as part of a row-value comparison between columns with different collation sequences. [forum:/forumpost/6ceca07fc3 | Forum post 6ceca07fc3].
drh [Wed, 25 Feb 2026 12:44:19 +0000 (12:44 +0000)]
Reinstant the work-around for [https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96270|GCC compiler bug 96270]
which is apparently still an issue as of gcc 13.3.0 with -m32 on Mint Linux.
The test case in the bug report is fixed, but the vdbeMemRenderNum() routine
in SQLite still shows the problem.
drh [Tue, 24 Feb 2026 19:37:16 +0000 (19:37 +0000)]
Rename SQLITE_ENABLE_MULTITHREADED_CHECKS to SQLITE_THREAD_MISUSE_WARNINGS.
Run test cases with that option. Also add the
SQLITE_THREAD_MISUSE_ABORT option.
drh [Mon, 23 Feb 2026 12:19:05 +0000 (12:19 +0000)]
Change the SQLAR archive extraction algorithm in the CLI so that it
uses the newly enhanced realpath() SQL function to guard against
attacks that use symlinks to try to write files outside of the
destination directory.
[forum:/forumpost/641b09daa17d9086|Forum post 641b09daa17d9086].
drh [Mon, 23 Feb 2026 01:34:14 +0000 (01:34 +0000)]
When doing an SQLAR archive extraction in the CLI, postpone creating symlinks until after
all files and directories have been created. This prevents a hostile archive from
creating a symlink through which it can subsequently write content outside of the target
directory. [forum:forumpost/9e176adfef91c207|Forum post 9e176adfef91c207].
drh [Sat, 21 Feb 2026 20:03:47 +0000 (20:03 +0000)]
Improved rounding of double→text conversions for rounding digits
of 17. Add the new sqlite3_db_config(), SQLITE_DBCONFIG_FP_DIGITS,
to specify the number of significant digits to preserve on conversions.
drh [Sat, 21 Feb 2026 19:26:58 +0000 (19:26 +0000)]
New sqlite3_db_config(SQLITE_DBCONFIG_FP_DIGITS) that let's the application
specify the number of significant digits that double→text conversions
will attempt to preserve.
drh [Sat, 21 Feb 2026 10:57:02 +0000 (10:57 +0000)]
Increase the precision of double → text conversions that happen in
sqlite3_column_text(), or sqlite3_value_text(), or in the CAST() operator,
or similar, so that round-tripping the value back to double results in
exactly the same value.
drh [Fri, 20 Feb 2026 16:21:58 +0000 (16:21 +0000)]
In the decimal extension, an optional second argument to the
decimal() and decimal_exp() functions rounds the value to the
number of significant digits specified by that argument.
drh [Mon, 16 Feb 2026 16:56:59 +0000 (16:56 +0000)]
Refactor the sqlite3AtoF() routine so that it requires a zero-terminated
UTF-8 input. When the need arises to convert UTF16 or non-terminated
strings, wrapper functions are used. Together, this makes the code slightly
smaller and faster.