]> git.ipfire.org Git - thirdparty/sqlite.git/log
thirdparty/sqlite.git
2 years agoTweaks to the Worker1 and Promiser APIs prompted by documenting them.
stephan [Fri, 30 Sep 2022 23:02:11 +0000 (23:02 +0000)] 
Tweaks to the Worker1 and Promiser APIs prompted by documenting them.

FossilOrigin-Name: c68b9aa160e2c1197ae7eb06a634017ac2b281393074afa4582762d5458c6889

2 years agoAdd JS wrapper for sqlite3_exec() which knows how to handle a JS callback. Add some...
stephan [Fri, 30 Sep 2022 20:35:37 +0000 (20:35 +0000)] 
Add JS wrapper for sqlite3_exec() which knows how to handle a JS callback. Add some console.error() reporting of module-load failures, as they otherwise often get silently swallowed up by the loader's mechanisms. Add 'flexible-string' JS-to-WASM argument converter which performs more X-to-string conversions than the 'string' arg converter does.

FossilOrigin-Name: 96818aa83f4ccc574f558231249ecbdd39763b4351cf4cf6d33f53774a3ee5e6

2 years agoAdd sqlite3.version object. Add more state to the Worker #1 config-get response,...
stephan [Fri, 30 Sep 2022 16:49:03 +0000 (16:49 +0000)] 
Add sqlite3.version object. Add more state to the Worker #1 config-get response, including sqlite3.version.

FossilOrigin-Name: 711f458d188a0dbe6612069c856ade29323ab426dfa4f80e7b82757ccc474cb8

2 years agoAdd oo1.JsStorageDb.clearStorage/storageSize() methods, copies of capi.sqlite3_web_kv...
stephan [Fri, 30 Sep 2022 15:46:08 +0000 (15:46 +0000)] 
Add oo1.JsStorageDb.clearStorage/storageSize() methods, copies of capi.sqlite3_web_kvvfs_clear/size().

FossilOrigin-Name: 1e09efe7fa15b8908f8b8353164a8361de778e27ea6c0b11c402bf4e1c56333d

2 years agobatch-runner.js: force WebSQL batches to not abort for a failed statement (necessary...
stephan [Fri, 30 Sep 2022 15:24:58 +0000 (15:24 +0000)] 
batch-runner.js: force WebSQL batches to not abort for a failed statement (necessary for apples-to-apples-ish benchmark comparisons).

FossilOrigin-Name: 60f0c5cb04d2baf8431e523434d8753e39377f4b2c6bad225c2e5487a4be419b

2 years agoExpose sqlite3_msize() to wasm.
stephan [Fri, 30 Sep 2022 12:10:36 +0000 (12:10 +0000)] 
Expose sqlite3_msize() to wasm.

FossilOrigin-Name: aa6ad34f3b693b554be58ee2895f79397162c5b8877c709653c8508fc625e8f4

2 years agoAdd oo1.JsStorageDb() as a convenience wrapper for oo1.DB(...,'kvvfs'). Minor doc...
stephan [Fri, 30 Sep 2022 11:01:44 +0000 (11:01 +0000)] 
Add oo1.JsStorageDb() as a convenience wrapper for oo1.DB(...,'kvvfs'). Minor doc cleanups.

FossilOrigin-Name: 8a7998709f859a562cf6829485cb9921f8823af0efabe003741348ab1169fb89

2 years agowasm: expose sqlite3_de/serialize(), sqlite3_malloc/free() and friends, noting that...
stephan [Fri, 30 Sep 2022 10:55:28 +0000 (10:55 +0000)] 
wasm: expose sqlite3_de/serialize(), sqlite3_malloc/free() and friends, noting that the former explicitly lies on use of the latter for memory management so is not generically safe for use in wasm.

FossilOrigin-Name: fbc0edb5d31aa0dea92460e853f15f08c642451a7878994116b530cf172325cc

2 years agoAdd JS infrastructure to ostensibly allow us to customize the wasm imports, which...
stephan [Thu, 29 Sep 2022 22:08:22 +0000 (22:08 +0000)] 
Add JS infrastructure to ostensibly allow us to customize the wasm imports, which will hypothetically allow us to eliminate the dependency on EM_JS(), but the corresponding Emscripten glue-level feature currently breaks fatally with WASMFS builds so it's disabled.

FossilOrigin-Name: 88d9253b0db5494bf1c9b6d24f22524eeec856b89e64a66ffb30d945f0df21d3

2 years agoEnsure that the wasm builds always, even when passing -Ox flags from the CLI, use...
stephan [Thu, 29 Sep 2022 17:35:25 +0000 (17:35 +0000)] 
Ensure that the wasm builds always, even when passing -Ox flags from the CLI, use -g3 to avoid problems caused by symbol minification with optimization levels -O2 and higher.

FossilOrigin-Name: 27d24d5d270e272af07d96cf07bdbe79ba07a9c69ba3c70beb76ebf4f90fc97a

2 years agoFinish eliminating explicit Emscripten module dependencies in test code and fiddle...
stephan [Thu, 29 Sep 2022 16:54:23 +0000 (16:54 +0000)] 
Finish eliminating explicit Emscripten module dependencies in test code and fiddle. The only remnant in public code is the Emscripten-generated module load/init interface.

FossilOrigin-Name: 7be78dd4efc410f13ff1ceda1fad82b309cc24de2d5106c8bd6b2adeaa13b106

2 years agoRework the Emscripten-emitted module loader/init function such that it passes on...
stephan [Thu, 29 Sep 2022 13:17:50 +0000 (13:17 +0000)] 
Rework the Emscripten-emitted module loader/init function such that it passes on the sqlite3 module, instead of the Emscripten module, to the first then() of sqlite3InitModule()'s returned Promise. This eliminates any need to mention the Emscripten module object in client-side code unless they want to configure it in advance for loading-status reports.

FossilOrigin-Name: 0dbaa0e2b5abf5c23e2039ec90a3055ebb3c063aaf4e556c42546defe6fbb86d

2 years agoWasm: expose sqlite3_exec() and use it to simplify the db-reset logic in batch-runner...
stephan [Wed, 28 Sep 2022 18:10:50 +0000 (18:10 +0000)] 
Wasm: expose sqlite3_exec() and use it to simplify the db-reset logic in batch-runner.js a bit.

FossilOrigin-Name: 2e2821f782511b9d2274a89a5a922582aba18c7e9dc7ce01080e713942a56d7d

2 years agoMore work on batch-runner.html/js to facilitate speed comparisons between various...
stephan [Wed, 28 Sep 2022 17:52:52 +0000 (17:52 +0000)] 
More work on batch-runner.html/js to facilitate speed comparisons between various VFSes and WebSQL.

FossilOrigin-Name: 3bd1bc240676e56da87fc49f3c67a1edc4fafcf2a2416298d19ae4f80b676a72

2 years agoCorrect duplicate copies of sqlite3-api.js being embedded in the wasmfs-based builds.
stephan [Wed, 28 Sep 2022 13:01:49 +0000 (13:01 +0000)] 
Correct duplicate copies of sqlite3-api.js being embedded in the wasmfs-based builds.

FossilOrigin-Name: bbfcfba260f39a9c91e82d87e06b1c2cb297c03498b4530aa3e7e01ef9916012

2 years agoWork around broken -Os wasm builds by adding the -g3 flag. Unrelated documentation...
stephan [Wed, 28 Sep 2022 07:53:47 +0000 (07:53 +0000)] 
Work around broken -Os wasm builds by adding the -g3 flag. Unrelated documentation tweaks.

FossilOrigin-Name: f5d6bf8616341037fa3e229edf820d19acef3e0a6207a652b2b143de0a493214

2 years agoMinor doc correction.
stephan [Tue, 27 Sep 2022 17:03:01 +0000 (17:03 +0000)] 
Minor doc correction.

FossilOrigin-Name: 093f6e4b36db294e8e47df2fc75a4bc4fc101e2b6ff0201c912ccd1dcf394479

2 years agoComment the -O2 wasm builds as being broken (mysteriously missing JS bits).
stephan [Tue, 27 Sep 2022 14:31:34 +0000 (14:31 +0000)] 
Comment the -O2 wasm builds as being broken (mysteriously missing JS bits).

FossilOrigin-Name: f61475ef8244fd1f6553eca9cc2317e5de2cc04443e24afb7b91ebd3fc1bd402

2 years agoWASM API renaming. Reworked JS API bootstrap's async post-init into a generic mechani...
stephan [Tue, 27 Sep 2022 13:40:12 +0000 (13:40 +0000)] 
WASM API renaming. Reworked JS API bootstrap's async post-init into a generic mechanism, no longer OPFS-specific.

FossilOrigin-Name: c42a8cb090cad1108dfd6be574202d744c59e053b505bc4c17252dc6b65d26bf

2 years agowasm/js: rename /persistent to /opfs to account for potential future persistent stora...
stephan [Tue, 27 Sep 2022 09:17:37 +0000 (09:17 +0000)] 
wasm/js: rename /persistent to /opfs to account for potential future persistent storage options. Minor flag-handling cleanups in the speedtest1 pages. Minor API tweaks in oo1.

FossilOrigin-Name: 4dc972a3656b2a9ec915bfb3f653136560c753ce4024c3f0d0d0c28f66db7a0a

2 years agoFiddle: replace db export routine with a C-side one which works for both Emscripten...
stephan [Mon, 26 Sep 2022 13:55:10 +0000 (13:55 +0000)] 
Fiddle: replace db export routine with a C-side one which works for both Emscripten FS-hosted and OPFS-hosted db files. Minor code-adjacent cleanups.

FossilOrigin-Name: 3579a8d6f1f6cd3cd8aad9949536870c5fe7bae8c1778f700dd85d763e266b94

2 years agoGet fiddle db export working for OPFS VFS. Add root dir handle to the main OPFS VFS...
stephan [Mon, 26 Sep 2022 11:38:58 +0000 (11:38 +0000)] 
Get fiddle db export working for OPFS VFS. Add root dir handle to the main OPFS VFS worker to enable creation of certain utility functions without delegating to the async worker. Add sqlite3.capi.sqlite3_wasm_rc_str() to map integer result codes back to their SQLITE_xxx counterparts. Minor doc touchups.

FossilOrigin-Name: 9b2244e1c8a40efe6547094a1b57acc8f2173145a8731abb0e36268ce0a8ef41

2 years agowasm: change StructBinder signature for sqlite3_file::pMethods from 'P' to 'p' to...
stephan [Mon, 26 Sep 2022 11:34:31 +0000 (11:34 +0000)] 
wasm: change StructBinder signature for sqlite3_file::pMethods from 'P' to 'p' to eliminate an unnecessary and inconsistent level of magic.

FossilOrigin-Name: 85f2e877e53899860af4dc6630044b471a9c7c82faba1f4e1e60ae991460b943

2 years agoFiddle: move, rather than copy, data buffers between the threads. Fix the case of...
stephan [Sat, 24 Sep 2022 11:32:00 +0000 (11:32 +0000)] 
Fiddle: move, rather than copy, data buffers between the threads. Fix the case of an uploaded db failing to install because its filename is the same as the opened db.

FossilOrigin-Name: bcec4f964a7b02f59be05286ff715bac654a32b19f05a743e402f4cdb207cab8

2 years agoResolve "already configured" warnings from shell's main() when starting it up in...
stephan [Sat, 24 Sep 2022 10:15:08 +0000 (10:15 +0000)] 
Resolve "already configured" warnings from shell's main() when starting it up in fiddle mode.

FossilOrigin-Name: 114ef3552af977b272a0baddeb9a2326484b60acfc75284e43c55530f86b413f

2 years agoReworked out the OPFS async proxy metrics are fetched so that they play more nicely...
stephan [Sat, 24 Sep 2022 10:12:19 +0000 (10:12 +0000)] 
Reworked out the OPFS async proxy metrics are fetched so that they play more nicely with the tight event-polling loop.

FossilOrigin-Name: ef503ced5c2ca842be9aea9ef13719a378ed3020e884032db09afee1b8eba0a1

2 years agoRefactoring towards getting fiddle to support OPFS as a first-class citizen. Certain...
stephan [Sat, 24 Sep 2022 07:36:45 +0000 (07:36 +0000)] 
Refactoring towards getting fiddle to support OPFS as a first-class citizen. Certain operations, e.g. import, export, and unlink, are not OPFS-aware.

FossilOrigin-Name: 1b923ed6438d7fef4508936e0c4bc026a368721698b1539961e3fb3140a185cb

2 years agoReformulate some JS to work around a buggy/broken code transformation in one of the...
stephan [Wed, 21 Sep 2022 20:24:12 +0000 (20:24 +0000)] 
Reformulate some JS to work around a buggy/broken code transformation in one of the Emscripten-driven code optimizers.

FossilOrigin-Name: e1249369d5ec1c582c280b1f578b35d53637fdf1cbd97c16d5ed95b136b83e56

2 years agoPut pieces in place for fiddle to support opfs, but more cleanup is required in the...
stephan [Wed, 21 Sep 2022 19:51:25 +0000 (19:51 +0000)] 
Put pieces in place for fiddle to support opfs, but more cleanup is required in the sqlite3.js/fiddle connection. bigIntEnabled now defaults to whether the Emscripten's module has bigint enabled. Add wasm-sensible defaults for several SQLITE_ENABLE/OMIT flags in sqlite3-wasm.c.

FossilOrigin-Name: 7c7fb7535e86b3960eea7f29ab7e6d5197c166b4ee64ad4a9bc0749f2869badc

2 years agoshell.c.in: when building in fiddle mode, define _POSIX_SOURCE (ifndef) so that emcc...
stephan [Wed, 21 Sep 2022 16:21:21 +0000 (16:21 +0000)] 
shell.c.in: when building in fiddle mode, define _POSIX_SOURCE (ifndef) so that emcc's string.h reveals strdup().

FossilOrigin-Name: fb85b269c43147f153977606dd8ede7b93744bf955e4c1a2b198907fd8a94620

2 years agoCorrect mistyped --shrink-memory flag in speedtest1-worker. Minor OPFS proxy cleanups.
stephan [Wed, 21 Sep 2022 14:02:47 +0000 (14:02 +0000)] 
Correct mistyped --shrink-memory flag in speedtest1-worker. Minor OPFS proxy cleanups.

FossilOrigin-Name: 86e2b55ec9483fa5add51a479c6509d73461f1ac6fca5d49e057b1c66f4314d2

2 years agoDoc cleanups and additions. Add a way for the OPFS async runner to propagate exceptio...
stephan [Wed, 21 Sep 2022 12:27:35 +0000 (12:27 +0000)] 
Doc cleanups and additions. Add a way for the OPFS async runner to propagate exception text to the calling thread.

FossilOrigin-Name: 5c5e80652825cf883e6c17809cb98f2bf17d5feac2d263f6f492479154730dab

2 years agoAdded some structure to the wasm demo link list.
stephan [Wed, 21 Sep 2022 12:25:40 +0000 (12:25 +0000)] 
Added some structure to the wasm demo link list.

FossilOrigin-Name: 777077c4c2249e1ec78390d4f65aaf281c1fbefcef4bcc7609199e995645ceb6

2 years agoAdd a link to fiddle.html in the wasm test app index.html.
stephan [Wed, 21 Sep 2022 08:41:40 +0000 (08:41 +0000)] 
Add a link to fiddle.html in the wasm test app index.html.

FossilOrigin-Name: 96c734c07acfbea153d9aaf293a28a2d49d78de19fec4fb90c4c08fb1fd0ddbd

2 years agoMove fiddle build rules into the wasm-centric build files. Add rule to push wasm...
stephan [Wed, 21 Sep 2022 08:39:03 +0000 (08:39 +0000)] 
Move fiddle build rules into the wasm-centric build files. Add rule to push wasm bits to the wasm test server.

FossilOrigin-Name: 113f8204dc4ac849d5632d3de1680b6e0da871e107ef484c8d7273799bee3d88

2 years agoCorrect a too-strict is-opfs-available check.
stephan [Tue, 20 Sep 2022 16:20:35 +0000 (16:20 +0000)] 
Correct a too-strict is-opfs-available check.

FossilOrigin-Name: 1b5f1b4a6c1457f98c258459e23e321fc59793de298fecb84031b87f02156cd5

2 years agoMerge kv-vfs-magic-names branch into fiddle-opfs branch and make some kvvfs-relevant...
stephan [Tue, 20 Sep 2022 16:10:39 +0000 (16:10 +0000)] 
Merge kv-vfs-magic-names branch into fiddle-opfs branch and make some kvvfs-relevant tweaks.

FossilOrigin-Name: e3d36dcdd37e59f17a07d3611d08744eb86f439fab82a648490dd608bcaa3185

2 years agoExport the sqlite3_uri_...() family of functions to wasm.
stephan [Tue, 20 Sep 2022 14:52:26 +0000 (14:52 +0000)] 
Export the sqlite3_uri_...() family of functions to wasm.

FossilOrigin-Name: 72bebc848fce5c3b4766017d016ccb2360de2bd0cb3e47e710c80dbcb6b8b707

2 years agoMerge opfs-proxy-atomics branch into the fiddle-opfs branch.
stephan [Tue, 20 Sep 2022 14:39:54 +0000 (14:39 +0000)] 
Merge opfs-proxy-atomics branch into the fiddle-opfs branch.

FossilOrigin-Name: 25a36920d44544547a84161681cd41e292b4a70df60ac3630791873a79237d98

2 years agoWhen compiled with SQLITE_OS_KV_OPTIONAL, the magic names ":localStorage:" kv-vfs-magic-names
drh [Tue, 20 Sep 2022 14:36:53 +0000 (14:36 +0000)] 
When compiled with SQLITE_OS_KV_OPTIONAL, the magic names ":localStorage:"
and ":sessionStorage:" are recognized and converted to use the kv-vfs.

FossilOrigin-Name: c5db9262d0388ccb0e84c6a4b4e2e786dd634f13874e4034ba7b175befa4ce90

2 years agoRemove an obsolete file. opfs-proxy-atomics
stephan [Tue, 20 Sep 2022 14:21:58 +0000 (14:21 +0000)] 
Remove an obsolete file.

FossilOrigin-Name: 1e43855498b4329f733bd6e0731841dc884e885de7d2d338d402d05d54309427

2 years agoSpeed up de/serialization of func args and return values in the OPFS VFS proxy.
stephan [Tue, 20 Sep 2022 13:25:39 +0000 (13:25 +0000)] 
Speed up de/serialization of func args and return values in the OPFS VFS proxy.

FossilOrigin-Name: 5bf235bbe035e4ace7a54851e190742528af6b4266328a1b8bbb9fb3dd7f2118

2 years agoOPFS proxy: remove one sanity-checking-only level of proxy function to shave off...
stephan [Tue, 20 Sep 2022 10:47:36 +0000 (10:47 +0000)] 
OPFS proxy: remove one sanity-checking-only level of proxy function to shave off a few microseconds.

FossilOrigin-Name: b534831f3efb8910a17e29956e3e87cc80055ea66e15dbef992b6a556ff042f8

2 years agoRemove extraneous result-code SAB slots. Doc additions.
stephan [Tue, 20 Sep 2022 10:11:52 +0000 (10:11 +0000)] 
Remove extraneous result-code SAB slots. Doc additions.

FossilOrigin-Name: 88de20712bbe3e11ea74af95edc34e9ab9f346f9aa0a30931e5a9e1f96ce57b0

2 years agoAn alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayB...
stephan [Tue, 20 Sep 2022 08:27:57 +0000 (08:27 +0000)] 
An alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayBuffer and Atomics, instead of worker messages, for communication (only the initial one-time handshake during initialization uses worker messages). It runs speedtest1 approx. 15-20% faster but still 20-ish% slower than WASMFS.

FossilOrigin-Name: a83ee3082d89439ea3ad5737e63e25bebb0f91895aca006ce5fecf5b93a2651a

2 years agoOPFS VFS: further internal refactoring towards experimenting with a new comms model.
stephan [Tue, 20 Sep 2022 03:31:02 +0000 (03:31 +0000)] 
OPFS VFS: further internal refactoring towards experimenting with a new comms model.

FossilOrigin-Name: 5ca412ced24b4e3af5f467e710a597ed440badf7b8335346aade11d3cad3d1a1

2 years agoOPFS VFS: moved i/o buffer from per-file to the VFS, and related refactoring, in...
stephan [Tue, 20 Sep 2022 01:28:47 +0000 (01:28 +0000)] 
OPFS VFS: moved i/o buffer from per-file to the VFS, and related refactoring, in prep for experimentation with a new inter-worker comms model.

FossilOrigin-Name: d4d63e4580ad8d497310608175308c03c517e051d7865cb66aa0b10356612d7d

2 years agoFurther metrics and buffer-copy optimizations in the OPFS proxy, but with little...
stephan [Mon, 19 Sep 2022 18:22:29 +0000 (18:22 +0000)] 
Further metrics and buffer-copy optimizations in the OPFS proxy, but with little effect.

FossilOrigin-Name: d1f1fe6f1c60640f7770dfb9245c459a09b8d24ec2ddf664dff77c810bd51f96

2 years agoCut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead...
stephan [Mon, 19 Sep 2022 17:09:09 +0000 (17:09 +0000)] 
Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th.

FossilOrigin-Name: fb7f287310d74a3e236125ae9c49b859f9263c29ae85161c1bcf9dd0778d8a51

2 years agoRemove an obsolete file. Update wasmfs build for today's changes. Fix the case that...
stephan [Mon, 19 Sep 2022 14:56:13 +0000 (14:56 +0000)] 
Remove an obsolete file. Update wasmfs build for today's changes. Fix the case that OPFS.xTruncate() is passed a BigInt instead of a Number.

FossilOrigin-Name: 976053925013bf1975f5f9222e28ba648af28e305bb6bdae600eb24d0e136bec

2 years agoAdd sqlite3_web_vfs_list() to JS API. Corrected OPFS VFS's registering itself as...
stephan [Mon, 19 Sep 2022 13:44:23 +0000 (13:44 +0000)] 
Add sqlite3_web_vfs_list() to JS API. Corrected OPFS VFS's registering itself as the default VFS. speedtest1-worker now uses the xDelete() of both the default VFS and OPFS, to avoid that it starts up with a persistent OPFS test db (the native app calls unlink(), but that unlink call operates on a different virtual filesystem than the OPFS VFS).

FossilOrigin-Name: 2ec7e09139a510b9fd29e4c97283b20740a00f369193c6fecbb734f187e81b48

2 years agoMerge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with...
stephan [Mon, 19 Sep 2022 13:16:35 +0000 (13:16 +0000)] 
Merge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with the new method of handling kvvfs. Adjust speedtest1 JS build to include sqlite3-api.js so that it can use kvvfs and opfs VFSes. Permit passing of the vfs as a URL parameter to certain demo/test apps. Milestone: speedtest-worker.html?vfs=opfs runs with the standalone OPFS impl.

FossilOrigin-Name: ec09f32f7ae2249aaf27388ad2062982afa8bbbb5f88d236d6d9068bf33ad93d

2 years agospeedtest1: add --vfs NAME flag and eliminate code duplication in argc size verificat...
stephan [Mon, 19 Sep 2022 11:47:38 +0000 (11:47 +0000)] 
speedtest1: add --vfs NAME flag and eliminate code duplication in argc size verification for all flags which take a value.

FossilOrigin-Name: 281d09867134e0a057cfadd9bfcbf0e21b8ac9737a278c41973d33a3101be7bc

2 years agoMinor doc improvements in demo-123.js.
stephan [Mon, 19 Sep 2022 10:13:01 +0000 (10:13 +0000)] 
Minor doc improvements in demo-123.js.

FossilOrigin-Name: 9370ce94780e974da03d5c74cf9a5983eed333e7b3bc54e605c5f30b3fa99287

2 years agoGeneric cleanups in the OPFS VFS proxies.
stephan [Mon, 19 Sep 2022 09:58:01 +0000 (09:58 +0000)] 
Generic cleanups in the OPFS VFS proxies.

FossilOrigin-Name: 7d5f6adc3b964413fc96ad8d2735312c3e58348024cabdd2099682cbf696eaf7

2 years agoCorrect OPFS VFS xRead() to copy the result buffer if the result code is SQLITE_IOERR...
stephan [Mon, 19 Sep 2022 09:25:25 +0000 (09:25 +0000)] 
Correct OPFS VFS xRead() to copy the result buffer if the result code is SQLITE_IOERR_SHORT_READ.

FossilOrigin-Name: 56668f9902c6e896b6c63621a444444c6f58ee20f88a5feae97f1699be35892d

2 years agoReplace OPFS VFS xSleep() impl with a more efficient one (no Worker round-trip needed).
stephan [Mon, 19 Sep 2022 05:19:04 +0000 (05:19 +0000)] 
Replace OPFS VFS xSleep() impl with a more efficient one (no Worker round-trip needed).

FossilOrigin-Name: b9773f164878b0a1b7c88cc7a6d1374ea95f64920065e8b2b178a1afffd82fe5

2 years agoRename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same...
stephan [Mon, 19 Sep 2022 03:57:31 +0000 (03:57 +0000)] 
Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes.

FossilOrigin-Name: 2e4a005bd35424caeaa99ace23162cf79e2ebdb159475ffad92b85dc864ad764

2 years agoDisable build of wasmfs-using components on aarch64 (ARM), as wasmfs won't build...
stephan [Mon, 19 Sep 2022 00:40:53 +0000 (00:40 +0000)] 
Disable build of wasmfs-using components on aarch64 (ARM), as wasmfs won't build there.

FossilOrigin-Name: 9a9eeebc2c27b734041089131b4952d7c0440df48ef32f355641aca61d4b30a0

2 years agoNumerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose...
stephan [Sun, 18 Sep 2022 17:32:35 +0000 (17:32 +0000)] 
Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities.

FossilOrigin-Name: 26e625d05d9820033b23536f18ad3ddc59ed712ad507d4b0c7fe88abd15d2be8

2 years agoMove the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init...
stephan [Sun, 18 Sep 2022 03:05:55 +0000 (03:05 +0000)] 
Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. Rename x-sync-async.* to test-opfs-vfs.* Milestone: first successful test of OPFS without WASMFS.

FossilOrigin-Name: b2abf60dbfa6648f671a3932cb65feb28d05a0d5b7f792351d14f9c13d9798c5

2 years agoMove the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init...
stephan [Sun, 18 Sep 2022 02:35:30 +0000 (02:35 +0000)] 
Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client.

FossilOrigin-Name: 1c660970d0f62bcfd6e698a72b050d99972a1e39f45a5ac24194a190f8f78ab3

2 years agoPlug OPFS methods in to their sqlite3_vfs/io_methods counterparts. Add URL args to...
stephan [Sun, 18 Sep 2022 00:16:12 +0000 (00:16 +0000)] 
Plug OPFS methods in to their sqlite3_vfs/io_methods counterparts. Add URL args to control debug output and running of sanity-checks in the OPFS init code.

FossilOrigin-Name: a0e93ed20b2463606a63b03ce8ca41ec1fb22886db5c5c898ace86ba24636f70

2 years agoCorrect OPFS xRead() and xWrite() impls and add a very basic sanity test for them.
stephan [Sat, 17 Sep 2022 23:47:20 +0000 (23:47 +0000)] 
Correct OPFS xRead() and xWrite() impls and add a very basic sanity test for them.

FossilOrigin-Name: cd06cc670029763955cf60ffcf944b36d41cb005b859d9b9fd0eea1b6741d0e9

2 years agoImplement OPFS xAccess(), albeit with more limited semantics than the VFS API calls...
stephan [Sat, 17 Sep 2022 23:29:27 +0000 (23:29 +0000)] 
Implement OPFS xAccess(), albeit with more limited semantics than the VFS API calls for. Add a way for OPFS xDelete() to optionally recursively remove empty dirs left over after deleting a file.

FossilOrigin-Name: c342b5d745f301104c59851c753287ebbbe95a69a56cb522d376d0f3e352c30f

2 years agoGeneric cleanups in the OPFS sync/async proxy.
stephan [Sat, 17 Sep 2022 21:13:26 +0000 (21:13 +0000)] 
Generic cleanups in the OPFS sync/async proxy.

FossilOrigin-Name: f36bddbe54c3acbfaa958042e4d24724f130bdca551401033f9bc63f3da73492

2 years agoAdd the remaining vfs/io_methods wrappers to the OPFS sync/async proxy, but most...
stephan [Sat, 17 Sep 2022 20:50:12 +0000 (20:50 +0000)] 
Add the remaining vfs/io_methods wrappers to the OPFS sync/async proxy, but most are not yet tested.

FossilOrigin-Name: 44db9132145b3072488ea91db53f6c06be74544beccad5fd07efd22c0f03dc04

2 years agoFix an uninitialized variable in the decoder kv-vfs.
drh [Sat, 17 Sep 2022 18:31:31 +0000 (18:31 +0000)] 
Fix an uninitialized variable in the decoder kv-vfs.

FossilOrigin-Name: 354726aa6c399053785f29104de15091629ce4bc275b9e2205cb3656a9e81cd7

2 years agoInclude the kv-vfs as an optional VFS on unix builds if the
drh [Sat, 17 Sep 2022 18:29:49 +0000 (18:29 +0000)] 
Include the kv-vfs as an optional VFS on unix builds if the
SQLITE_OS_KV_OPTIONAL compile-time option is present.

FossilOrigin-Name: 852812d1e2ec3c53ad7c6c64662b37d861fefcf1baeee3d58eba88bcb3f6d8df

2 years agoMerge the latest trunk enhancements into the kv-vfs branch.
drh [Sat, 17 Sep 2022 17:26:44 +0000 (17:26 +0000)] 
Merge the latest trunk enhancements into the kv-vfs branch.

FossilOrigin-Name: 37c7ca25a6e837e7f83b96a32e2b1a455730e48a48012c458ae2e734a3ccdd1f

2 years agoAdd initial bits of an experimental async-impl-via-synchronous-interface proxy intend...
stephan [Sat, 17 Sep 2022 15:08:22 +0000 (15:08 +0000)] 
Add initial bits of an experimental async-impl-via-synchronous-interface proxy intended to marshal OPFS via sqlite3_vfs API.

FossilOrigin-Name: 38da059b472415da52f57de7332fbeb8a91e3add1f4be3ff9c1924b52672f77c

2 years agoMerge kv-vfs branch into fiddle-opfs branch. Adjust speedtest1 --size flags to accoun...
stephan [Fri, 16 Sep 2022 20:16:50 +0000 (20:16 +0000)] 
Merge kv-vfs branch into fiddle-opfs branch. Adjust speedtest1 --size flags to account for new size limit.

FossilOrigin-Name: afb79050e635f3c698e51f06c346cbf23b096cfda7d0f1d8e68514ea0c25b7b7

2 years agoCorrections to the xRead method for databases in the os_kv.c VFS so that it
drh [Fri, 16 Sep 2022 15:19:04 +0000 (15:19 +0000)] 
Corrections to the xRead method for databases in the os_kv.c VFS so that it
correctly reads the header, and thus avoids unnecessary cache flushes.  This
changes also includes an optimization to the header read logic.

FossilOrigin-Name: ef54961ce69fddb4cfeeff0860288de2858a6f7a5aa396691e8e99933eb9af54

2 years agoWhen calculating kvvfs sessionStorage/localStorage size, multiply it by 2 to account...
stephan [Fri, 16 Sep 2022 12:49:45 +0000 (12:49 +0000)] 
When calculating kvvfs sessionStorage/localStorage size, multiply it by 2 to account for JS using 16-bit characters.

FossilOrigin-Name: 52d1b185b9f6cee1eb3dec436f47e0f52e4621a127abfad8c27f92fd78147889

2 years agoMerge kv-vfs branch into fiddle-opfs branch for [21915af560b1|synchronous=off fix...
stephan [Fri, 16 Sep 2022 11:45:09 +0000 (11:45 +0000)] 
Merge kv-vfs branch into fiddle-opfs branch for [21915af560b1|synchronous=off fix]. Remove some duplicate debug output in OPFS test code.

FossilOrigin-Name: 13899bb98c80525276d2484598b94e4206358f243f06d45c02700024f7e226fd

2 years agoFix os_kv.c so that it uses SQLITE_FCNTL_SYNC and hence
drh [Fri, 16 Sep 2022 11:37:01 +0000 (11:37 +0000)] 
Fix os_kv.c so that it uses SQLITE_FCNTL_SYNC and hence
works even with PRAGMA synchronous=OFF.

FossilOrigin-Name: 21915af560b111aeeaee751790356151a5f063c2fc703dd4b35b22dc393409fb

2 years agoAdd batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce...
stephan [Fri, 16 Sep 2022 02:30:49 +0000 (02:30 +0000)] 
Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it.

FossilOrigin-Name: d8df25920a047d5cf2093cc6233128c5e6057a9104d0c4397e643645bd646fe1

2 years agoRemove --nosync flag from speedtest1-kvvfs.html, as that seems to be the trigger...
stephan [Fri, 16 Sep 2022 02:28:03 +0000 (02:28 +0000)] 
Remove --nosync flag from speedtest1-kvvfs.html, as that seems to be the trigger for the 'no such table: z1' error. That app now warns if that flag is passed to it via URL args.

FossilOrigin-Name: ad0677e8abcc077636d1cf1d8485be4943506382581edf832e6b8a2021560040

2 years agoPull the src/os_kv.c part of [13839759f8f4] into the kv-vfs branch.
stephan [Fri, 16 Sep 2022 01:08:06 +0000 (01:08 +0000)] 
Pull the src/os_kv.c part of [13839759f8f4] into the kv-vfs branch.

FossilOrigin-Name: e334449912d5176e355d02024a07ed867741f71c9d10ce6744ca800414bf3eeb

2 years agoAdd return value checks to kvstorageWrite(), necessary for recognizing storage-full...
stephan [Fri, 16 Sep 2022 01:05:19 +0000 (01:05 +0000)] 
Add return value checks to kvstorageWrite(), necessary for recognizing storage-full errors in the JS binding. speedtest1-kvvfs.html runs successfully on localStorage with --size 1 to 6 and starts failing with storage shortage at --size 7.

FossilOrigin-Name: 13839759f8f45e4eb0aa6a5052801f3964d5b2dc2e427e5e91ee6692176381eb

2 years agoMore work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks...
stephan [Thu, 15 Sep 2022 06:42:41 +0000 (06:42 +0000)] 
More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html.

FossilOrigin-Name: 00ee49a3a2c148480f614e49a0ee5b35a255758c0a53693f0b464b31e7a4045b

2 years agoRemove some stale comments. Get scratchpad-opfs-worker2.js running again in prep...
stephan [Thu, 15 Sep 2022 03:16:49 +0000 (03:16 +0000)] 
Remove some stale comments. Get scratchpad-opfs-worker2.js running again in prep for reuse in experimenting with the OPFS all-synchronous API.

FossilOrigin-Name: 4cbbd370186f84e440f496e7f11c641d7dc1723acc46e4a31483392e0eb046e9

2 years agoSplit wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enablin...
stephan [Thu, 15 Sep 2022 03:09:00 +0000 (03:09 +0000)] 
Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread.

FossilOrigin-Name: 08476f3c218d45846e7496bdae0b06e2122466111fdf2aa2aabb1805b15ef982

2 years agoClean up some JS documentation.
stephan [Thu, 15 Sep 2022 02:27:48 +0000 (02:27 +0000)] 
Clean up some JS documentation.

FossilOrigin-Name: 925fdbfc6c1f9f06951346bc0d4fb81ffb5797d5f1123728931973f6cb1a0efa

2 years agoAdd/apply various kvvfs-specific utility APIs to the JS layer to assist in testing...
stephan [Tue, 13 Sep 2022 19:27:03 +0000 (19:27 +0000)] 
Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum.

FossilOrigin-Name: 0d78961870ee9f22f1ba16d423377d28dcc36e04b1e31ffd57f3e2fd51f8f0f2

2 years agoGet testing1.js working with a kvvfs build.
stephan [Mon, 12 Sep 2022 22:27:00 +0000 (22:27 +0000)] 
Get testing1.js working with a kvvfs build.

FossilOrigin-Name: 333a45725d1708e0fefa559c33ce1c7eeb425cdb04f594b1f2b48166c1478c79

2 years agoRemove an extraneous key-copy op in the EM_JS impl of kvstorageDelete().
stephan [Mon, 12 Sep 2022 20:21:34 +0000 (20:21 +0000)] 
Remove an extraneous key-copy op in the EM_JS impl of kvstorageDelete().

FossilOrigin-Name: 1c5aeee45564c14e7e2e7730f3f52106339ea3148fb5aa786fa767a35db46f51

2 years agoMerge kv-vfs branch into fiddle-opfs and add speedtest1-kvvfs.html test app (which...
stephan [Mon, 12 Sep 2022 20:18:28 +0000 (20:18 +0000)] 
Merge kv-vfs branch into fiddle-opfs and add speedtest1-kvvfs.html test app (which currently fails early on).

FossilOrigin-Name: 8b1608a9a37131121cb3a0cb1e8dbad5e39dc1fc2a341d056f09537f179b2e7a

2 years agoFix the <sys/time.h> include in os_kv.c.
drh [Mon, 12 Sep 2022 19:33:20 +0000 (19:33 +0000)] 
Fix the <sys/time.h> include in os_kv.c.

FossilOrigin-Name: 878cc93e779c3857adc711f212520f007256274fcd3f6f3d72c754a937fcd198

2 years agoAdd a gettimeofday()-based implementation of xCurrentTimeInt64() to os_kv.c.
drh [Mon, 12 Sep 2022 19:29:34 +0000 (19:29 +0000)] 
Add a gettimeofday()-based implementation of xCurrentTimeInt64() to os_kv.c.

FossilOrigin-Name: e393ed650ef124143f84e9d787fb996e308dd7af6b8f50df72a6f085b67bf9c3

2 years agoFix a debugging/testing edit in the previous check-in.
drh [Mon, 12 Sep 2022 18:10:41 +0000 (18:10 +0000)] 
Fix a debugging/testing edit in the previous check-in.

FossilOrigin-Name: 6fc8d34c0ae1f8277544be741f2f0835fad8e475d35bd24573224ccc1699b8bd

2 years agoAdd sqlite3.oo1.DB.clearKvvfsStorage(). Add controls to kvvfs1.js demo to reset and...
stephan [Mon, 12 Sep 2022 17:59:04 +0000 (17:59 +0000)] 
Add sqlite3.oo1.DB.clearKvvfsStorage(). Add controls to kvvfs1.js demo to reset and query the db without requiring the dev console.

FossilOrigin-Name: d845c6c22bd5d3fffc66e0566df346d690dd8bd1fc1688e312161b1a1edcfd79

2 years agoFaster text decoder for kv_os.c.
drh [Mon, 12 Sep 2022 17:44:35 +0000 (17:44 +0000)] 
Faster text decoder for kv_os.c.

FossilOrigin-Name: 3354a2edb762d70ccc31d4d25f81b70e644d76e01cb1e81d2e97b5414d809d30

2 years agoMerge kv-vfs branch into fiddle-opfs branch to add kvvfs-based wasm build and demo.
stephan [Mon, 12 Sep 2022 16:09:50 +0000 (16:09 +0000)] 
Merge kv-vfs branch into fiddle-opfs branch to add kvvfs-based wasm build and demo.

FossilOrigin-Name: a7d8b26acd3c1ae344369e4d70804c0cab45272c0983cfd32d616a0a7b28acb9

2 years agoNew run-time options to the speed-check.sh script.
drh [Mon, 12 Sep 2022 16:07:24 +0000 (16:07 +0000)] 
New run-time options to the speed-check.sh script.

FossilOrigin-Name: ff46bc9d2a844dde9dafa157114f4d9ee157205ca5c08ed9ee32f018de310e62

2 years agoFix uninitialized variable in rollback-journal processing in os_kv.c
drh [Mon, 12 Sep 2022 15:59:35 +0000 (15:59 +0000)] 
Fix uninitialized variable in rollback-journal processing in os_kv.c

FossilOrigin-Name: e49682c5eac91958f143e639c5656ca54560d14f5805d514bf4aa0c206e63844

2 years agoResolve missing kvvfs JS test: sqlite3_initialize() must be called by the client...
stephan [Mon, 12 Sep 2022 13:05:54 +0000 (13:05 +0000)] 
Resolve missing kvvfs JS test: sqlite3_initialize() must be called by the client in this build. Add basic sanity test of kvvfs using sessionStorage().

FossilOrigin-Name: 250a935aeb94d3fadec0d3fe22de85de4e658e2fdb3be3aa9a8bbc8f7b7d8414

2 years agoFix the amalgamation so that it builds with the standard unix backend again.
drh [Mon, 12 Sep 2022 12:41:07 +0000 (12:41 +0000)] 
Fix the amalgamation so that it builds with the standard unix backend again.

FossilOrigin-Name: 0adf158f4e5791339988185cc29347c75056b062e0db65b5999fdfa1f433c1ed

2 years agoInitial build of kvvfs in wasm. It loads but cannot find the VFS for as-yet-unknown...
stephan [Mon, 12 Sep 2022 12:39:28 +0000 (12:39 +0000)] 
Initial build of kvvfs in wasm. It loads but cannot find the VFS for as-yet-unknown reasons (sqlite3 shell works fine), and most APIs throw "null function or function signature mismatch" from deep within wasm, presumably as a side effect of the "missing" VFS.

FossilOrigin-Name: 1a2f24a0bdfc6eaae478916b8f4f9c6b63ead9964534fc2951fb4e995ffe61f1

2 years agoMinor cleanups and documentation in the wasm pieces.
stephan [Sun, 11 Sep 2022 16:59:40 +0000 (16:59 +0000)] 
Minor cleanups and documentation in the wasm pieces.

FossilOrigin-Name: 4e6ce329872eb733ba2f7f7879747c52761ae97790fd8ed169a25a79854cc3d9