stephan [Fri, 30 Sep 2022 20:35:37 +0000 (20:35 +0000)]
Add JS wrapper for sqlite3_exec() which knows how to handle a JS callback. Add some console.error() reporting of module-load failures, as they otherwise often get silently swallowed up by the loader's mechanisms. Add 'flexible-string' JS-to-WASM argument converter which performs more X-to-string conversions than the 'string' arg converter does.
stephan [Fri, 30 Sep 2022 10:55:28 +0000 (10:55 +0000)]
wasm: expose sqlite3_de/serialize(), sqlite3_malloc/free() and friends, noting that the former explicitly lies on use of the latter for memory management so is not generically safe for use in wasm.
stephan [Thu, 29 Sep 2022 22:08:22 +0000 (22:08 +0000)]
Add JS infrastructure to ostensibly allow us to customize the wasm imports, which will hypothetically allow us to eliminate the dependency on EM_JS(), but the corresponding Emscripten glue-level feature currently breaks fatally with WASMFS builds so it's disabled.
stephan [Thu, 29 Sep 2022 17:35:25 +0000 (17:35 +0000)]
Ensure that the wasm builds always, even when passing -Ox flags from the CLI, use -g3 to avoid problems caused by symbol minification with optimization levels -O2 and higher.
stephan [Thu, 29 Sep 2022 16:54:23 +0000 (16:54 +0000)]
Finish eliminating explicit Emscripten module dependencies in test code and fiddle. The only remnant in public code is the Emscripten-generated module load/init interface.
stephan [Thu, 29 Sep 2022 13:17:50 +0000 (13:17 +0000)]
Rework the Emscripten-emitted module loader/init function such that it passes on the sqlite3 module, instead of the Emscripten module, to the first then() of sqlite3InitModule()'s returned Promise. This eliminates any need to mention the Emscripten module object in client-side code unless they want to configure it in advance for loading-status reports.
stephan [Tue, 27 Sep 2022 09:17:37 +0000 (09:17 +0000)]
wasm/js: rename /persistent to /opfs to account for potential future persistent storage options. Minor flag-handling cleanups in the speedtest1 pages. Minor API tweaks in oo1.
stephan [Mon, 26 Sep 2022 11:38:58 +0000 (11:38 +0000)]
Get fiddle db export working for OPFS VFS. Add root dir handle to the main OPFS VFS worker to enable creation of certain utility functions without delegating to the async worker. Add sqlite3.capi.sqlite3_wasm_rc_str() to map integer result codes back to their SQLITE_xxx counterparts. Minor doc touchups.
stephan [Sat, 24 Sep 2022 11:32:00 +0000 (11:32 +0000)]
Fiddle: move, rather than copy, data buffers between the threads. Fix the case of an uploaded db failing to install because its filename is the same as the opened db.
stephan [Wed, 21 Sep 2022 19:51:25 +0000 (19:51 +0000)]
Put pieces in place for fiddle to support opfs, but more cleanup is required in the sqlite3.js/fiddle connection. bigIntEnabled now defaults to whether the Emscripten's module has bigint enabled. Add wasm-sensible defaults for several SQLITE_ENABLE/OMIT flags in sqlite3-wasm.c.
stephan [Tue, 20 Sep 2022 08:27:57 +0000 (08:27 +0000)]
An alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayBuffer and Atomics, instead of worker messages, for communication (only the initial one-time handshake during initialization uses worker messages). It runs speedtest1 approx. 15-20% faster but still 20-ish% slower than WASMFS.
stephan [Mon, 19 Sep 2022 17:09:09 +0000 (17:09 +0000)]
Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th.
stephan [Mon, 19 Sep 2022 13:44:23 +0000 (13:44 +0000)]
Add sqlite3_web_vfs_list() to JS API. Corrected OPFS VFS's registering itself as the default VFS. speedtest1-worker now uses the xDelete() of both the default VFS and OPFS, to avoid that it starts up with a persistent OPFS test db (the native app calls unlink(), but that unlink call operates on a different virtual filesystem than the OPFS VFS).
stephan [Mon, 19 Sep 2022 13:16:35 +0000 (13:16 +0000)]
Merge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with the new method of handling kvvfs. Adjust speedtest1 JS build to include sqlite3-api.js so that it can use kvvfs and opfs VFSes. Permit passing of the vfs as a URL parameter to certain demo/test apps. Milestone: speedtest-worker.html?vfs=opfs runs with the standalone OPFS impl.
stephan [Sun, 18 Sep 2022 03:05:55 +0000 (03:05 +0000)]
Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. Rename x-sync-async.* to test-opfs-vfs.* Milestone: first successful test of OPFS without WASMFS.
stephan [Sun, 18 Sep 2022 02:35:30 +0000 (02:35 +0000)]
Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client.
stephan [Sun, 18 Sep 2022 00:16:12 +0000 (00:16 +0000)]
Plug OPFS methods in to their sqlite3_vfs/io_methods counterparts. Add URL args to control debug output and running of sanity-checks in the OPFS init code.
stephan [Sat, 17 Sep 2022 23:29:27 +0000 (23:29 +0000)]
Implement OPFS xAccess(), albeit with more limited semantics than the VFS API calls for. Add a way for OPFS xDelete() to optionally recursively remove empty dirs left over after deleting a file.
Corrections to the xRead method for databases in the os_kv.c VFS so that it
correctly reads the header, and thus avoids unnecessary cache flushes. This
changes also includes an optimization to the header read logic.
stephan [Fri, 16 Sep 2022 02:30:49 +0000 (02:30 +0000)]
Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it.
stephan [Fri, 16 Sep 2022 02:28:03 +0000 (02:28 +0000)]
Remove --nosync flag from speedtest1-kvvfs.html, as that seems to be the trigger for the 'no such table: z1' error. That app now warns if that flag is passed to it via URL args.
stephan [Fri, 16 Sep 2022 01:05:19 +0000 (01:05 +0000)]
Add return value checks to kvstorageWrite(), necessary for recognizing storage-full errors in the JS binding. speedtest1-kvvfs.html runs successfully on localStorage with --size 1 to 6 and starts failing with storage shortage at --size 7.
stephan [Thu, 15 Sep 2022 06:42:41 +0000 (06:42 +0000)]
More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html.
stephan [Tue, 13 Sep 2022 19:27:03 +0000 (19:27 +0000)]
Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum.
stephan [Mon, 12 Sep 2022 12:39:28 +0000 (12:39 +0000)]
Initial build of kvvfs in wasm. It loads but cannot find the VFS for as-yet-unknown reasons (sqlite3 shell works fine), and most APIs throw "null function or function signature mismatch" from deep within wasm, presumably as a side effect of the "missing" VFS.