]> git.ipfire.org Git - thirdparty/sqlite.git/log
thirdparty/sqlite.git
2 years agoAdd optional feature: A CLI continuation prompt which reflects open lexemes and paren... dynamic_prompt
larrybr [Tue, 6 Dec 2022 05:09:51 +0000 (05:09 +0000)] 
Add optional feature: A CLI continuation prompt which reflects open lexemes and parens, similarly to PG shell.

FossilOrigin-Name: dac2ddc287db7a68d0cd49b785060f62290868fbb1aa2ee09e54d3b1acfbf55f

2 years agoAdd test case that should have been part of previous commit.
dan [Mon, 5 Dec 2022 14:20:54 +0000 (14:20 +0000)] 
Add test case that should have been part of previous commit.

FossilOrigin-Name: dc7dd2d3e50e7cc474b22f1b5b219da32bcd7aa1ba56864d1dbcf0d3a6fa06f2

2 years agoFix a problem in the memdb vfs xLock() function allowing clients to upgrade to EXCLUS...
dan [Mon, 5 Dec 2022 14:12:14 +0000 (14:12 +0000)] 
Fix a problem in the memdb vfs xLock() function allowing clients to upgrade to EXCLUSIVE locks when other connections are holding SHARED.

FossilOrigin-Name: 15f0be8a640e7bfa4130edd4650a745337bd96083b119a1553f9abf9ff066806

2 years agoRemove SQLITE_EXPERIMENTAL tag from sqlite3_vtab_collation() and fix a related doc...
stephan [Mon, 5 Dec 2022 13:07:06 +0000 (13:07 +0000)] 
Remove SQLITE_EXPERIMENTAL tag from sqlite3_vtab_collation() and fix a related doc typo.

FossilOrigin-Name: 35d670b3593a46e13ded541ef477fa35dac6bcb9c31e6cf4b90bd7fa550a8ee2

2 years agoRemove obsolete "experimental" comments on the virtual-table interface
drh [Mon, 5 Dec 2022 10:50:06 +0000 (10:50 +0000)] 
Remove obsolete "experimental" comments on the virtual-table interface
documentation in sqlite.h.in.

FossilOrigin-Name: 28fc32abdcb1c6af19f0926a3a7317e15930a0ba89645d5ffbf6ce99d8463273

2 years agoImproved query planner cost estimates. Fix for ticket [e8b674241947eb3b].
drh [Mon, 5 Dec 2022 02:52:37 +0000 (02:52 +0000)] 
Improved query planner cost estimates.  Fix for ticket [e8b674241947eb3b].

FossilOrigin-Name: 1b779afa3ed2f35a110e460fc6ed13cba744db85b9924149ab028b100d1e1e12

2 years agoUse the smaller estimated row size for searching IPK tables, but use the qp-tuning
drh [Mon, 5 Dec 2022 02:42:30 +0000 (02:42 +0000)] 
Use the smaller estimated row size for searching IPK tables, but use the
original larger row size estimate for scanning, since the leaves can have large
rows.

FossilOrigin-Name: df3818997b822743ac407dde45c5fd75845ca40f461e31350d86963dffec6cd6

2 years agoFix safe mode authorizer callback to reject disallowed UDFs. Reported at [forum:...
larrybr [Sun, 4 Dec 2022 23:20:38 +0000 (23:20 +0000)] 
Fix safe mode authorizer callback to reject disallowed UDFs. Reported at [forum:/forumpost/07beac8056151b2f|Forum post 07beac8056151b2f].

FossilOrigin-Name: cefc032473ac5ad244c0b6402c541b2f76c0c65a041bda03bfbe7c0e2c11fac2

2 years agoExport sqlite3_result_zeroblob/zeroblob64() to wasm.
stephan [Sun, 4 Dec 2022 08:16:33 +0000 (08:16 +0000)] 
Export sqlite3_result_zeroblob/zeroblob64() to wasm.

FossilOrigin-Name: a60e56627fc0ef8831429941d429ee02c6ee51ce5a2c1af581dc5bc5a00d911e

2 years agoIncrease the nominal row size for IPK index lookups slightly, for better
drh [Sat, 3 Dec 2022 19:04:09 +0000 (19:04 +0000)] 
Increase the nominal row size for IPK index lookups slightly, for better
balance.

FossilOrigin-Name: 1a61c500add4a2bfe80c0c691d559cfca166dc5f8262651a58da7ec16a51d430

2 years agoAdd TOTAL rows to the output generated by tool/vdbe_profile.tcl.
drh [Sat, 3 Dec 2022 18:21:30 +0000 (18:21 +0000)] 
Add TOTAL rows to the output generated by tool/vdbe_profile.tcl.

FossilOrigin-Name: 9800586393c9d3b82459ef657620d245a7985ef5fa389b8a9ea633d6a29c7299

2 years agoAdd a test case to show that ticket [e8b674241947eb3b] has been fixed.
drh [Sat, 3 Dec 2022 17:23:29 +0000 (17:23 +0000)] 
Add a test case to show that ticket [e8b674241947eb3b] has been fixed.

FossilOrigin-Name: e20de6d450c2b4d5bde737f625de16ff53262c22ce7aa6917b64f1665170d33f

2 years agoFurther improvements to the estimated cost of sorting. Take into account
drh [Sat, 3 Dec 2022 17:09:15 +0000 (17:09 +0000)] 
Further improvements to the estimated cost of sorting.  Take into account
the number of columns to be sorted.

FossilOrigin-Name: f3290cf83b7c02d17d85d8942954f052b486c370cd5ec732969da9061dc1d19a

2 years agoCorrect a memory leak in tester1.js.
stephan [Sat, 3 Dec 2022 15:41:29 +0000 (15:41 +0000)] 
Correct a memory leak in tester1.js.

FossilOrigin-Name: e42b052dd754e577b8429e2430821c5f88096170ffa58b4e5e3cb7ee19b6c37d

2 years agosqlite3.wasm.allocFromTypedArray() now optionally accepts an ArrayBuffer as its argument.
stephan [Sat, 3 Dec 2022 14:58:45 +0000 (14:58 +0000)] 
sqlite3.wasm.allocFromTypedArray() now optionally accepts an ArrayBuffer as its argument.

FossilOrigin-Name: 75a1a796f86d289c7275666fab19013934775dcccaed44a1a61d1749a6cb99c9

2 years agoRename wasm.xWrap.resultAdapter() X:free entries to X:dealloc for consistency with...
stephan [Sat, 3 Dec 2022 13:10:58 +0000 (13:10 +0000)] 
Rename wasm.xWrap.resultAdapter() X:free entries to X:dealloc for consistency with wasm.dealloc(). Add an undocumented feature to replace wasm.alloc/dealloc/realloc() with the C-standard allocators (after an allocator misuse led down a several-hour rabbit hole trying to discover a mis-free() violation). Related test updates.

FossilOrigin-Name: d9807656f8a7c2a893d3f68ee5592f44826b8e999ae66f7d9000674b5c1b0207

2 years agoDoc typo fix for SQLITE_MAX_ALLOCATION_SIZE in malloc.c. No code changes.
stephan [Sat, 3 Dec 2022 13:05:33 +0000 (13:05 +0000)] 
Doc typo fix for SQLITE_MAX_ALLOCATION_SIZE in malloc.c. No code changes.

FossilOrigin-Name: ed1ed21221b048ac5a5275cdfc4d9b2a406acdc7d4b648c3b61bcc822d88d955

2 years agoMove kvvfs xRead()/xWrite() buffer from function-local stack memory to heap memory...
stephan [Sat, 3 Dec 2022 11:51:29 +0000 (11:51 +0000)] 
Move kvvfs xRead()/xWrite() buffer from function-local stack memory to heap memory to accommodate changes in emsdk 3.1.27.

FossilOrigin-Name: f1da32410ca7b808b3bef5f5a59766e7281e9e6ea343c8b979599bf1fc1060f5

2 years agoJavaScript: add sqlite3.wasm.realloc(), sqlite3.capi.SQLITE_MAX_ALLOCATION_SIZE,...
stephan [Sat, 3 Dec 2022 11:16:55 +0000 (11:16 +0000)] 
JavaScript: add sqlite3.wasm.realloc(), sqlite3.capi.SQLITE_MAX_ALLOCATION_SIZE, and related tests.

FossilOrigin-Name: eeb84ba5de1152ef0f42105b8b285fdee9f5ad58281e60a4e0c8b1d6de1dead8

2 years agowasm build: rename the path to the wasm docs checkout, for clarity and consistency.
stephan [Sat, 3 Dec 2022 03:06:16 +0000 (03:06 +0000)] 
wasm build: rename the path to the wasm docs checkout, for clarity and consistency.

FossilOrigin-Name: b820db32365b2ca8e2397fd6ea85883e4555ffd82948e248a0f98415b7328349

2 years agowasm snapshot and doc-related build automation tweaks.
stephan [Sat, 3 Dec 2022 02:42:21 +0000 (02:42 +0000)] 
wasm snapshot and doc-related build automation tweaks.

FossilOrigin-Name: 13eb1abd06f55fb88fc7f7be6149fd94b12057d9c38cc6b97bec3940e7e01f04

2 years agoRework the oo1.DB's distinct-per-VFS post-open() step to accept either a batch of...
stephan [Sat, 3 Dec 2022 01:59:03 +0000 (01:59 +0000)] 
Rework the oo1.DB's distinct-per-VFS post-open() step to accept either a batch of SQL or a callback function. Increase OPFS's busy timeout to 10s.

FossilOrigin-Name: 9feefe253ac487cb52be6bdf91bdd305963266716baa08f2bf9505954ee76321

2 years agoTuning the query planner by adjusting the weights that predict the relative
drh [Sat, 3 Dec 2022 00:52:21 +0000 (00:52 +0000)] 
Tuning the query planner by adjusting the weights that predict the relative
performance of sorting and index lookup.

FossilOrigin-Name: 9f2806da4d88beceac2e81e05421f00481dd3dd100b096cd2ae6c828adb42ca7

2 years agoMinor internal tweaks to the OPFS VFS. Resolve a missing result code which lead to...
stephan [Fri, 2 Dec 2022 18:56:37 +0000 (18:56 +0000)] 
Minor internal tweaks to the OPFS VFS. Resolve a missing result code which lead to a null deref in xFileSize().

FossilOrigin-Name: 57dd593ef0efa17dfb3a9f4eac36d5b8b879e271de817d8cd94a8c8b56d31870

2 years agoOPFS VFS: translate createSyncAccessHandle() exceptions which appear to be locking...
stephan [Fri, 2 Dec 2022 18:06:26 +0000 (18:06 +0000)] 
OPFS VFS: translate createSyncAccessHandle() exceptions which appear to be locking violations to SQLITE_BUSY. This seems to improve concurrency considerably even with a reduced retry count of 5 (was 6).

FossilOrigin-Name: 0d36021d107d3afca190ad61c3380536ad0cc2d493d345d48f9f9c1191741128

2 years agoFor the sqlite3_bind and sqlite3_result interfaces for UTF16 strings, round
drh [Fri, 2 Dec 2022 17:52:52 +0000 (17:52 +0000)] 
For the sqlite3_bind and sqlite3_result interfaces for UTF16 strings, round
the number of bytes down to the next even number, to avoid creating a UTF16
string that is an odd number of bytes.
[forum:/forumpost/411199488d065f83|Forum post 411199488d065f83].

FossilOrigin-Name: b57e3c3db00a6bc6db20c82530479f9eba7e37b731f0da6fe81682e84c7ac916

2 years agoUse sqlite3_result_int64() instead of sqlite3_result_int() when returning
drh [Fri, 2 Dec 2022 15:31:47 +0000 (15:31 +0000)] 
Use sqlite3_result_int64() instead of sqlite3_result_int() when returning
potentially large values from the DBSTAT virtual table, to avoid integer
overflows in the result.
[forum:/forumpost/ada2ab044f|Forum post ada2ab044f].

FossilOrigin-Name: 5652154a8c93cf3b1ff6c2e55e94abbe995b0bb625f733461df20e006c2f13f8

2 years agoMinor touchups to the JS test index page and test server push rules.
stephan [Fri, 2 Dec 2022 11:35:21 +0000 (11:35 +0000)] 
Minor touchups to the JS test index page and test server push rules.

FossilOrigin-Name: 0881f3e92364b3dc81443220451e853c0763efaf97653a6348bc39bd9bdb23ad

2 years agowasm builds: explicitly set a default stack size because emsdk 3.1.27 reduced it...
stephan [Fri, 2 Dec 2022 10:43:14 +0000 (10:43 +0000)] 
wasm builds: explicitly set a default stack size because emsdk 3.1.27 reduced it from 4MB to only 64kb, leading to memory corruption when kvvfs is used (it requires at least twice that for I/O).

FossilOrigin-Name: 758112460fb624198a144ed7d8d2324497fa72a2eee80010bd22b621e064b389

2 years agowasm: after building snapshot zip file, emit instructions for pushing it to the test...
stephan [Fri, 2 Dec 2022 09:23:38 +0000 (09:23 +0000)] 
wasm: after building snapshot zip file, emit instructions for pushing it to the test server.

FossilOrigin-Name: 9615c77919fee60d708d72729c741b373a89900aabc318e0ceb4154638cdf339

2 years agoCorrect the problem which triggered the rollback in [7eec635562f6]: an incorrect...
stephan [Fri, 2 Dec 2022 08:51:22 +0000 (08:51 +0000)] 
Correct the problem which triggered the rollback in [7eec635562f6]: an incorrect default db page size (not a multiple of 512 bytes).

FossilOrigin-Name: e06e490c240aa56b616e6f0380b5d08abf06f35f9e683e5150c5ac464eae9e55

2 years agoRoll back the SQLITE_DEFAULT_PAGE_SIZE part of [c260895faacb34] because kvvfs does...
stephan [Fri, 2 Dec 2022 08:38:04 +0000 (08:38 +0000)] 
Roll back the SQLITE_DEFAULT_PAGE_SIZE part of [c260895faacb34] because kvvfs does not work at all with a page size of 8kb.

FossilOrigin-Name: 7eec635562f65592121d7ff0a31a7b4ff6cdfe38e657f326302ef851b3b37c3f

2 years agosqlite3-wasm.c: code legibility and coding style tweaks. Increase SQLITE_DEFAULT_PAGE...
stephan [Fri, 2 Dec 2022 08:29:03 +0000 (08:29 +0000)] 
sqlite3-wasm.c: code legibility and coding style tweaks. Increase SQLITE_DEFAULT_PAGE_SIZE from 4k to 8k, as that improves OPFS speedtest1 performance by roughly 12%.

FossilOrigin-Name: c260895faacb3458c557778630756d02a8520c0f1864bddcf86cdd27ef4a42bd

2 years agoExpand JS tests for db export/import and document reason it cannot currently work...
stephan [Fri, 2 Dec 2022 07:14:56 +0000 (07:14 +0000)] 
Expand JS tests for db export/import and document reason it cannot currently work with kvvfs. Fix a minor JS build dependencies bug. Update page title with PASS/FAIL prefix for tester1.js to improve overview when launching multiple test tabs. Add ability of tester1 should-run-test predicates to report why a given test is disabled.

FossilOrigin-Name: 75f610d3a4cf3d972220f9abc27cdf5990451e3835ceb9cf66973934004dfc5c

2 years agoRemove extraneous/unused sqlite3.oo1.version object. Add httpd makefile target.
stephan [Fri, 2 Dec 2022 03:37:49 +0000 (03:37 +0000)] 
Remove extraneous/unused sqlite3.oo1.version object. Add httpd makefile target.

FossilOrigin-Name: 8e4d30ac033a6d9019a7eeedfe788dc0120f565cef2ae8f09d2bf32eb94d8a33

2 years agosqlite3_js_create_file() now accepts an ArrayBuffer data source. Add test for OPFS...
stephan [Thu, 1 Dec 2022 15:22:03 +0000 (15:22 +0000)] 
sqlite3_js_create_file() now accepts an ArrayBuffer data source. Add test for OPFS-based export/re-import. The (sqlite3*) argument converter now optionally accepts sqlite3.oo1.DB instances.

FossilOrigin-Name: 14a84b67fb17e16a5691ea4bf7f374123ac73a361a5d3d0efca53788d2001e3a

2 years agoReformulate and simplify some JS tests related to the previous checkin.
stephan [Thu, 1 Dec 2022 04:45:51 +0000 (04:45 +0000)] 
Reformulate and simplify some JS tests related to the previous checkin.

FossilOrigin-Name: 9ea2d3dcf798393a7fd231e199c0e2c6302949fe2a7f2573178fb0e50c78a2f4

2 years agoExpand "sqlite3_vfs*" JS-to-WASM function argument conversions to accept VFS names...
stephan [Thu, 1 Dec 2022 03:55:28 +0000 (03:55 +0000)] 
Expand "sqlite3_vfs*" JS-to-WASM function argument conversions to accept VFS names (JS strings) and capi.sqlite3_vfs instances. Implement sqlite3_js_vfs_create_file() to facilitate creation of file-upload features which store the file in VFS-specific storage (where possible, e.g. "unix" and "opfs" VFSes). Correct an argument type check in the SQLite3Error and WasmAllocError constructors.

FossilOrigin-Name: e1009b16d351b23676ad7bffab0c91b373a92132eb855c9af61991b50cd237ed

2 years agoAdd a testcase() macro to verify that the case of a NOT NULL error message
drh [Wed, 30 Nov 2022 21:18:23 +0000 (21:18 +0000)] 
Add a testcase() macro to verify that the case of a NOT NULL error message
hitting the string length limit.

FossilOrigin-Name: 91f50964c10fb12d889bda7d597d8edf475d97d2d8b534b4400e0fed1d753c6a

2 years agoFix an over-zealous assert() reported by Yong Heng.
drh [Wed, 30 Nov 2022 20:22:31 +0000 (20:22 +0000)] 
Fix an over-zealous assert() reported by Yong Heng.

FossilOrigin-Name: 6ee61f8cede4998f0c838d6506b058c6b09f34b3d7f30ed296100785c93f8d00

2 years agoAttempt to rationalize the bits associated with ".wheretrace". Provide
drh [Wed, 30 Nov 2022 19:11:31 +0000 (19:11 +0000)] 
Attempt to rationalize the bits associated with ".wheretrace".  Provide
a decoder key in sqliteInt.h for what each bit is intended to do.

FossilOrigin-Name: 8ec361695a107a94f2cf6a7fe509656a99d85d49bd7c74133c69903f059a2675

2 years agoRename some JS files from X.js to X.c-pp.js to keep the maintainer, and downstream...
stephan [Wed, 30 Nov 2022 18:21:01 +0000 (18:21 +0000)] 
Rename some JS files from X.js to X.c-pp.js to keep the maintainer, and downstream build customizers, aware that those files contain constructs specific to the c-pp preprocessor and will not run as-is in JS.

FossilOrigin-Name: 2eade7c7b17a186735c72974c11a34798a08364861d0f307e897ba765c0a93c7

2 years agoAlways use nanosleep() (instead of usleep() or sleep) if the
drh [Wed, 30 Nov 2022 13:44:31 +0000 (13:44 +0000)] 
Always use nanosleep() (instead of usleep() or sleep) if the
_POSIX_C_SOURCE macro says it should be available.

FossilOrigin-Name: 6620c57b9d3eae7226a412318b43393196df069b5b90aae0cf1743fdd2d102dd

2 years agoInstall sqlite3_malloc/sqlite3_free() as the JS-side WASM allocator (as opposed to...
stephan [Wed, 30 Nov 2022 11:50:16 +0000 (11:50 +0000)] 
Install sqlite3_malloc/sqlite3_free() as the JS-side WASM allocator (as opposed to replacing C-level's malloc()/free() with them). All tests work and this eliminates the potential for allocator discrepancies when using the (de)serialize APIs.

FossilOrigin-Name: 95c78f6b46e0d8efa4313061f47677479f48610b7a7261dc8d0fb1859aca2ad9

2 years agoSimplify how the OPFS VFS async proxy copies initial state sent to it from the synchr...
stephan [Wed, 30 Nov 2022 08:37:17 +0000 (08:37 +0000)] 
Simplify how the OPFS VFS async proxy copies initial state sent to it from the synchronous side of the connection. Make the lock-wait time a multiple of the wait-loop timeout interval.

FossilOrigin-Name: eddafafffa634a42ceeed70aa3fc58be130527612157a4bf4ff9e65c7f6dc26c

2 years agoOPFS VFS: remove an invalid TODO and fix a property name typo which caused xCheckRese...
stephan [Wed, 30 Nov 2022 07:48:35 +0000 (07:48 +0000)] 
OPFS VFS: remove an invalid TODO and fix a property name typo which caused xCheckReservedLock() to always report false.

FossilOrigin-Name: 3b037caa2fa07b6c44c485574e9e5dc71f4a8e82bc902c1321bb0b918b139c74

2 years agoRename some OPFS JS files. Prevent JS bindings of sqlite3_uri_...() from performing...
stephan [Wed, 30 Nov 2022 07:17:29 +0000 (07:17 +0000)] 
Rename some OPFS JS files. Prevent JS bindings of sqlite3_uri_...() from performing JS-to-C-string argument conversion on their first argument, as doing so is specifically illegal.

FossilOrigin-Name: 79832808de2cbdba140ed9e0558f1502b51d131ab4315265315922cda7b748cb

2 years agoRefactor a significant chunk of the OPFS sqlite3_vfs init code into sqlite3.VfsHelper...
stephan [Wed, 30 Nov 2022 05:27:36 +0000 (05:27 +0000)] 
Refactor a significant chunk of the OPFS sqlite3_vfs init code into sqlite3.VfsHelper, and internal-use-only API encapsulating code relevant to creating new VFSes in JS. Intended to assist in pending experimentation with an alternative OPFS VFS.

FossilOrigin-Name: e25d7b080a807e35b32cb885ea75b384130e5c6e936dfef783c5b45d9bfe77d8

2 years agoDoc and logging text tweaks in the OPFS async proxy and test app.
stephan [Wed, 30 Nov 2022 03:08:50 +0000 (03:08 +0000)] 
Doc and logging text tweaks in the OPFS async proxy and test app.

FossilOrigin-Name: 7ce8608e221924d2c7067687eb6eef0f3cab181d5b4132e55a67d8514b6ce94b

2 years agosqlite3.oo1.OpfsDb: default to journal_mode=persist, as current tests show it to...
stephan [Tue, 29 Nov 2022 18:28:40 +0000 (18:28 +0000)] 
sqlite3.oo1.OpfsDb: default to journal_mode=persist, as current tests show it to be marginally faster than truncate/delete in Chrome v109. Also increase default busy_timeout from 2 seconds to 3, admittedly on a whim.

FossilOrigin-Name: d0c8fa30a31c691bc1be5e98d806eeb1e23a8fc6cd54d87e5c1b720aa936e707

2 years agoOPFS speedtest1: hide a currently-broken/to-fix WASMFS-build link.
stephan [Tue, 29 Nov 2022 18:00:01 +0000 (18:00 +0000)] 
OPFS speedtest1: hide a currently-broken/to-fix WASMFS-build link.

FossilOrigin-Name: cde95d382f8debcbca27c4aaf08470ffda35ab10d723a887786669367590ad3c

2 years agoChange the handling of hwtime.h to make it easier to compile performance
drh [Tue, 29 Nov 2022 17:52:04 +0000 (17:52 +0000)] 
Change the handling of hwtime.h to make it easier to compile performance
measurement builds that make use of hwtime.h.  This should not affect
productions builds.

FossilOrigin-Name: f64a224244743ab121371abd516fccbfc93c110e0952211764bd1b217e792c1b

2 years agoFix legacy build system main.mk to include basexx.c in testfixture builds.
dan [Tue, 29 Nov 2022 11:28:45 +0000 (11:28 +0000)] 
Fix legacy build system main.mk to include basexx.c in testfixture builds.

FossilOrigin-Name: 8c9200b7e156206c4270ff60631c57d5898e2769f7f68294ce652bfec4cfa7c2

2 years agoRename one instance of opfs.OpfsDb to oo1.OpfsDb, as per [0c5c51f4fb04].
stephan [Tue, 29 Nov 2022 06:56:09 +0000 (06:56 +0000)] 
Rename one instance of opfs.OpfsDb to oo1.OpfsDb, as per [0c5c51f4fb04].

FossilOrigin-Name: 75b04c9b302ec66749d8f072a49e61de9640b868bec99eadeb5b4bdef354c336

2 years agoMinor internal cleanups and docs in the OPFS sqlite3_vfs.
stephan [Tue, 29 Nov 2022 06:09:32 +0000 (06:09 +0000)] 
Minor internal cleanups and docs in the OPFS sqlite3_vfs.

FossilOrigin-Name: 61799b05ff232c2ac349169c27bfe7f8d9277366093b0c9dd2739828993b3066

2 years agoInternal restructuring of the OPFS sqlite3_vfs in order to facilitate certain experim...
stephan [Tue, 29 Nov 2022 05:25:08 +0000 (05:25 +0000)] 
Internal restructuring of the OPFS sqlite3_vfs in order to facilitate certain experimentation and improve error reporting/hints if it cannot be activated. Deprecate the name sqlite3.opfs.OpfsDb, preferring sqlite3.oo1.OpfsDb for consistency with JsStorageDb and any future DB subclasses.

FossilOrigin-Name: 0c5c51f4fb04a4b90c50ec9704cfea9a3fb7d7d0ee55c1b0d4476129188217a6

2 years agoAdd an explicit warning about the current API-instability of the sqlite3.opfs namespa...
stephan [Tue, 29 Nov 2022 02:23:12 +0000 (02:23 +0000)] 
Add an explicit warning about the current API-instability of the sqlite3.opfs namespace, which may need to be eliminated based on re-thinking of how the OPFS sqlite3_vfs is registered. Comment changes only - no code.

FossilOrigin-Name: 0cb2fd14179397051a25d066256a553fc198656d5668c7010c016f2b8f495bf4

2 years agoCheck-in [8d5b76593d82b3a5] contained an error that was causing some obscure
drh [Mon, 28 Nov 2022 21:17:30 +0000 (21:17 +0000)] 
Check-in [8d5b76593d82b3a5] contained an error that was causing some obscure
error codes to be lost.  Fixed here.

FossilOrigin-Name: 46cdd3637d6a206ad2bcf8653cc6f2c7a886a16cc7685c45967938609941a755

2 years agoGet ext/misc/basexx.c into the testfixture.exe build for MSC.
larrybr [Mon, 28 Nov 2022 21:09:49 +0000 (21:09 +0000)] 
Get ext/misc/basexx.c into the testfixture.exe build for MSC.

FossilOrigin-Name: bb2c5d088e4784f6763eb0ea2a8542ad2129529f181fa92963c7231d7a68f25c

2 years agoFix harmless compiler warnings.
drh [Mon, 28 Nov 2022 20:08:15 +0000 (20:08 +0000)] 
Fix harmless compiler warnings.

FossilOrigin-Name: 8d5b76593d82b3a57bc904ced33c24cd49369868b5dd2fe7f2c7b114f5aee2f6

2 years agoImplementation of enhancement request [695a1a53dea6b240]: The query planner
drh [Mon, 28 Nov 2022 19:42:48 +0000 (19:42 +0000)] 
Implementation of enhancement request [695a1a53dea6b240]: The query planner
has improved awareness of when indexes on expressions are covering and adjusts
their costs accordingly.

FossilOrigin-Name: e3474d79b27298e96e7686e5a6f9a8e736b8a6568a0a08fb1abe5bcca038ca63

2 years agoAdd the SQLITE_FCNTL_RESET_CACHE verb. Use it to ensure that the page cache is purged...
dan [Mon, 28 Nov 2022 18:41:41 +0000 (18:41 +0000)] 
Add the SQLITE_FCNTL_RESET_CACHE verb. Use it to ensure that the page cache is purged before and after a the recovery extension is run.

FossilOrigin-Name: 6db0bc4bc0d272b610bef2aeeae43f539ed6e7cc0a9cc767d5af85ecb0019d5f

2 years agoTest cases added derived from the enhancment ticket. covering-indexed-expr
drh [Mon, 28 Nov 2022 18:17:42 +0000 (18:17 +0000)] 
Test cases added derived from the enhancment ticket.

FossilOrigin-Name: c022c0152ad61a4f56e36f4062609073e2273fbf6f826c20652159be229c2d46

2 years agoMerge trunk changes into covering-index-expr branch.
drh [Mon, 28 Nov 2022 16:11:10 +0000 (16:11 +0000)] 
Merge trunk changes into covering-index-expr branch.

FossilOrigin-Name: 89d775ada39aed4dc532374ace02156d07dc06e8ae54a194608af0c5a582d20f

2 years agoRework the covering index checking routine, whereIsCoveringIndex(), so that
drh [Mon, 28 Nov 2022 15:23:53 +0000 (15:23 +0000)] 
Rework the covering index checking routine, whereIsCoveringIndex(), so that
it can return a "maybe" result for aggregate queries where we are not exactly
sure.  The index is scored as if it is covering, but the main table is
still opened.

FossilOrigin-Name: b8eec4214363192e6f3e12b3faa5810d8269a5fdaecab3ec09b02e5002cf798a

2 years agoAdd (optional) base64 and base85 UDF extensions.
larrybr [Mon, 28 Nov 2022 14:51:50 +0000 (14:51 +0000)] 
Add (optional) base64 and base85 UDF extensions.

FossilOrigin-Name: b44ab10c49bc2895483a9d40813be3798710ee713cc4bf04e449dce55a68452a

2 years agoSync w/trunk, zap surplus space. base_convert
larrybr [Mon, 28 Nov 2022 14:11:48 +0000 (14:11 +0000)] 
Sync w/trunk, zap surplus space.

FossilOrigin-Name: b8345630a2a322234bda49ee4b996f6ba20e2b080621e229a2ec5e820892a663

2 years agoEnsure the RETURNING clause is honoured when a row of a temp table is updated by...
dan [Mon, 28 Nov 2022 13:47:27 +0000 (13:47 +0000)] 
Ensure the RETURNING clause is honoured when a row of a temp table is updated by an ON CONFLICT clause.

FossilOrigin-Name: a2449bcc2c71d0f4c3289621fbf1cb97f0f407c9f7b5bf18245b7854a07c6cfa

2 years agoConform CLI .trace arg handling to its help.
larrybr [Mon, 28 Nov 2022 02:28:44 +0000 (02:28 +0000)] 
Conform CLI .trace arg handling to its help.

FossilOrigin-Name: 31546ea320e8daa020f8a9f491718ffc3dde0e32954018afed88e8494a6aff6c

2 years agoDocument an OPFS API change in Chrome v108 which does not break our code but does...
stephan [Sun, 27 Nov 2022 00:57:56 +0000 (00:57 +0000)] 
Document an OPFS API change in Chrome v108 which does not break our code but does change several formerly async methods to synchronous. No code changes.

FossilOrigin-Name: e9dd87e28a96eb6560943b9a2cb3cb0ff14698c96fe63865944dcfed73b67bd3

2 years agoBetter reporting of when the WHERE clause analysis thinks that an index
drh [Sat, 26 Nov 2022 20:52:38 +0000 (20:52 +0000)] 
Better reporting of when the WHERE clause analysis thinks that an index
is covering.

FossilOrigin-Name: 17ebcf316b91042c823eea7bb6f1309325023cb5c70538cdb2ce932caee2ef05

2 years agoShow which AggInfo.aCol an Expr node represents in the tree trace debugging
drh [Sat, 26 Nov 2022 20:48:56 +0000 (20:48 +0000)] 
Show which AggInfo.aCol an Expr node represents in the tree trace debugging
logic.  Does not affect production builds.

FossilOrigin-Name: b8076f4dc55810855f668dfd4221bdb555cc78c61df200cf0cb87eb137e22850

2 years agoMerge trunk fixes into the covering-indexed-expr branch.
drh [Sat, 26 Nov 2022 20:13:54 +0000 (20:13 +0000)] 
Merge trunk fixes into the covering-indexed-expr branch.

FossilOrigin-Name: 9ac67ff968e874cf955e46e3993e3215c766feec3d5bdd38d77884eedd86b59e

2 years agoFix an incorrect assertion-fault in the "TREETRACE" debugging logic on
drh [Sat, 26 Nov 2022 20:12:02 +0000 (20:12 +0000)] 
Fix an incorrect assertion-fault in the "TREETRACE" debugging logic on
select.c.  The problem does not affect production builds.

FossilOrigin-Name: f11bb42292d94e7b7687bd2134f72afe1353182238cb4837fc0a6d78a021dd7e

2 years agoOPFS VFS: add the opfs-unlock-asap=1 URI flag which tells the VFS to release implicit...
stephan [Sat, 26 Nov 2022 15:24:58 +0000 (15:24 +0000)] 
OPFS VFS: add the opfs-unlock-asap=1 URI flag which tells the VFS to release implicit locks ASAP instead of during VFS idle time. This improves concurrency notably in the test app but brings a significant performance penalty in speedtest1 (roughly 4x slowdown). This is not the final word in OPFS concurrency, but gets us a step further.

FossilOrigin-Name: 9542f9ce9e023b489e2d93661f719fb0751c1e28f72fded9d3c2156d5777e7b1

2 years agoExperimental changes to help the query planner detect when an expression
drh [Sat, 26 Nov 2022 14:19:47 +0000 (14:19 +0000)] 
Experimental changes to help the query planner detect when an expression
index is coverting.  Works somewhat, but there are tests that fail.

FossilOrigin-Name: 968c189bcf29a9b25305251a58d09b7d52ab9dd08f5057dc3ab1f7ad1a5316a0

2 years agoRelax restriction (8) on the push-down optimization so that it only applies
drh [Fri, 25 Nov 2022 17:05:55 +0000 (17:05 +0000)] 
Relax restriction (8) on the push-down optimization so that it only applies
if one or more columns uses a collating sequence other than BINARY.
See [forum:/forumpost/3824ced748baa808|forum post 3824ced748baa808] and
check-in [346a3b12b861ce7b].

FossilOrigin-Name: adbca3448e2099f0d6149a073978f230ed9a92a2f384779879ef89e672231bcf

2 years agoFix #ifdefs so that restrictions (8) and (9) of the push-down optimization
drh [Fri, 25 Nov 2022 16:32:59 +0000 (16:32 +0000)] 
Fix #ifdefs so that restrictions (8) and (9) of the push-down optimization
are still enforced even if compiled with SQLITE_OMIT_WINDOWFUNC.  This
fixes a bug introduced by check-in [346a3b12b861ce7b].

FossilOrigin-Name: 09e1e42e0ff26f9a71cbd128169f060a66425828d637bf8f781490ca38d99103

2 years agoImmprove the query planner such that it is able to make use of indexed
drh [Fri, 25 Nov 2022 16:10:48 +0000 (16:10 +0000)] 
Immprove the query planner such that it is able to make use of indexed
expressions within an aggregate query with GROUP BY.  This implements
enhancement request [99378177930f87bd].

FossilOrigin-Name: b9190d3da70c41717eb188474fd225ee43d0b46646e1b03de5967bd332553870

2 years agoAdd restriction (9) to the push-down optimization: If the subquery is
drh [Fri, 25 Nov 2022 15:52:00 +0000 (15:52 +0000)] 
Add restriction (9) to the push-down optimization:  If the subquery is
a compound then all arms of the compound must have the same affinity.
dbsqlfuzz 3a548de406a50e896c1bf7142692d35d339d697f.

FossilOrigin-Name: 1ad41840c5e0fa702ba2c0df77a3ea126bd695b910b5d1271fa3129c38c58f5f

2 years agoRemove a redundant assert() statement. agg-with-indexed-expr
drh [Fri, 25 Nov 2022 13:26:51 +0000 (13:26 +0000)] 
Remove a redundant assert() statement.

FossilOrigin-Name: e3e1b453dc47884ddc9e51c4302fa2f4d40bca3d69ac7c13d8e2ae1adc81ac56

2 years agoImprove the robustness of the updateAccumulator() routine against OOM.
drh [Fri, 25 Nov 2022 13:15:48 +0000 (13:15 +0000)] 
Improve the robustness of the updateAccumulator() routine against OOM.

FossilOrigin-Name: d2844f577b69fdc16a0a2568c0958fc3d8aff33e9a0ef80e0f58e92f01097432

2 years agoImproved comments. Add assert()s to verify that the AggInfo structure
drh [Fri, 25 Nov 2022 13:08:20 +0000 (13:08 +0000)] 
Improved comments.  Add assert()s to verify that the AggInfo structure
is unchanged after registers have been assigned.

FossilOrigin-Name: 5200b84195ee1ccaa387f7032eae3d463724c48cb53ba0251bbc79e927dd9752

2 years agoCorrectly deal with IF-NULL-ROW operators when dealing with
drh [Thu, 24 Nov 2022 23:35:27 +0000 (23:35 +0000)] 
Correctly deal with IF-NULL-ROW operators when dealing with
indexed expressions in aggregates.

FossilOrigin-Name: 939cb47025354e2df047de7654c0b06f791957cfe4e904abe8892207cea90215

2 years agoSpeed up base85() conversions and sync w/trunk.
larrybr [Thu, 24 Nov 2022 20:11:34 +0000 (20:11 +0000)] 
Speed up base85() conversions and sync w/trunk.

FossilOrigin-Name: 17b823500a2ed135c1f40aa7f4d87ba5b2eab35c0abd9e0856041cf0f510cbee

2 years agoTake care not to try to add comments to a TK_AGG_COLUMN opcode that does
drh [Thu, 24 Nov 2022 19:02:40 +0000 (19:02 +0000)] 
Take care not to try to add comments to a TK_AGG_COLUMN opcode that does
not have an associated Table object because it is a reference to an indexed
expression.

FossilOrigin-Name: 5fc23863e4a6388884ce061cd69546757b4c39d731bbb58a8121b92fd5144f7b

2 years agoChange a NEVER() into an assert().
drh [Thu, 24 Nov 2022 18:45:56 +0000 (18:45 +0000)] 
Change a NEVER() into an assert().

FossilOrigin-Name: a0fd44f4e5a0ec83465203a0009f307ca50223833575895a3ebf8a289515714f

2 years agoFix a test case in fts3expr4.test to account for different locales.
dan [Thu, 24 Nov 2022 17:58:55 +0000 (17:58 +0000)] 
Fix a test case in fts3expr4.test to account for different locales.

FossilOrigin-Name: a2b6883ac2ef878f525ee847b170beb9f9ab9d1fa67557101be2cdae1e7f7a57

2 years agoMore work on the OPFS concurrency testing app. opfs-unlock-asap
stephan [Thu, 24 Nov 2022 17:53:09 +0000 (17:53 +0000)] 
More work on the OPFS concurrency testing app.

FossilOrigin-Name: c0458caca3508d5d252f9b5198bda4f51a5c1874540f014b17e409f2daab1706

2 years agoJS documentation cleanups. No code changes.
stephan [Thu, 24 Nov 2022 16:03:49 +0000 (16:03 +0000)] 
JS documentation cleanups. No code changes.

FossilOrigin-Name: 18e89a436daa18a8c972caf06b298da43c97a6ea3e2e5229dccb6920c27bfdb9

2 years agoUpdate multiplex3.test to account for the fact that the multiplexor xDelete method...
dan [Thu, 24 Nov 2022 15:32:00 +0000 (15:32 +0000)] 
Update multiplex3.test to account for the fact that the multiplexor xDelete method may return an error even if it manages to delete the first chunk of a file.

FossilOrigin-Name: 1a7f3254735054ed8ca32d5ec7c8cde9195a64702638bdc50392007e396fead2

2 years agoAdd NEVER() and ALWAYS() macros on branches that are believed to be
drh [Thu, 24 Nov 2022 15:04:23 +0000 (15:04 +0000)] 
Add NEVER() and ALWAYS() macros on branches that are believed to be
unreachable.

FossilOrigin-Name: 3a901e88c87fc76c7fe42e47a976a5706830f0dbd6027605663e4d55f4f33590

2 years agoNew test cases. Fix the logic so that it works for GROUP BY aggregates
drh [Thu, 24 Nov 2022 13:19:25 +0000 (13:19 +0000)] 
New test cases.  Fix the logic so that it works for GROUP BY aggregates
that do not require sorting.

FossilOrigin-Name: ef6ebe7922f56c1584a005deedc85ca1070b4fe5082ada8bbf8d06df54f1c9ef

2 years agoSpeed up base64 conversions, and add test with more data for the baseNN conversion...
larrybr [Thu, 24 Nov 2022 02:59:33 +0000 (02:59 +0000)] 
Speed up base64 conversions, and add test with more data for the baseNN conversion to grind.

FossilOrigin-Name: 6c84ae4ba83713c751fddff8be41686bbcb525ac8135e1520436c62d0bc23d2c

2 years agoAdd sqlite3.oo1.DB.prototype.checkRc() and tests for both that method and its class...
stephan [Thu, 24 Nov 2022 02:35:03 +0000 (02:35 +0000)] 
Add sqlite3.oo1.DB.prototype.checkRc() and tests for both that method and its class-level counterpart.

FossilOrigin-Name: f7eaa6ba2147bfd6dbdc2444d0f919d846aa7f9b68cccab17ef585ffdacf3d60

2 years agoMerge the latest trunk changes into the agg-with-indexed-expr branch to
drh [Thu, 24 Nov 2022 01:41:44 +0000 (01:41 +0000)] 
Merge the latest trunk changes into the agg-with-indexed-expr branch to
simplify diffs.

FossilOrigin-Name: 38c3d3f1ed0fd2bb62aa8a7e5a27f2b247123e094e2fdb0a2475d788c3dfbc04

2 years agoAdd explanatory comment to the new optimization. And add a test case.
drh [Thu, 24 Nov 2022 01:40:20 +0000 (01:40 +0000)] 
Add explanatory comment to the new optimization.  And add a test case.

FossilOrigin-Name: e6c20f61de7d048eee65c8e74a3eb36760ab9747ebd1ab50e49642b777c10306

2 years agoMinor JS doc updates.
stephan [Wed, 23 Nov 2022 21:09:51 +0000 (21:09 +0000)] 
Minor JS doc updates.

FossilOrigin-Name: 27efd63ad7fb3bf8d0d07f2c9f48652c8cacc4e697c229c8085120a8e6b50a39

2 years agoAdd optional zSchema argument to sqlite3_js_db_export().
stephan [Wed, 23 Nov 2022 21:03:22 +0000 (21:03 +0000)] 
Add optional zSchema argument to sqlite3_js_db_export().

FossilOrigin-Name: 9c23644b1e5bf44bfb431a35fd1674c11ccb99e9eb0989f10175b0cb2a858eaa

2 years agoOPFS concurrency test: add a URL flag to enable/disable unlock-asap mode.
stephan [Wed, 23 Nov 2022 20:49:08 +0000 (20:49 +0000)] 
OPFS concurrency test: add a URL flag to enable/disable unlock-asap mode.

FossilOrigin-Name: 1c1bf22eadae2a5a7d4358e7cdd22641c2efb9296f42e7376749293b3a58b114