]> git.ipfire.org Git - thirdparty/sqlite.git/log
thirdparty/sqlite.git
2 years agoMerge from trunk prepratory to branch end. sha3sum_text_validation
larrybr [Tue, 6 Dec 2022 19:25:07 +0000 (19:25 +0000)] 
Merge from trunk prepratory to branch end.

FossilOrigin-Name: 1d01f8483af7b6a4e93a49fd8bbb6a2b9300e969b574efeb96d56081a33055c4

2 years agoAvoid several -Wall warnings in textfixture build.
larrybr [Tue, 6 Dec 2022 19:20:49 +0000 (19:20 +0000)] 
Avoid several -Wall warnings in textfixture build.

FossilOrigin-Name: 03ae8680e430c6d2c39ca26fa2cb6eed46e3c3e52b53f7a8bf548560446c6d3d

2 years agoFor CLI .sha3sum, emit warning to stderr for any invalidly encoded text fields.
larrybr [Tue, 6 Dec 2022 18:48:37 +0000 (18:48 +0000)] 
For CLI .sha3sum, emit warning to stderr for any invalidly encoded text fields.

FossilOrigin-Name: 8e833ecc81367658e81acd1d3a0242ab954a62d6f719af56f1d185656d0b73e6

2 years agoHave sqlite3_stmt_scanstatus_v2() return an NCYCLE value for all loops, not just...
dan [Tue, 6 Dec 2022 18:48:06 +0000 (18:48 +0000)] 
Have sqlite3_stmt_scanstatus_v2() return an NCYCLE value for all loops, not just virtual tables ones. The value returned is the sum of the NCYCLE counts for the various opcodes that move or read data from the table or index cursor associated with the loop.

FossilOrigin-Name: 9499b2f51e8174c6b8a67840c92ba23b7dd1dc8dc2b91fca0c5dc07b71662149

2 years agoClear a few more -Wall warnings and simplify dynaprompt feature keep/omit macros.
larrybr [Tue, 6 Dec 2022 17:59:05 +0000 (17:59 +0000)] 
Clear a few more -Wall warnings and simplify dynaprompt feature keep/omit macros.

FossilOrigin-Name: 540e895d877fab1ea138786e56923a202018f68c78199a89adfc296c75735b30

2 years agoFix compiler warnings in the new dynamic continuation prompt logic of
drh [Tue, 6 Dec 2022 15:11:13 +0000 (15:11 +0000)] 
Fix compiler warnings in the new dynamic continuation prompt logic of
the CLI.

FossilOrigin-Name: 0d80500d358fa1c9b5867c2c8250d278ba813bf2ad81bb0bc3f820a71489b374

2 years agoIn the unix backend, when implementing the defenses against small file
drh [Tue, 6 Dec 2022 13:12:33 +0000 (13:12 +0000)] 
In the unix backend, when implementing the defenses against small file
descriptors, delete a file just created if it was opened with
O_EXCL|O_CREAT so that it can be created again the next time through the
loop.  Fix for the problem described by
[forum:/forumpost/699af709ab3a8ccf|forum post 699af709ab3a8ccf].

FossilOrigin-Name: c0cfe0582add87981826d124a0763482f51fae4b105b5a970dd56919f1d04d60

2 years agoSupport an SQLITE_SCANSTAT_NCYCLE statistic for "CO-ROUTINE" elements.
dan [Tue, 6 Dec 2022 11:41:05 +0000 (11:41 +0000)] 
Support an SQLITE_SCANSTAT_NCYCLE statistic for "CO-ROUTINE" elements.

FossilOrigin-Name: d3f6a207fd1b3f53688fa1ff35eb71e1c5bcc4115c0c35bb187d22ccf7100b71

2 years agoAdd optional feature: A CLI continuation prompt which reflects open lexemes and paren...
larrybr [Tue, 6 Dec 2022 05:31:20 +0000 (05:31 +0000)] 
Add optional feature: A CLI continuation prompt which reflects open lexemes and parens, similarly to PG shell.

FossilOrigin-Name: f41f18b1c3c950565ee3c237aebb51cfc3241813c6425813a8217e07aa0153a6

2 years agoAdd optional feature: A CLI continuation prompt which reflects open lexemes and paren... dynamic_prompt
larrybr [Tue, 6 Dec 2022 05:09:51 +0000 (05:09 +0000)] 
Add optional feature: A CLI continuation prompt which reflects open lexemes and parens, similarly to PG shell.

FossilOrigin-Name: dac2ddc287db7a68d0cd49b785060f62290868fbb1aa2ee09e54d3b1acfbf55f

2 years agoEnhance the sqlite3_stmt_scanstatus() API and add sqlite3_stmt_scanstatus_v2(). For...
dan [Mon, 5 Dec 2022 19:16:23 +0000 (19:16 +0000)] 
Enhance the sqlite3_stmt_scanstatus() API and add sqlite3_stmt_scanstatus_v2(). For creation of enhanced query performance reports.

FossilOrigin-Name: 4893b4e3eafc7c9c22b24717f90a585862203f987cf108b079ce6e946093e675

2 years agoUpdate comments in sqlite.h.in to account for sqlite3_stmt_scanstatus_v2(). scanstatus_v2
dan [Mon, 5 Dec 2022 18:52:12 +0000 (18:52 +0000)] 
Update comments in sqlite.h.in to account for sqlite3_stmt_scanstatus_v2().

FossilOrigin-Name: 009462f2344b1f468cf9440343a47fec68d783a2bfb4fa6168bb227ec910b918

2 years agoMerge latest trunk changes.
dan [Mon, 5 Dec 2022 18:26:37 +0000 (18:26 +0000)] 
Merge latest trunk changes.

FossilOrigin-Name: 1a72777b1279f74f212fb2f675a4594a238e5d28f048879d7f5ad5287673c3c4

2 years agoAdd loops and rows counters to "USE TEMP B-TREE FOR ORDER BY" records. Also fix the...
dan [Mon, 5 Dec 2022 18:19:56 +0000 (18:19 +0000)] 
Add loops and rows counters to "USE TEMP B-TREE FOR ORDER BY" records. Also fix the sqliteHwtime() function so that it returns a 64-bit value.

FossilOrigin-Name: 41a0e05e8c0fca3b803fe4bd017a157c172b2ca518356a2a4d4ed4f12d01a1e3

2 years agoAdd test case that should have been part of previous commit.
dan [Mon, 5 Dec 2022 14:20:54 +0000 (14:20 +0000)] 
Add test case that should have been part of previous commit.

FossilOrigin-Name: dc7dd2d3e50e7cc474b22f1b5b219da32bcd7aa1ba56864d1dbcf0d3a6fa06f2

2 years agoFix a problem in the memdb vfs xLock() function allowing clients to upgrade to EXCLUS...
dan [Mon, 5 Dec 2022 14:12:14 +0000 (14:12 +0000)] 
Fix a problem in the memdb vfs xLock() function allowing clients to upgrade to EXCLUSIVE locks when other connections are holding SHARED.

FossilOrigin-Name: 15f0be8a640e7bfa4130edd4650a745337bd96083b119a1553f9abf9ff066806

2 years agoRemove SQLITE_EXPERIMENTAL tag from sqlite3_vtab_collation() and fix a related doc...
stephan [Mon, 5 Dec 2022 13:07:06 +0000 (13:07 +0000)] 
Remove SQLITE_EXPERIMENTAL tag from sqlite3_vtab_collation() and fix a related doc typo.

FossilOrigin-Name: 35d670b3593a46e13ded541ef477fa35dac6bcb9c31e6cf4b90bd7fa550a8ee2

2 years agoRemove obsolete "experimental" comments on the virtual-table interface
drh [Mon, 5 Dec 2022 10:50:06 +0000 (10:50 +0000)] 
Remove obsolete "experimental" comments on the virtual-table interface
documentation in sqlite.h.in.

FossilOrigin-Name: 28fc32abdcb1c6af19f0926a3a7317e15930a0ba89645d5ffbf6ce99d8463273

2 years agoImproved query planner cost estimates. Fix for ticket [e8b674241947eb3b].
drh [Mon, 5 Dec 2022 02:52:37 +0000 (02:52 +0000)] 
Improved query planner cost estimates.  Fix for ticket [e8b674241947eb3b].

FossilOrigin-Name: 1b779afa3ed2f35a110e460fc6ed13cba744db85b9924149ab028b100d1e1e12

2 years agoUse the smaller estimated row size for searching IPK tables, but use the qp-tuning
drh [Mon, 5 Dec 2022 02:42:30 +0000 (02:42 +0000)] 
Use the smaller estimated row size for searching IPK tables, but use the
original larger row size estimate for scanning, since the leaves can have large
rows.

FossilOrigin-Name: df3818997b822743ac407dde45c5fd75845ca40f461e31350d86963dffec6cd6

2 years agoFix safe mode authorizer callback to reject disallowed UDFs. Reported at [forum:...
larrybr [Sun, 4 Dec 2022 23:20:38 +0000 (23:20 +0000)] 
Fix safe mode authorizer callback to reject disallowed UDFs. Reported at [forum:/forumpost/07beac8056151b2f|Forum post 07beac8056151b2f].

FossilOrigin-Name: cefc032473ac5ad244c0b6402c541b2f76c0c65a041bda03bfbe7c0e2c11fac2

2 years agoExport sqlite3_result_zeroblob/zeroblob64() to wasm.
stephan [Sun, 4 Dec 2022 08:16:33 +0000 (08:16 +0000)] 
Export sqlite3_result_zeroblob/zeroblob64() to wasm.

FossilOrigin-Name: a60e56627fc0ef8831429941d429ee02c6ee51ce5a2c1af581dc5bc5a00d911e

2 years agoEnhance SQLITE_SCANSTAT_NCYCLE so that it reports on virtual tables.
dan [Sat, 3 Dec 2022 21:24:26 +0000 (21:24 +0000)] 
Enhance SQLITE_SCANSTAT_NCYCLE so that it reports on virtual tables.

FossilOrigin-Name: 622d8eb3724bee617b55d6fb71f1a2d683db6858065adced6bf3ce9525bcd6b5

2 years agoIncrease the nominal row size for IPK index lookups slightly, for better
drh [Sat, 3 Dec 2022 19:04:09 +0000 (19:04 +0000)] 
Increase the nominal row size for IPK index lookups slightly, for better
balance.

FossilOrigin-Name: 1a61c500add4a2bfe80c0c691d559cfca166dc5f8262651a58da7ec16a51d430

2 years agoAdd TOTAL rows to the output generated by tool/vdbe_profile.tcl.
drh [Sat, 3 Dec 2022 18:21:30 +0000 (18:21 +0000)] 
Add TOTAL rows to the output generated by tool/vdbe_profile.tcl.

FossilOrigin-Name: 9800586393c9d3b82459ef657620d245a7985ef5fa389b8a9ea633d6a29c7299

2 years agoAdd CYCLES scanstat measurement to "USE TEMP B-TREE FOR ORDER BY" lines.
dan [Sat, 3 Dec 2022 18:16:25 +0000 (18:16 +0000)] 
Add CYCLES scanstat measurement to "USE TEMP B-TREE FOR ORDER BY" lines.

FossilOrigin-Name: 365011ae8b7e3fcaa2c4ea9601231a3ef2223e60d7a53ec33013109dca22ad58

2 years agoAdd a test case to show that ticket [e8b674241947eb3b] has been fixed.
drh [Sat, 3 Dec 2022 17:23:29 +0000 (17:23 +0000)] 
Add a test case to show that ticket [e8b674241947eb3b] has been fixed.

FossilOrigin-Name: e20de6d450c2b4d5bde737f625de16ff53262c22ce7aa6917b64f1665170d33f

2 years agoFurther improvements to the estimated cost of sorting. Take into account
drh [Sat, 3 Dec 2022 17:09:15 +0000 (17:09 +0000)] 
Further improvements to the estimated cost of sorting.  Take into account
the number of columns to be sorted.

FossilOrigin-Name: f3290cf83b7c02d17d85d8942954f052b486c370cd5ec732969da9061dc1d19a

2 years agoCause CLI .sha3sum to warn of text fields that do not survive CAST(CAST(t as BLOB...
larrybr [Sat, 3 Dec 2022 16:09:32 +0000 (16:09 +0000)] 
Cause CLI .sha3sum to warn of text fields that do not survive CAST(CAST(t as BLOB) AS TEXT) due to invalid UTF encoding.

FossilOrigin-Name: 123f2a0785790addf9c60a0fd09077dda9cb84d33a2594901a2b22bb555be491

2 years agoCorrect a memory leak in tester1.js.
stephan [Sat, 3 Dec 2022 15:41:29 +0000 (15:41 +0000)] 
Correct a memory leak in tester1.js.

FossilOrigin-Name: e42b052dd754e577b8429e2430821c5f88096170ffa58b4e5e3cb7ee19b6c37d

2 years agosqlite3.wasm.allocFromTypedArray() now optionally accepts an ArrayBuffer as its argument.
stephan [Sat, 3 Dec 2022 14:58:45 +0000 (14:58 +0000)] 
sqlite3.wasm.allocFromTypedArray() now optionally accepts an ArrayBuffer as its argument.

FossilOrigin-Name: 75a1a796f86d289c7275666fab19013934775dcccaed44a1a61d1749a6cb99c9

2 years agoRename wasm.xWrap.resultAdapter() X:free entries to X:dealloc for consistency with...
stephan [Sat, 3 Dec 2022 13:10:58 +0000 (13:10 +0000)] 
Rename wasm.xWrap.resultAdapter() X:free entries to X:dealloc for consistency with wasm.dealloc(). Add an undocumented feature to replace wasm.alloc/dealloc/realloc() with the C-standard allocators (after an allocator misuse led down a several-hour rabbit hole trying to discover a mis-free() violation). Related test updates.

FossilOrigin-Name: d9807656f8a7c2a893d3f68ee5592f44826b8e999ae66f7d9000674b5c1b0207

2 years agoDoc typo fix for SQLITE_MAX_ALLOCATION_SIZE in malloc.c. No code changes.
stephan [Sat, 3 Dec 2022 13:05:33 +0000 (13:05 +0000)] 
Doc typo fix for SQLITE_MAX_ALLOCATION_SIZE in malloc.c. No code changes.

FossilOrigin-Name: ed1ed21221b048ac5a5275cdfc4d9b2a406acdc7d4b648c3b61bcc822d88d955

2 years agoMove kvvfs xRead()/xWrite() buffer from function-local stack memory to heap memory...
stephan [Sat, 3 Dec 2022 11:51:29 +0000 (11:51 +0000)] 
Move kvvfs xRead()/xWrite() buffer from function-local stack memory to heap memory to accommodate changes in emsdk 3.1.27.

FossilOrigin-Name: f1da32410ca7b808b3bef5f5a59766e7281e9e6ea343c8b979599bf1fc1060f5

2 years agoJavaScript: add sqlite3.wasm.realloc(), sqlite3.capi.SQLITE_MAX_ALLOCATION_SIZE,...
stephan [Sat, 3 Dec 2022 11:16:55 +0000 (11:16 +0000)] 
JavaScript: add sqlite3.wasm.realloc(), sqlite3.capi.SQLITE_MAX_ALLOCATION_SIZE, and related tests.

FossilOrigin-Name: eeb84ba5de1152ef0f42105b8b285fdee9f5ad58281e60a4e0c8b1d6de1dead8

2 years agowasm build: rename the path to the wasm docs checkout, for clarity and consistency.
stephan [Sat, 3 Dec 2022 03:06:16 +0000 (03:06 +0000)] 
wasm build: rename the path to the wasm docs checkout, for clarity and consistency.

FossilOrigin-Name: b820db32365b2ca8e2397fd6ea85883e4555ffd82948e248a0f98415b7328349

2 years agowasm snapshot and doc-related build automation tweaks.
stephan [Sat, 3 Dec 2022 02:42:21 +0000 (02:42 +0000)] 
wasm snapshot and doc-related build automation tweaks.

FossilOrigin-Name: 13eb1abd06f55fb88fc7f7be6149fd94b12057d9c38cc6b97bec3940e7e01f04

2 years agoRework the oo1.DB's distinct-per-VFS post-open() step to accept either a batch of...
stephan [Sat, 3 Dec 2022 01:59:03 +0000 (01:59 +0000)] 
Rework the oo1.DB's distinct-per-VFS post-open() step to accept either a batch of SQL or a callback function. Increase OPFS's busy timeout to 10s.

FossilOrigin-Name: 9feefe253ac487cb52be6bdf91bdd305963266716baa08f2bf9505954ee76321

2 years agoTuning the query planner by adjusting the weights that predict the relative
drh [Sat, 3 Dec 2022 00:52:21 +0000 (00:52 +0000)] 
Tuning the query planner by adjusting the weights that predict the relative
performance of sorting and index lookup.

FossilOrigin-Name: 9f2806da4d88beceac2e81e05421f00481dd3dd100b096cd2ae6c828adb42ca7

2 years agoEnhance the sqlite3_stmt_scanstatus() API and add sqlite3_stmt_scanstatus_v2(). For...
dan [Fri, 2 Dec 2022 20:32:22 +0000 (20:32 +0000)] 
Enhance the sqlite3_stmt_scanstatus() API and add sqlite3_stmt_scanstatus_v2(). For creation of easier to read query performance reports.

FossilOrigin-Name: 55800833645739efeddcacef464c623931cb6aeb43f4219b4e4faf473c25c8bb

2 years agoMinor internal tweaks to the OPFS VFS. Resolve a missing result code which lead to...
stephan [Fri, 2 Dec 2022 18:56:37 +0000 (18:56 +0000)] 
Minor internal tweaks to the OPFS VFS. Resolve a missing result code which lead to a null deref in xFileSize().

FossilOrigin-Name: 57dd593ef0efa17dfb3a9f4eac36d5b8b879e271de817d8cd94a8c8b56d31870

2 years agoOPFS VFS: translate createSyncAccessHandle() exceptions which appear to be locking...
stephan [Fri, 2 Dec 2022 18:06:26 +0000 (18:06 +0000)] 
OPFS VFS: translate createSyncAccessHandle() exceptions which appear to be locking violations to SQLITE_BUSY. This seems to improve concurrency considerably even with a reduced retry count of 5 (was 6).

FossilOrigin-Name: 0d36021d107d3afca190ad61c3380536ad0cc2d493d345d48f9f9c1191741128

2 years agoFor the sqlite3_bind and sqlite3_result interfaces for UTF16 strings, round
drh [Fri, 2 Dec 2022 17:52:52 +0000 (17:52 +0000)] 
For the sqlite3_bind and sqlite3_result interfaces for UTF16 strings, round
the number of bytes down to the next even number, to avoid creating a UTF16
string that is an odd number of bytes.
[forum:/forumpost/411199488d065f83|Forum post 411199488d065f83].

FossilOrigin-Name: b57e3c3db00a6bc6db20c82530479f9eba7e37b731f0da6fe81682e84c7ac916

2 years agoUse sqlite3_result_int64() instead of sqlite3_result_int() when returning
drh [Fri, 2 Dec 2022 15:31:47 +0000 (15:31 +0000)] 
Use sqlite3_result_int64() instead of sqlite3_result_int() when returning
potentially large values from the DBSTAT virtual table, to avoid integer
overflows in the result.
[forum:/forumpost/ada2ab044f|Forum post ada2ab044f].

FossilOrigin-Name: 5652154a8c93cf3b1ff6c2e55e94abbe995b0bb625f733461df20e006c2f13f8

2 years agoMinor touchups to the JS test index page and test server push rules.
stephan [Fri, 2 Dec 2022 11:35:21 +0000 (11:35 +0000)] 
Minor touchups to the JS test index page and test server push rules.

FossilOrigin-Name: 0881f3e92364b3dc81443220451e853c0763efaf97653a6348bc39bd9bdb23ad

2 years agowasm builds: explicitly set a default stack size because emsdk 3.1.27 reduced it...
stephan [Fri, 2 Dec 2022 10:43:14 +0000 (10:43 +0000)] 
wasm builds: explicitly set a default stack size because emsdk 3.1.27 reduced it from 4MB to only 64kb, leading to memory corruption when kvvfs is used (it requires at least twice that for I/O).

FossilOrigin-Name: 758112460fb624198a144ed7d8d2324497fa72a2eee80010bd22b621e064b389

2 years agowasm: after building snapshot zip file, emit instructions for pushing it to the test...
stephan [Fri, 2 Dec 2022 09:23:38 +0000 (09:23 +0000)] 
wasm: after building snapshot zip file, emit instructions for pushing it to the test server.

FossilOrigin-Name: 9615c77919fee60d708d72729c741b373a89900aabc318e0ceb4154638cdf339

2 years agoCorrect the problem which triggered the rollback in [7eec635562f6]: an incorrect...
stephan [Fri, 2 Dec 2022 08:51:22 +0000 (08:51 +0000)] 
Correct the problem which triggered the rollback in [7eec635562f6]: an incorrect default db page size (not a multiple of 512 bytes).

FossilOrigin-Name: e06e490c240aa56b616e6f0380b5d08abf06f35f9e683e5150c5ac464eae9e55

2 years agoRoll back the SQLITE_DEFAULT_PAGE_SIZE part of [c260895faacb34] because kvvfs does...
stephan [Fri, 2 Dec 2022 08:38:04 +0000 (08:38 +0000)] 
Roll back the SQLITE_DEFAULT_PAGE_SIZE part of [c260895faacb34] because kvvfs does not work at all with a page size of 8kb.

FossilOrigin-Name: 7eec635562f65592121d7ff0a31a7b4ff6cdfe38e657f326302ef851b3b37c3f

2 years agosqlite3-wasm.c: code legibility and coding style tweaks. Increase SQLITE_DEFAULT_PAGE...
stephan [Fri, 2 Dec 2022 08:29:03 +0000 (08:29 +0000)] 
sqlite3-wasm.c: code legibility and coding style tweaks. Increase SQLITE_DEFAULT_PAGE_SIZE from 4k to 8k, as that improves OPFS speedtest1 performance by roughly 12%.

FossilOrigin-Name: c260895faacb3458c557778630756d02a8520c0f1864bddcf86cdd27ef4a42bd

2 years agoExpand JS tests for db export/import and document reason it cannot currently work...
stephan [Fri, 2 Dec 2022 07:14:56 +0000 (07:14 +0000)] 
Expand JS tests for db export/import and document reason it cannot currently work with kvvfs. Fix a minor JS build dependencies bug. Update page title with PASS/FAIL prefix for tester1.js to improve overview when launching multiple test tabs. Add ability of tester1 should-run-test predicates to report why a given test is disabled.

FossilOrigin-Name: 75f610d3a4cf3d972220f9abc27cdf5990451e3835ceb9cf66973934004dfc5c

2 years agoRemove extraneous/unused sqlite3.oo1.version object. Add httpd makefile target.
stephan [Fri, 2 Dec 2022 03:37:49 +0000 (03:37 +0000)] 
Remove extraneous/unused sqlite3.oo1.version object. Add httpd makefile target.

FossilOrigin-Name: 8e4d30ac033a6d9019a7eeedfe788dc0120f565cef2ae8f09d2bf32eb94d8a33

2 years agosqlite3_js_create_file() now accepts an ArrayBuffer data source. Add test for OPFS...
stephan [Thu, 1 Dec 2022 15:22:03 +0000 (15:22 +0000)] 
sqlite3_js_create_file() now accepts an ArrayBuffer data source. Add test for OPFS-based export/re-import. The (sqlite3*) argument converter now optionally accepts sqlite3.oo1.DB instances.

FossilOrigin-Name: 14a84b67fb17e16a5691ea4bf7f374123ac73a361a5d3d0efca53788d2001e3a

2 years agoReformulate and simplify some JS tests related to the previous checkin.
stephan [Thu, 1 Dec 2022 04:45:51 +0000 (04:45 +0000)] 
Reformulate and simplify some JS tests related to the previous checkin.

FossilOrigin-Name: 9ea2d3dcf798393a7fd231e199c0e2c6302949fe2a7f2573178fb0e50c78a2f4

2 years agoExpand "sqlite3_vfs*" JS-to-WASM function argument conversions to accept VFS names...
stephan [Thu, 1 Dec 2022 03:55:28 +0000 (03:55 +0000)] 
Expand "sqlite3_vfs*" JS-to-WASM function argument conversions to accept VFS names (JS strings) and capi.sqlite3_vfs instances. Implement sqlite3_js_vfs_create_file() to facilitate creation of file-upload features which store the file in VFS-specific storage (where possible, e.g. "unix" and "opfs" VFSes). Correct an argument type check in the SQLite3Error and WasmAllocError constructors.

FossilOrigin-Name: e1009b16d351b23676ad7bffab0c91b373a92132eb855c9af61991b50cd237ed

2 years agoAdd a testcase() macro to verify that the case of a NOT NULL error message
drh [Wed, 30 Nov 2022 21:18:23 +0000 (21:18 +0000)] 
Add a testcase() macro to verify that the case of a NOT NULL error message
hitting the string length limit.

FossilOrigin-Name: 91f50964c10fb12d889bda7d597d8edf475d97d2d8b534b4400e0fed1d753c6a

2 years agoFix an over-zealous assert() reported by Yong Heng.
drh [Wed, 30 Nov 2022 20:22:31 +0000 (20:22 +0000)] 
Fix an over-zealous assert() reported by Yong Heng.

FossilOrigin-Name: 6ee61f8cede4998f0c838d6506b058c6b09f34b3d7f30ed296100785c93f8d00

2 years agoAttempt to rationalize the bits associated with ".wheretrace". Provide
drh [Wed, 30 Nov 2022 19:11:31 +0000 (19:11 +0000)] 
Attempt to rationalize the bits associated with ".wheretrace".  Provide
a decoder key in sqliteInt.h for what each bit is intended to do.

FossilOrigin-Name: 8ec361695a107a94f2cf6a7fe509656a99d85d49bd7c74133c69903f059a2675

2 years agoRename some JS files from X.js to X.c-pp.js to keep the maintainer, and downstream...
stephan [Wed, 30 Nov 2022 18:21:01 +0000 (18:21 +0000)] 
Rename some JS files from X.js to X.c-pp.js to keep the maintainer, and downstream build customizers, aware that those files contain constructs specific to the c-pp preprocessor and will not run as-is in JS.

FossilOrigin-Name: 2eade7c7b17a186735c72974c11a34798a08364861d0f307e897ba765c0a93c7

2 years agoAlways use nanosleep() (instead of usleep() or sleep) if the
drh [Wed, 30 Nov 2022 13:44:31 +0000 (13:44 +0000)] 
Always use nanosleep() (instead of usleep() or sleep) if the
_POSIX_C_SOURCE macro says it should be available.

FossilOrigin-Name: 6620c57b9d3eae7226a412318b43393196df069b5b90aae0cf1743fdd2d102dd

2 years agoInstall sqlite3_malloc/sqlite3_free() as the JS-side WASM allocator (as opposed to...
stephan [Wed, 30 Nov 2022 11:50:16 +0000 (11:50 +0000)] 
Install sqlite3_malloc/sqlite3_free() as the JS-side WASM allocator (as opposed to replacing C-level's malloc()/free() with them). All tests work and this eliminates the potential for allocator discrepancies when using the (de)serialize APIs.

FossilOrigin-Name: 95c78f6b46e0d8efa4313061f47677479f48610b7a7261dc8d0fb1859aca2ad9

2 years agoSimplify how the OPFS VFS async proxy copies initial state sent to it from the synchr...
stephan [Wed, 30 Nov 2022 08:37:17 +0000 (08:37 +0000)] 
Simplify how the OPFS VFS async proxy copies initial state sent to it from the synchronous side of the connection. Make the lock-wait time a multiple of the wait-loop timeout interval.

FossilOrigin-Name: eddafafffa634a42ceeed70aa3fc58be130527612157a4bf4ff9e65c7f6dc26c

2 years agoOPFS VFS: remove an invalid TODO and fix a property name typo which caused xCheckRese...
stephan [Wed, 30 Nov 2022 07:48:35 +0000 (07:48 +0000)] 
OPFS VFS: remove an invalid TODO and fix a property name typo which caused xCheckReservedLock() to always report false.

FossilOrigin-Name: 3b037caa2fa07b6c44c485574e9e5dc71f4a8e82bc902c1321bb0b918b139c74

2 years agoRename some OPFS JS files. Prevent JS bindings of sqlite3_uri_...() from performing...
stephan [Wed, 30 Nov 2022 07:17:29 +0000 (07:17 +0000)] 
Rename some OPFS JS files. Prevent JS bindings of sqlite3_uri_...() from performing JS-to-C-string argument conversion on their first argument, as doing so is specifically illegal.

FossilOrigin-Name: 79832808de2cbdba140ed9e0558f1502b51d131ab4315265315922cda7b748cb

2 years agoRefactor a significant chunk of the OPFS sqlite3_vfs init code into sqlite3.VfsHelper...
stephan [Wed, 30 Nov 2022 05:27:36 +0000 (05:27 +0000)] 
Refactor a significant chunk of the OPFS sqlite3_vfs init code into sqlite3.VfsHelper, and internal-use-only API encapsulating code relevant to creating new VFSes in JS. Intended to assist in pending experimentation with an alternative OPFS VFS.

FossilOrigin-Name: e25d7b080a807e35b32cb885ea75b384130e5c6e936dfef783c5b45d9bfe77d8

2 years agoDoc and logging text tweaks in the OPFS async proxy and test app.
stephan [Wed, 30 Nov 2022 03:08:50 +0000 (03:08 +0000)] 
Doc and logging text tweaks in the OPFS async proxy and test app.

FossilOrigin-Name: 7ce8608e221924d2c7067687eb6eef0f3cab181d5b4132e55a67d8514b6ce94b

2 years agosqlite3.oo1.OpfsDb: default to journal_mode=persist, as current tests show it to...
stephan [Tue, 29 Nov 2022 18:28:40 +0000 (18:28 +0000)] 
sqlite3.oo1.OpfsDb: default to journal_mode=persist, as current tests show it to be marginally faster than truncate/delete in Chrome v109. Also increase default busy_timeout from 2 seconds to 3, admittedly on a whim.

FossilOrigin-Name: d0c8fa30a31c691bc1be5e98d806eeb1e23a8fc6cd54d87e5c1b720aa936e707

2 years agoOPFS speedtest1: hide a currently-broken/to-fix WASMFS-build link.
stephan [Tue, 29 Nov 2022 18:00:01 +0000 (18:00 +0000)] 
OPFS speedtest1: hide a currently-broken/to-fix WASMFS-build link.

FossilOrigin-Name: cde95d382f8debcbca27c4aaf08470ffda35ab10d723a887786669367590ad3c

2 years agoChange the handling of hwtime.h to make it easier to compile performance
drh [Tue, 29 Nov 2022 17:52:04 +0000 (17:52 +0000)] 
Change the handling of hwtime.h to make it easier to compile performance
measurement builds that make use of hwtime.h.  This should not affect
productions builds.

FossilOrigin-Name: f64a224244743ab121371abd516fccbfc93c110e0952211764bd1b217e792c1b

2 years agoFix legacy build system main.mk to include basexx.c in testfixture builds.
dan [Tue, 29 Nov 2022 11:28:45 +0000 (11:28 +0000)] 
Fix legacy build system main.mk to include basexx.c in testfixture builds.

FossilOrigin-Name: 8c9200b7e156206c4270ff60631c57d5898e2769f7f68294ce652bfec4cfa7c2

2 years agoRename one instance of opfs.OpfsDb to oo1.OpfsDb, as per [0c5c51f4fb04].
stephan [Tue, 29 Nov 2022 06:56:09 +0000 (06:56 +0000)] 
Rename one instance of opfs.OpfsDb to oo1.OpfsDb, as per [0c5c51f4fb04].

FossilOrigin-Name: 75b04c9b302ec66749d8f072a49e61de9640b868bec99eadeb5b4bdef354c336

2 years agoMinor internal cleanups and docs in the OPFS sqlite3_vfs.
stephan [Tue, 29 Nov 2022 06:09:32 +0000 (06:09 +0000)] 
Minor internal cleanups and docs in the OPFS sqlite3_vfs.

FossilOrigin-Name: 61799b05ff232c2ac349169c27bfe7f8d9277366093b0c9dd2739828993b3066

2 years agoInternal restructuring of the OPFS sqlite3_vfs in order to facilitate certain experim...
stephan [Tue, 29 Nov 2022 05:25:08 +0000 (05:25 +0000)] 
Internal restructuring of the OPFS sqlite3_vfs in order to facilitate certain experimentation and improve error reporting/hints if it cannot be activated. Deprecate the name sqlite3.opfs.OpfsDb, preferring sqlite3.oo1.OpfsDb for consistency with JsStorageDb and any future DB subclasses.

FossilOrigin-Name: 0c5c51f4fb04a4b90c50ec9704cfea9a3fb7d7d0ee55c1b0d4476129188217a6

2 years agoAdd an explicit warning about the current API-instability of the sqlite3.opfs namespa...
stephan [Tue, 29 Nov 2022 02:23:12 +0000 (02:23 +0000)] 
Add an explicit warning about the current API-instability of the sqlite3.opfs namespace, which may need to be eliminated based on re-thinking of how the OPFS sqlite3_vfs is registered. Comment changes only - no code.

FossilOrigin-Name: 0cb2fd14179397051a25d066256a553fc198656d5668c7010c016f2b8f495bf4

2 years agoCheck-in [8d5b76593d82b3a5] contained an error that was causing some obscure
drh [Mon, 28 Nov 2022 21:17:30 +0000 (21:17 +0000)] 
Check-in [8d5b76593d82b3a5] contained an error that was causing some obscure
error codes to be lost.  Fixed here.

FossilOrigin-Name: 46cdd3637d6a206ad2bcf8653cc6f2c7a886a16cc7685c45967938609941a755

2 years agoGet ext/misc/basexx.c into the testfixture.exe build for MSC.
larrybr [Mon, 28 Nov 2022 21:09:49 +0000 (21:09 +0000)] 
Get ext/misc/basexx.c into the testfixture.exe build for MSC.

FossilOrigin-Name: bb2c5d088e4784f6763eb0ea2a8542ad2129529f181fa92963c7231d7a68f25c

2 years agoFix harmless compiler warnings.
drh [Mon, 28 Nov 2022 20:08:15 +0000 (20:08 +0000)] 
Fix harmless compiler warnings.

FossilOrigin-Name: 8d5b76593d82b3a57bc904ced33c24cd49369868b5dd2fe7f2c7b114f5aee2f6

2 years agoImplementation of enhancement request [695a1a53dea6b240]: The query planner
drh [Mon, 28 Nov 2022 19:42:48 +0000 (19:42 +0000)] 
Implementation of enhancement request [695a1a53dea6b240]: The query planner
has improved awareness of when indexes on expressions are covering and adjusts
their costs accordingly.

FossilOrigin-Name: e3474d79b27298e96e7686e5a6f9a8e736b8a6568a0a08fb1abe5bcca038ca63

2 years agoAdd the SQLITE_FCNTL_RESET_CACHE verb. Use it to ensure that the page cache is purged...
dan [Mon, 28 Nov 2022 18:41:41 +0000 (18:41 +0000)] 
Add the SQLITE_FCNTL_RESET_CACHE verb. Use it to ensure that the page cache is purged before and after a the recovery extension is run.

FossilOrigin-Name: 6db0bc4bc0d272b610bef2aeeae43f539ed6e7cc0a9cc767d5af85ecb0019d5f

2 years agoTest cases added derived from the enhancment ticket. covering-indexed-expr
drh [Mon, 28 Nov 2022 18:17:42 +0000 (18:17 +0000)] 
Test cases added derived from the enhancment ticket.

FossilOrigin-Name: c022c0152ad61a4f56e36f4062609073e2273fbf6f826c20652159be229c2d46

2 years agoMerge trunk changes into covering-index-expr branch.
drh [Mon, 28 Nov 2022 16:11:10 +0000 (16:11 +0000)] 
Merge trunk changes into covering-index-expr branch.

FossilOrigin-Name: 89d775ada39aed4dc532374ace02156d07dc06e8ae54a194608af0c5a582d20f

2 years agoRework the covering index checking routine, whereIsCoveringIndex(), so that
drh [Mon, 28 Nov 2022 15:23:53 +0000 (15:23 +0000)] 
Rework the covering index checking routine, whereIsCoveringIndex(), so that
it can return a "maybe" result for aggregate queries where we are not exactly
sure.  The index is scored as if it is covering, but the main table is
still opened.

FossilOrigin-Name: b8eec4214363192e6f3e12b3faa5810d8269a5fdaecab3ec09b02e5002cf798a

2 years agoAdd (optional) base64 and base85 UDF extensions.
larrybr [Mon, 28 Nov 2022 14:51:50 +0000 (14:51 +0000)] 
Add (optional) base64 and base85 UDF extensions.

FossilOrigin-Name: b44ab10c49bc2895483a9d40813be3798710ee713cc4bf04e449dce55a68452a

2 years agoSync w/trunk, zap surplus space. base_convert
larrybr [Mon, 28 Nov 2022 14:11:48 +0000 (14:11 +0000)] 
Sync w/trunk, zap surplus space.

FossilOrigin-Name: b8345630a2a322234bda49ee4b996f6ba20e2b080621e229a2ec5e820892a663

2 years agoEnsure the RETURNING clause is honoured when a row of a temp table is updated by...
dan [Mon, 28 Nov 2022 13:47:27 +0000 (13:47 +0000)] 
Ensure the RETURNING clause is honoured when a row of a temp table is updated by an ON CONFLICT clause.

FossilOrigin-Name: a2449bcc2c71d0f4c3289621fbf1cb97f0f407c9f7b5bf18245b7854a07c6cfa

2 years agoConform CLI .trace arg handling to its help.
larrybr [Mon, 28 Nov 2022 02:28:44 +0000 (02:28 +0000)] 
Conform CLI .trace arg handling to its help.

FossilOrigin-Name: 31546ea320e8daa020f8a9f491718ffc3dde0e32954018afed88e8494a6aff6c

2 years agoDocument an OPFS API change in Chrome v108 which does not break our code but does...
stephan [Sun, 27 Nov 2022 00:57:56 +0000 (00:57 +0000)] 
Document an OPFS API change in Chrome v108 which does not break our code but does change several formerly async methods to synchronous. No code changes.

FossilOrigin-Name: e9dd87e28a96eb6560943b9a2cb3cb0ff14698c96fe63865944dcfed73b67bd3

2 years agoBetter reporting of when the WHERE clause analysis thinks that an index
drh [Sat, 26 Nov 2022 20:52:38 +0000 (20:52 +0000)] 
Better reporting of when the WHERE clause analysis thinks that an index
is covering.

FossilOrigin-Name: 17ebcf316b91042c823eea7bb6f1309325023cb5c70538cdb2ce932caee2ef05

2 years agoShow which AggInfo.aCol an Expr node represents in the tree trace debugging
drh [Sat, 26 Nov 2022 20:48:56 +0000 (20:48 +0000)] 
Show which AggInfo.aCol an Expr node represents in the tree trace debugging
logic.  Does not affect production builds.

FossilOrigin-Name: b8076f4dc55810855f668dfd4221bdb555cc78c61df200cf0cb87eb137e22850

2 years agoMerge trunk fixes into the covering-indexed-expr branch.
drh [Sat, 26 Nov 2022 20:13:54 +0000 (20:13 +0000)] 
Merge trunk fixes into the covering-indexed-expr branch.

FossilOrigin-Name: 9ac67ff968e874cf955e46e3993e3215c766feec3d5bdd38d77884eedd86b59e

2 years agoFix an incorrect assertion-fault in the "TREETRACE" debugging logic on
drh [Sat, 26 Nov 2022 20:12:02 +0000 (20:12 +0000)] 
Fix an incorrect assertion-fault in the "TREETRACE" debugging logic on
select.c.  The problem does not affect production builds.

FossilOrigin-Name: f11bb42292d94e7b7687bd2134f72afe1353182238cb4837fc0a6d78a021dd7e

2 years agoOPFS VFS: add the opfs-unlock-asap=1 URI flag which tells the VFS to release implicit...
stephan [Sat, 26 Nov 2022 15:24:58 +0000 (15:24 +0000)] 
OPFS VFS: add the opfs-unlock-asap=1 URI flag which tells the VFS to release implicit locks ASAP instead of during VFS idle time. This improves concurrency notably in the test app but brings a significant performance penalty in speedtest1 (roughly 4x slowdown). This is not the final word in OPFS concurrency, but gets us a step further.

FossilOrigin-Name: 9542f9ce9e023b489e2d93661f719fb0751c1e28f72fded9d3c2156d5777e7b1

2 years agoExperimental changes to help the query planner detect when an expression
drh [Sat, 26 Nov 2022 14:19:47 +0000 (14:19 +0000)] 
Experimental changes to help the query planner detect when an expression
index is coverting.  Works somewhat, but there are tests that fail.

FossilOrigin-Name: 968c189bcf29a9b25305251a58d09b7d52ab9dd08f5057dc3ab1f7ad1a5316a0

2 years agoRelax restriction (8) on the push-down optimization so that it only applies
drh [Fri, 25 Nov 2022 17:05:55 +0000 (17:05 +0000)] 
Relax restriction (8) on the push-down optimization so that it only applies
if one or more columns uses a collating sequence other than BINARY.
See [forum:/forumpost/3824ced748baa808|forum post 3824ced748baa808] and
check-in [346a3b12b861ce7b].

FossilOrigin-Name: adbca3448e2099f0d6149a073978f230ed9a92a2f384779879ef89e672231bcf

2 years agoFix #ifdefs so that restrictions (8) and (9) of the push-down optimization
drh [Fri, 25 Nov 2022 16:32:59 +0000 (16:32 +0000)] 
Fix #ifdefs so that restrictions (8) and (9) of the push-down optimization
are still enforced even if compiled with SQLITE_OMIT_WINDOWFUNC.  This
fixes a bug introduced by check-in [346a3b12b861ce7b].

FossilOrigin-Name: 09e1e42e0ff26f9a71cbd128169f060a66425828d637bf8f781490ca38d99103

2 years agoImmprove the query planner such that it is able to make use of indexed
drh [Fri, 25 Nov 2022 16:10:48 +0000 (16:10 +0000)] 
Immprove the query planner such that it is able to make use of indexed
expressions within an aggregate query with GROUP BY.  This implements
enhancement request [99378177930f87bd].

FossilOrigin-Name: b9190d3da70c41717eb188474fd225ee43d0b46646e1b03de5967bd332553870

2 years agoAdd restriction (9) to the push-down optimization: If the subquery is
drh [Fri, 25 Nov 2022 15:52:00 +0000 (15:52 +0000)] 
Add restriction (9) to the push-down optimization:  If the subquery is
a compound then all arms of the compound must have the same affinity.
dbsqlfuzz 3a548de406a50e896c1bf7142692d35d339d697f.

FossilOrigin-Name: 1ad41840c5e0fa702ba2c0df77a3ea126bd695b910b5d1271fa3129c38c58f5f

2 years agoRemove a redundant assert() statement. agg-with-indexed-expr
drh [Fri, 25 Nov 2022 13:26:51 +0000 (13:26 +0000)] 
Remove a redundant assert() statement.

FossilOrigin-Name: e3e1b453dc47884ddc9e51c4302fa2f4d40bca3d69ac7c13d8e2ae1adc81ac56

2 years agoImprove the robustness of the updateAccumulator() routine against OOM.
drh [Fri, 25 Nov 2022 13:15:48 +0000 (13:15 +0000)] 
Improve the robustness of the updateAccumulator() routine against OOM.

FossilOrigin-Name: d2844f577b69fdc16a0a2568c0958fc3d8aff33e9a0ef80e0f58e92f01097432

2 years agoImproved comments. Add assert()s to verify that the AggInfo structure
drh [Fri, 25 Nov 2022 13:08:20 +0000 (13:08 +0000)] 
Improved comments.  Add assert()s to verify that the AggInfo structure
is unchanged after registers have been assigned.

FossilOrigin-Name: 5200b84195ee1ccaa387f7032eae3d463724c48cb53ba0251bbc79e927dd9752