drh [Thu, 31 Mar 2022 11:12:56 +0000 (11:12 +0000)]
Refinement to [3f9887d4a58cbfdb]: The SQLITE_CONSTRAINT error should only
be returned if the prepared statement came from sqlite3_prepare_v2(). The
legacy sqlite3_prepare() is documented to return only SQLITE_ERROR.
drh [Tue, 29 Mar 2022 22:57:00 +0000 (22:57 +0000)]
In setResultStrOrError(), if the input string pointer is NULL and hence the
value gets set to an SQL NULL, then the Mem.enc field is uninitialized. So
do not read it. This is a harmless OSSFuzz/ASAN found problem.
drh [Tue, 29 Mar 2022 01:43:09 +0000 (01:43 +0000)]
Fix the sqlite3_result_xxxxx() routines so that they all check for and
perform any necessary text encoding conversions and check for oversize
strings and BLOBs. Thus those checks can be done where they are most
efficient and avoided in cases like OP_Function where they are more
expensive.
drh [Fri, 25 Mar 2022 01:23:37 +0000 (01:23 +0000)]
Fix the Bloom-filter optimization so that it does not use IS NULL or
IS NOT NULL constraints from the WHERE clause when operating on a LEFT JOIN.
[forum:/forumpost/031e262a89b6a9d2|Forum thread 031e262a89b6a9d2].
drh [Wed, 23 Mar 2022 10:04:52 +0000 (10:04 +0000)]
Use trickery to code the UTF-8 BOM for the --bom option on .output, .once,
and .excel in the CLI, to work around a warning in some Microsoft compilers.
[https://fossil-scm.org/forum/forumpost/bd0844268f0fab71|Fossil forum post bd0844268f0fab71].
drh [Mon, 21 Mar 2022 18:17:09 +0000 (18:17 +0000)]
Strengthen the defenses against corrupt databases in the
sqlite3BtreeInsert() function of the btree module.
[forum:/forumpost/c7ec29905f|Forum post c7ec29905f].
drh [Mon, 21 Mar 2022 13:47:15 +0000 (13:47 +0000)]
Fix the assert() that attempts to verify that the table-reference to
index-reference translator finds all required translations.
[forum:/forumpost/929168fdd6|Forum post 929168fdd6].
drh [Mon, 14 Mar 2022 16:54:05 +0000 (16:54 +0000)]
Foreign key constraint failures should return SQLITE_CONSTRAINT_FOREIGNKEY
even if there is a RETURNING clause. See
[forum:/forumpost/e6be6e82c86aa59b|forum thread e6be6e82c86aa59b].
drh [Thu, 10 Mar 2022 16:26:00 +0000 (16:26 +0000)]
Prevent a NULL-pointer dereference when trying to parse a illegal
schema entry that contains a window function while doing a RENAME COLUMN.
[forum:/forumpost/ec2a2e0deb|Forum post ec2a2e0deb].
drh [Wed, 9 Mar 2022 12:20:40 +0000 (12:20 +0000)]
Improve the defenses against bad pathnames input into the findCreateFileMode()
function of os_unix.c in order to quiet static-analyzer warnings. There
are no demonstrated problems in the prior code, but this change makes the code
easier to prove correct and more robust against future changes.
drh [Mon, 7 Mar 2022 17:19:40 +0000 (17:19 +0000)]
In the stay-on-last-page optimization for sqlite3BtreeIndexMoveto()
(check-in [0057bbb508e7662b] about 16 hours ago), be sure
to clear the BTCF_ValidOvfl flag, since the overflow cache is invalidated
by the search on the last page. OSSFuzz issue 45329.
drh [Mon, 7 Mar 2022 16:22:31 +0000 (16:22 +0000)]
Do not allocate new Trigger objects in the parser following a syntax error,
to avoid violating invariants associated with Expr nodes. See
[forum:/forumpost/2024e94071ef1531|forum thread 2024e94071ef1531] for more
information.
drh [Mon, 7 Mar 2022 14:51:29 +0000 (14:51 +0000)]
Fix the code generated for vector IN operator constraints on virtual tables
so that they work even if the "omit" field in the sqlite3_index_info object
is off. This has apparently never worked correctly before. Presumably, nobody
has ever before written a virtual table that can use vector IN operator
constraints and that relies on bytecode to double-check the constraints.
Test cases in TH3. Problem discovered by
dbsqlfuzz cab8e26194a40147627094f3c6849c0a7b1e0310.
drh [Mon, 7 Mar 2022 01:29:36 +0000 (01:29 +0000)]
Optimizations to sqlite3BtreeIndexMoveto() avoid unnecessary comparisons if
the cursor is already near the end of the table and is not moving far. This
case is more common that you would expect. The optimization saves almost
4 million CPU cycles.
drh [Sat, 5 Mar 2022 20:12:53 +0000 (20:12 +0000)]
Fix the 'localtime' modifier in date/time functions so that it preserves
fractional seconds.
[forum:/forumpost/2ffbaa2c3fd7fb82|Forum post 2ffbaa2c3fd7fb82].
drh [Sat, 5 Mar 2022 14:44:12 +0000 (14:44 +0000)]
Fix the sqlite_offset() function so that it gives the correct answer even
if the argument is a virtual column in an index-only query. Test cases
in TH3.
drh [Fri, 4 Mar 2022 20:54:09 +0000 (20:54 +0000)]
Do try to evaluate the Bloom filter on a LEFT JOIN early, as doing so
essentially converts it into an INNER JOIN. See
[forum:/forumpost/544af7eee2|forum thread 544af7eee2].
drh [Fri, 4 Mar 2022 16:28:24 +0000 (16:28 +0000)]
When setting an sqlite3_value object to a pointer value, make sure any prior
memory allocations associated with that object have been cleared first.
dbsqlfuzz 33f842d1a09afaad5f078c3e1162a54b78e5f2ab.
drh [Thu, 3 Mar 2022 16:48:35 +0000 (16:48 +0000)]
Make sure the xParseCell and xCellSize methods of the MemPage object are
initialized consistently even if the page is detected as being corrupt.
dbsqlfuzz fd21f341f3b4f582401d2feb2a1c0c4cc2c26caa.
drh [Thu, 3 Mar 2022 15:59:22 +0000 (15:59 +0000)]
Fix for the problem identified in
[forum:/forumpost/0cd8e058bf|forum post 0cd8e058bf]:
When evaluating an multi-index OR, do not push down auxiliary WHERE clause
terms that involve subqueries into the OR-subqueries. Otherwise, the
covering-index optimizer might convert table-references into index-references
for the particular OR index that is active for the branch in which the
subquery subroutine is coded, and those index-references
will not work if the subquery subroutine is invoked from a different OR branch
that uses a different index.
drh [Thu, 3 Mar 2022 15:00:44 +0000 (15:00 +0000)]
Add the new OP_BeginSubrtn opcode (which is really an alias for OP_Integer)
and make other changes so that the span of a subroutine that implements
a subquery is more readily apparent in bytecode listings.
drh [Wed, 2 Mar 2022 21:04:10 +0000 (21:04 +0000)]
Bloom filter pull-down optimization is incompatible with Skip-Scan.
Make sure the query planner does not try to to both.
[forum:/info/50a1bbe08ce4c29c|Forum post 50a1bbe08ce4c29c].
drh [Tue, 1 Mar 2022 20:15:04 +0000 (20:15 +0000)]
The MemPage.aDataEnd field should point to the end of the data buffer for
the page, not just the end of the usable portion of that buffer. The purpose
aDataEnd is to detect cells that overflow the page, and that won't work on a
page with reserved bytes and a cell that starts in the reserved region, unless
the boundary is at the very end of the page. Chromium issue 1276294.
drh [Tue, 1 Mar 2022 16:22:54 +0000 (16:22 +0000)]
Fix two assert() statements in btree.c which were not true in the case of
a corrupt database file.
[forum:/forumpost/14819f2063|Forum post 14819f2063].
drh [Tue, 1 Mar 2022 14:13:32 +0000 (14:13 +0000)]
Increase the max_page_count on ROLLBACK, if necessary, so that it is sufficient
to cover the entire database. Fix for the problem identified by
[forum:/forumpost/3b9e894312|forum post 3b9e894312].
drh [Mon, 28 Feb 2022 16:44:58 +0000 (16:44 +0000)]
Fix the Xfer-optimization on the INSERT statement so that it is omitted if
there is a RETURNING clause, since that optimization is not able to deal
with RETURNING. See [forum:/forumpost/595e132f71|forum thread 595e132f71]
for details.
drh [Mon, 28 Feb 2022 12:08:09 +0000 (12:08 +0000)]
The performance increase in the previous check-in of this branch was due to
the revised loop in initMemArray() and reordering fields of Mem - not the
call the memcpy(). Changing the code to avoid memcpy() results in an even
better gain, and code that is far less dodgy.