drh [Fri, 20 Mar 2020 16:13:41 +0000 (16:13 +0000)]
Revamp the EXPLAIN infrastructure to facilitate sqlite3_stmt_mode(). The
currently code mostly works, but there are test failures. This is an
incremental check-in.
dan [Mon, 16 Mar 2020 18:52:53 +0000 (18:52 +0000)]
Fix handling of window functions in aggregate queries that have no GROUP BY clause. Also remove a faulty assert causing the error reported in [618156e3].
drh [Mon, 16 Mar 2020 03:07:53 +0000 (03:07 +0000)]
At the end of the right-hand table loop of a LEFT JOIN that uses an IN
operator in the ON clause, put the OP_IfNoHope operator after the
OP_IfNotOpen operator, not before, to avoid a (harmless) uninitialized
register reference. Ticket [82b588d342d515d1]
drh [Wed, 11 Mar 2020 19:56:26 +0000 (19:56 +0000)]
Do not factor out constant functions into the initialization section of a
prepared statement, because even though they are constant, they can still
throw exceptions. Instead, put such functions in an OP_Once block. This
fixes ticket [3c9eadd2a6ba0aa5] and causes COALESCE() and CASE...END to
be short-circuit.
drh [Wed, 11 Mar 2020 17:58:27 +0000 (17:58 +0000)]
Do not factor out constant functions into the initialization section at the
end of the prepared statement, be cause if they throw an exception, it will
abort the statement even if the function is never called. Better to put
constant functions in an OP_Once block.
drh [Tue, 10 Mar 2020 18:55:41 +0000 (18:55 +0000)]
Further changes to ensure that expressions held in table and index definitions
do not get passed down into code generator logic where they might be modified.
drh [Tue, 10 Mar 2020 02:57:37 +0000 (02:57 +0000)]
The sqlite3ExprCodeFactorable() routine should make a copy of non-factorable
expressions, as they might be coming from a DEFAULT or generated column
in a table constraint.
dan [Thu, 5 Mar 2020 18:04:09 +0000 (18:04 +0000)]
Report an error if the main, or any other, database encoding is modified by an external process (perhaps using the backup API) after the db has been opened.
drh [Thu, 5 Mar 2020 16:13:24 +0000 (16:13 +0000)]
Change the sqlite3.pDfltColl (the default collating sequence for the
database connection) so that it is the collating sequence appropriate for
the database encoding, not the UTF8 collating sequence. This helps to
ensure that the database encoding collation is always used, even for
expressions that do not have an defined collating sequence.
Ticket [1b8d7264567eb6fc].
drh [Tue, 3 Mar 2020 20:04:29 +0000 (20:04 +0000)]
Remove an invalid assert() on the debugging logic that checks to ensure that
register values are not used after they go stale. Ticket [d165ad781b39d574].
drh [Fri, 28 Feb 2020 16:04:28 +0000 (16:04 +0000)]
The RTREE extension behaves has if data columns have type REAL, so we
should actually declare them as REAL so that automatic indexes handle
them correctly. Ticket [e63b4d1a65546532]
dan [Thu, 27 Feb 2020 17:16:45 +0000 (17:16 +0000)]
In sqlite3changeset_apply(), ensure that DELETE and UPDATE changes are always executed on main database tables, not similarly named temp tables, as documented. INSERT statements are already being handled correctly.
dan [Thu, 27 Feb 2020 15:07:16 +0000 (15:07 +0000)]
Optimization for "SELECT min(x) FROM tbl" where "x" is indexed and NOT NULL. This also allows similar queries on NOT NULL virtual table columns to be optimized.
drh [Thu, 27 Feb 2020 13:54:18 +0000 (13:54 +0000)]
Extra zero terminators on the end of the blank filename returned by
sqlite3PagerFilename() for an in-memory database. This helps the result
work better with sqlite3_filename_journal() and similar functions.
drh [Thu, 27 Feb 2020 11:32:14 +0000 (11:32 +0000)]
Ensure that the filename passed into the xFullPathname method of the VFS is
acceptable as an argument to sqlite3_uri_parameter(). The interface spec does
not guarantee this, but it has been so historically and some applications
have come to depends on it.
drh [Tue, 25 Feb 2020 20:05:58 +0000 (20:05 +0000)]
Update the fuzzcheck test module so that it avoids inserting text values
that contain embedded NULs in the XSQL table. Fix some legacy entries in
the test/fuzzdata8.db that had embedded NULs. Add in new dbsqlfuzz cases
that have accumulated over on the dbsqlfuzz project for a while.
drh [Mon, 24 Feb 2020 17:05:09 +0000 (17:05 +0000)]
If STAT4 determines that a WHERE clause term that is not used by an index
has very high probability of being true, then do not use that term to reduce
the estimated output row count.
drh [Mon, 24 Feb 2020 16:46:08 +0000 (16:46 +0000)]
Rework this changes so that instead of setting the WhereTerm.truthProb when
a term is seen to be of low selectivity, it merely sets a new flag
(the TERM_HIGHTRUTH flag) which causes whereLoopOutputAdjust() to ignore
that term.
drh [Sat, 22 Feb 2020 18:27:48 +0000 (18:27 +0000)]
Do not activate the truthProb adjustment mechanism if the truth probability
is less than the heuristic value, as there could be correlations unknown to
stat4. Also add additional tracing output to make truthProb adjustments more
visible.
drh [Sat, 22 Feb 2020 13:01:19 +0000 (13:01 +0000)]
In the OP_Column opcode, if the cursor is marked NullRow (due to being the
right table of a LEFT JOIN that does not match) and the cursor is the table
cursor for an OR-optimization with a covering index, then do not substitute
the covering index cursor, since the covering index cursor does not have
the NullRow flag set. Ticket [aa4378693018aa99]
drh [Mon, 17 Feb 2020 00:12:04 +0000 (00:12 +0000)]
Take care when checking the table of a TK_COLUMN expression node to see if the
table is a virtual table to first ensure that the Expr.y.pTab pointer is not
null due to generated column optimizations. Ticket [4374860b29383380].
dan [Wed, 12 Feb 2020 11:57:35 +0000 (11:57 +0000)]
When determining whether an == or IS constraint in a WHERE clause makes an ORDER BY term redundant, consider the collation sequence used by the == or IS comparison, not the collation sequence of the comparison expression itself. Possible fix for [fb8c538a8f].
drh [Wed, 5 Feb 2020 18:28:17 +0000 (18:28 +0000)]
Small size reduction and performance improvement in the
sqlite3VdbeMemFromBtree() interface used to pull content out of the b-tree
and into an sqlite3_value object.
drh [Tue, 4 Feb 2020 01:41:44 +0000 (01:41 +0000)]
Extend the OP_Copy-coalesce optimization fix of check-in [b36126c1889e323c]
so that it is also correctly disabled by the CASE operator.
Ticket [9d3666754ac37d5a].
drh [Mon, 3 Feb 2020 19:56:51 +0000 (19:56 +0000)]
Cosmetic change to the xfer-optimization to put the OP_RowData opcode closer
to the corresponding OP_Insert opcode. This makes it slightly easier to read
and understand the code.
drh [Sat, 1 Feb 2020 17:38:24 +0000 (17:38 +0000)]
Fix the comment display for the OP_Function opcode. And at the same time,
improve the comment generating logic to make use of the newer
sqlite3_str_appendf() interface.
dan [Wed, 29 Jan 2020 15:03:01 +0000 (15:03 +0000)]
Fix a problem with the processing of IN(...) constraints handled by virtual table implementations that do not set the "omit" flag when the virtual table column contains at least one NULL value.
drh [Tue, 28 Jan 2020 18:09:53 +0000 (18:09 +0000)]
Reinstate the optimization of converting "a IN (C)" into "a=C" but only
if C is a constant. If the RHS is a table column, the complications of
managing affinity and collations become too involved to mess with.
drh [Mon, 27 Jan 2020 14:40:44 +0000 (14:40 +0000)]
Revise the layout of filenames in the Pager object so that it is unchanged
from prior versions. It turns out that some important 3rd-party software
does questionable pointer manipulations on those filenames that depend on
that legacy layout. Technical this is a misuse of SQLite by the 3rd-party
software, but we want to avoid unnecessary breakage.