]> git.ipfire.org Git - thirdparty/sqlite.git/log
thirdparty/sqlite.git
5 years agoEnhancements to aid testing and debugging:
drh [Thu, 2 Jan 2020 14:42:42 +0000 (14:42 +0000)] 
Enhancements to aid testing and debugging:
In PRAGMA vdbe_trace=on output, show pScopyFrom dependencies on register
values.  Add the sqlite3VdbeRegisterDump() procedure, callable from a
debugger, that shows the values of all registers.  Pass the VDBE pointer
into test_trace_breakpoint() so that sqlite3VdbeRegisterDump() is callable
from the breakpoint.

FossilOrigin-Name: 9886cb4b7987f720aa9d701222ab0987caa0ab8c5d216cb6e523c4a45366dfe5

5 years agoAdd the test_trace_breakpoint() subroutine that is invoked after each
drh [Thu, 2 Jan 2020 13:26:49 +0000 (13:26 +0000)] 
Add the test_trace_breakpoint() subroutine that is invoked after each
instruction is printed while running PRAGMA vdbe_trace=on.  Only works for
SQLITE_DEBUG builds.  Also add parameters "pc" and "pOp" to
test_addop_breakpoint() to make it easier to set conditionals.

FossilOrigin-Name: 49a6368c384178653cb3ccb58cc8eff93327c16929bf79eeefeb13a4ce897153

5 years agoUse OP_Copy instead of OP_SCopy to move the results of a scalar subquery.
drh [Thu, 2 Jan 2020 02:50:45 +0000 (02:50 +0000)] 
Use OP_Copy instead of OP_SCopy to move the results of a scalar subquery.

FossilOrigin-Name: 435c272dcf6ed5f3acb564b8f959557145f117b869547b670258cf5a1908ab6b

5 years agoThe sqlite3WhereEnd() call now unwinds all Expr modifications made by the
drh [Thu, 2 Jan 2020 00:45:38 +0000 (00:45 +0000)] 
The sqlite3WhereEnd() call now unwinds all Expr modifications made by the
sqlite3WhereBegin().

FossilOrigin-Name: 7bfd42f1dc0c94f9bb74516e62fec8e39c20f4749f4e0972f66c2886d3c64f73

5 years agoProvide the -DSQLITE_ENABLE_INTERNAL_FUNCTIONS=1 compile-time option. Fix
drh [Wed, 1 Jan 2020 23:02:35 +0000 (23:02 +0000)] 
Provide the -DSQLITE_ENABLE_INTERNAL_FUNCTIONS=1 compile-time option.  Fix
the ".testctrl internal_function" command in the CLI so that it does not
signal an error on a valid input.

FossilOrigin-Name: 8ee2ce92c082771675d0e8be597043cf9f0fd4f8a73d6a1498bf8743d6b3904a

5 years agoWhen generating code for a subquery, make a copy of the Select object and
drh [Wed, 1 Jan 2020 21:14:30 +0000 (21:14 +0000)] 
When generating code for a subquery, make a copy of the Select object and
generate the code out of the copy, in case the code generator makes
modifications to expression and the Select object needs to be reused.

FossilOrigin-Name: 4edddcc0bc8d71e9b8abac67bc3766f1d9143dddd1f59264859ce65e5aa9b8c6

5 years agoEnsure that when code for a scalar SELECT featuring window functions is generated...
dan [Wed, 1 Jan 2020 20:17:15 +0000 (20:17 +0000)] 
Ensure that when code for a scalar SELECT featuring window functions is generated more than once by the planner, separate ephemeral tables are opened for each instance.

FossilOrigin-Name: ce1417325273aba866767349b55d9bbfb61a08e716bebda2122918a9657ee38c

5 years agoFix the sqlite3ExprImpliesNonNullRow() routine so that it correctly
drh [Wed, 1 Jan 2020 16:43:41 +0000 (16:43 +0000)] 
Fix the sqlite3ExprImpliesNonNullRow() routine so that it correctly
handles a numeric comparison of two AND subexpressions.

FossilOrigin-Name: 07e504d5174a1a8b1aa05ec5b44ac81ab3ea706c5a098caa88deb3d489cec8fb

5 years agoNew test-only SQL functions: implies_nonnull_row(), expr_compare(), and
drh [Wed, 1 Jan 2020 15:43:30 +0000 (15:43 +0000)] 
New test-only SQL functions: implies_nonnull_row(), expr_compare(), and
expr_implies_expr().  The SQLITE_TESTCTRL_INTERNAL_FUNCTIONS test-control
is modified to toggle internal function access on and off for a single
database connection.

FossilOrigin-Name: 473892a8eceacf24d57fd0c72ff2a0b8be4e0d75e0af7a30bdb24fbc3b453601

5 years agoFactor out code generation for in-line SQL functions into a separate routine
drh [Wed, 1 Jan 2020 13:55:08 +0000 (13:55 +0000)] 
Factor out code generation for in-line SQL functions into a separate routine
inside of expr.c.

FossilOrigin-Name: 586a65a28fc6cac77309612abc32a2e1017c65e0387f9f438f5d8ac8406da8eb

5 years agoIssue OP_ReleaseReg opcodes against the array of registers used to compute
drh [Wed, 1 Jan 2020 01:26:05 +0000 (01:26 +0000)] 
Issue OP_ReleaseReg opcodes against the array of registers used to compute
index records after the index record has been computed.

FossilOrigin-Name: e3330861b4deab7c7cc09ad8b936d633c600b9fb4e274801efeaaf94e5c860b4

5 years agoRemove an incorrect assert() statement. Ticket [46fcd700b855e6d1]
drh [Tue, 31 Dec 2019 23:17:35 +0000 (23:17 +0000)] 
Remove an incorrect assert() statement.  Ticket [46fcd700b855e6d1]

FossilOrigin-Name: eca7ec9cda4606c4104bcb8da315070b615713db9f3444579c6a3e96b0aad02f

5 years agoAlso set the SQLITE_DIRECTONLY flag on the load_extension() function.
drh [Tue, 31 Dec 2019 18:39:23 +0000 (18:39 +0000)] 
Also set the SQLITE_DIRECTONLY flag on the load_extension() function.

FossilOrigin-Name: 3bd095a53119c368fe30e539983588b27957203344cf427405b9a64784b8eba7

5 years agoSet the SQLITE_DIRECTONLY flag on the fts3_tokenizer() function of FTS3,
drh [Tue, 31 Dec 2019 18:12:36 +0000 (18:12 +0000)] 
Set the SQLITE_DIRECTONLY flag on the fts3_tokenizer() function of FTS3,
thus preventing that function from being called from within a trigger or
view.

FossilOrigin-Name: f3171dc22e4d3a40ca17fe609391d92fc6a997b775bfefa5947aec69c889aa73

5 years agoThe OP_ResultRow opcode releases the SCopy dependences on all its registers,
drh [Tue, 31 Dec 2019 12:18:24 +0000 (12:18 +0000)] 
The OP_ResultRow opcode releases the SCopy dependences on all its registers,
as the values in those registers will not be reused.

FossilOrigin-Name: 1dc83c5d54ca2890112e735e336c209adb8d067d2f647e9f8ae5d58f84a52461

5 years agoBack out the NEVER() from check-in [40d10e7aad5b8992] because it is reachable
drh [Mon, 30 Dec 2019 23:50:19 +0000 (23:50 +0000)] 
Back out the NEVER() from check-in [40d10e7aad5b8992] because it is reachable
after all. Ticket [892575cdba4e1e36]

FossilOrigin-Name: f481636f1a0333c6e98906e5d4433bf1d046380bc29c760e8e7e2c2b3530b956

5 years agoAdd an ALWAYS() to a conditional which is apparently always true.
drh [Mon, 30 Dec 2019 23:41:59 +0000 (23:41 +0000)] 
Add an ALWAYS() to a conditional which is apparently always true.

FossilOrigin-Name: ea44c607b8038999a6bda4d761d95056aa4058d6c2891b7eee0715a50e703490

5 years agoPut a NEVER() on a defensive branch.
drh [Mon, 30 Dec 2019 20:42:53 +0000 (20:42 +0000)] 
Put a NEVER() on a defensive branch.

FossilOrigin-Name: 40d10e7aad5b899206e45d1f8606470a90a07049e15d94c3811cfe695f3a402e

5 years agoFix a typo in a comment. No changes to code.
drh [Mon, 30 Dec 2019 20:42:17 +0000 (20:42 +0000)] 
Fix a typo in a comment.  No changes to code.

FossilOrigin-Name: d4813a8d813f234e819f8ebefbd3ff96d3d69593cc7d716cf0ad6c1a126da827

5 years agoDo not use HIDDEN columns for NATURAL joins. Fix for [7c0e06b16].
dan [Mon, 30 Dec 2019 14:32:27 +0000 (14:32 +0000)] 
Do not use HIDDEN columns for NATURAL joins. Fix for [7c0e06b16].

FossilOrigin-Name: ab09ef427181130be09a087b7e572ad4cfb6b3e1b459769ee5ebf046b3ead682

5 years agoIn ALTER TABLE, rename columns and tables in expressions that are optimized out by...
dan [Mon, 30 Dec 2019 06:55:31 +0000 (06:55 +0000)] 
In ALTER TABLE, rename columns and tables in expressions that are optimized out by the "AND 0" optimization. Doing this also fixes an otherwise harmless assert() failure.

FossilOrigin-Name: a9e0354c992b0287608ddd18fd35fe7e6102a8f293d6e6e1b3488644bcda8168

5 years agoDo not allow triggers that run as part of REPLACE conflict resolution
drh [Sun, 29 Dec 2019 22:08:20 +0000 (22:08 +0000)] 
Do not allow triggers that run as part of REPLACE conflict resolution
during an UPDATE to modify the the table being updated.  Otherwise, those
triggers might delete content out from under the update operation, leading
to all kinds of problems.  Ticket [314cc133e5ada126]

FossilOrigin-Name: db4b7e1dc399c1f16b827ac087aa37c0815f4b2f41f1ffad59963eead2ab5562

5 years agoAdd the OP_FinishSeek opcode which completes an OP_DeferredSeek if the seek
drh [Sun, 29 Dec 2019 00:52:41 +0000 (00:52 +0000)] 
Add the OP_FinishSeek opcode which completes an OP_DeferredSeek if the seek
has not already completed.  Also add the sqlite3WhereUsesDeferredSeek()
interface to the query planner.  The UPDATE implementation adds an
OP_FinishSeek before running the final OP_Insert if one is needed.
Ticket [ec8abb025e78f40c] and also an assertion fault reported by Yongheng.

FossilOrigin-Name: 21ef6e99331210b80fa7c71b4f02e8f768a748d01aef884368af2f6b51a067e0

5 years agoDo not attempt to flatten compound sub-queries in a FROM clause into the parent if...
dan [Sat, 28 Dec 2019 18:25:51 +0000 (18:25 +0000)] 
Do not attempt to flatten compound sub-queries in a FROM clause into the parent if any component of the sub-query uses a window function.

FossilOrigin-Name: eeb76f621de2f930a548db0fbb9fe25b4479b73581826b8dfa2e63cd1f1ab783

5 years agoChange an assert() in where.c to a testcase() macro, since the condition may be false...
dan [Sat, 28 Dec 2019 18:08:39 +0000 (18:08 +0000)] 
Change an assert() in where.c to a testcase() macro, since the condition may be false. This was a problem with the assert() only, there is no bug in release builds that omit assert().

FossilOrigin-Name: 82be135dee7ccfde5f8a67f3621b7ced449dce89bae9cadf025154a4de848c11

5 years agoDisable early coding of transitive constraints at the end of each loop in
drh [Sat, 28 Dec 2019 16:20:23 +0000 (16:20 +0000)] 
Disable early coding of transitive constraints at the end of each loop in
the WHERE clause processing if the loop being coded is for a LEFT JOIN,
even if the loop is part of an OR-clause optimization for virtual tables.
Test cases in TH3.

FossilOrigin-Name: 9421b442cad9858ec21050c106aa935a1c0723ab08ef9a867a79638b040b6e68

5 years agoFix an instance where the planner might choose to use the OR-optimization when it...
dan [Sat, 28 Dec 2019 15:24:02 +0000 (15:24 +0000)] 
Fix an instance where the planner might choose to use the OR-optimization when it adds no benefit. The same quirk causes an assert() to fail. This is not a bug in released versions - without the assert() the library still gets the right answer, it just does so less efficiently than it should.

FossilOrigin-Name: f4bed1d7af8a94c6facd567dec5afae8865a5ad76b8834493099e5e30bed1132

5 years agoFurther improvements to .wheretrace during loop code generation.
drh [Sat, 28 Dec 2019 14:33:26 +0000 (14:33 +0000)] 
Further improvements to .wheretrace during loop code generation.

FossilOrigin-Name: c4d5b75c9381255ec8d9a284eedb6b27be46ca868cae5985cf8a5769b15290c3

5 years agoNew enhancements to .wheretrace. The 0x20000 flag shows the WHERE clause
drh [Sat, 28 Dec 2019 14:07:22 +0000 (14:07 +0000)] 
New enhancements to .wheretrace.  The 0x20000 flag shows the WHERE clause
before and after coding each loop.  The 0x800 flag shows status at the start
and at the end of each loop.  An extra "C" tag is shown on coded terms.

FossilOrigin-Name: 59cc46e5a6d8dbb030f27716ad5446ecccf81cf0cfff95338b9133777f2059e7

5 years agoExpose some of the Where data structure debug printing routines to the
drh [Sat, 28 Dec 2019 13:39:47 +0000 (13:39 +0000)] 
Expose some of the Where data structure debug printing routines to the
entire WHERE-clause processing module.

FossilOrigin-Name: 85e76887761b5d7e290fb2bac7f9121648f79d67fdd92f624d81632f28620518

5 years agoAdd the --enable-all option to the main configure script as a short-hand
drh [Sat, 28 Dec 2019 13:17:11 +0000 (13:17 +0000)] 
Add the --enable-all option to the main configure script as a short-hand
to enable FTS4, FTS5, Geopoly/Rtree, JSON, and Sessions.  In the
amalgamation-autoconf, the --enable-rtree option (which is enabled by
default) also now activates Geopoly.

FossilOrigin-Name: 52ea0672d7494346c636cd7b05ca42161eb3b07d9378fa9959be9a11de87dbbb

5 years agoSimplifications to the initialization of the sqlite3_index_info structure
drh [Sat, 28 Dec 2019 13:01:52 +0000 (13:01 +0000)] 
Simplifications to the initialization of the sqlite3_index_info structure
that is used to communicate with virtual table modules.  Avoid adding
unused constraints to the sqlite3_index_info structure.  Extra constraints
are harmless, but might be confusing to people trying to understand the code.

FossilOrigin-Name: 5e6357fc953a955d8ebb5c1fcd72e04e4ae5e8bf5941810015c2fbc50de70535

5 years agoIf the ".wheretrace 0x10000" bit is set, print WhereTerm.prereqAll
drh [Sat, 28 Dec 2019 12:33:35 +0000 (12:33 +0000)] 
If the ".wheretrace 0x10000" bit is set, print WhereTerm.prereqAll
and .prereqRight fields in the WhereTerm trace output.

FossilOrigin-Name: a4f330b133a36356681dce7d2a65d8ab792620494d6cd4ef4934a97cf3a8e6fc

5 years agoTwo new test cases added to test/fuzzdata8.db.
drh [Sat, 28 Dec 2019 11:55:44 +0000 (11:55 +0000)] 
Two new test cases added to test/fuzzdata8.db.

FossilOrigin-Name: 1be2c18f4021e1d0b8412ecb68a599475c717dd97c7db209fc4203358957fd70

5 years agoSecond attempt to fix a problem with unwinding the WITH stack of the Parse object...
dan [Sat, 28 Dec 2019 09:08:27 +0000 (09:08 +0000)] 
Second attempt to fix a problem with unwinding the WITH stack of the Parse object following an error.

FossilOrigin-Name: 315d1f1a503e8c186739658c85b92fa3d12514e5baaa26925c85664b5e85d1c0

5 years agoMerge latest trunk changes with this branch. better-error-handling-1
dan [Sat, 28 Dec 2019 08:33:46 +0000 (08:33 +0000)] 
Merge latest trunk changes with this branch.

FossilOrigin-Name: d693be375380fbfca426999a77eeecb8453fa77b77f608bfe266945dee1da41d

5 years agoFix an assert() failure in altertable3-22.4.
dan [Sat, 28 Dec 2019 08:26:47 +0000 (08:26 +0000)] 
Fix an assert() failure in altertable3-22.4.

FossilOrigin-Name: c566a91d5c4290b0a491a01e85bf8f9fbebf2910e0d478bafc2d22da7f749cae

5 years agoFix a faulty assert() associated with query search limiting query plans.
drh [Sat, 28 Dec 2019 03:55:50 +0000 (03:55 +0000)] 
Fix a faulty assert() associated with query search limiting query plans.

FossilOrigin-Name: 0d743585c242ebd898b232f42937a2306752aaed0a6756705e3ef1cf7593aa97

5 years agoConvert an assert() back into a conditional. The conditional was converted
drh [Sat, 28 Dec 2019 02:40:49 +0000 (02:40 +0000)] 
Convert an assert() back into a conditional.  The conditional was converted
into an assert() by check-in [6ae4ad6ebee4db88] (2009-05-28) because we were
unable to find a test case for it.  Yongheng's fuzzer just now
found that test case.

FossilOrigin-Name: 4d0b9109f7a5312d4e136395e08b11dad64d746bc106ad44d47675e5b1dcb4ef

5 years agoWhen an INSERT is receiving content from a SELECT, run an OP_ReleaseReg opcode
drh [Sat, 28 Dec 2019 01:52:46 +0000 (01:52 +0000)] 
When an INSERT is receiving content from a SELECT, run an OP_ReleaseReg opcode
at the top of each iteration of the loop in order to prevent spurious
OP_SCopy misuse complaints.  Ticket [de4b04149b9fdeae]

FossilOrigin-Name: 6afadd3b3a40b0ef29fd14fb24c2a4b9479483e5f8b9125ce02d8daae662207f

5 years agoRecompute the values for all generated columns after
drh [Sat, 28 Dec 2019 00:36:51 +0000 (00:36 +0000)] 
Recompute the values for all generated columns after
NOT NULL ON CONFLICT REPLACE constraints fire.
Tickets [37823501c68a09f9] and [5fbc159eeb092130].

FossilOrigin-Name: 4cc12c18860bc4801a407cf45e88e23d3d40391f01a461fbac2cac5f102100e1

5 years agoDo not attempt to unwind the WITH stack in the Parse object following an error. This...
dan [Fri, 27 Dec 2019 20:54:42 +0000 (20:54 +0000)] 
Do not attempt to unwind the WITH stack in the Parse object following an error. This fixes a separate case to [de6e6d68].

FossilOrigin-Name: d29edef93451cc67a5d69c1cce1b1832d9ca8fff1f600afdd51338b74d077b92

5 years agoRemove a NEVER() that is no longer true. Fix for [36ffedcb9].
dan [Fri, 27 Dec 2019 20:06:32 +0000 (20:06 +0000)] 
Remove a NEVER() that is no longer true. Fix for [36ffedcb9].

FossilOrigin-Name: 597896ed0ae9e2960a8f39576bd7f77a11dccc1da84b6a44ebb5c38d90ebc330

5 years agoFix a problem involving window function aliases being referenced from sub-selects.
dan [Fri, 27 Dec 2019 19:46:07 +0000 (19:46 +0000)] 
Fix a problem involving window function aliases being referenced from sub-selects.

FossilOrigin-Name: e3b5fc05c00fc58be7a7c94ce1d97a5b05113f39aba03df64aab08364f85616b

5 years agoAdd a missing "ifcapable rtree {...}" line to the window1.test test module.
drh [Fri, 27 Dec 2019 18:15:04 +0000 (18:15 +0000)] 
Add a missing "ifcapable rtree {...}" line to the window1.test test module.

FossilOrigin-Name: 4e6fbb1a5100fe29209bcf668fbecf3a733034b84925265d8a15b0a84baf6cf0

5 years agoFix a problem with window functions in aggregate queries that do not have GROUP BY...
dan [Fri, 27 Dec 2019 16:25:56 +0000 (16:25 +0000)] 
Fix a problem with window functions in aggregate queries that do not have GROUP BY clauses.

FossilOrigin-Name: 99609786f485653464d1caef1d207ae299d28c925bff3e8e94891759b4765dcc

5 years agoDo not mistake constant integers in a PARTITION BY expression for references to ORDER...
dan [Fri, 27 Dec 2019 15:31:47 +0000 (15:31 +0000)] 
Do not mistake constant integers in a PARTITION BY expression for references to ORDER BY style references to values returned by the SELECT statement.

FossilOrigin-Name: 45c64d39d5afed7fdc6b1de19408559740d1fba1f5e0269ab68b132366a289e4

5 years agoFollow-up to check-in [c8c6dd0e6582ec91] - change the xAccess() method to
drh [Fri, 27 Dec 2019 13:30:46 +0000 (13:30 +0000)] 
Follow-up to check-in [c8c6dd0e6582ec91] - change the xAccess() method to
return true if the file exists and it is anything other than a regular file,
or if it is a regular file with a non-zero file size.

FossilOrigin-Name: 8a39803ef8db4d8cb0d231e66299525fad4e61266ca29b3704aebb88df1c745b

5 years agoWhen determining if an aggregate within a sub-query should be processed as part of...
dan [Fri, 27 Dec 2019 08:57:08 +0000 (08:57 +0000)] 
When determining if an aggregate within a sub-query should be processed as part of the sub-query or an outer query, consider any FILTER clause in the same way as the arguments to the aggregate.

FossilOrigin-Name: 1ffc045d2bbeeda9c13a3a3585538ff6434fc852b0ae6679d3c8b9f606bf92e6

5 years agoEnsure that the Pager.changeCountDone flag is cleared whenever dropping
drh [Fri, 27 Dec 2019 01:50:46 +0000 (01:50 +0000)] 
Ensure that the Pager.changeCountDone flag is cleared whenever dropping
the write lock, even when transitioning from EXCLUSIVE locking mode into
NORMAL locking mode while in WAL mode.  Ticket [fb3b3024ea238d5c].

FossilOrigin-Name: 846b1de6e5a9e418f225273dc033234c64c116fcd89c8261522b4902c248451f

5 years agoFix buffer underflows in the zipfile extension associated with zero-length
drh [Fri, 27 Dec 2019 00:19:53 +0000 (00:19 +0000)] 
Fix buffer underflows in the zipfile extension associated with zero-length
or NULL filename in the ZIP archive.  But report on the mailing list by
Yongheng and Rui.

FossilOrigin-Name: 465a15c5c2077011befa854a77f9d295bb751ef20b7d2937ac0ba47cc84587c9

5 years agoIf an UPSERT can cause an Abort due to a constraint failure, make sure
drh [Thu, 26 Dec 2019 23:40:33 +0000 (23:40 +0000)] 
If an UPSERT can cause an Abort due to a constraint failure, make sure
the query planner knows this.  Ticket [7c13db5c3bf74001].

FossilOrigin-Name: f14ce948662f3445a06d84d4f32d9b81143e0cffaf773d8aa4d3a609bfb9682b

5 years agoAn UPDATE of a table that is indexed by a constant virtual column that uses
drh [Thu, 26 Dec 2019 23:16:18 +0000 (23:16 +0000)] 
An UPDATE of a table that is indexed by a constant virtual column that uses
the one-pass optimization might cause the table seek to be omitted before
reaching row DELETE/INSERT.  Fix this by coding an extra OP_Column in that
circumstance.  Ticket [ec8abb025e78f40c]

FossilOrigin-Name: e54560495926fbb8a2ce829c677a2dd0066e46b7a8d4ada9d8a34a3426959836

5 years agoFix an assert() in fts5 that could fail if an xSavepoint() call on another vtab fails.
dan [Thu, 26 Dec 2019 14:36:31 +0000 (14:36 +0000)] 
Fix an assert() in fts5 that could fail if an xSavepoint() call on another vtab fails.

FossilOrigin-Name: a5d7f5d24a239f729de0b8aa5becf5af35ba87160565ee0713c335b8d1bbf12d

5 years agoMakefile.in fix so that it works on systems that require a .EXE suffix
drh [Thu, 26 Dec 2019 01:10:17 +0000 (01:10 +0000)] 
Makefile.in fix so that it works on systems that require a .EXE suffix
on executables.

FossilOrigin-Name: f482a4cdfa768941e22c399de8ec29a55e729529eeae86d3832077ad1bef22f3

5 years agoAdd two new fuzzer test cases from dbsqlfuzz.
drh [Thu, 26 Dec 2019 01:02:02 +0000 (01:02 +0000)] 
Add two new fuzzer test cases from dbsqlfuzz.

FossilOrigin-Name: f8d5586a8ceca985dcb2d63d0fbad9e271e9e1664720ee698d4947acd5dcc543

5 years agoIn the xAccess() method of the unix VFS, return true if the named object
drh [Thu, 26 Dec 2019 00:56:50 +0000 (00:56 +0000)] 
In the xAccess() method of the unix VFS, return true if the named object
is a directory, regardless of what stat() reports as the st_size for the
object.  Different filesystems report st_size differently for directories.
Problem reported on the mailing list by Stefan Brüns.

FossilOrigin-Name: c8c6dd0e6582ec9103d007b294c42bb1820be1fa7dab85d873b04e0b90571626

5 years agoFix an FTS3 test case that depends on the ICU extension so that it only
drh [Thu, 26 Dec 2019 00:54:39 +0000 (00:54 +0000)] 
Fix an FTS3 test case that depends on the ICU extension so that it only
runs if SQLite is compiled with ICU.

FossilOrigin-Name: 19c6240bdbb022b2af463e59d873280d0f2385bf30e22b9aad5fc9677a99f251

5 years agoFix an incorrect assert() added by check-in [fa58aad48a788802].
drh [Thu, 26 Dec 2019 00:53:43 +0000 (00:53 +0000)] 
Fix an incorrect assert() added by check-in [fa58aad48a788802].
Problem found by dbsqlfuzz.

FossilOrigin-Name: a17b29f08c888301911c35096899f10d24192c8ecec467cddde5401b6bd1903f

5 years agoDisables the optimization that tries to pull the value of an expression
drh [Thu, 26 Dec 2019 00:20:56 +0000 (00:20 +0000)] 
Disables the optimization that tries to pull the value of an expression
from an index on that expression if the expression is a constant.

FossilOrigin-Name: e5fd8b50500f9225e435ef5afee98e3c2cccd22785f99c718d7e6c9b5b653c56

5 years agoWhen the sqlite3WindowRewrite() routine detects and error, have it convert
drh [Wed, 25 Dec 2019 23:54:21 +0000 (23:54 +0000)] 
When the sqlite3WindowRewrite() routine detects and error, have it convert
the SELECT statement into just "SELECT null" so that it does not leave the
parse tree in a goofy state that can cause problems with subsequent code
before the stack has a chance to unwind and report the error.
Ticket [d87336c81c7d0873]

FossilOrigin-Name: fa58aad48a788802b13a819e49f9b8787f713bbe395c46c7295e821c52c81738

5 years agoFix a minor performance regression from check-in [401c9d30e06191d9]
drh [Tue, 24 Dec 2019 21:42:22 +0000 (21:42 +0000)] 
Fix a minor performance regression from check-in [401c9d30e06191d9]

FossilOrigin-Name: 76f54ee86777cbf530654323c953388ef64d0608516722d2522be6c859fa1382

5 years agoBackout the early VTable cursor close change from yesterday, as Yongheng and
drh [Tue, 24 Dec 2019 21:01:37 +0000 (21:01 +0000)] 
Backout the early VTable cursor close change from yesterday, as Yongheng and
Rui have found a test case for which it fails.  The new test case is added
to fuzzcheck.

FossilOrigin-Name: ddb10f0374e66886d0681937b14cf0b4f100f03d3955b45b6c508dc6d9e36976

5 years agoFix a problem causing fts5 integrity-check failures if numeric values are inserted...
dan [Tue, 24 Dec 2019 20:51:01 +0000 (20:51 +0000)] 
Fix a problem causing fts5 integrity-check failures if numeric values are inserted into a table within a utf-16 database. Fix for [752fdbf6].

FossilOrigin-Name: bae060f382e4386c0793db4aa280fe9cd4893307d17a1801a17218ae4dc031c2

5 years agoMake the zipfile() extension function more robust against zero-length filenames.
drh [Tue, 24 Dec 2019 18:53:13 +0000 (18:53 +0000)] 
Make the zipfile() extension function more robust against zero-length filenames.

FossilOrigin-Name: b9c2005f0291d58d7f3bfd4a2286eb6563e9c9433519c3329035f325fc723445

5 years agoFix another case where malformed utf-8 was being mishandled in fts5. Fix for [df46a6f3].
dan [Tue, 24 Dec 2019 16:20:05 +0000 (16:20 +0000)] 
Fix another case where malformed utf-8 was being mishandled in fts5. Fix for [df46a6f3].

FossilOrigin-Name: 1c0a05b09a97e6e2e9b11c31ed6ec7e6484686614b587ebfd0cfe27d973ba461

5 years agoFix an assert() added as part of commit [a11b393dc] that can fail if fts5 database...
dan [Tue, 24 Dec 2019 15:35:53 +0000 (15:35 +0000)] 
Fix an assert() added as part of commit [a11b393dc] that can fail if fts5 database records are corrupt.

FossilOrigin-Name: 4630c1ec013a4c2e54a34c9a64b2c803ac3912450de660497eb34ee21c91f426

5 years agoExtra defense against problems following an OOM. dbsqlfuzz find. Also
drh [Tue, 24 Dec 2019 15:01:17 +0000 (15:01 +0000)] 
Extra defense against problems following an OOM.  dbsqlfuzz find.  Also
import the latest dbsqlfuzz test cases.

FossilOrigin-Name: 0a70f5ddaf3567f335cc0fd0fcd5303e2afb5e38a6388faddf5cab1f2a74e3dd

5 years agoFix a spurious report of corruption that could be made by the fts5 integrity-check...
dan [Tue, 24 Dec 2019 14:27:03 +0000 (14:27 +0000)] 
Fix a spurious report of corruption that could be made by the fts5 integrity-check in SQLITE_DEBUG builds if the fts5 index contains malformed utf text.

FossilOrigin-Name: a11b393dc2c882cf0b3c47c3405bf43ca1d6459605bd39cccce4d32da653a72d

5 years agoConvert an ALWAYS() into an assert() with an extra error term.
drh [Tue, 24 Dec 2019 13:41:33 +0000 (13:41 +0000)] 
Convert an ALWAYS() into an assert() with an extra error term.
Dbsqlfuzz find, with test case in TH3.

FossilOrigin-Name: b473ad35c5ce355853e1805a5c0658bda1500775f22f59c6b6759ae990e65aca

5 years agoAllow comparison operators of a register against itself.
drh [Tue, 24 Dec 2019 01:53:05 +0000 (01:53 +0000)] 
Allow comparison operators of a register against itself.
Ticket [188f912b51cd802a],

FossilOrigin-Name: 401c9d30e06191d938503aae024bc453d960fa64dc812ed86c661f94533247fd

5 years agoTest case for the zipfile-extension bug fix of the previous check-in.
drh [Mon, 23 Dec 2019 21:11:15 +0000 (21:11 +0000)] 
Test case for the zipfile-extension bug fix of the previous check-in.

FossilOrigin-Name: bc8bfc7fcdf33f6855584e10e9260073430517ff3268cf0c7988dcc4cd785391

5 years agoFix the zipfile() function in the zipfile extension so that it is able to
drh [Mon, 23 Dec 2019 21:04:33 +0000 (21:04 +0000)] 
Fix the zipfile() function in the zipfile extension so that it is able to
deal with goofy filenames that contain embedded zeros.

FossilOrigin-Name: cc0fb00a128fd0773db5ff7891f7aa577a3671d570166d2cbb30df922344adcf

5 years agoThoroughly reset the rtree cursor at the start of each VFilter operation,
drh [Mon, 23 Dec 2019 20:41:39 +0000 (20:41 +0000)] 
Thoroughly reset the rtree cursor at the start of each VFilter operation,
including clearing its cache.  This prevents left over pages in the cache
which can cause problems on shutdown after a LEFT JOIN.
Ticket [5eadca17c4dde90c]

FossilOrigin-Name: 4c50afafce8416369f89477ba7fe7d9b047399a5ee5754c73d0e67bbea8d877c

5 years agoRemove an extra deflateInit2() call accidently left in
drh [Mon, 23 Dec 2019 20:07:28 +0000 (20:07 +0000)] 
Remove an extra deflateInit2() call accidently left in
check-in [f5ee30426e8876e7]

FossilOrigin-Name: 953e6aa6d9ca09e549c88bc728c322fabdcef581f3b4a7763d95488dddbbd6a5

5 years agoDo an early close of virtual table cursors to avoid unnecessary cursor
drh [Mon, 23 Dec 2019 19:28:34 +0000 (19:28 +0000)] 
Do an early close of virtual table cursors to avoid unnecessary cursor
contention in UPDATE for some virtual table implementations.
Ticket [56a74875be799b85]

FossilOrigin-Name: eb95dac7f6482c36853a23234c5ce7af37c7beed8e3675d6a49c4d3a8307e076

5 years agoEarly detection of database corruption in balance_deeper().
drh [Mon, 23 Dec 2019 18:02:15 +0000 (18:02 +0000)] 
Early detection of database corruption in balance_deeper().

FossilOrigin-Name: 61c2233654158e65a3d3baeea947903a919a569fcc4a5b342b2e9a68cec1b6f3

5 years agoFix a case in which SQLite could fail to identify "x BETWEEN ? AND ?" being true...
dan [Mon, 23 Dec 2019 15:17:11 +0000 (15:17 +0000)] 
Fix a case in which SQLite could fail to identify "x BETWEEN ? AND ?" being true as implying that x is not null. Ticket [dfd66334].

FossilOrigin-Name: 2f17974912ec5e99089dc0da803e7ff1bf033377a49762d2689a812c005f2641

5 years agoFor expressions like (x, y) IN (SELECT ...) where the SELECT uses window-functions...
dan [Mon, 23 Dec 2019 14:20:46 +0000 (14:20 +0000)] 
For expressions like (x, y) IN (SELECT ...) where the SELECT uses window-functions, require that all columns on the LHS be indexed before an index can be used. Fix for [d9ed4ebe].

FossilOrigin-Name: 0b1dbd60f5db3abe2097dbc0b6de9671685ca5eaf7d3fc8e3f87ff5065a9d114

5 years agoFix a shift-overflow problem in yesterday's check-in [36fdeb4f0a66970a]
drh [Mon, 23 Dec 2019 13:24:34 +0000 (13:24 +0000)] 
Fix a shift-overflow problem in yesterday's check-in [36fdeb4f0a66970a]
that OSSFuzz helpfully discovered overnight.  Thanks Google.

FossilOrigin-Name: bff38e2b5318ed032aaf1d350903c3494b4531f2dc59a6997144ec8e23defef4

5 years agoFix the OP_Cast operator so that when casting to TEXT, it always leaves
drh [Mon, 23 Dec 2019 03:37:46 +0000 (03:37 +0000)] 
Fix the OP_Cast operator so that when casting to TEXT, it always leaves
the result in the encoding of the database.  Ticket [0911b5d161b039c6].
Test cases in TH3.

FossilOrigin-Name: f347744e0d576f0250c29259cca755e57afded8956224114c01603c1bca5b3a4

5 years agoChange the assert() back into a testcase(). See also check-ins
drh [Mon, 23 Dec 2019 02:43:52 +0000 (02:43 +0000)] 
Change the assert() back into a testcase().  See also check-ins
[9ab985a9c8160b90] and [ddb17d92df194337] and other check-ins that those
reference.  Fix for ticket [9d708e474201c001]

FossilOrigin-Name: 2c44c73499154bc57634a54cb743642d0aacc93a1336fbb8bb3eb5dbbf616357

5 years agoEnhance the sqlite3VdbeMemAboutToChange() shallow-copy validation mechanism
drh [Mon, 23 Dec 2019 02:18:49 +0000 (02:18 +0000)] 
Enhance the sqlite3VdbeMemAboutToChange() shallow-copy validation mechanism
by adding the new OP_ReleaseReg opcode to tell MemAboutToChange() that a
range of registers is no longer needed so that the source register can be
freely changed.  This is a change to debugging and test builds only and
does not impact release builds.  Fix for ticket
[c62c5e58524b204d] and [5ad2aa6921faa1ee].  The previous fix to ticket
[5ad2aa6921faa1ee] is backed out by this change since this change is a better
fix.

FossilOrigin-Name: 36fdeb4f0a66970a35de688b617f90899c89cfdfab659f864df99aa7ebf854ea

5 years agoChange the code generator for the IN operator so that it avoids creating
drh [Sun, 22 Dec 2019 23:48:36 +0000 (23:48 +0000)] 
Change the code generator for the IN operator so that it avoids creating
OP_Eq and OP_Ne opcode with the same P1 and P3 arguments.  This enables us
to back out check-in [ddb17d92df194337] and also fix ticket [188f912b51cd802].

FossilOrigin-Name: 9ab985a9c8160b905730678f40ed440a246cdec549c798bafefaed5abbc0437f

5 years agoMake a hard copy of strings in constraint checks prior to applying
drh [Sun, 22 Dec 2019 20:29:25 +0000 (20:29 +0000)] 
Make a hard copy of strings in constraint checks prior to applying
OP_RealAffinity, to avoid problems with a pointer accounting assert.
This change is not strictly necessary - the correct answer is obtained
without it and no UB occurs - however the pointer accounting asserts are
useful to prevent other problems so it is a simple matter to bring this
piece into compliance.  Ticket [5ad2aa6921faa1ee]

FossilOrigin-Name: 89a9dad6330270a4c3b962f86a208088d2ea9883c7d291351a77f058e0ed8b0c

5 years agoWhen constructing the virtual MATCH term of the WHERE clause for a virtual
drh [Sun, 22 Dec 2019 20:03:29 +0000 (20:03 +0000)] 
When constructing the virtual MATCH term of the WHERE clause for a virtual
table that is in a LEFT JOIN, be sure to set the correct Expr.iRightJoinTable
value.  This value does not appear to ever be used, except inside of a single
assert().  But it is good to set it correctly, nevertheless.  This fixes
ticket [7929c1efb2d67e98], which as far as I can tell is completely harmless.

FossilOrigin-Name: ef604882a275d3d5ebd4d5a08e3fe43e148f169b7d5c3a81464fbe7f54f3582a

5 years agoIn the WHERE clause debugging output (the .wheretrace output) show the
drh [Sun, 22 Dec 2019 19:41:12 +0000 (19:41 +0000)] 
In the WHERE clause debugging output (the .wheretrace output) show the
parent index of any WhereTerm that is a child.

FossilOrigin-Name: 7fc733328c5914c747e048c830522ec13e433c0a86388bae47b198a3e220c6c2

5 years agoFix to the optimization of check-in [a47efb7c8520a011] that reads the values
drh [Sun, 22 Dec 2019 18:55:04 +0000 (18:55 +0000)] 
Fix to the optimization of check-in [a47efb7c8520a011] that reads the values
of expressions used in an index-on-expression directly from the index rather
than recomputing the value.  If the expression has a top-level COLLATE or
unlikely() operator, be sure to clear the corresponding flags prior to
converting it into a TK_COLUMN expression.  Failure to do this is most likely
harmless in production, but might cause an assertion fault in debugging builds.
Ticket [b0cb8aff561a6dcd].  Test cases in TH3.

FossilOrigin-Name: 56539e1c132632c075efc217ad5951a35e4459605ac128914aec3a7be1e25718

5 years agoBack out the asserts of check-ins [a500893b6f64aced] and [d9c9fe9f5ad3fc91]
drh [Sun, 22 Dec 2019 18:23:30 +0000 (18:23 +0000)] 
Back out the asserts of check-ins [a500893b6f64aced] and [d9c9fe9f5ad3fc91]
as ticket [1b06916e01c82b66] demonstrates a case that refutes them.

FossilOrigin-Name: ddb17d92df194337a103c561ef46ced00db5e67e4551e82de7cb5ad7f457dd5f

5 years agoWhen parsing a CREATE TABLE from the sqlite_master table, delete the CHECK
drh [Sun, 22 Dec 2019 18:06:49 +0000 (18:06 +0000)] 
When parsing a CREATE TABLE from the sqlite_master table, delete the CHECK
constraints if there are any errors, since there might otherwise be attempts
to use those CHECK constraints if PRAGMA writable_schema=ON is set.
This undoes the fix in check-in [ea721b34477ab8b4] for a more general
solution.

FossilOrigin-Name: a982e6434cd66bfbe94d455f538bcbc4360b91572062d92acae6b77e9560e65d

5 years agoEnsure sqlite3WindowRewrite() is called on a SELECT statement before any terms aremov...
dan [Sun, 22 Dec 2019 17:32:25 +0000 (17:32 +0000)] 
Ensure sqlite3WindowRewrite() is called on a SELECT statement before any terms aremoved from it as part of IN() clause processing. Fix for [f00d096ca].

FossilOrigin-Name: 8c856404b4e98d295449a4e89a41495dc007319a8e9c35c1a763718d7c5f67e8

5 years agoFix "PRAGMA data_version" so that it works the same way with locking_mode=PERSIST...
dan [Sun, 22 Dec 2019 14:29:55 +0000 (14:29 +0000)] 
Fix "PRAGMA data_version" so that it works the same way with locking_mode=PERSIST and journal_mode=PERSIST configured. Fix for 7a458c2a5f.

FossilOrigin-Name: 45748e2db028ffbda5d3e747493721a5a89af3fbc06823081a2f27f570e40e73

5 years agoFix a spurious report of corruption from the fts3/4 integrity-check triggered by...
dan [Sat, 21 Dec 2019 20:43:34 +0000 (20:43 +0000)] 
Fix a spurious report of corruption from the fts3/4 integrity-check triggered by using the languageid option.

FossilOrigin-Name: 70815e273f511481c310dc33b3b49079901d891078fad49b663fbce06c77ecc2

5 years agoWhen creating a new virtual table, ensure that the OP_ParseSchema opcode
drh [Sat, 21 Dec 2019 19:37:09 +0000 (19:37 +0000)] 
When creating a new virtual table, ensure that the OP_ParseSchema opcode
processes the correct entry in the sqlite_master table even if there is
a second entry with the same name and table values due to database corruption
and the use of writable_schema=ON.  Dbsqlfuzz find.

FossilOrigin-Name: 4dbb6e1cb094f3428c74ea8bdd86ab63341fecce978a062968ca01423f382e90

5 years agoWhen a corrupt schema is loaded using writable_schema=ON, the CHECK constraints
drh [Sat, 21 Dec 2019 14:09:30 +0000 (14:09 +0000)] 
When a corrupt schema is loaded using writable_schema=ON, the CHECK constraints
(or other expressions in the table definition) might not be fully resolved.
Ensure that the code generator can deal with this if the table is subsequently
used in a DML statement.  dbsqlfuzz find.

FossilOrigin-Name: ea721b34477ab8b49d182352c4bc198245933b850e9b6248b4f97600e80bb44b

5 years agoDo not try to access a generated column through an index if the collating
drh [Fri, 20 Dec 2019 22:46:41 +0000 (22:46 +0000)] 
Do not try to access a generated column through an index if the collating
sequence for the generated column is non-standard.
Part 2 of ticket [e0a8120553f4b082]

FossilOrigin-Name: 056bb8dcbdc45989c5c6e86d2966200062e3c01c382ec52aae37c828104b4496

5 years agoApply real affinity to generated columns of type REAL that are extract from
drh [Fri, 20 Dec 2019 20:45:02 +0000 (20:45 +0000)] 
Apply real affinity to generated columns of type REAL that are extract from
an index.  Ticket [e0a8120553f4b082]

FossilOrigin-Name: 728ad39e3bd07a2503a95c68ed5bbc3f28fd97551d59f12a5fab25dc68227e99

5 years agoDebugging enhancment: Show the Expr.y.pTab pointer on TK_COLUMN nodes of
drh [Fri, 20 Dec 2019 20:08:56 +0000 (20:08 +0000)] 
Debugging enhancment:  Show the Expr.y.pTab pointer on TK_COLUMN nodes of
an expression tree in the treeview.

FossilOrigin-Name: 64154ac450e4366d18e7e867841877a69c1f978d3ba9b8754cc133248966731d

5 years agoFix a bad interaction between RBU and [df51ae19].
dan [Fri, 20 Dec 2019 20:03:21 +0000 (20:03 +0000)] 
Fix a bad interaction between RBU and [df51ae19].

FossilOrigin-Name: 0b9d8a1202c4220fd2ef299b6194533c1bf4018a0cd2d13da7e22c1a7de05ffa

5 years agoFix two more cases in fts5 where sqlite3_value_bytes() was being called before sqlite...
dan [Fri, 20 Dec 2019 19:41:01 +0000 (19:41 +0000)] 
Fix two more cases in fts5 where sqlite3_value_bytes() was being called before sqlite3_value_text(). Fix for e431c355.

FossilOrigin-Name: a1ba9a37d7a68a6d31f8197c6350589ebe6a12f4e3c193a178dd7ead8bcd565a