]> git.ipfire.org Git - thirdparty/sqlite.git/log
thirdparty/sqlite.git
5 years agoMerge latest trunk changes with this branch. better-error-handling-1
dan [Sat, 28 Dec 2019 08:33:46 +0000 (08:33 +0000)] 
Merge latest trunk changes with this branch.

FossilOrigin-Name: d693be375380fbfca426999a77eeecb8453fa77b77f608bfe266945dee1da41d

5 years agoFix an assert() failure in altertable3-22.4.
dan [Sat, 28 Dec 2019 08:26:47 +0000 (08:26 +0000)] 
Fix an assert() failure in altertable3-22.4.

FossilOrigin-Name: c566a91d5c4290b0a491a01e85bf8f9fbebf2910e0d478bafc2d22da7f749cae

5 years agoFix a faulty assert() associated with query search limiting query plans.
drh [Sat, 28 Dec 2019 03:55:50 +0000 (03:55 +0000)] 
Fix a faulty assert() associated with query search limiting query plans.

FossilOrigin-Name: 0d743585c242ebd898b232f42937a2306752aaed0a6756705e3ef1cf7593aa97

5 years agoConvert an assert() back into a conditional. The conditional was converted
drh [Sat, 28 Dec 2019 02:40:49 +0000 (02:40 +0000)] 
Convert an assert() back into a conditional.  The conditional was converted
into an assert() by check-in [6ae4ad6ebee4db88] (2009-05-28) because we were
unable to find a test case for it.  Yongheng's fuzzer just now
found that test case.

FossilOrigin-Name: 4d0b9109f7a5312d4e136395e08b11dad64d746bc106ad44d47675e5b1dcb4ef

5 years agoWhen an INSERT is receiving content from a SELECT, run an OP_ReleaseReg opcode
drh [Sat, 28 Dec 2019 01:52:46 +0000 (01:52 +0000)] 
When an INSERT is receiving content from a SELECT, run an OP_ReleaseReg opcode
at the top of each iteration of the loop in order to prevent spurious
OP_SCopy misuse complaints.  Ticket [de4b04149b9fdeae]

FossilOrigin-Name: 6afadd3b3a40b0ef29fd14fb24c2a4b9479483e5f8b9125ce02d8daae662207f

5 years agoRecompute the values for all generated columns after
drh [Sat, 28 Dec 2019 00:36:51 +0000 (00:36 +0000)] 
Recompute the values for all generated columns after
NOT NULL ON CONFLICT REPLACE constraints fire.
Tickets [37823501c68a09f9] and [5fbc159eeb092130].

FossilOrigin-Name: 4cc12c18860bc4801a407cf45e88e23d3d40391f01a461fbac2cac5f102100e1

5 years agoDo not attempt to unwind the WITH stack in the Parse object following an error. This...
dan [Fri, 27 Dec 2019 20:54:42 +0000 (20:54 +0000)] 
Do not attempt to unwind the WITH stack in the Parse object following an error. This fixes a separate case to [de6e6d68].

FossilOrigin-Name: d29edef93451cc67a5d69c1cce1b1832d9ca8fff1f600afdd51338b74d077b92

5 years agoRemove a NEVER() that is no longer true. Fix for [36ffedcb9].
dan [Fri, 27 Dec 2019 20:06:32 +0000 (20:06 +0000)] 
Remove a NEVER() that is no longer true. Fix for [36ffedcb9].

FossilOrigin-Name: 597896ed0ae9e2960a8f39576bd7f77a11dccc1da84b6a44ebb5c38d90ebc330

5 years agoFix a problem involving window function aliases being referenced from sub-selects.
dan [Fri, 27 Dec 2019 19:46:07 +0000 (19:46 +0000)] 
Fix a problem involving window function aliases being referenced from sub-selects.

FossilOrigin-Name: e3b5fc05c00fc58be7a7c94ce1d97a5b05113f39aba03df64aab08364f85616b

5 years agoAdd a missing "ifcapable rtree {...}" line to the window1.test test module.
drh [Fri, 27 Dec 2019 18:15:04 +0000 (18:15 +0000)] 
Add a missing "ifcapable rtree {...}" line to the window1.test test module.

FossilOrigin-Name: 4e6fbb1a5100fe29209bcf668fbecf3a733034b84925265d8a15b0a84baf6cf0

5 years agoFix a problem with window functions in aggregate queries that do not have GROUP BY...
dan [Fri, 27 Dec 2019 16:25:56 +0000 (16:25 +0000)] 
Fix a problem with window functions in aggregate queries that do not have GROUP BY clauses.

FossilOrigin-Name: 99609786f485653464d1caef1d207ae299d28c925bff3e8e94891759b4765dcc

5 years agoDo not mistake constant integers in a PARTITION BY expression for references to ORDER...
dan [Fri, 27 Dec 2019 15:31:47 +0000 (15:31 +0000)] 
Do not mistake constant integers in a PARTITION BY expression for references to ORDER BY style references to values returned by the SELECT statement.

FossilOrigin-Name: 45c64d39d5afed7fdc6b1de19408559740d1fba1f5e0269ab68b132366a289e4

5 years agoFollow-up to check-in [c8c6dd0e6582ec91] - change the xAccess() method to
drh [Fri, 27 Dec 2019 13:30:46 +0000 (13:30 +0000)] 
Follow-up to check-in [c8c6dd0e6582ec91] - change the xAccess() method to
return true if the file exists and it is anything other than a regular file,
or if it is a regular file with a non-zero file size.

FossilOrigin-Name: 8a39803ef8db4d8cb0d231e66299525fad4e61266ca29b3704aebb88df1c745b

5 years agoWhen determining if an aggregate within a sub-query should be processed as part of...
dan [Fri, 27 Dec 2019 08:57:08 +0000 (08:57 +0000)] 
When determining if an aggregate within a sub-query should be processed as part of the sub-query or an outer query, consider any FILTER clause in the same way as the arguments to the aggregate.

FossilOrigin-Name: 1ffc045d2bbeeda9c13a3a3585538ff6434fc852b0ae6679d3c8b9f606bf92e6

5 years agoEnsure that the Pager.changeCountDone flag is cleared whenever dropping
drh [Fri, 27 Dec 2019 01:50:46 +0000 (01:50 +0000)] 
Ensure that the Pager.changeCountDone flag is cleared whenever dropping
the write lock, even when transitioning from EXCLUSIVE locking mode into
NORMAL locking mode while in WAL mode.  Ticket [fb3b3024ea238d5c].

FossilOrigin-Name: 846b1de6e5a9e418f225273dc033234c64c116fcd89c8261522b4902c248451f

5 years agoFix buffer underflows in the zipfile extension associated with zero-length
drh [Fri, 27 Dec 2019 00:19:53 +0000 (00:19 +0000)] 
Fix buffer underflows in the zipfile extension associated with zero-length
or NULL filename in the ZIP archive.  But report on the mailing list by
Yongheng and Rui.

FossilOrigin-Name: 465a15c5c2077011befa854a77f9d295bb751ef20b7d2937ac0ba47cc84587c9

5 years agoIf an UPSERT can cause an Abort due to a constraint failure, make sure
drh [Thu, 26 Dec 2019 23:40:33 +0000 (23:40 +0000)] 
If an UPSERT can cause an Abort due to a constraint failure, make sure
the query planner knows this.  Ticket [7c13db5c3bf74001].

FossilOrigin-Name: f14ce948662f3445a06d84d4f32d9b81143e0cffaf773d8aa4d3a609bfb9682b

5 years agoAn UPDATE of a table that is indexed by a constant virtual column that uses
drh [Thu, 26 Dec 2019 23:16:18 +0000 (23:16 +0000)] 
An UPDATE of a table that is indexed by a constant virtual column that uses
the one-pass optimization might cause the table seek to be omitted before
reaching row DELETE/INSERT.  Fix this by coding an extra OP_Column in that
circumstance.  Ticket [ec8abb025e78f40c]

FossilOrigin-Name: e54560495926fbb8a2ce829c677a2dd0066e46b7a8d4ada9d8a34a3426959836

5 years agoFix an assert() in fts5 that could fail if an xSavepoint() call on another vtab fails.
dan [Thu, 26 Dec 2019 14:36:31 +0000 (14:36 +0000)] 
Fix an assert() in fts5 that could fail if an xSavepoint() call on another vtab fails.

FossilOrigin-Name: a5d7f5d24a239f729de0b8aa5becf5af35ba87160565ee0713c335b8d1bbf12d

5 years agoMakefile.in fix so that it works on systems that require a .EXE suffix
drh [Thu, 26 Dec 2019 01:10:17 +0000 (01:10 +0000)] 
Makefile.in fix so that it works on systems that require a .EXE suffix
on executables.

FossilOrigin-Name: f482a4cdfa768941e22c399de8ec29a55e729529eeae86d3832077ad1bef22f3

5 years agoAdd two new fuzzer test cases from dbsqlfuzz.
drh [Thu, 26 Dec 2019 01:02:02 +0000 (01:02 +0000)] 
Add two new fuzzer test cases from dbsqlfuzz.

FossilOrigin-Name: f8d5586a8ceca985dcb2d63d0fbad9e271e9e1664720ee698d4947acd5dcc543

5 years agoIn the xAccess() method of the unix VFS, return true if the named object
drh [Thu, 26 Dec 2019 00:56:50 +0000 (00:56 +0000)] 
In the xAccess() method of the unix VFS, return true if the named object
is a directory, regardless of what stat() reports as the st_size for the
object.  Different filesystems report st_size differently for directories.
Problem reported on the mailing list by Stefan BrĂ¼ns.

FossilOrigin-Name: c8c6dd0e6582ec9103d007b294c42bb1820be1fa7dab85d873b04e0b90571626

5 years agoFix an FTS3 test case that depends on the ICU extension so that it only
drh [Thu, 26 Dec 2019 00:54:39 +0000 (00:54 +0000)] 
Fix an FTS3 test case that depends on the ICU extension so that it only
runs if SQLite is compiled with ICU.

FossilOrigin-Name: 19c6240bdbb022b2af463e59d873280d0f2385bf30e22b9aad5fc9677a99f251

5 years agoFix an incorrect assert() added by check-in [fa58aad48a788802].
drh [Thu, 26 Dec 2019 00:53:43 +0000 (00:53 +0000)] 
Fix an incorrect assert() added by check-in [fa58aad48a788802].
Problem found by dbsqlfuzz.

FossilOrigin-Name: a17b29f08c888301911c35096899f10d24192c8ecec467cddde5401b6bd1903f

5 years agoDisables the optimization that tries to pull the value of an expression
drh [Thu, 26 Dec 2019 00:20:56 +0000 (00:20 +0000)] 
Disables the optimization that tries to pull the value of an expression
from an index on that expression if the expression is a constant.

FossilOrigin-Name: e5fd8b50500f9225e435ef5afee98e3c2cccd22785f99c718d7e6c9b5b653c56

5 years agoWhen the sqlite3WindowRewrite() routine detects and error, have it convert
drh [Wed, 25 Dec 2019 23:54:21 +0000 (23:54 +0000)] 
When the sqlite3WindowRewrite() routine detects and error, have it convert
the SELECT statement into just "SELECT null" so that it does not leave the
parse tree in a goofy state that can cause problems with subsequent code
before the stack has a chance to unwind and report the error.
Ticket [d87336c81c7d0873]

FossilOrigin-Name: fa58aad48a788802b13a819e49f9b8787f713bbe395c46c7295e821c52c81738

5 years agoFix a minor performance regression from check-in [401c9d30e06191d9]
drh [Tue, 24 Dec 2019 21:42:22 +0000 (21:42 +0000)] 
Fix a minor performance regression from check-in [401c9d30e06191d9]

FossilOrigin-Name: 76f54ee86777cbf530654323c953388ef64d0608516722d2522be6c859fa1382

5 years agoBackout the early VTable cursor close change from yesterday, as Yongheng and
drh [Tue, 24 Dec 2019 21:01:37 +0000 (21:01 +0000)] 
Backout the early VTable cursor close change from yesterday, as Yongheng and
Rui have found a test case for which it fails.  The new test case is added
to fuzzcheck.

FossilOrigin-Name: ddb10f0374e66886d0681937b14cf0b4f100f03d3955b45b6c508dc6d9e36976

5 years agoFix a problem causing fts5 integrity-check failures if numeric values are inserted...
dan [Tue, 24 Dec 2019 20:51:01 +0000 (20:51 +0000)] 
Fix a problem causing fts5 integrity-check failures if numeric values are inserted into a table within a utf-16 database. Fix for [752fdbf6].

FossilOrigin-Name: bae060f382e4386c0793db4aa280fe9cd4893307d17a1801a17218ae4dc031c2

5 years agoMake the zipfile() extension function more robust against zero-length filenames.
drh [Tue, 24 Dec 2019 18:53:13 +0000 (18:53 +0000)] 
Make the zipfile() extension function more robust against zero-length filenames.

FossilOrigin-Name: b9c2005f0291d58d7f3bfd4a2286eb6563e9c9433519c3329035f325fc723445

5 years agoFix another case where malformed utf-8 was being mishandled in fts5. Fix for [df46a6f3].
dan [Tue, 24 Dec 2019 16:20:05 +0000 (16:20 +0000)] 
Fix another case where malformed utf-8 was being mishandled in fts5. Fix for [df46a6f3].

FossilOrigin-Name: 1c0a05b09a97e6e2e9b11c31ed6ec7e6484686614b587ebfd0cfe27d973ba461

5 years agoFix an assert() added as part of commit [a11b393dc] that can fail if fts5 database...
dan [Tue, 24 Dec 2019 15:35:53 +0000 (15:35 +0000)] 
Fix an assert() added as part of commit [a11b393dc] that can fail if fts5 database records are corrupt.

FossilOrigin-Name: 4630c1ec013a4c2e54a34c9a64b2c803ac3912450de660497eb34ee21c91f426

5 years agoExtra defense against problems following an OOM. dbsqlfuzz find. Also
drh [Tue, 24 Dec 2019 15:01:17 +0000 (15:01 +0000)] 
Extra defense against problems following an OOM.  dbsqlfuzz find.  Also
import the latest dbsqlfuzz test cases.

FossilOrigin-Name: 0a70f5ddaf3567f335cc0fd0fcd5303e2afb5e38a6388faddf5cab1f2a74e3dd

5 years agoFix a spurious report of corruption that could be made by the fts5 integrity-check...
dan [Tue, 24 Dec 2019 14:27:03 +0000 (14:27 +0000)] 
Fix a spurious report of corruption that could be made by the fts5 integrity-check in SQLITE_DEBUG builds if the fts5 index contains malformed utf text.

FossilOrigin-Name: a11b393dc2c882cf0b3c47c3405bf43ca1d6459605bd39cccce4d32da653a72d

5 years agoConvert an ALWAYS() into an assert() with an extra error term.
drh [Tue, 24 Dec 2019 13:41:33 +0000 (13:41 +0000)] 
Convert an ALWAYS() into an assert() with an extra error term.
Dbsqlfuzz find, with test case in TH3.

FossilOrigin-Name: b473ad35c5ce355853e1805a5c0658bda1500775f22f59c6b6759ae990e65aca

5 years agoAllow comparison operators of a register against itself.
drh [Tue, 24 Dec 2019 01:53:05 +0000 (01:53 +0000)] 
Allow comparison operators of a register against itself.
Ticket [188f912b51cd802a],

FossilOrigin-Name: 401c9d30e06191d938503aae024bc453d960fa64dc812ed86c661f94533247fd

5 years agoTest case for the zipfile-extension bug fix of the previous check-in.
drh [Mon, 23 Dec 2019 21:11:15 +0000 (21:11 +0000)] 
Test case for the zipfile-extension bug fix of the previous check-in.

FossilOrigin-Name: bc8bfc7fcdf33f6855584e10e9260073430517ff3268cf0c7988dcc4cd785391

5 years agoFix the zipfile() function in the zipfile extension so that it is able to
drh [Mon, 23 Dec 2019 21:04:33 +0000 (21:04 +0000)] 
Fix the zipfile() function in the zipfile extension so that it is able to
deal with goofy filenames that contain embedded zeros.

FossilOrigin-Name: cc0fb00a128fd0773db5ff7891f7aa577a3671d570166d2cbb30df922344adcf

5 years agoThoroughly reset the rtree cursor at the start of each VFilter operation,
drh [Mon, 23 Dec 2019 20:41:39 +0000 (20:41 +0000)] 
Thoroughly reset the rtree cursor at the start of each VFilter operation,
including clearing its cache.  This prevents left over pages in the cache
which can cause problems on shutdown after a LEFT JOIN.
Ticket [5eadca17c4dde90c]

FossilOrigin-Name: 4c50afafce8416369f89477ba7fe7d9b047399a5ee5754c73d0e67bbea8d877c

5 years agoRemove an extra deflateInit2() call accidently left in
drh [Mon, 23 Dec 2019 20:07:28 +0000 (20:07 +0000)] 
Remove an extra deflateInit2() call accidently left in
check-in [f5ee30426e8876e7]

FossilOrigin-Name: 953e6aa6d9ca09e549c88bc728c322fabdcef581f3b4a7763d95488dddbbd6a5

5 years agoDo an early close of virtual table cursors to avoid unnecessary cursor
drh [Mon, 23 Dec 2019 19:28:34 +0000 (19:28 +0000)] 
Do an early close of virtual table cursors to avoid unnecessary cursor
contention in UPDATE for some virtual table implementations.
Ticket [56a74875be799b85]

FossilOrigin-Name: eb95dac7f6482c36853a23234c5ce7af37c7beed8e3675d6a49c4d3a8307e076

5 years agoEarly detection of database corruption in balance_deeper().
drh [Mon, 23 Dec 2019 18:02:15 +0000 (18:02 +0000)] 
Early detection of database corruption in balance_deeper().

FossilOrigin-Name: 61c2233654158e65a3d3baeea947903a919a569fcc4a5b342b2e9a68cec1b6f3

5 years agoFix a case in which SQLite could fail to identify "x BETWEEN ? AND ?" being true...
dan [Mon, 23 Dec 2019 15:17:11 +0000 (15:17 +0000)] 
Fix a case in which SQLite could fail to identify "x BETWEEN ? AND ?" being true as implying that x is not null. Ticket [dfd66334].

FossilOrigin-Name: 2f17974912ec5e99089dc0da803e7ff1bf033377a49762d2689a812c005f2641

5 years agoFor expressions like (x, y) IN (SELECT ...) where the SELECT uses window-functions...
dan [Mon, 23 Dec 2019 14:20:46 +0000 (14:20 +0000)] 
For expressions like (x, y) IN (SELECT ...) where the SELECT uses window-functions, require that all columns on the LHS be indexed before an index can be used. Fix for [d9ed4ebe].

FossilOrigin-Name: 0b1dbd60f5db3abe2097dbc0b6de9671685ca5eaf7d3fc8e3f87ff5065a9d114

5 years agoFix a shift-overflow problem in yesterday's check-in [36fdeb4f0a66970a]
drh [Mon, 23 Dec 2019 13:24:34 +0000 (13:24 +0000)] 
Fix a shift-overflow problem in yesterday's check-in [36fdeb4f0a66970a]
that OSSFuzz helpfully discovered overnight.  Thanks Google.

FossilOrigin-Name: bff38e2b5318ed032aaf1d350903c3494b4531f2dc59a6997144ec8e23defef4

5 years agoFix the OP_Cast operator so that when casting to TEXT, it always leaves
drh [Mon, 23 Dec 2019 03:37:46 +0000 (03:37 +0000)] 
Fix the OP_Cast operator so that when casting to TEXT, it always leaves
the result in the encoding of the database.  Ticket [0911b5d161b039c6].
Test cases in TH3.

FossilOrigin-Name: f347744e0d576f0250c29259cca755e57afded8956224114c01603c1bca5b3a4

5 years agoChange the assert() back into a testcase(). See also check-ins
drh [Mon, 23 Dec 2019 02:43:52 +0000 (02:43 +0000)] 
Change the assert() back into a testcase().  See also check-ins
[9ab985a9c8160b90] and [ddb17d92df194337] and other check-ins that those
reference.  Fix for ticket [9d708e474201c001]

FossilOrigin-Name: 2c44c73499154bc57634a54cb743642d0aacc93a1336fbb8bb3eb5dbbf616357

5 years agoEnhance the sqlite3VdbeMemAboutToChange() shallow-copy validation mechanism
drh [Mon, 23 Dec 2019 02:18:49 +0000 (02:18 +0000)] 
Enhance the sqlite3VdbeMemAboutToChange() shallow-copy validation mechanism
by adding the new OP_ReleaseReg opcode to tell MemAboutToChange() that a
range of registers is no longer needed so that the source register can be
freely changed.  This is a change to debugging and test builds only and
does not impact release builds.  Fix for ticket
[c62c5e58524b204d] and [5ad2aa6921faa1ee].  The previous fix to ticket
[5ad2aa6921faa1ee] is backed out by this change since this change is a better
fix.

FossilOrigin-Name: 36fdeb4f0a66970a35de688b617f90899c89cfdfab659f864df99aa7ebf854ea

5 years agoChange the code generator for the IN operator so that it avoids creating
drh [Sun, 22 Dec 2019 23:48:36 +0000 (23:48 +0000)] 
Change the code generator for the IN operator so that it avoids creating
OP_Eq and OP_Ne opcode with the same P1 and P3 arguments.  This enables us
to back out check-in [ddb17d92df194337] and also fix ticket [188f912b51cd802].

FossilOrigin-Name: 9ab985a9c8160b905730678f40ed440a246cdec549c798bafefaed5abbc0437f

5 years agoMake a hard copy of strings in constraint checks prior to applying
drh [Sun, 22 Dec 2019 20:29:25 +0000 (20:29 +0000)] 
Make a hard copy of strings in constraint checks prior to applying
OP_RealAffinity, to avoid problems with a pointer accounting assert.
This change is not strictly necessary - the correct answer is obtained
without it and no UB occurs - however the pointer accounting asserts are
useful to prevent other problems so it is a simple matter to bring this
piece into compliance.  Ticket [5ad2aa6921faa1ee]

FossilOrigin-Name: 89a9dad6330270a4c3b962f86a208088d2ea9883c7d291351a77f058e0ed8b0c

5 years agoWhen constructing the virtual MATCH term of the WHERE clause for a virtual
drh [Sun, 22 Dec 2019 20:03:29 +0000 (20:03 +0000)] 
When constructing the virtual MATCH term of the WHERE clause for a virtual
table that is in a LEFT JOIN, be sure to set the correct Expr.iRightJoinTable
value.  This value does not appear to ever be used, except inside of a single
assert().  But it is good to set it correctly, nevertheless.  This fixes
ticket [7929c1efb2d67e98], which as far as I can tell is completely harmless.

FossilOrigin-Name: ef604882a275d3d5ebd4d5a08e3fe43e148f169b7d5c3a81464fbe7f54f3582a

5 years agoIn the WHERE clause debugging output (the .wheretrace output) show the
drh [Sun, 22 Dec 2019 19:41:12 +0000 (19:41 +0000)] 
In the WHERE clause debugging output (the .wheretrace output) show the
parent index of any WhereTerm that is a child.

FossilOrigin-Name: 7fc733328c5914c747e048c830522ec13e433c0a86388bae47b198a3e220c6c2

5 years agoFix to the optimization of check-in [a47efb7c8520a011] that reads the values
drh [Sun, 22 Dec 2019 18:55:04 +0000 (18:55 +0000)] 
Fix to the optimization of check-in [a47efb7c8520a011] that reads the values
of expressions used in an index-on-expression directly from the index rather
than recomputing the value.  If the expression has a top-level COLLATE or
unlikely() operator, be sure to clear the corresponding flags prior to
converting it into a TK_COLUMN expression.  Failure to do this is most likely
harmless in production, but might cause an assertion fault in debugging builds.
Ticket [b0cb8aff561a6dcd].  Test cases in TH3.

FossilOrigin-Name: 56539e1c132632c075efc217ad5951a35e4459605ac128914aec3a7be1e25718

5 years agoBack out the asserts of check-ins [a500893b6f64aced] and [d9c9fe9f5ad3fc91]
drh [Sun, 22 Dec 2019 18:23:30 +0000 (18:23 +0000)] 
Back out the asserts of check-ins [a500893b6f64aced] and [d9c9fe9f5ad3fc91]
as ticket [1b06916e01c82b66] demonstrates a case that refutes them.

FossilOrigin-Name: ddb17d92df194337a103c561ef46ced00db5e67e4551e82de7cb5ad7f457dd5f

5 years agoWhen parsing a CREATE TABLE from the sqlite_master table, delete the CHECK
drh [Sun, 22 Dec 2019 18:06:49 +0000 (18:06 +0000)] 
When parsing a CREATE TABLE from the sqlite_master table, delete the CHECK
constraints if there are any errors, since there might otherwise be attempts
to use those CHECK constraints if PRAGMA writable_schema=ON is set.
This undoes the fix in check-in [ea721b34477ab8b4] for a more general
solution.

FossilOrigin-Name: a982e6434cd66bfbe94d455f538bcbc4360b91572062d92acae6b77e9560e65d

5 years agoEnsure sqlite3WindowRewrite() is called on a SELECT statement before any terms aremov...
dan [Sun, 22 Dec 2019 17:32:25 +0000 (17:32 +0000)] 
Ensure sqlite3WindowRewrite() is called on a SELECT statement before any terms aremoved from it as part of IN() clause processing. Fix for [f00d096ca].

FossilOrigin-Name: 8c856404b4e98d295449a4e89a41495dc007319a8e9c35c1a763718d7c5f67e8

5 years agoFix "PRAGMA data_version" so that it works the same way with locking_mode=PERSIST...
dan [Sun, 22 Dec 2019 14:29:55 +0000 (14:29 +0000)] 
Fix "PRAGMA data_version" so that it works the same way with locking_mode=PERSIST and journal_mode=PERSIST configured. Fix for 7a458c2a5f.

FossilOrigin-Name: 45748e2db028ffbda5d3e747493721a5a89af3fbc06823081a2f27f570e40e73

5 years agoFix a spurious report of corruption from the fts3/4 integrity-check triggered by...
dan [Sat, 21 Dec 2019 20:43:34 +0000 (20:43 +0000)] 
Fix a spurious report of corruption from the fts3/4 integrity-check triggered by using the languageid option.

FossilOrigin-Name: 70815e273f511481c310dc33b3b49079901d891078fad49b663fbce06c77ecc2

5 years agoWhen creating a new virtual table, ensure that the OP_ParseSchema opcode
drh [Sat, 21 Dec 2019 19:37:09 +0000 (19:37 +0000)] 
When creating a new virtual table, ensure that the OP_ParseSchema opcode
processes the correct entry in the sqlite_master table even if there is
a second entry with the same name and table values due to database corruption
and the use of writable_schema=ON.  Dbsqlfuzz find.

FossilOrigin-Name: 4dbb6e1cb094f3428c74ea8bdd86ab63341fecce978a062968ca01423f382e90

5 years agoWhen a corrupt schema is loaded using writable_schema=ON, the CHECK constraints
drh [Sat, 21 Dec 2019 14:09:30 +0000 (14:09 +0000)] 
When a corrupt schema is loaded using writable_schema=ON, the CHECK constraints
(or other expressions in the table definition) might not be fully resolved.
Ensure that the code generator can deal with this if the table is subsequently
used in a DML statement.  dbsqlfuzz find.

FossilOrigin-Name: ea721b34477ab8b49d182352c4bc198245933b850e9b6248b4f97600e80bb44b

5 years agoDo not try to access a generated column through an index if the collating
drh [Fri, 20 Dec 2019 22:46:41 +0000 (22:46 +0000)] 
Do not try to access a generated column through an index if the collating
sequence for the generated column is non-standard.
Part 2 of ticket [e0a8120553f4b082]

FossilOrigin-Name: 056bb8dcbdc45989c5c6e86d2966200062e3c01c382ec52aae37c828104b4496

5 years agoApply real affinity to generated columns of type REAL that are extract from
drh [Fri, 20 Dec 2019 20:45:02 +0000 (20:45 +0000)] 
Apply real affinity to generated columns of type REAL that are extract from
an index.  Ticket [e0a8120553f4b082]

FossilOrigin-Name: 728ad39e3bd07a2503a95c68ed5bbc3f28fd97551d59f12a5fab25dc68227e99

5 years agoDebugging enhancment: Show the Expr.y.pTab pointer on TK_COLUMN nodes of
drh [Fri, 20 Dec 2019 20:08:56 +0000 (20:08 +0000)] 
Debugging enhancment:  Show the Expr.y.pTab pointer on TK_COLUMN nodes of
an expression tree in the treeview.

FossilOrigin-Name: 64154ac450e4366d18e7e867841877a69c1f978d3ba9b8754cc133248966731d

5 years agoFix a bad interaction between RBU and [df51ae19].
dan [Fri, 20 Dec 2019 20:03:21 +0000 (20:03 +0000)] 
Fix a bad interaction between RBU and [df51ae19].

FossilOrigin-Name: 0b9d8a1202c4220fd2ef299b6194533c1bf4018a0cd2d13da7e22c1a7de05ffa

5 years agoFix two more cases in fts5 where sqlite3_value_bytes() was being called before sqlite...
dan [Fri, 20 Dec 2019 19:41:01 +0000 (19:41 +0000)] 
Fix two more cases in fts5 where sqlite3_value_bytes() was being called before sqlite3_value_text(). Fix for e431c355.

FossilOrigin-Name: a1ba9a37d7a68a6d31f8197c6350589ebe6a12f4e3c193a178dd7ead8bcd565a

5 years agoExport the public RBU entry points from the Win32 DLL.
mistachkin [Fri, 20 Dec 2019 17:41:15 +0000 (17:41 +0000)] 
Export the public RBU entry points from the Win32 DLL.

FossilOrigin-Name: e62d1791f428b7a9bb14a59483f12d8ee08afbf4061a5e3be8f64787444301e7

5 years agoDebugging improvement: when tracing the VDBE, output the register value for
drh [Fri, 20 Dec 2019 17:25:10 +0000 (17:25 +0000)] 
Debugging improvement: when tracing the VDBE, output the register value for
OP_Cast after the cast is completed.

FossilOrigin-Name: dc5f1d282d1e75f8a3791b0c95cb8f143b2a956323cb6c889d76d20db7f89257

5 years agoWhen computing dependencies on WHERE-clause terms, be sure to take into
drh [Fri, 20 Dec 2019 15:35:56 +0000 (15:35 +0000)] 
When computing dependencies on WHERE-clause terms, be sure to take into
account the FILTER clause of aggregate functions.  Problem reported by
Manuel Rigger.

FossilOrigin-Name: 3cc2b5709e66ef605c3b85ed13ae6cc9c3090e864ebc18859db36284b8f715b6

5 years agoImproved assert() statements on the OP_Column opcode.
drh [Fri, 20 Dec 2019 14:37:56 +0000 (14:37 +0000)] 
Improved assert() statements on the OP_Column opcode.

FossilOrigin-Name: c538601e4c979ee59f18d19e6eb6296cebfa580fb029a18d5a3f0dd9393acb75

5 years agoLoad new dbsqlfuzz test cases into test/fuzzdata8.db.
drh [Fri, 20 Dec 2019 14:24:26 +0000 (14:24 +0000)] 
Load new dbsqlfuzz test cases into test/fuzzdata8.db.

FossilOrigin-Name: 01aeeb5c4c70a0f19651448e94ad5c6810cab3d6957b5bc7a9b4b8e0b887f3de

5 years agoFix a potential problem with ALTER TABLE commands on schemas that contains a triggers...
dan [Fri, 20 Dec 2019 14:18:29 +0000 (14:18 +0000)] 
Fix a potential problem with ALTER TABLE commands on schemas that contains a triggers or view featuring a sub-query that uses an ON clause with a join that is not a LEFT JOIN.

FossilOrigin-Name: 365dd8b663d5a28af33556b2b3820c6b5482114ef741a1d82b55c6e154f3043a

5 years agoFix harmless compiler warnings.
drh [Fri, 20 Dec 2019 14:08:34 +0000 (14:08 +0000)] 
Fix harmless compiler warnings.

FossilOrigin-Name: 2bbd014c900e0d71fd30ec647739b82c5dc3aa237a0203f5a1db50593dae199e

5 years agoEnsure fts5 consistently calls sqlite3_value_text() before sqlite3_value_bytes()...
dan [Fri, 20 Dec 2019 13:24:11 +0000 (13:24 +0000)] 
Ensure fts5 consistently calls sqlite3_value_text() before sqlite3_value_bytes(). This matters when storing blobs in fts5 tables within utf-16 databases. Fix for [cf36cb4e02].

FossilOrigin-Name: c16305eba0eb14368f2631b9a82b7eb4483c93f3b31257ee7c33f9b00c344379

5 years agoDo not continue an ALTER TABLE tree walk of a subquery if a prior error
drh [Fri, 20 Dec 2019 12:55:21 +0000 (12:55 +0000)] 
Do not continue an ALTER TABLE tree walk of a subquery if a prior error
is seen in the tree.  Report the problem immediately, without continuing.

FossilOrigin-Name: 01ca865f31c0c817ede7f357401ef76cf311ae662397908464f53e1c147f5b6b

5 years agoFix an assert() in btree.c that can fail if an INSERT is attempted on
drh [Fri, 20 Dec 2019 12:33:17 +0000 (12:33 +0000)] 
Fix an assert() in btree.c that can fail if an INSERT is attempted on
a corrupted database while in PRAGMA writable_schema=ON.

FossilOrigin-Name: 07beb3629e648e7feae4e786191514339ac1d8f6ba5bb6e7cc8a04bc1f1cd6ab

5 years agoContinuation of [e2bddcd4c55ba3cb]: Add another spot where it is necessary
drh [Thu, 19 Dec 2019 22:08:19 +0000 (22:08 +0000)] 
Continuation of [e2bddcd4c55ba3cb]: Add another spot where it is necessary
to abort early due to prior errors in sqlite3WindowRewrite().

FossilOrigin-Name: cba2a2a44cdf138a629109bb0ad088ed4ef67fc66bed3e0373554681a39615d2

5 years agoFix the regexp extension so that it correctly translates all over-length
drh [Thu, 19 Dec 2019 21:11:48 +0000 (21:11 +0000)] 
Fix the regexp extension so that it correctly translates all over-length
3-byte UTF8 sequences into 0xfffd.

FossilOrigin-Name: 3d4c0bf8904135fa68c75801bfa738715cacc3b19dc8ad6ef550b11798d4b121

5 years agoWhen an error occurs while rewriting the parser tree for window functions
drh [Thu, 19 Dec 2019 20:37:32 +0000 (20:37 +0000)] 
When an error occurs while rewriting the parser tree for window functions
in the sqlite3WindowRewrite() routine, make sure that pParse->nErr is set,
and make sure that this shuts down any subsequent code generation that might
depend on the transformations that were implemented.  This fixes a problem
discovered by the Yongheng and Rui fuzzer.

FossilOrigin-Name: e2bddcd4c55ba3cbe0130332679ff4b048630d0ced9a8899982edb5a3569ba7f

5 years agoDetect the invalid use of NULLS LAST on an INTEGER PRIMARY KEY definition
drh [Thu, 19 Dec 2019 17:42:27 +0000 (17:42 +0000)] 
Detect the invalid use of NULLS LAST on an INTEGER PRIMARY KEY definition
of a WITHOUT ROWID table.

FossilOrigin-Name: 4417c5bf0aabb34ed174f01afd981c924ae965a42128719d8d6735536631d12f

5 years agoFix the zipfile extension so that INSERT works even if the pathname of
drh [Thu, 19 Dec 2019 15:15:40 +0000 (15:15 +0000)] 
Fix the zipfile extension so that INSERT works even if the pathname of
the file being inserted is a NULL.  Bug discovered by the
Yongheng and Rui fuzzer.

FossilOrigin-Name: a80f84b511231204658304226de3e075a55afc2e3f39ac063716f7a57f585c06

5 years agoFix incorrect SQL generated by the Lemon change of the previous check-in.
drh [Thu, 19 Dec 2019 13:17:07 +0000 (13:17 +0000)] 
Fix incorrect SQL generated by the Lemon change of the previous check-in.

FossilOrigin-Name: fccfb8a9ed3c1df9f23762bb8df6fdf36a21118899e3fae41f451169a5f2c08e

5 years agoModify the parse.sql output file from Lemon so that the RULE table contains
drh [Thu, 19 Dec 2019 12:29:31 +0000 (12:29 +0000)] 
Modify the parse.sql output file from Lemon so that the RULE table contains
a column with the complete text of the rule.

FossilOrigin-Name: 329cbb372d4d77f774f01a87d104c40ef42d713afbe76195b65f5ca205c5ea7b

5 years agoChange an unreachable testcase() into an assert()
drh [Wed, 18 Dec 2019 21:22:40 +0000 (21:22 +0000)] 
Change an unreachable testcase() into an assert()

FossilOrigin-Name: 062e00c4d7e30155299339eaefe6cbe01953894baebf0af6d7ab3295b7436eb7

5 years agoContinue to back away from the LEFT JOIN optimization of check-in [41c27bc0ff1d3135]
drh [Wed, 18 Dec 2019 20:51:58 +0000 (20:51 +0000)] 
Continue to back away from the LEFT JOIN optimization of check-in [41c27bc0ff1d3135]
by disallowing query flattening if the outer query is DISTINCT.  Without this fix,
if an index scan is run on the table within the view on the right-hand side of the
LEFT JOIN, stale result registers might be accessed yielding incorrect results,
and/or an OP_IfNullRow opcode might be invoked on the un-opened table, resulting
in a NULL-pointer dereference.  This problem was found by the Yongheng and Rui fuzzer.

FossilOrigin-Name: 862974312edf00e9d1068115d1a39b7235b7db68b6d86b81d38a12f025a4748e

5 years agoAdd the --vdbe-debug option to fuzzcheck.
drh [Wed, 18 Dec 2019 13:42:04 +0000 (13:42 +0000)] 
Add the --vdbe-debug option to fuzzcheck.

FossilOrigin-Name: 289158aa24b066c453d2bce4bc2dead1c56fb0b23c3f7c4810b34b13627cef34

5 years agoEnhancements to fuzzcheck on unix so that it captures signals and prints
drh [Wed, 18 Dec 2019 13:02:18 +0000 (13:02 +0000)] 
Enhancements to fuzzcheck on unix so that it captures signals and prints
the current test file and test number prior to terminating.

FossilOrigin-Name: ae7cbb246bff3717c283869ccf27ce83611422f3801176819465a8d96f6481bf

5 years agoAdd a test case for fix in check-in [1ca0bd982ab1183b]
drh [Wed, 18 Dec 2019 09:17:55 +0000 (09:17 +0000)] 
Add a test case for fix in check-in [1ca0bd982ab1183b]

FossilOrigin-Name: 519864da8bb671941a64bf333c2086ad9a068b5f85759e7abd368f0706d3aefd

5 years agoCorrectly unwind the savepoint stack if the database goes read-only in
drh [Wed, 18 Dec 2019 02:12:04 +0000 (02:12 +0000)] 
Correctly unwind the savepoint stack if the database goes read-only in
the middle of a savepoint.

FossilOrigin-Name: a96d02b4ab8c70cc20b322e4fa3a4b8814f05c51f5d8b071ec9d69e7379f3ea5

5 years agoRemove an assert() in the code generator that can fail due to an invalid
drh [Wed, 18 Dec 2019 01:31:21 +0000 (01:31 +0000)] 
Remove an assert() in the code generator that can fail due to an invalid
INSERT statement.  The same assert() appears again in the VDBE and will fire
there if the problem persists in an SQL statement that does not contain
errors.  Gramfuzz find.

FossilOrigin-Name: 9bdd57cb9ad8ac1f4a0f1a3942d94ca5193109eeb1975343c4f0c7abf7135946

5 years agoWhen processing constant integer values in ORDER BY clauses of window
drh [Wed, 18 Dec 2019 00:05:50 +0000 (00:05 +0000)] 
When processing constant integer values in ORDER BY clauses of window
definitions (see check-in [7e4809eadfe99ebf]) be sure to fully disable
the constant value to avoid an invalid pointer dereference if the expression
is ever duplicated. This fixes a crash report from Yongheng and Rui.

FossilOrigin-Name: 1ca0bd982ab1183bbafce0d260e4dceda5eb766ed2e7793374a88d1ae0bdd2ca

5 years agoClean up the ExprList that holds the names of columns in a CTE before checking
drh [Tue, 17 Dec 2019 12:03:30 +0000 (12:03 +0000)] 
Clean up the ExprList that holds the names of columns in a CTE before checking
for unused references in the ALTER TABLE implementation.

FossilOrigin-Name: 8223e79f987feda5c8e51ec52cec6798cca16d070b10558939e2888ca1a25b8e

5 years agoWhen a table is part of a LEFT JOIN and should be a completely NULL row due to
drh [Mon, 16 Dec 2019 16:52:22 +0000 (16:52 +0000)] 
When a table is part of a LEFT JOIN and should be a completely NULL row due to
the semantics of a LEFT JOIN, make sure any generated columns on that row
evaluate to NULL.  Ticket [3b84b42943644d6f]

FossilOrigin-Name: 0271491438ad2a985aeff355173a8d0f1e5813954c82147bc68cb26cca5804c8

5 years agoEnsure that all ON CONFLICT REPLACE indexes are sorted to the end of the list
drh [Sun, 15 Dec 2019 02:49:32 +0000 (02:49 +0000)] 
Ensure that all ON CONFLICT REPLACE indexes are sorted to the end of the list
of indexes for a table, even for weird cases where the same UNIQUE constraint
occurs twice with the ON CONFLICT REPLACE clause only on the second one.  This
avoids an out-of-order contraint processing problem that can arise due
to the optimization of check-in [469a62ca33081854].

FossilOrigin-Name: 1e3918ca2f2c1cfcfa44249b1d7b847d52cbb8d302a8d4a335c090cfdf22d7a1

5 years agoAbort the OP_Savepoint opcode early if a virtual table xSync method fails,
drh [Sun, 15 Dec 2019 00:36:33 +0000 (00:36 +0000)] 
Abort the OP_Savepoint opcode early if a virtual table xSync method fails,
perhaps due to an interrupt.

FossilOrigin-Name: 672e749aef7351de3c69b365c1f80c756fda4e261b5d2ac1faa01d3a7d5a4c49

5 years agoWhen a generated column expression is initially parsed, if it contains errors
drh [Sat, 14 Dec 2019 19:55:31 +0000 (19:55 +0000)] 
When a generated column expression is initially parsed, if it contains errors
convert the expression to a simple NULL.  This prevents additional pieces of
the expression tree that are allocated from lookaside from leaking into the
schema, where lookaside allocations are not allowed.

FossilOrigin-Name: a89b38605661e36d192df1d074383b704b34a959fcbc169837c0d4f6848f3208

5 years agoDue to the previous change, the p5 parameter to OP_VColumn no longer ever
drh [Sat, 14 Dec 2019 18:08:22 +0000 (18:08 +0000)] 
Due to the previous change, the p5 parameter to OP_VColumn no longer ever
contains extraneous bits, so change a testcase() into an assert() to show
as much.

FossilOrigin-Name: 5b4a88cd3b3a32b1242e0eb733cc0bc4becb830a887b8e6604a4c7452050e163

5 years agoDo not set OP_Column flags on the instructions generated by
drh [Sat, 14 Dec 2019 17:43:37 +0000 (17:43 +0000)] 
Do not set OP_Column flags on the instructions generated by
sqlite3ExprCodeGetColumn() if the opcode generated is not really an
OP_Column, which might happen if the column is virtual.
Fix for ticket [b439bfcfb7deedc6]

FossilOrigin-Name: 2401e04730a156aa48787b91af4e516406cb7635145e430be62fd16481816237

5 years agoMake the sqlite3ExprCodeTarget() expression code generator routine robust
drh [Sat, 14 Dec 2019 15:01:55 +0000 (15:01 +0000)] 
Make the sqlite3ExprCodeTarget() expression code generator routine robust
in the face of unknown opcodes - it simply generates a NULL value.

FossilOrigin-Name: f8e876c82a246ceed32b166f64e05dfe5ce4ab4c6820be60404109b43d36bb80

5 years agoFix problems in fts3 with processing corrupt records and undefined integer overflows.
dan [Sat, 14 Dec 2019 14:15:21 +0000 (14:15 +0000)] 
Fix problems in fts3 with processing corrupt records and undefined integer overflows.

FossilOrigin-Name: 3b873029ef1903f76442445343af3786cd59ee66b0e5e0e6b76a0b6b1b5eb88a

5 years agoFix an utterly harmless "applying zero offset to null pointer" UB warning
drh [Fri, 13 Dec 2019 23:38:57 +0000 (23:38 +0000)] 
Fix an utterly harmless "applying zero offset to null pointer" UB warning
in sqlite3AtoF():

FossilOrigin-Name: 052fdf5e58b41ccadaa5aac293ceb4d309ced661d46f3a52be9eb8d01d347a82