Commit Graph

20409 Commits

Author SHA1 Message Date
drh
88862d49a6 Enhance dbfuzz2 so that with the -v option it shows the return code and
error message for any failing SQL statements.

FossilOrigin-Name: 3a127ef9f7feafe6ba8c75e4eb29e28aa61a30249082cc8767ada1ec0cc0b7f1
2019-02-04 19:45:26 +00:00
drh
f2305414cd Ensure that the sqlite3_exec() callback gets the correct number of columns
for a query, even if the schema changes out from under it.

FossilOrigin-Name: a16ffb5a4b30f1ac4a8384fccde0a0f69e49545b6f9be9267928609328c4a151
2019-02-04 19:12:54 +00:00
dan
4fc4ba771c Do not run shmlock.test as part of the inmemory_journal permutation.
FossilOrigin-Name: d3fea69cbb071867a90ad56ca05123ebf8e44689a92577a525a883c176c791bd
2019-02-04 16:42:57 +00:00
drh
e2d2ad6ece Update the test/fuzzdata8.db database with new interesting cases from
dbsqlfuzz.

FossilOrigin-Name: e467e4c8fa256ddd6c26c1f21bc26289a21f18b781737eaf706bc813ed8234fa
2019-02-04 16:36:24 +00:00
dan
83a635a267 Fix problems with SAVEPOINT and related commands within transactions on a corrupt database that include fts5 or some other virtual tables.
FossilOrigin-Name: b0da6e0dd1f9a9ba7646ee6bb3c14945e570303c9a42d20fa8a653570e12c3f9
2019-02-04 16:25:40 +00:00
drh
3f83f59154 Add the ".progress" command to the CLI.
FossilOrigin-Name: ad84ca24ff760f7a2b128f2eeb4d799a97f84b8b8a8bb8d9657c4c555f6ccf47
2019-02-04 14:53:18 +00:00
dan
0062ddd4c9 Fix another potential buffer overread in fts5.
FossilOrigin-Name: 14fba4cc5db2bc206f024bfa5dd82fffdbc7577162da2c3b15a91cc1c91f3846
2019-02-04 14:00:20 +00:00
dan
81f44cdbfb Fix another corruption case in fts5.
FossilOrigin-Name: e608085d3f7e3cc0015fde626e205ee9842ef3cd06dc879fbbc7f2067c592965
2019-02-04 11:54:22 +00:00
dan
41fc545147 Fix an assertion failure in fts3 triggered by a corrupt database.
FossilOrigin-Name: 560ebd4269fad07b8e988d070454017123593eb6762597d08e8c32d7b4fb92ac
2019-02-03 07:46:07 +00:00
drh
150edadf58 New testcase added to test/fuzzdata8.db.
FossilOrigin-Name: 2f5c1925b94d85489533549c58191feb1d803ad9d58b43f47fef42b72c531a38
2019-02-02 18:24:28 +00:00
drh
e5989723ba Honor key query parameters for SEE on the URI filename for ATTACH
and VACUUM INTO.

FossilOrigin-Name: 2e01096b8933a2315e3dffcd7d0de84c744c1b4be1d909388c61f0fd636ddb99
2019-02-02 15:59:49 +00:00
drh
9b2bd91d9a Give a link to the URI Filename document in the documentation to the
various sqlite3_uri_parameter() interfaces.

FossilOrigin-Name: 69b0b2ee536fda8975f60643f7f1891db4649958aca476822ea2a632ce731992
2019-02-02 15:05:25 +00:00
dan
8f5b14a5c2 Fix a buffer overread in fts5 debugging scalar function fts5_decode().
FossilOrigin-Name: 54f2399fb2a626602d405c857297f2da833f9f048cbc478f9110bed2e9bda299
2019-02-02 13:47:25 +00:00
mistachkin
844b900491 Fix harmless compiler warning.
FossilOrigin-Name: dddda685f3443d8a38901f758543fcde73d7b8cfe72b0ad5f419cd7459343bf5
2019-02-02 01:27:45 +00:00
drh
7ba03ea15a Slight adjustment to the printf formatter large memory allocation detector
so that it does not overestimate the amount of space needed for
oversize %d conversions.

FossilOrigin-Name: 1aee70d6de8a9b17ebb74a7cb1dad65139cde1b615dcce4d15d3a476fda8676b
2019-02-01 21:08:27 +00:00
drh
2964225247 Prevent the printf formatter from doing large memory allocations - larger
than either the size of the static buffer for interfaces like
sqlite3_snprintf(), or larger than SQLITE_LIMIT_LENGTH for interfaces that
are associated with a database connection.  This helps to prevent DOS
attacks on products that let hostile sources inject arbitrary SQL.  It also
helps fuzzers run faster and more effectively.

FossilOrigin-Name: 179e5d46054e5c86f53a79b7a0823d9a383da8391ad1d3c3b22645927a1e052b
2019-02-01 20:29:04 +00:00
drh
9a6d01bff5 Performance improvement in the parsing of options to %-formats in the
printf implementation.

FossilOrigin-Name: 40d8f8ae87abf928542c4e558a4c3a3eab18776a3e8db7ca1c5e5f744ca0bce3
2019-02-01 18:46:41 +00:00
drh
9c46c66c46 Ensure that the Walker.pParse structure is initialized when walking the
source tree to gather the WINDOW clauses on a query with window functions.

FossilOrigin-Name: 4ca9d5d53d41d08fbce29f9da8cc0948df9c4c3136210af88b499cf889b5ccb8
2019-02-01 15:06:27 +00:00
drh
38a28a7434 New test cases added to test/fuzzdata8.db.
FossilOrigin-Name: e5924939c9c8f39367900360fa294cdb1220e9f68d183e32882e0926f775869d
2019-02-01 14:54:18 +00:00
drh
26b7ec8b24 Improve the strict enforcement of cell sizes in balancing from
check-in [12713f320b2c1def] so that it also works with table-btrees
in addition to index-btrees.

FossilOrigin-Name: ef27e7a08728aa7447ae19812803ac5c4a9d80c97541014bd292485792005a3e
2019-02-01 14:50:43 +00:00
dan
1c7d389a23 Fix an assert() in fts5 that could fail if the database is corrupt.
FossilOrigin-Name: 55f06aa3f890dc1dc5c2ed0e6777e46d609a6f9d223d1d4b5813097ce4f4e797
2019-02-01 14:40:44 +00:00
dan
abd7449478 Fix another corrupt-database-handling problem in fts5.
FossilOrigin-Name: 9876ae1b802a449aacbbdd42778653edc19e29315dc627c23ab7493d6492c800
2019-02-01 13:34:00 +00:00
drh
3bf0d7a881 Remove an incorrect assert() from the deserialize in-memory database VFS.
FossilOrigin-Name: 80151d7e3b9fd84086874a1cebed024b9e14005ea2d0108cf6efcd07373fcf51
2019-01-31 16:09:48 +00:00
drh
23a885956e Add the SQLITE_CONFIG_MEMDB_MAXSIZE configuration option for configuring
the default maximum size of an in-memory database created using
sqlite3_deserialize().  This is necessary to make the interface reasonably
testable.

FossilOrigin-Name: cb72ee0478ce98c48aae059fd5de4e36caf2b8c953e08fcb799bfd119ad46b73
2019-01-31 15:38:53 +00:00
dan
b92b019fe3 Fix another buffer overrun that could occur when quering a corrupt database using an fts5vocab table.
FossilOrigin-Name: b80119bd6044c0993210115732b6ae7b4b0b29c1471b0723bef24f2052c8e671
2019-01-31 14:37:18 +00:00
drh
6a0db87a2f If the amalgamation file "sqlite3.c" is generated using the
SQLITE_ENABLE_UPDATE_DELETE_LIMIT option, but then subsequently (mis-)compiled
without that option, and then a particular syntax error could cause
a memory leak, but for this fix.

FossilOrigin-Name: 1bb17c863a269ab8b28df94b9fdaac3ee70777ccd0a30300131bf67df6722052
2019-01-31 02:42:47 +00:00
drh
9b747068ba Improvements to "PRAGMA parser_trace=ON": Make it a flag pragma, and then
use the flag to show the complete SQL text at the beginning of the parse.

FossilOrigin-Name: 507c43537f00d089efb3c3020fb6e826e8f89f3efa2caaa12c5309d35dd0c22e
2019-01-31 01:39:01 +00:00
drh
8bf4126c84 Change the error message slightly for when two or more indexes have the
same rootpage.

FossilOrigin-Name: 916c52dad414b224695d455a42af36f85dff216504c2aa67d1d2c5818cdca8d6
2019-01-30 19:50:07 +00:00
dan
da7a4c0f7a When parsing the schema, check for duplicate index root page numbers before adding the index to the schema hash table.
FossilOrigin-Name: 6b360bc0b9ad18ccda9d58d43a8388d19999432bfdf294abcf5beb44ef990cb4
2019-01-30 19:12:13 +00:00
dan
56b99ebc32 Add test case for the previous commit.
FossilOrigin-Name: 197edb235b785c68fda7be788efbc61137d9452886599415a927b959ade93c65
2019-01-30 18:47:35 +00:00
drh
8d40673c84 Detect if two indexes of the same table share a common rootpage while
parsing the schema, and throw an error immediately.

FossilOrigin-Name: f21c6f1343edc4ba68f9f8e68caac34da0d1c651526ecaf61c0e2d66593a555e
2019-01-30 18:33:33 +00:00
drh
ba7bd02002 Fix an off-by-one error when parsing the names of indexes that do not have
arguments in the index_usage utility.

FossilOrigin-Name: dc794d8f518ed46536928a77a560d46e4f9365c30a00cd108f000d8556a66c00
2019-01-30 16:58:04 +00:00
drh
a8614259c1 Add the --progress, --using, and -q options to the index_usage utility program.
FossilOrigin-Name: a5e6be7cbc5d931308ddcc073c9cd6275c9711cae055d72a7c4aa71c1d15914c
2019-01-30 15:47:38 +00:00
drh
5a9c6bcc15 Enhancements to the index_usage utility program.
FossilOrigin-Name: 19c739b4a8a43d894e37a99fa34838f3e3fa1fe0d019aefbc33f1d38d76af1a4
2019-01-30 14:01:43 +00:00
dan
dd579233e8 Fix another buffer overread in fts5 that may occur when accessing a corrupt database.
FossilOrigin-Name: 760d14374d40bcd9ce3a89771c18dc236c9728553c4747c9b7452ee7b24f4140
2019-01-30 12:15:27 +00:00
drh
3f4f6823ac Fix harmless compiler warnings.
FossilOrigin-Name: 9a93c68a7673bda2e6c308d3b2798d0fb12fde399996cae24b1f0d5864adf355
2019-01-29 16:54:31 +00:00
drh
4551005332 Load all the latest dbsqlfuzz finds into test/fuzzdata8.db.
FossilOrigin-Name: e744d2dd93a78235e6c1d938b08232183b5ce7e355d1b7daf2199e585b03c106
2019-01-29 16:47:46 +00:00
drh
a046643820 More robust handling of corrupt database file in the rebalance operation of
the btree logic.

FossilOrigin-Name: 97704cb7d29fa7cc4ea9a6761a7844c1946d637ea2b22d287fc787ae0f63c407
2019-01-29 16:41:13 +00:00
dan
e25646d64c Avoid an integer overflow in the fts5 snippet() function triggered by a corrupt database record.
FossilOrigin-Name: 7c862c4655841e1263cf548d01f5b4331685ab7d396494891cad0a5f50c63e16
2019-01-29 16:34:45 +00:00
dan
eb2ed10ec0 Avoid a crash when the fts5 snippet() function (or similar) is used with a special query pattern, like '*id' or '*reads'.
FossilOrigin-Name: 9d58a15737f7379fed724749cfd5b4de5b44231870e99fc664683ad483c3e6ba
2019-01-29 15:30:03 +00:00
dan
3993397e6c Fix a buffer overrun triggered by a merge operation on a corrupt fts5 database.
FossilOrigin-Name: 7ee3cd5b2cdf8db35f70400b3df58f7c421c78ae77c50f22b0ee22d5a3039429
2019-01-29 11:42:43 +00:00
drh
4a7e9a25b6 Set a low limit on the length of strings and blobs in the OSSFuzz module,
in an effort to avoid timeouts when the fuzzer does things like
"randomblob(1e12)".

FossilOrigin-Name: 119d1609fa415894fdd0aa19440ca1c658bd01910838a4cfadbe0e304b033bfd
2019-01-29 02:37:22 +00:00
drh
3967d946c7 Fix the query planner so that it is able to use an index on a CAST expression.
FossilOrigin-Name: 2c886f3d49c9569e9f6ee28b89b57e2d4004c8972d11da0c23f7b2d2e0c9900a
2019-01-28 19:06:17 +00:00
drh
e86974c619 Fix a performance regression caused by the previous commit.
FossilOrigin-Name: c4db0ad12d4f3d2800d36404f391b325cdc4aa7f8dcea93b2d63a489d9095ad4
2019-01-28 18:58:54 +00:00
dan
95d5a88058 Make indexes on CAST(...) expressions work.
FossilOrigin-Name: 3ef711d98fb239cf24472f124e7b36b0dde33355de5a2c9a3a978bbdd042a735
2019-01-28 18:08:59 +00:00
dan
c7def600bd Fix a buffer overread in fts3 that could occur when accessing a corrupt database.
FossilOrigin-Name: a9faf9033910927c74553e66c474d84ed3386f263cef3eec299e25d8306f410d
2019-01-28 16:50:42 +00:00
dan
6c43dbad98 Do not assume that text parameters passed to fts4aux queries do not contain embedded nul characters.
FossilOrigin-Name: df04859a995571cd6faf1abe088725708f35e81195760274df9e2ec9bd47f69f
2019-01-28 13:27:25 +00:00
drh
9e35a61e59 Add a new test case to fuzzdata8.db.
FossilOrigin-Name: ce8e279a7d37594e865c2d9197a5efe5d647e11f074dbaf4a4ee31c74c71dbc9
2019-01-28 11:54:13 +00:00
drh
0c873bf28a Change the definition of the BtCursor.skipNext field so that it is undefined
for states CURSOR_VALID and CURSOR_INVALID.

FossilOrigin-Name: 6089d5dba73437d787259033e94a38cf1b0c87cac4a89c27e69d8239c22f496e
2019-01-28 00:42:06 +00:00
drh
c07eee7a50 Improvements to the ".eqp trace" command in the CLI so that it ensures that
the schema has been read prior to activing the tracing feature, to avoid
cluttering the output with a trace of the schema parse.

FossilOrigin-Name: 42687d45aa1a2b2a32ddfc5711b4ae9cf76b8cfbeb20b57d6705ecaf5824fcb6
2019-01-27 19:50:56 +00:00
drh
05d49d1442 New test cases form dbsqlfuzz
FossilOrigin-Name: 9cf8ebd141aa2eb661d457624c76433bd9e4abfdef04aa52e28bc169172c2c8f
2019-01-27 02:45:32 +00:00
drh
211a0857b9 Window functions that can abort should indicate this, so that if they are
used in DML statement, a statement journal will be used.

FossilOrigin-Name: 0ea05a0eb96acb860d8af2e788e29a5ae3b161e7892ab62f1ff2f3e722c0dcb6
2019-01-27 02:41:34 +00:00
drh
e5da9356e0 Update fuzzcheck so that with the --load-dbsql options it screens its
inputs to ensure that they are valid dbsqlfuzz cases.  Add new dbsqlfuzz
finds to the test/fuzzdata8.db database.

FossilOrigin-Name: 004f7d9bbeee049c22a7e1c82c6cc0ea79a237967de3ff57686acea3060d5f45
2019-01-27 01:11:40 +00:00
drh
e6a9efb976 Ensure that the btree cursor is correctly set up prior to backing it up
as part of a delete operation on a divider cell in an index.

FossilOrigin-Name: a3ea1a822d3a110f4f186f2fc8550f435c8c98635d058096b7be9d4df7066b8b
2019-01-26 23:34:50 +00:00
dan
32bb700a95 Fix a broken assert() in fts3. Also some test script issues causing failures with builds that do not support fts3.
FossilOrigin-Name: d59567dda231e7ffec437d5e055676cf42d791196406cdc696cafa2583eb483b
2019-01-26 21:05:54 +00:00
drh
b4e5039316 Add the ".eqp trace" command to the CLI when using SQLITE_DEBUG, as a
convenient shorthand for "PRAGMA vdbe_debug=ON" but with automatic indentation
feature for program listings provided by the CLI.

FossilOrigin-Name: 626502faa17b5b7a0fa36cabfd12e463eb09aec048d01c587a18d3977cf04662
2019-01-26 15:40:04 +00:00
drh
0814acd981 Do not accidently truncate zeroblob values when doing an arithmetic
operation.  Fix for ticket [bb4bdb9f7f654b0bb9f34cfba].

FossilOrigin-Name: 13f6942eb0da2d92a0830f18640ce64208bd0cd6ff6d0c97e4a4c57ac3d65ba6
2019-01-25 20:09:04 +00:00
drh
b2e61bc66b Reinitialize debugging information on registers of a trigger prior to each
invocation of the trigger.  This prevents false-positives from the
sqlite3VdbeMemAboutToChange() test logic.

FossilOrigin-Name: 98b3b97573ccb3d986d1552f97f96e15088a598945923adcac77010c1e8b7fe4
2019-01-25 19:29:01 +00:00
drh
f69af053dc Fix harmless compiler warnings.
FossilOrigin-Name: 3ff220ad378d547538b4501e6b2201e835a2c66891e5e40856db377c101a50e5
2019-01-25 18:17:37 +00:00
drh
0578084507 Fix the assert_fts3_nc() macro so that it works for test builds that omit
SQLITE_DEBUG.

FossilOrigin-Name: b10df2b569e356551cf51ca179f307e4f8827413ee9feb886db1f1e0650e6c5f
2019-01-25 18:17:03 +00:00
drh
ed457037c4 Fix harmless compiler warnings in fuzzcheck
FossilOrigin-Name: a82539665718c8183d9d0bb7bcb3adcb1a23294b8c30f364829a982dab89a332
2019-01-25 17:51:06 +00:00
dan
c2ea77e5b2 Fix a segfault that could follow an OOM when querying a table that has one or more columns with default values "true" or "false".
FossilOrigin-Name: 202f9919c222ee933924c63c15ad36ec4481457b1e4d9179af14c9f284117c0c
2019-01-25 17:26:59 +00:00
dan
e88609f23e Fix asan warnings in fts5 triggered by corrupt databases - passing NULL to memcmp, out-of-range left-shift values and signed integer overflow.
FossilOrigin-Name: 93f8ec146d63af13f04e337ada4fa75e9254f72b1394df09701ae12e185f27e2
2019-01-25 16:54:06 +00:00
dan
c1f6618217 Add an assert_fts3_nc() macro to fts3 - for assert() conditions that are only true when it is guaranteed that the fts3 database is not corrupt.
FossilOrigin-Name: 3498908cd7f3d0e35b70796537124e3da0bd99d48750ba51bcb9eba87e28ed4d
2019-01-25 14:48:18 +00:00
drh
31ab7d8e5c Extend fuzzcheck so that it can process dbsqlfuzz cases. Add a collection
of interesting dbsqlfuzz cases to the standard test suite.

FossilOrigin-Name: e2991a7ecf5dd241c4bab7c7cadc4901043fb0c3ea1fac70b684f6d8332677f1
2019-01-25 14:23:36 +00:00
drh
94f0a83421 Fix the xFetch method of the "memdb" VFS (used by deserialize) so that it
is robust against corrupt database file.

FossilOrigin-Name: 2c1ef40e787a6bc355b50168527a47eb09acd30d0d88cff8336a434ad554115d
2019-01-25 14:16:01 +00:00
dan
ba14c69b4f Fix a couple of assert() statments in btree.c that could fail with corrupt databases.
FossilOrigin-Name: 5eb5e8289fa71e5e29e081d33d4a59c64b463bf6b06d7070c05c46d77d808ad4
2019-01-25 13:42:12 +00:00
drh
725a9c7f6c In fuzzcheck, activate vdbe_debug for dbsqlfuzz cases when using the -vvvvv
verbosity level or above.

FossilOrigin-Name: 2e6f7c2aced49824a38b3494b796a8ec73aa7a90b51159f670596df15ed1c5ab
2019-01-25 13:03:38 +00:00
drh
df21659a40 Improved automatic detection of dbsqlfuzz cases in fuzzcheck.
FossilOrigin-Name: 1ef24e89c9630fd383ba32f5aefcf9c27907f27f5072f3537a1cfd75a093a8d7
2019-01-25 04:43:26 +00:00
drh
a47e709e89 Add the ability to process dbsqlfuzz cases in fuzzcheck and add an initial
set of interesting dbsqlfuzz cases.

FossilOrigin-Name: fb9074ff450a67feaa62ca61d19154de26d5c8a8d147409ee6d1fbd667b2914f
2019-01-25 04:00:14 +00:00
dan
813ed78054 Fix a buffer overread in fts3 that could occur in a prefix query on a corrupted database.
FossilOrigin-Name: d0d5689371577b2861d4a9464443d055f3256f3f51d89e0388233a4cbe2601ee
2019-01-24 17:41:12 +00:00
dan
f467744d5f Fix a problem with running ALTER TABLE on a schema that contains expressions of the type "col IN ()" (empty set on RHS of IN operator).
FossilOrigin-Name: 2d9cd06715092c312c8c0ec392696a0e90ed090b074e2082e0b830f1399aa941
2019-01-24 16:27:10 +00:00
drh
3cc9af2140 Omit deprecated PRAGMAs when compiling with the -DSQLITE_OMIT_DEPRECATED option.
FossilOrigin-Name: 008112bcef561a8a3ebfb007cbef36cbc0071f547e6ebdba3d3bbb39e2c48c7a
2019-01-24 16:07:18 +00:00
drh
491b6d897d Make sure the column name flags are restored correctly after an error
inside of sqlite3ResultSetOfSelect().

FossilOrigin-Name: b1601db7adba2dccd4f90fbbe5099c801a2b0d97af087fbe82ed98d05496b38c
2019-01-24 15:51:03 +00:00
dan
b0c4c94996 Fix a potential problem with "INSERT INTO ... SELECT * FROM" (or VACUUM) statements on a corrupted database.
FossilOrigin-Name: db4b4c2c1e9f1adacfb1b2fedb717a4d8bb0a299c3b11835404a99fcd67bf24b
2019-01-24 15:16:17 +00:00
drh
936ade4dba Change a integer variable in sqlite3VdbeRecordUnpack() to unsigned in order
to avoid any possibility of an integer overflow.

FossilOrigin-Name: 1b536f6fd8d58800042f130842f0586aaa357841ee0d1b690a9815c865d50826
2019-01-24 14:16:20 +00:00
drh
55469bbafb Use memmove() instead of memcpy() in a place where buffers might overlap
if the database file is badly corrupted, to prevent warnings
from ASAN and valgrind.

FossilOrigin-Name: 65ad6c55f1ba9bc2f75afffa3adaf19f145fad7ac9a00ccce6372e9a2cc4341b
2019-01-24 13:36:47 +00:00
drh
6a02f237e3 Remove an unreachable branch.
FossilOrigin-Name: 4c976f171342ba2d39779279c28f3bc088572c440922e3e4421a3bf541263360
2019-01-24 04:44:54 +00:00
dan
b4738ddb20 Fix an assert() in vdbemem.c that could fire if the database was corrupt.
FossilOrigin-Name: a70958cd7d5cf750c293537856918e5789013b70f6a827f724d83559b38629c8
2019-01-23 20:31:56 +00:00
dan
a5f9f42a0e Fix a problem with renaming a table within a schema that contains a composite query that uses a column alias as an ORDER BY term.
FossilOrigin-Name: 2ca6b8f84ec07e313aa4e1c0894827401b418dcc4221e9c54c384f1c3893952a
2019-01-23 19:50:46 +00:00
drh
e3dadac591 Stricter enforcement of cell sizes when doing balancing operations on the
btree, in order to catch file corruption sooner.

FossilOrigin-Name: 12713f320b2c1def273dd8b7833dddaaad5331aba779d4b1ec9aa949814f38fe
2019-01-23 19:25:59 +00:00
dan
f8f47dd8c5 Fix another fts5 crash that can occur if the database is corrupted.
FossilOrigin-Name: 44ce8baa47192be03c8f11777904c3c07fa5cc5c97b6d8e81572d380995ac688
2019-01-23 19:17:05 +00:00
dan
d99950310f Fix problems with sub-selects in WINDOW definitions. Also rename-column operations when the column being renamed appears in a WINDOW definition that is part of a VIEW or TRIGGER.
FossilOrigin-Name: 0387cb3add992b2028efe4f2100188d8f9fdfdcb233329857aa4b46a293cfc97
2019-01-23 16:59:24 +00:00
dan
4c7bdd1597 Fix a buffer overwrite triggered by a prefix query on a corrupt fts5 table.
FossilOrigin-Name: 1d8172a94bce2d067027685e5aa1dc8f5d5d55b5da0832ad2836f6939de68c8f
2019-01-23 12:19:22 +00:00
dan
ccfa550922 Fix a buffer overrun that could occur in fts5 if a prefix query is made on a corrupt database.
FossilOrigin-Name: 1abc4415648e69362061e9f9a4f2c1d419ba33801999b377650d8b9a4d2d3a7c
2019-01-22 21:17:40 +00:00
dan
8e4fe01dda Do not run shmlock.test as part of the journaltest permutation.
FossilOrigin-Name: 5b7d0c784e6467154d9549af0163d788b69d5e5a6478a735834cdea945e7dbb7
2019-01-22 20:18:29 +00:00
drh
c5ea2d474d Use _strdup() instead of strdup() on Windows builds of the CLI, to avoid
a compiler warning reported on the mailing list.

FossilOrigin-Name: a7126a4f4fb4e3e25be95d673323f835252bb2b427469616046a90b2acf64071
2019-01-22 16:44:59 +00:00
drh
f186f0b058 Fix the SQLITE_DESERIALIZE_READONLY feature so that it does not cause
an assertion fault in the pager.

FossilOrigin-Name: b9eccef7825c61980678599358b62bc394283124653061ce163ead0c653f481d
2019-01-22 16:43:47 +00:00
drh
178edcd774 Update dbfuzz2 to set a maximum database size of 100MiB by default, but
with the new --max-db-size N option to change that limit.

FossilOrigin-Name: 21d6bb78ef2979d011b917d2d6519d7cd0009fcad83ed23ab2e9a5e02d8e51ab
2019-01-22 16:11:31 +00:00
drh
6ca644818b Enhancements to deserialize: (1) Add the SQLITE_FCNTL_SIZE_LIMIT file control
to set a maximum size for an in-memory database, defaulting to 
SQLITE_MEMDB_DEFAULT_MAXSIZE or 1GiB.  (2) Honor the SQLITE_DESERIALIZE_READONLY
flag. (3) Enhance the TCL interface to support -maxsize N and -readonly BOOLEAN.
(4) Add the --maxsize option to the ".open" command and on the command-line for
the CLI.

FossilOrigin-Name: 30f08d58882819a69e353bcc1b6b349664bbfbe00aa1c115ba44a9fd899fcc5b
2019-01-22 16:06:20 +00:00
drh
247c1b4a0b Make sure cursors are opened on all indexes for an UPDATE OR REPLACE
regardless of whether or not the indexes are partial or contain columns
that might need to be updated.

FossilOrigin-Name: e148cdad35520e6684cfeba23b003f60b55f83a6bf621aff16be8aa5612cdcee
2019-01-22 13:45:48 +00:00
dan
2c7a73eaea Fix another segfault caused by a corrupt fts3 database.
FossilOrigin-Name: ba3b8412726548a0716c1a2d67260c3b7e31956474f4cd4ce607cf2cebc667dd
2019-01-22 12:21:28 +00:00
drh
9b13365bba Ensure that a key comparison does not read a collating sequence past the
end of the KeyInfo, even if the key field of an index is corrupted by having
a string in the last column instead of the ROWID.

FossilOrigin-Name: 058a8006dceda78a894ea9446f057aa60b6d38e96506d4d91bda0ee2f9314ba3
2019-01-22 02:34:35 +00:00
drh
87c3ad4537 Enhance the btree search routine so that it does early detection of
impossibly large keys and thereby avoids a large malloc() call.

FossilOrigin-Name: 3ecaaee69f49e43d38047b7d53b82689eba7f7d33541fcac3c32b6dc8a568458
2019-01-21 23:18:22 +00:00
dan
129371553c Avoid deferencing a freed pointer following an OOM or SQLITE_CORRUPT error in
the fts3 xDestroy method.

FossilOrigin-Name: 505ed9a47825240979338a24044559613fbbd2a7850bdff70c7164da054ec63d
2019-01-21 17:57:31 +00:00
dan
ac30553f76 Remove a faulty assert() from fts3.
FossilOrigin-Name: 6c33a303ebbb0f5193ead535280ba63118e14fb4f9977ce80dc716a0b082ec99
2019-01-21 16:12:20 +00:00
drh
e6c628e98c Make sure the extra bytes allocated for the saved cursor position by
check-in [160b1e31c0f27257] are initialized to zero.

FossilOrigin-Name: 2737564929e86ead84a3ff6512369268198d38b46e3e02866f8ce34babc76cb9
2019-01-21 16:01:17 +00:00
drh
50b910a8b5 Add the --memtrace option to the CLI.
FossilOrigin-Name: a1e12fa2a8eb5648a96cc2d8d39899d6f87c5e2269cec7de486964c8b915e724
2019-01-21 14:55:03 +00:00
drh
7c04c69204 Minor fix the fallocate.test module change from [7cd56cad5efead5]
FossilOrigin-Name: 94fb7a47003c3c1a52e833e98d27399cfec5382afde2b3990e8223c8bff7cfa0
2019-01-21 14:49:14 +00:00
drh
5976552a72 Add the --max-data and --max-as options to dbfuzz2. Also cause dbfuzz2 to
show its maximum RSS size upon exit in standalone mode with the -v option.

FossilOrigin-Name: 7ce93e824a954d1e0cf8d7343e59a2660175f42bd4dac02aed8ad77644e7eb2f
2019-01-21 13:47:55 +00:00
drh
8ed07d1274 Add the --max-stack option to dbfuzz2.
FossilOrigin-Name: c11ae4fed89484f0a0061002861b5d599bbda8e40a1f0c33fdbe8cb072134d5c
2019-01-20 00:03:59 +00:00
dan
27a037354c Avoid a division-by-zero error in fts5 caused by a corrupt database.
FossilOrigin-Name: 928e622178bd004be52b2271ac554176aebe9d6e115ecac4e7072eccbb4262b5
2019-01-19 15:55:27 +00:00
dan
b8852ae083 Remove a broken assert() triggered by a "PRAGMA max_page_count = N"
invocation, where N is larger than the number of pages in the database image,
but smaller than the number of pages in the database file.

FossilOrigin-Name: 7cd56cad5efead5097e9918c87313bb46e583fe62bae0df1be60e10117e2c3cb
2019-01-19 15:27:09 +00:00
dan
93b6c1666d Fix a problem with using ALTER TABLE to rename a table or column when the
database schema contains a trigger or view that itself contains an expression
 "<column> AND 0".

FossilOrigin-Name: 908ff7fffa302255a74e2334ca3a1779ed43acb0268fce5a83a56fd16794dc88
2019-01-19 14:07:37 +00:00
dan
225b05795f Fix a memory leak introduced by [55c5d72a].
FossilOrigin-Name: fbd681dce2905d8b8ad7948ba684b0b62d6c04be0f7a2992ee7d6836a99010de
2019-01-18 21:17:08 +00:00
dan
6e0a75a2b8 Fix a crash in the fts5vocab module caused by including a "term < NULL" term
in a WHERE clause.

FossilOrigin-Name: 9e717c4377c0116a5d36815fbc30f8b8803f14770d30be361feb27cc5b5b537b
2019-01-18 21:12:32 +00:00
dan
a86b20820d Fix an infinite loop caused by a corrupt database in fts3. Also an undefined
left-shift in fts5.

FossilOrigin-Name: 55c5d72af9510e2f27c33544d804a58d4282b0efb384ead38484129ce91b574f
2019-01-18 21:03:15 +00:00
dan
4c1c79035f Avoid passing a NULL pointer to memcpy in fts5, even if the database is corrupt.
FossilOrigin-Name: acccc9808f52009b2c7794770c8b7efe2ce861fa184d32e287e7114c74e2b522
2019-01-18 20:15:02 +00:00
drh
d00ad9a656 Typecasts added to the GeoPoly extension to avoid harmless UBSAN warnings.
FossilOrigin-Name: a1f6a093ac4a2de8c5b02c30fe57e09770460fd5bdddfe9c7a9a24fb83a6b491
2019-01-18 19:33:56 +00:00
dan
451297752c Fix problems causing undefined left-shift operations in the fts3 snippet()
function.

FossilOrigin-Name: b90dbaed3092236e97f9796fa63989a3648060e16189e1267c430f4a7e799fac
2019-01-18 19:26:48 +00:00
drh
ed968fa4cb Avoid integer overflow when computing the array of a bounding box with
the rtree_i32 virtual table.

FossilOrigin-Name: b352f1590d20a574b0681e011ececcf4f41fa5b157503d330e03939404aca0e9
2019-01-18 18:52:17 +00:00
drh
e3b89d2c5f Handle oversize floating point values carefully when converting to integers
for the '%' binary operator.

FossilOrigin-Name: 048add13fc10e69ae504a49c4663612381d928b2cf1f9cdab4ff34bd40f601a3
2019-01-18 17:53:50 +00:00
dan
0e14e98706 Fix a fairly obscure problem allowing an "ALTER TABLE RENAME col TO ..." statement to modify the schema in such a way as to break a reference within a trigger program.
FossilOrigin-Name: 64bec9e6214c6932fab5a3fb8c569ae14cd2d603bd0f8b26104815c3bb9d396a
2019-01-18 16:06:18 +00:00
drh
3cb79202f8 Use the full 64-bit integer value in the argument to randomblob().
FossilOrigin-Name: 05df5f7aeaf2132810452e8871132d1e66650a841991cb2a9ce32f2d03e20bf4
2019-01-18 14:53:15 +00:00
drh
21908b2166 Disable the xfer-optimization if the two tables have the same root page
due to schema corruption.

FossilOrigin-Name: f31b3bd2a6a8aa35a6454f37f3a4b4595e2ad74256cd180439019ab4c6c2059e
2019-01-17 20:19:35 +00:00
dan
934d4f7bac Fix another corruption related crash in fts5.
FossilOrigin-Name: 4538d9afe4e6360e20c7b474e77314e5ad293453e1edbe356bf6ea84c05857ee
2019-01-17 20:06:56 +00:00
drh
aa0f2d0efa Add the SQLITE_ENABLE_EARLY_CURSOR_CLOSE compile-time option which causes
read cursors to be closed after their usefulness ends during a two-pass
UPDATE.

FossilOrigin-Name: 7def6c8edd85f19ee09038e01541f75b1f71ca39b9fb782b8f0fcac89207c353
2019-01-17 19:33:16 +00:00
dan
f66da6ce8e Fix a buffer overwrite that could occur when running an fts5 prefix query
against a corrupt database.

FossilOrigin-Name: 3910b5639d5c96d1840d4feeea64d3a55073531b7365a4e75d9cda6f119f3cfc
2019-01-17 19:11:10 +00:00
dan
25fb50674f Fix problems with joining two or more fts5_vocab tables that access the same
underlying fts5 table.

FossilOrigin-Name: 49956395e14b61f6bf839e59ae7dd95eb32ebf32f3d16388844de6621b9c2d98
2019-01-17 17:39:15 +00:00
drh
29c992cb04 Revamp the SrcList allocator routines to be methods of Parse instead of
being methods of the "sqlite3" object, so that they can leave better error
messages when the SrcList object grows too large.

FossilOrigin-Name: df08d472b090b212fb77ce2aae0e1ffe79ae5db4b1accf55e6fdb18e8b0a7098
2019-01-17 15:40:41 +00:00
drh
0ad7aa8182 Limit the size of SrcList objects to 200 entries (compile-time configurable
using -DSQLITE_MAX_SRCLIST=n).  The maximum number of tables in a join has
always been 64, so this is not a real constraint on capability.  Limiting the
size of a SrcList prevents DOS attacks (discovered by OSSFuzz) using crazy
nexted CTE joins.

FossilOrigin-Name: 7cac614d5df55eb092b863163483b6782b942b21bd15fd787576fef5619fa849
2019-01-17 14:34:46 +00:00
drh
04fcef00ee Fix a corner-case for the logic that cause an insert of a NULL into an
INTEGER PRIMARY KEY column to be converted into a valid integer key,
when the NULL results from a CASE expression that lacks an ELSE clause.

FossilOrigin-Name: 9a425051e7ba59e797636f5cf32b5f6efafdb21c8d5300e099b8008b829c1439
2019-01-17 04:40:04 +00:00
drh
30fdb45da0 Ensure that the variable-length integer decoder in FTS3 never tries to
left-shift a negative number.

FossilOrigin-Name: 10ffc1fe006a1766ff2f1ffc6129786afb86b6b1f8d3a17334e8ecb838d37b9c
2019-01-17 03:43:38 +00:00
drh
f20bbc5f69 Fix a failure in sqlite3ExprCompare()
FossilOrigin-Name: 835e2cc55feea2f2426c99816c27f9c35d2fd4cdd9bf1c7f46f889ecfd222778
2019-01-17 01:06:00 +00:00
dan
8b20e4a3eb Fix a problem with fix [b4b57413].
FossilOrigin-Name: ca7b7aaed010ec67bec5355a9ac1d7f229d6d5f2ba2d796b76f5e908b1267688
2019-01-16 20:48:13 +00:00
dan
e011ee2830 Fix a problem in the fts3 matchinfo() function with corrupt database handling.
FossilOrigin-Name: 24ed5fb6aea30b098d2faf3cf9d638933e518657217e903239ffaa1e25d34f66
2019-01-16 19:44:09 +00:00
drh
375afb8bda Fix a problem in the code generator for sorting results with SRT_EphemTab
and a LIMIT clause.

FossilOrigin-Name: 49fcde2f1f981ac0c75728bed3444e7c1d3167311db282d3375c96ce090b2210
2019-01-16 19:26:31 +00:00
dan
5e970a8f40 Fix a problem with renaming a column that is used as part of an ORDER BY on a
compound SELECT within a database view or trigger.

FossilOrigin-Name: b4b5741366578b25ec6e4c415ab8239215e53b1c900be613575f40a826cfccc9
2019-01-16 14:58:37 +00:00
dan
fb8ac325d7 Avoid a dangling pointer comparison when renaming a table that has a trigger
that itself contains a window function with an (illegal) column reference in a
FOLLOWING expression.

FossilOrigin-Name: d45bee36f2c1091a2d32c16ca8921bf4e7c9e40c46d0a36fbcb179ecfafcfbf0
2019-01-16 12:05:22 +00:00
dan
4ccb41fc99 Fix a memory leak that could occur in fts3 when handling a corrupt database.
FossilOrigin-Name: 65cebb06a0afcbcb4157c3d518a62ed188b1e90d9e9b69d88fece484bcb6e380
2019-01-16 11:38:06 +00:00
dan
dfb5c963a0 Fix a problem with ALTER TABLE and vector assignments in UPDATE statements
within triggers.

FossilOrigin-Name: cc6cd7531fee39b4c2a9f522f1089c1d79254a9e25acae59468322031f94c25a
2019-01-15 20:51:35 +00:00
dan
1a39d2bb5c Handle SQL NULL values without crashing in the fts5 snippet() and highlight()
functions.

FossilOrigin-Name: a5e9cc794f17566160c4b2b5791601a744a21ace987b19ff8a5816fd78dce56a
2019-01-15 18:14:27 +00:00
dan
a783931794 Fix a buffer overread in fts3 caused by a corrupt record.
FossilOrigin-Name: e54efd60c20bbfc5d70ddb9e1dd6ffb68cbbcfa57b4f07cb104767ef08c1a559
2019-01-15 16:14:31 +00:00
dan
2d4e720579 Have fts5 reject attempts to insert a non-integer, non-null value into a rowid
column with SQLITE_MISMATCH.

FossilOrigin-Name: 4a9483f81e9ca1aa41d6ca33cb3171370418ccc71fdee9741bbf6f694bf4fbaa
2019-01-15 15:18:58 +00:00
drh
e2dcc42e1f Fix a harmless memory leak in the Lemon parser generator utility program.
FossilOrigin-Name: 1caff0fb0b2051e205e7a8cfd5dadf92680c52a88a441ded95b1ea6542db5f83
2019-01-15 14:44:23 +00:00
dan
58ed374370 Fix a problem triggered by DELETE statements with WHERE clauses that use the
OR-optimization on some virtual tables.

FossilOrigin-Name: ecf5caa7e9825a8b03d15ee525ec68be78c55926ddfaca27a040a7614caf0e85
2019-01-15 14:31:01 +00:00
dan
9c039d9f2e Add scalar function "prefix_length()" to ext/misc/prefixes.c.
FossilOrigin-Name: 0bf1550507d9d3c8a41f8a50db3a59bf808f0a4e24637dc0905d35579305eca7
2019-01-14 20:44:00 +00:00
dan
9274ad898f Fix a problem causing some Tcl test cases to fail with errors like "expected: [<multiline-whitespace>], got: []".
FossilOrigin-Name: 11b8a4cba747ca214fe3506caa3b711c87be4d55412754e673cc0603afefb086
2019-01-14 19:13:30 +00:00
drh
4ead96348e Add the "prefixes" table-valued function in the ext/misc folder.
FossilOrigin-Name: f0fd2163fc23a555ef03af43294a14fcabae6921f52e64c979286c745b4f6218
2019-01-14 16:16:30 +00:00
dan
f8a488bb6b Fix a problem causing a crash if an fts5vocab table was created to query an
fts3/4 FTS index.

FossilOrigin-Name: 9cd64ce453e4435d80117b8610c22a0a4b53cd6e84b39815e70e684f6b0295c8
2019-01-14 15:35:15 +00:00
dan
94ec92a1f7 Fix a problem with querying a corrupt fts3 database.
FossilOrigin-Name: 65e50706a017751d72d0f9c7f575955194977decd0acf13aaca90c3cb925f20e
2019-01-14 15:09:00 +00:00
drh
4067ce7d54 When parsing a corrupt record using sqlite3VdbeRecordUnpack(), make sure
that the resulting UnpackedRecord object is completely initialized, to avoid
(harmless) MSAN warnings.

FossilOrigin-Name: ddc3697efd61830fad55c6bf7f9115f4a7f53e08838eba5a82e1406c7a721665
2019-01-14 13:32:15 +00:00
dan
68c1f9ce9a Have fts3 ignore empty sets of parenthesis if built with
SQLITE_ENABLE_FTS3_PARENTHESIS.

FossilOrigin-Name: c93c6b45a317c40eb5c0abb6620d21f5821a601632c791e11e5ce62e039eccda
2019-01-14 11:56:13 +00:00
drh
d12db3dabb Avoid reading off the front of a page buffer when balancing a corrupt
btree page.

FossilOrigin-Name: cb50509020d952fa9efed8df7fa08b07b71ae9bdbdefea216b6e660863291039
2019-01-14 05:48:10 +00:00
drh
b10a50e7f8 In dbfuzz2, avoid using a malloc in the LLVMFuzzerInitialize() initializer
routine, so that no memory leaks are reported.  Also, show the version of
SQLite being used when the -v option is on.

FossilOrigin-Name: 824f93246988ffa213bbd41a7de08886999b1a8ae00fdf6b9767acb6e3ec6a1f
2019-01-13 20:23:34 +00:00
drh
f202c6cdcf Relax the minimum size database file constraint on the dbtotxt utility program.
FossilOrigin-Name: 97e723d746eeb2159f5bf1701532271ac6a4620879c82d496f4499c178b64479
2019-01-13 20:17:52 +00:00
drh
0f1bf4c1ac Improved detection of database corruption while balancing pages from an
auto_vacuum database with overflow pages.  Test cases in TH3.

FossilOrigin-Name: 35f04235c477501390acea126d07a730d81d03cdf7abcd82d861e397b3f75b0f
2019-01-13 20:17:21 +00:00
drh
a1bd5c6aae Move a local variable declaration into the outermost scope in which it
is used. This fixes an ASAN warning.

FossilOrigin-Name: ac3b6021d9437ab1c027850d321f0a3e575b008763d8d515e2347f7d4e7c294b
2019-01-13 00:58:57 +00:00
drh
a79bcf358b Prevent unsigned 32-bit integer overflow from leading to a buffer overread
inside of an assert().  The problem fixed here is no reachable in production
code.

FossilOrigin-Name: 0f850a25d67a752fe1e9059c0c3f78e00c222113e556a7605fd3c50817b573cb
2019-01-12 21:30:26 +00:00
dan
951d165125 Fix another problem with handling corrupt records in fts5_decode().
FossilOrigin-Name: 726e398b9ddc15ea1b67c40e2a12ddf5d04428a866699588c5fa1af75af846e1
2019-01-12 20:55:36 +00:00
drh
6214d93993 Fix an off-by-one error on a Goto in the code generator, that only
causes problems for a REPLACE on an INTEGER PRIMARY KEY in non-debug
builds.  Test case in TH3.

FossilOrigin-Name: e35eb8776ed539afe1d5db099470ab1124d8dd2db73ee5db7c811f8df9a9576e
2019-01-12 16:19:23 +00:00
dan
b4439eef10 Fix a problem with corrupt database handling in the fts3 matchinfo() function.
FossilOrigin-Name: 703646b1b5c84d550fe0d74e399c0eeb729da1d263e4693320f69e6509678985
2019-01-12 14:58:35 +00:00
drh
5d4589014a Improved detection of shadow table corruption in the fts5_decode() SQL function.
FossilOrigin-Name: b74e5f3f3057ee7a98ebcb14ca0751048eacbec8fca3e11e241883029a57ecdf
2019-01-12 00:45:20 +00:00
drh
3e534eac19 Improved shadow table corruption detection in the matchinfo() function of FTS3.
FossilOrigin-Name: 567be3bb1e8b6477f3bf1c7b4cd6ec066fba69d0dcf8785632e244ce25db639f
2019-01-12 00:12:33 +00:00
drh
a2dc7494ef Indicate that the database may be corrupt in the fts3corrupt4.test test script.
FossilOrigin-Name: 473626d5579dd19023abccaf7c1822ac0c883a0b98904837ea096fa16e4f41c4
2019-01-12 00:07:48 +00:00
drh
73bce66701 Use 64-bit APIs in the fileio.c extension.
FossilOrigin-Name: b49d56a0faf012978c50fb8662125ea21bdf5054fddf5975644cbc941c153e70
2019-01-11 23:08:56 +00:00
dan
90294961da Fix a segfault in fts3 prompted by a corrupted database.
FossilOrigin-Name: 2d7b1d1d41ff69d5465fcb861420816ecb066c25a9015eefdab5fae15a2e1c9f
2019-01-11 21:34:25 +00:00
dan
0ad63e5e0c Fix another problem with corrupt databases in fts5.
FossilOrigin-Name: 9bf811ea89a133d481a86407683a7518e909eccb7aa3aa02f7932bdb33387572
2019-01-11 19:27:55 +00:00
dan
e6370e9c29 Fix a problem handling a negative value in the "number-of-pages" database
header field. Also a problem with running "REINDEX tbl" against a virtual
table for which the SQL passed to sqlite3_declare_vtab() contains PRIMARY KEY
or UNIQUE constraints.

FossilOrigin-Name: 556dd8922f65af93084ee725c638b8dc696b611dd508c212a3b507d6ca474031
2019-01-11 17:41:23 +00:00
drh
1735f91df4 Improved detection and reporting of errors in the readfile() extension SQL
function.

FossilOrigin-Name: d2f0b5a483869d39f5c5814c9d6df3d3b8a46d582b4dcf0cf11e23b707c4213e
2019-01-11 17:20:25 +00:00
drh
2bd207ff6d Omit errors about missing SAVEPOINTs when aborting the .archive command
in the CLI.

FossilOrigin-Name: 2a47387ba6aa3c294607b7641aa1c4cf70a7b27a861e1098c2f79a38e5b7036a
2019-01-11 17:19:59 +00:00
drh
2a58dbde30 Improved detection of cell corruption in sqlite3VdbeRecordCompareWithSkip().
FossilOrigin-Name: fa47f4c6589c431cf678560ac33dea6b695052012bea2096b2c92869ed51c688
2019-01-11 16:44:16 +00:00
drh
f10ce63651 Fix PRAGMA integrity_check so that it does not cancel the PRAGMA vdbe_debug
setting.

FossilOrigin-Name: aaa3a19f8cf5ba7003634e4610abc7832354af91d7c7f65469218678f66bcd46
2019-01-11 14:46:44 +00:00
drh
1972c8cf07 Add the --vdbe-debug command-line option to dbfuzz2.
FossilOrigin-Name: 599b4df43c64ce295517c804b6bb4c3ab77d77e0ba585dc0657274f55489a3b1
2019-01-11 14:38:47 +00:00
drh
26391eeefe Add the "dbfuzz2" target to main.mk. Remove an unused local variable
from dbfuzz2.c.

FossilOrigin-Name: 05c7609cfd02f49114876267d7090f4e0ea2467f3aa7c65019c5986ec8f60a87
2019-01-11 14:22:33 +00:00
drh
51d35b0fa9 The keywords TRUE and FALSE should only act as boolean literal values if
unquoted.

FossilOrigin-Name: 5547f39de993c708f72301ef25df190a2f007e0c4253799bdd9e86bb1ae41777
2019-01-11 13:32:23 +00:00
drh
e65b9c6a5e Separate makefile targets for "dbfuzz2", "dbfuzz2-asan", and "dbfuzz2-msan".
FossilOrigin-Name: ea119641a7edcd0e1ec14371d93e35f860971888bd98cb0019253358a2ef123b
2019-01-11 13:03:06 +00:00
dan
6ca98ad6b1 Fix a memory leak in fts5 that could occur if scalar function fts5_decode()
was passed a corrupt record.

FossilOrigin-Name: 240f1c0c92cad8a1c19438b38489aedb831f62a4603c53f22367e876d8f644ff
2019-01-10 19:26:52 +00:00
dan
f7d687c9d3 Fix a memory leak in fts5.
FossilOrigin-Name: ff3b011f17f4f02e486f3c8de607f62eddf3b86562e2df82dba57ea166994b65
2019-01-10 18:35:09 +00:00
dan
25e3073741 Avoid use-after-free and double-free errors that could occur if an fts5 table
is modified in certain ways while there are active cursors.

FossilOrigin-Name: 3291b2a6fe6f38ae91b933e5cd2bf7d97432374b4fb1fccd92b4bd759b02ee06
2019-01-10 17:08:20 +00:00
dan
c80864d77a Fix further problems with fts5 handling corrupt databases.
FossilOrigin-Name: 83c467d7af63bd2e7800aff4fe9b09dbd75557460b75a9e07205dfae7e28312c
2019-01-10 15:17:32 +00:00
drh
69ab18d213 More aggressive early detection of orphaned and malformed autoindexes when
parsing the schema.

FossilOrigin-Name: 10f9e39d6ed2413fa9abc6c82da3ed48f32a42b6190b6219fca7faf850d05113
2019-01-10 14:33:15 +00:00
drh
5f913ecb62 Use the new SQLITE_IDXTYPE_IPK values (3) on Index.idxType to indicate the
fake INTEGER PRIMARY KEY index used during query planning.

FossilOrigin-Name: e22d2f905fe840bea51b536ebedc9b637190ea0a37f16559668d99a61e971411
2019-01-10 13:56:08 +00:00
drh
c49d43a787 Fix the fts3DecodeIntArray() function so that it will not read off the end
of the buffer it is handed.  Any unread integers are set to zero.

FossilOrigin-Name: 666cf8f6b39ae1f72e82b45e9cacba23caf61370ca0c695b3b14452accbb1a0d
2019-01-10 01:12:43 +00:00
dan
a47e45f734 Fix an out-of-bounds read in SQL function fts5_decode() that could occur if it
was passed a corrupt record.

FossilOrigin-Name: 931278b257b12ac14fc8fbc82c6dc88ce4ac4b8e0d668543e68f0289d825daa1
2019-01-09 21:12:23 +00:00
drh
1757fed87f Adjustments to the page cache to try to avoid harmless TSAN warnings.
FossilOrigin-Name: 383437be276719ac1c72688ae81017c63907ac561f2ffb8e42c43b012248f294
2019-01-09 14:49:58 +00:00
drh
8eaf565feb Improved database corruption detection in the dbstat virtual table.
FossilOrigin-Name: fc5ecc88f8a0855ba90039177ee284721b2035e4de56a1024e323216f011f0af
2019-01-09 11:19:41 +00:00
drh
3e08ba4521 Small simplification to the dbstat virtual table.
FossilOrigin-Name: 9d4c156fa26f80bb8f88b6ad8f24520a15af296de9c110604f6bd42c257455a0
2019-01-09 11:06:03 +00:00
drh
885eeb676b Enhance the ALTER TABLE RENAME COLUMN feature so that it works on tables
that have redundant UNIQUE and/or PRIMARY KEY constraints.
Fix for ticket [bc8d94f0fbd633fd9a051e3]

FossilOrigin-Name: f09aa3248e2b4a32ff5b5d37084cb2a27445322cb06a3354cef723f4f0b2cd34
2019-01-09 02:02:24 +00:00
drh
2d77d80a65 Use 64-bit math to compute the sizes of memory allocations in extensions.
FossilOrigin-Name: ca67f2ec0e294384c397db438605df1b47aae5f348a8de94f97286997625d169
2019-01-08 20:02:48 +00:00
drh
c930b405f0 Performance improvement on the instr() function, especially for large
haystacks.

FossilOrigin-Name: ce51f1a2b6a1789a5876e01cf829e45d84f3851d135a2fa5c44a56f948673a60
2019-01-08 15:18:24 +00:00
drh
c0f162020e Fix a possible memory leak when trying to UPDATE a corrupt RTREE index.
FossilOrigin-Name: 63eb803dbc27077007dbee8def659d1523724eb73f1def1cdb68027e5c20843a
2019-01-08 14:28:02 +00:00
dan
174c21ff06 Fix further problems with fts5 and corrupt databases causing integer overflow.
FossilOrigin-Name: 673a7dd698eca05bb2909fa01f962a0667361732a6d0e61cf7780ead4a07c967
2019-01-08 08:02:12 +00:00
dan
d9d7d253fc Fix another potential buffer overread in fts5.
FossilOrigin-Name: d0e943d9574c7cab1af4a0fe753a0b5f91ea0e884b91a85b13239144b71f067c
2019-01-07 16:52:00 +00:00
dan
418454c63f For SQLITE_ENABLE_PREUPDATE_HOOK builds, disable the optimization for a
REPLACE command on a WITHOUT ROWID table with no indexes, triggers or foreign
key constraints that causes SQLite to clobber any existing row without
separately checking for it. This optimization causes SQLite to omit the
expected pre-update-hook callbacks.

FossilOrigin-Name: 6281ef974c0ac7a7133c9e1e04c09fdbd627c019c6ff3227293548caf8283864
2019-01-07 15:57:35 +00:00
drh
dcc2700d7e The page size becomes fixed as soon as page1 from a non-empty database file
is read.

FossilOrigin-Name: 6064584d578f0ac7aba3f67912a5d9eb1a3c6001469f6b9becbe68871131f887
2019-01-06 02:06:31 +00:00
drh
a8e057618a Add the exprNodeCopy() routine that will safely memcpy() an Expr node that
might be a size-reduced node.

FossilOrigin-Name: a874c649960ba2e2b2fd380d08c02a45884a1060d3922be8847729008ca6766e
2019-01-05 21:56:12 +00:00
drh
6b86e51eea If the OP_ParseSchema opcode with a non-NULL P4 operand does not parse any
rows out of the sqlite_master table, that indicates that the sqlite_master
table is corrupt, so raise an SQLITE_CORRUPT error.

FossilOrigin-Name: 598d7358e7329f0de6e3defc217665909e46874258ac29592ee2fd53e6411cda
2019-01-05 21:09:37 +00:00
dan
c210c08891 Fix two more problems with corrupt database handling in fts5.
FossilOrigin-Name: 444c7c99beb9f8b82963e9784aa2be54124c7011f4771122b4f608c02aa26408
2019-01-05 07:17:56 +00:00
drh
b042d92128 Give a meaningful error on an attempt to use UPSERT on a virtual table,
rather than simply not working.

FossilOrigin-Name: fa98748f956cff95fcfda3a513cea4df27aa573ae795a6855c40bfeca661d0b2
2019-01-04 23:39:37 +00:00
drh
f25cc4f3c5 If the ".open" command fails in the CLI, reopen to :memory: so that at least
some database is available for subsequent commands.

FossilOrigin-Name: d7f55c62c3fa053b9b930a044b60076a8278cbf679d8e5116b95f2ff19247f6b
2019-01-04 14:29:21 +00:00
drh
48cd0f28c5 Remove debugging pragmas accidently left in by a prior change.
FossilOrigin-Name: 26275fc7e89eb2542a65d01fd271cef016a4a1815bc3845c599cf2f4fbf7513d
2019-01-04 12:46:01 +00:00
drh
dfe3b58315 Fix a typo in a comment used to generate documentation. No functional changes.
FossilOrigin-Name: ccc4f7424bc8f37c168333091dd9e3d6f80ccff9e2b06f182df65be59a7e3349
2019-01-04 12:35:50 +00:00
dan
2f36262dbf Fix another fts5 assert() that may fail if the database is corrupt.
FossilOrigin-Name: 0888fc2e881d56ccd5222383d1a08cdf16fbd15c40da7ebfc14346be69ffb26a
2019-01-04 11:20:14 +00:00
dan
e9ad07c0a3 Fix some problems with sqlite3_interrupt() handling in fts5.
FossilOrigin-Name: c0eb839a3c23612b93002d2e1fbc41bf5db140775267989f85db52146144750b
2019-01-03 19:12:21 +00:00
drh
1baae57a72 Fix the csv01.test module so that it works on systems with \r\n line endings.
FossilOrigin-Name: 36eaf5a5e51b20699328aae67c2bfd325f81bea342e2b167fddd859a50a26e28
2019-01-03 16:03:48 +00:00
drh
94e02d9c2c Fix the shmlock.test script so that it works with the Windows restriction
that UnlockFile must exactly correspond to a prior LockFile.

FossilOrigin-Name: df939c89fa90b7f9ccf961027ca4eca4f987c49eabf530b5719a83e5ab0d346d
2019-01-03 16:00:23 +00:00
drh
20c0082371 Fix a harmless compiler warning in the CSV extension.
FossilOrigin-Name: fbcd72565f4425016cebbbf5dfd6aa510234cfb31c785cf364f04fff444aacae
2019-01-03 15:22:33 +00:00
dan
0d756cc83a Fix a problem with the second and subsequent sqlite3rbu_savestate() calls made
on an RBU vacuum handle.

FossilOrigin-Name: bef216dfa1456a787e3d9c74936ee1c6600827d4e252bd13e7a93046ce07469f
2019-01-03 15:17:01 +00:00
drh
232f278084 Fix the geopoly_svg() function so that it returns NULL when given zero
arguments.

FossilOrigin-Name: 120cb1768338eb86412fb69d4c7677247775bb37875ab3a98766cffd535f04cf
2019-01-03 00:44:03 +00:00
drh
8fc4a11c94 Fix harmless compiler warnings in the unicode2 logic of FTS3 and FTS5.
FossilOrigin-Name: 703029ac6d24860230a8c30fcbf5e7e1da619e84f1cc9b9e65ebc74879a184d2
2019-01-02 23:49:47 +00:00
dan
ca076bf26e Add another test case to vacuum-into.test.
FossilOrigin-Name: 0465d2fc0d3f8beaa1b6b0bd7bd51d69a524f30d889c9402e7d02cc06164a310
2019-01-02 16:01:59 +00:00
dan
992e927281 Add test cases for commit [6c6fb1c6].
FossilOrigin-Name: 3961995cb56b43f1753c53fc869edf336f1c17ec994d67dc74e44e8e02493894
2019-01-02 15:27:27 +00:00
drh
ee751fab4c Raise an error if the argument to VACUUM INTO is an identifier.
FossilOrigin-Name: 6c6fb1c6ea452c70b56fd4ac796affebee21512adba0455ae81c291b715f544f
2019-01-02 14:34:46 +00:00
dan
9b0da8319c Fix another fts5 problem caused by a corrupt database.
FossilOrigin-Name: 911342f7512145a8a189bb0ec59e70e4c9ddc9f20f3e84816ab25139ead5dc45
2019-01-01 19:17:42 +00:00
dan
97c8cb3ed8 Ensure that when a new cursor is opened by OP_OpenDup, any existing cursor
with the same id opened by a previous OP_OpenDup is closed first.

FossilOrigin-Name: 5c188361a91407805c0feb4bf6d3214522ce3e55013efcf63a4613ecd416bcbc
2019-01-01 18:00:17 +00:00
dan
fff1dc8e05 Fix another case in fts5 where a corrupt database could cause a buffer
overread.

FossilOrigin-Name: f7e6cdc5625664f449d0edbe39af2d45910c4137bfd856ae1f770dd826c138ff
2019-01-01 13:59:34 +00:00
drh
f9231c34eb Fix harmless compiler warnings.
FossilOrigin-Name: b57c545a384ab5d62becf3164945b32b1e108b2fb4c8dbd939a1706c2079e18b
2018-12-31 21:43:55 +00:00
drh
f01dae1f32 Additional steps to help ensure that scalar subqueries are only evaluated
once even if they are used in multiple places within the query.  This fixes
a performance regression reported on the mailing list and caused by
check-in [531eca6104e41e43] which was a fix for ticket
[787fa716be3a7f650c]. Think of this check-in as an improved fix for
that ticket.

FossilOrigin-Name: e130319317e761193890ef75787a3c114b61166f518df019697be8e2c820ec54
2018-12-31 20:39:37 +00:00
drh
d471bcb396 Remove an optimization that can no longer occur, being superceded by the
subquery-reuse optimization.  Put an assert in place of the optimization
to detect if the need for this optimization ever returns.

FossilOrigin-Name: 4fcdc7a2939457e51aab37cab168677cd7f464a9a0aff4f8981e864a30828b3b
2018-12-31 20:13:12 +00:00
drh
0de6efe59e Merge enhancements and bug fixes from trunk.
FossilOrigin-Name: 9fb646f29c05bca5f677a2c7c4f45c36bfe0a0c6a88cb7968b4a0459bdd63bb2
2018-12-31 18:30:41 +00:00
drh
d0fb7960ce Small changes to the OP_OpenEphemeral opcode to improve testability.
FossilOrigin-Name: f856676c8438dbf52d299e78f6dd6148d929755dc05cdcabafd17d9a86439435
2018-12-31 17:58:05 +00:00
drh
4afdfa195b Fix the OP_OpenEphemeral opcode in the bytecode engine so that if it is called
a second or subsequent time, it merely clears the existing table rather than
creating a new one.  Proposed fix for ticket [d0866b26f83e9c55e30de0821f5d].

FossilOrigin-Name: 4678cb1044f0b4dc813e48f3bd0f85240a66e2ecf8763280d66726cc031c93a7
2018-12-31 16:36:42 +00:00
drh
d1d158bf5a Additional small performance increase and size reduction to the
sqlite3VdbeMakeLabel() mechanism.

FossilOrigin-Name: 1bdee199a71e0a6c247b85e72de9e3a3099b7179c33288735721facef3b96459
2018-12-29 14:23:22 +00:00
drh
ec4ccdbcb1 A new implementation of sqlite3VdbeMakeLabel() is faster and makes fewer
memory allocations by deferring memory allocation until 
sqlite3VdbeResolveLabel() is called, at which point the code generator has
a better idea of how big the relocation table needs to be.
The sqlite3VdbeMakeLabel() routine now takes a Parse* parameter instead of
Vdbe*.

FossilOrigin-Name: 4a0929ac76d8aa5dd65eac3b83d6bbf41e505e01d175ca0fb2b19ba02d439415
2018-12-29 02:26:59 +00:00
drh
01e92df41e Merge performance enhancements from trunk.
FossilOrigin-Name: 0f1b9ff9e1e6f13e03045fcb7d0907227085054f9eb0b0b8471fb26b0094b13a
2018-12-28 21:32:35 +00:00
drh
81f9159b5e Faster allocation of new sqlite3_stmt objects.
FossilOrigin-Name: 891f1f72187f0f9ec0d24fda98cc08be3ae3c3ff8b27c4e409ee7135c3106398
2018-12-28 20:48:07 +00:00
drh
b6991796b4 Move the nOpAlloc field from Parse into Vdbe to avoid an extra pointer
deference on the fast path in sqlite3VdbeAddOp3().

FossilOrigin-Name: 8f10efc29dea7b816b1ba401726c268950d6671d890f686911269082a241d8d9
2018-12-28 20:14:03 +00:00
drh
64919e9b00 Merge patches and enhancements from trunk.
FossilOrigin-Name: da53f3d3ff26cc32d3a5ac788b6785baa3d1688e50d9354025a3130dc7e71a38
2018-12-28 19:23:20 +00:00
dan
4177c8a2b7 Fix another problem with loading the structure record from a corrupt fts5
database.

FossilOrigin-Name: c4d44542d259bbec11aea60ae94fcb4acd53e97e125723cae078cf0f8873f8ef
2018-12-28 18:09:45 +00:00
dan
1d4b16405b Fix a problem causing the sqlite_master entry corresponding to a virtual table
to be removed by a DROP TABLE even if the call to the vtabs xDestroy() method
failed.

FossilOrigin-Name: 0140f6dbfbea93eadcd7f727d84064a0c0d1f0806dbe3e5ef1017da603157e3b
2018-12-28 17:45:08 +00:00
dan
eefc72d12f Avoid an undefined left-shift operation in fts5 caused by malformed utf-8
text.

FossilOrigin-Name: c3a3a11194586bef80a9d7ca54caae8af30d4e7b464b8bb3d257ba2d2ec4791f
2018-12-28 14:33:55 +00:00
dan
666636b6de Fix a buffer overwrite in fts5 triggered by a corrupt database.
FossilOrigin-Name: a385298df264dbfa6765f63ad8708f74bc8e8a1404239c1049890b39a1bda888
2018-12-28 13:57:30 +00:00
dan
b163b57212 Fix problems in fts5 found by ASAN.
FossilOrigin-Name: c564bf870106faef297594a51995619c80311d06bd5f8a0c7644f666f22ba576
2018-12-28 07:37:22 +00:00
dan
593c716781 Fix another problem with corrupt database handling in fts5.
FossilOrigin-Name: fb0d7fba07865ff557bfcdacfda7b625d099ac8b4dcbe0c08cd2d4b84a068bb9
2018-12-27 20:12:02 +00:00
drh
aa243aa7cd Modify the btree02.test module so that it does not depend on whether TCL
interprets a value as a string or as an integer.

FossilOrigin-Name: d9e4f9574659c6285deba3dca68eca9989e615971cea3a901ffcda12fb4ee36f
2018-12-27 16:55:01 +00:00
dan
e80834f6ff Fix another problem in fts5 caused by a corrupt schema record.
FossilOrigin-Name: ff45f9dd69fe1e1c7875e5bb73ac3dde2a31746475d3765cda9233f8e06e26ca
2018-12-27 15:36:16 +00:00
drh
57e141bc28 Fix the filename normalization in the dbtotxt utility program.
FossilOrigin-Name: 29a00a05dd40ff59be340969e1aca4fee41e092f2faa1cea85a3448265a83c2c
2018-12-27 13:52:45 +00:00
dan
cd80dc3667 Fix a null-pointer dereference that could occur in fts5 with a corrupt
database.

FossilOrigin-Name: b0a49d5e6df116bb5eacb60a0bb0f482760aec38bd417d4250d9e57d43bdbb13
2018-12-27 07:46:37 +00:00
drh
bdefaf08ee Fix a problem in which nested CTEs with the same table name trick the
code generator into thinking they are the same CTE, which then tries to
use the manifest them both into the same transient table.

FossilOrigin-Name: 202dd033019dd27428e3cc5f6e164c95b37efe39e2753515112b201ddefca67b
2018-12-27 02:16:01 +00:00
drh
7eb2c9176a In the treeview debugging output,
provide additional details for the Table object
associated with each FROM clause term.

FossilOrigin-Name: 11d4682d2eec133ccca99ca9cf2620cd249b4afe55918f3ebf93b454431f9c55
2018-12-27 00:30:42 +00:00
dan
f6febee0e2 Update the coverage-wal permutation to cover branches enabled by
SQLITE_ENABLE_SNAPSHOT.

FossilOrigin-Name: 6821c61f1d71be2be7f867e59fd94582a1eaf45a4d1fb88be995807a77cc22ea
2018-12-26 18:34:56 +00:00
dan
89dec01e48 Add a few more test cases to walvfs.test.
FossilOrigin-Name: 14a64df36efe0dc36795311d58385f9c65dc465b214059e07c2ee6b95ea99c87
2018-12-26 17:49:57 +00:00
dan
0f55a571fc Avoid an "unitialized value read" problem in fts5 that can occur with a
corrupt database.

FossilOrigin-Name: 6cae81cd1ffb0180e0260d5ba921502c7dee2f54637bff3558a4d1cb5dfd3f32
2018-12-26 16:57:47 +00:00
dan
c2909bf8ac Fix a problem with handling corrupt databases in the fts5 integrity-check
function.

FossilOrigin-Name: 9bad745d373f4686a2726c3f77a2ccad60e7181645d7ed483a2e43075a4ef30c
2018-12-26 16:04:54 +00:00
dan
d8f3b4c320 Fix a problem with corrupt fts3 database detection introduced by [95a9a39f].
FossilOrigin-Name: c80bb15b44e9b6a3c0c5a49d5e93e66376c3520ecefc1db7425c8a7e09d3eade
2018-12-26 14:15:09 +00:00
drh
8625221444 Fix a problem with the CSV extension when it uses the "header" option on
a real file.

FossilOrigin-Name: 7acaed08f946633fca3ab8434dd6349ea6f1bac4fd9ec21f476db013ee06de30
2018-12-26 12:50:47 +00:00
drh
e4c26c41b4 Fix a typo in a comment that is used to generate session documentation.
FossilOrigin-Name: a1f50d57ce76f6c08a0ddbdd3c3fe2732d0e57dda14cb9aa3da00a14ceb4bc44
2018-12-25 11:17:50 +00:00
drh
47df8a2c40 Increase the number of parameters to sqlite3WhereCodeOneLoopStart() to
encourage the compiler to inline that routine within sqlite3WhereBegin().
This gives a performance improvement.

FossilOrigin-Name: 3c2ad0e915e835b1cb2962879beff50b2745d0cefe1cfc89ef90b7879c139ce6
2018-12-25 00:15:37 +00:00
drh
bd462bcc10 Improvements to EXPLAIN QUERY PLAN formatting. The MULTI-INDEX OR now shows
a separate "INDEX" subtree for each index.  SCALAR SUBQUERY entries provide
a subquery number that is related back to the .selecttrace output.

FossilOrigin-Name: 7153552bac51295c56a1c42ca79d57195851e232509f9e9610375692f48c7e86
2018-12-24 20:21:06 +00:00
dan
92107a38dc Add a few more test cases to walfault2.test and walvfs.test.
FossilOrigin-Name: 3873941c4fb9aa2d0de1129722c5fcdf2be1ea261677d4841ed99f896c778459
2018-12-24 20:00:27 +00:00
dan
1ab76166e3 Change another fts5 assert() that may fail with a corrupt database to an
assert_nc().

FossilOrigin-Name: d9653380c455b4bf6c371540d677cbde6d17664c58662b5cc9a12929f46192dd
2018-12-24 19:59:37 +00:00
dan
76e4990b25 Add test cases to test file walvfs.test.
FossilOrigin-Name: d697f6f3ca47350dad30feb529914fc5254fa662ece783408eea963d912918c1
2018-12-24 18:51:13 +00:00
drh
c310c53e01 Always include OP_Explain opcodes in the bytecode when SQLITE_DEBUG is enabled.
FossilOrigin-Name: d8c1235c02b745138ac2b12e8bbb00dda4cac321176be92a2fa9547be029b3d0
2018-12-24 18:10:39 +00:00
dan
94e95ea46e Copy some extra test infrastructure from the mutexfree-shmlock branch to trunk.
FossilOrigin-Name: 883337ffdb434b6856ceaada121c0be67e2fdec7b447675f45e85568ef28d7d9
2018-12-24 15:22:47 +00:00
drh
a599e150ca Restore a line of code that was previously commented out for debugging.
FossilOrigin-Name: e53781f5bdf5c7b7d91ffe76565ea2fe5d1b3a9b162b3a88a7017bbe785d4c72
2018-12-24 14:30:11 +00:00
dan
29e9af8848 Change the way a comparison used to detect corrupt databases in fts3 is done
to avoid potential pointer overflow in 32-bit builds.

FossilOrigin-Name: 95a9a39ff784b960b62dd6298e722a620ba3f9c8b76839a49bbef35d7bc84a8e
2018-12-24 13:34:36 +00:00
drh
5198ff5767 Do not generate subroutines for non-static SELECT and EXISTS expressions.
Fix up some test cases to account for the minor changes in EXPLAIN QUERY PLAN
output.

FossilOrigin-Name: 06de44ec9e173992ca9afb89dd2b4e40d2a7e35512c7959603cdceb606f5dfbd
2018-12-24 12:09:47 +00:00
drh
efb699fc7c Prevent the use of subroutines to implement the RHS of IN operators within
CHECK constraints.

FossilOrigin-Name: 6b24d3fb94e787d25e49501714780dfb0c0c99077102056aa68445f4808cdd57
2018-12-24 11:55:44 +00:00
drh
2c04131ca7 Experimental code that tries to put the computation of subqueries inside a
subroutine, and reuse that subroutine if the same subquery is evaluated more
than once.  Current code does not work for CHECK constraints.

FossilOrigin-Name: 6c44838adbe5dc482bc010e91a6dd7a0f777c989f443dd600740d2c783208e0d
2018-12-24 02:34:49 +00:00