Commit Graph

221 Commits

Author SHA1 Message Date
dan
cd80dc3667 Fix a null-pointer dereference that could occur in fts5 with a corrupt
database.

FossilOrigin-Name: b0a49d5e6df116bb5eacb60a0bb0f482760aec38bd417d4250d9e57d43bdbb13
2018-12-27 07:46:37 +00:00
dan
0f55a571fc Avoid an "unitialized value read" problem in fts5 that can occur with a
corrupt database.

FossilOrigin-Name: 6cae81cd1ffb0180e0260d5ba921502c7dee2f54637bff3558a4d1cb5dfd3f32
2018-12-26 16:57:47 +00:00
dan
c2909bf8ac Fix a problem with handling corrupt databases in the fts5 integrity-check
function.

FossilOrigin-Name: 9bad745d373f4686a2726c3f77a2ccad60e7181645d7ed483a2e43075a4ef30c
2018-12-26 16:04:54 +00:00
dan
1ab76166e3 Change another fts5 assert() that may fail with a corrupt database to an
assert_nc().

FossilOrigin-Name: d9653380c455b4bf6c371540d677cbde6d17664c58662b5cc9a12929f46192dd
2018-12-24 19:59:37 +00:00
dan
6a509d56bc Catch another case of a corrupted structure record in fts5.
FossilOrigin-Name: fb17fea4b9779fbd3adb6ff9500da83a6ca4fa7cba379aa70074e4328814a7f2
2018-12-22 19:54:17 +00:00
dan
862faf28bb Change a faulty assert() in fts5 that could fail on a corrupt db to an if() statement.
FossilOrigin-Name: 1daaba9fb46706af8ed0669a661c304df999a69cf639d29d93208f422c515d8c
2018-12-22 19:09:46 +00:00
dan
b63d6070ee Fix an assert() in fts5 that could fail with a corrupt database.
FossilOrigin-Name: 83935b950619855fb3bf05486091804ac2a61fb993c6ec826c89f943264fabc3
2018-12-22 15:21:31 +00:00
dan
634ac14488 Fix a "jump or move depends on uninitialized value" valgrind error that could
occur in fts5 when dealing with a corrupt database.

FossilOrigin-Name: 3518c09b944c92974939b1de244401afd34de5d9aa75050e5ad55581a94e1e71
2018-12-22 13:34:52 +00:00
dan
1ea0443c2d Add new sqlite3_prepare_v3() flag SQLITE_PREPARE_NO_VTAB, for preparing
statements that are not allowed to use any virtual tables. Use this to prevent
circular references in triggers on virtual table shadow tables from causing
resource leaks.

FossilOrigin-Name: 25666e3d03950caf753295cdb55df162e07dbcf6840b05875c6e0b127c469ecb
2018-12-21 19:29:11 +00:00
dan
2639ddc474 Fix a use-after-free problem in the fts5vocab virtual table.
FossilOrigin-Name: ac69d169c61c973448b1d3eef5b8ff0de2d932d7bb74068344cb1b79bd330578
2018-11-12 13:20:00 +00:00
dan
42b2653871 Fix a buffer overread in fts5.
FossilOrigin-Name: 0e3de8abbb0c7ae64e637776cb055ce79736f99a103e00e44d17a6b091b98c81
2018-07-25 15:25:55 +00:00
dan
eab0e10304 In extensions rtree, fts3 and fts5, ensure that when dynamic buffers are bound
to persistent SQL statements using SQLITE_STATIC, the binding is replaced with
an SQL NULL before the buffer is freed. Otherwise, a user may obtain a pointer
to the persistent statement using sqlite3_next_stmt() and attempt to access
the freed buffer using sqlite3_expanded_sql() or similar.

FossilOrigin-Name: 2a5f813bc61f9e780f2ccbda425611f65ad523b6d486a1e5e2b9d5e9f1d260a2
2018-02-07 18:02:50 +00:00
dan
2e6ca188c4 Fix a buffer overwrite in fts5 that could occur when processing a prefix
query.

FossilOrigin-Name: 92fc146bc2b781e7e2d7138b00e5ea649c6fee1c2b8449420460a1b3e5c9661b
2017-12-11 17:20:37 +00:00
drh
2c2f392dca Add interfaces sqlite3_prepare_v3() and sqlite3_prepare16_v3() with the
extra prepFlags argument.  Add the SQLITE_PREPARE_PERSISTENT option as one
bit in that argument.

FossilOrigin-Name: 4a25c5883380fe5990d8180adb58c3bdc7a3d081bc4c69cd4de3cd57074fb251
2017-06-01 00:54:35 +00:00
dan
22025ef198 Fix some problems in fts5 code detected by -fsanitize=undefined.
FossilOrigin-Name: 35f721045dfe3f82e016938ab1a668bfc37b6b57b8813cc963ef640ec82de58d
2017-05-19 12:32:32 +00:00
dan
6b4fbabd6a Do not save the state of an fts5 merge operation mid-way through an input
term, even if no previous entry for that term has caused any output. Doing so
may corrupt the FTS index.

FossilOrigin-Name: 9a2de4f05fabf7e725666fb8824ae1adad56834d7ae4f2e118375bb6f9c0ae77
2017-05-11 18:14:06 +00:00
dan
0619838c55 Fix an FTS5 bug that could cause a prefix-query without a prefix-index on a
database that contains delete-markers to return extra, non-matching, rows.

FossilOrigin-Name: 840042cb2bed2924e2263f21887317f661e5a585c4466c3af25b91ed57a6b49b
2017-04-21 16:04:18 +00:00
dan
03155f659f Update fts5 to support "<colset> : ( <expr> )" for column filtering, as well
as "<colset> : NEAR(...)" and "<colset> : <phrase>".

FossilOrigin-Name: c847543f8bb1376fef52bca72b4191162a32eb7e6c5f0cd1aa0ab116b3183396
2017-04-12 17:50:12 +00:00
dan
be0bc8bb4d Have fts5 close any open blob-handle when a new savepoint is opened. This
ensures that fts5 does not prevent DROP TABLE statements (which always open a
savepoint) from succeeding.

FossilOrigin-Name: a921ada89050ce1d162fd1b0056939573635e2cec7ac0c2a99ae924b3ae593f7
2017-04-08 09:12:20 +00:00
dan
1439a1e80f Improve handling of corrupt data in fts5.
FossilOrigin-Name: 609ac1c73f7c2b48a571c178a72383996a9538fe
2017-01-06 11:55:03 +00:00
dan
b3e845ba33 Fix an fts5 problem causing a crash in phrase queries where the first token of
the phrase matches one or more rows but some other token within the phrase
matches zero.

FossilOrigin-Name: e78f6f3bbf4781850960fe6741f7cba480f2c27d
2016-11-14 08:19:37 +00:00
dan
460d38f15e Fix an fts5 problem with corrupt database handling found by address-sanitizer.
FossilOrigin-Name: e22252e1da4cd9e41b970970a1c4f466aa6cc133
2016-08-13 10:34:12 +00:00
drh
c73bd0a5cc Fix a harmless compiler warning in FTS5.
FossilOrigin-Name: 2ea0c8b46eefd4874f021f8dfd344be751f2034f
2016-08-11 01:24:56 +00:00
dan
882ef0b8c0 Have fts5 interpret column lists that begin with a "-" character as "match any column except" lists.
FossilOrigin-Name: e517545650631d1e8a7ee63c6646a8b183a0a894
2016-08-09 19:26:57 +00:00
mistachkin
77fac879d2 More harmless compiler warning fixes.
FossilOrigin-Name: ab69527c1608da0b668f3b49e967661dd99cc3d4
2016-04-12 20:05:06 +00:00
drh
997de998a8 Remove an unused local variable from FTS5.
FossilOrigin-Name: 0ed693c29f184223cde3b3d51f0e06273e586803
2016-03-23 15:53:45 +00:00
dan
848b190e40 Explicitly limit the size of fts5 tokens to 32768 bytes.
FossilOrigin-Name: 70fc69eed9b09159899d7cbd1416a59d04210a63
2016-03-23 15:04:00 +00:00
dan
0a2f2b546a Fix some errors in fts5 test scripts.
FossilOrigin-Name: e1ab2d376a72786098125a41c1ea8140fcbd15c6
2016-03-22 15:01:54 +00:00
drh
13969f5af0 Fix harmless compiler warnings.
FossilOrigin-Name: 25d776e4523aefeec007943fe29aa17c23ccb301
2016-03-21 22:28:51 +00:00
dan
f5268268b7 Have fts5 cache the structure of its index in main memory. Use "PRAGMA data_version" to figure out when this cache should be invalidated.
FossilOrigin-Name: 902f221754f3cc6fb4ae049c135f85efce604ed3
2016-03-21 15:30:50 +00:00
dan
3c6707f72b Rearrange code so that tests pass whether SQLITE_DEBUG is defined or not.
FossilOrigin-Name: 89296a46c3b891ee0b637ff6f89d828d6e46e7b5
2016-03-21 15:18:13 +00:00
dan
ef480d37dc Change the way fts5 internally allocates segment ids in order to eliminated non-determinism from the module.
FossilOrigin-Name: d6e2637df16764aa9723a30ea2eb8a631d28cb2b
2016-03-21 09:56:19 +00:00
dan
c9460ab989 Fix some OOM-handling issues in the fts5 changes on this branch.
FossilOrigin-Name: 020a0bda59fe93b9361ceeed0d5a8ba4177380c8
2016-03-17 12:39:56 +00:00
dan
f55fb6615b Have fts5 cache the decoded structure of fts5 indexes in memory. Use "PRAGMA data_version" to detect stale caches.
FossilOrigin-Name: 33ef2210ef19e55c8d460bfe9d3dc146034c8acc
2016-03-16 19:48:10 +00:00
dan
1bb85f3ed3 Add tests to cover untested branches in recent fts5 changes.
FossilOrigin-Name: fa0033edf6ddf3c6631fa95b343225dbc8ac9429
2016-03-10 15:12:47 +00:00
drh
6459ca0b8c Remove an unused local variable.
FossilOrigin-Name: 3c343c3d012367942037e64a9855b825cb459844
2016-03-10 14:22:42 +00:00
dan
4dbc65b29a Add an incremental optimize capability to fts5. Make the 'merge' command independent of the 'automerge' settings.
FossilOrigin-Name: 556671444c03e3afca072d0f5e9bea2657de6fd3
2016-03-09 20:54:14 +00:00
dan
4e9d0d5418 Fix a problem in fts5 where a corrupt db could lead to a (huge) buffer overread.
FossilOrigin-Name: c9a30e117f2c6c9ef0cc0c6ca5227d2961715b8f
2016-02-29 17:34:16 +00:00
dan
437a54eaba Fix an fts5 problem causing 'optimize' to corrupt the fts index under some circumstances.
FossilOrigin-Name: 251d6473f7c9ad95adbdcc49cb8eaf7c0956764b
2016-02-29 17:16:26 +00:00
drh
7ff89807e0 Fix a harmless compiler warning in FTS5.
FossilOrigin-Name: e52c90758c53cbb0f74e0d8582e4c3c1bee04150
2016-02-15 23:46:36 +00:00
drh
5d2e2f1267 Remove the unused fts5BlobCompare() routine.
FossilOrigin-Name: defc762dd11144402abd591278819768a080d5ea
2016-02-11 21:53:57 +00:00
dan
2a44b3d877 Avoid a buffer overread when reading a corrupt fts5 structure record.
FossilOrigin-Name: facbc424e555061135aced7b134bf6c19f54e484
2016-02-11 18:08:38 +00:00
dan
22e8356368 Handle parser stack overflow when parsing fts5 query expressions. Fix some compiler warnings in fts5 code.
FossilOrigin-Name: bc3f7900d5a06829d123814a5ac7b951bcfc1560
2016-02-11 17:01:32 +00:00
dan
d120a1e3ab Add tests for and remove unreachable branches from fts5 in order to restore test coverage.
FossilOrigin-Name: 22589018ac3321f7bd89ce9fa69e57eae395e324
2016-02-05 21:09:26 +00:00
dan
b80b8f482c Further streamlining of fts5 prefix query code.
FossilOrigin-Name: ca91bd8ac70a5b3fef127364f73ec675e58bb92c
2016-02-05 19:18:02 +00:00
dan
4786cf5a20 Further improve performance of unindexed fts5 prefix queries.
FossilOrigin-Name: c9c6457d8ea911f6cc63967127e58da3146fd3ef
2016-02-04 19:45:19 +00:00
dan
eb715c62f9 Improve performance of fts5 prefix queries on detail=col tables.
FossilOrigin-Name: ca11f46db047e7f131cef3893f73824758a2076b
2016-02-03 20:04:59 +00:00
dan
735ff4a809 Enhance the performance of fts5 AND and OR queries.
FossilOrigin-Name: 62ea9e5ab8bc1a20245beebceb5ea62dcd7ec84e
2016-02-02 17:40:41 +00:00
dan
52f5d1ba6b Performance enhancement for fts5 column filter queries on detail=full tables.
FossilOrigin-Name: 13fb4aa6a87c5c6258979953da82eedc1a7bf14f
2016-01-30 19:16:11 +00:00
dan
18205c781d Further minor performance improvements and code-size reductions related to fts5 column filters on detail=col tables.
FossilOrigin-Name: b4ac61aeee976296e7719949cd4fb496147a29e8
2016-01-26 20:08:50 +00:00