mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
29,082 Commits 1,487 Branches 166 Tags 416 MiB
f3d279595e
Commit Graph

920 Commits

Author SHA1 Message Date
dan
d05bf0fe61 Fix various problems with fts5 contentless_delete=1 tables. 
FossilOrigin-Name: 0d005112b8aca9e9eca9d86d5fed9168f6a0218fd290b5489b9e7b05714610f4
 2023-07-18 19:52:32 +00:00
drh
3f47e7551c Merge the FTS5 memory leak fix from trunk. 
FossilOrigin-Name: fb65cb73d7ea22a8b20dccfa3abdaaa809eee4fcee6fe4846bd2e598ceb49aa4
 2023-07-18 17:43:47 +00:00
dan
a5b1f34125 Fix memory leak in fts5 introduced by [def41225]. 
FossilOrigin-Name: 4dcad2db743fdb9ef72871ca5a4d1384f76cb697161b0f5110e2670a83a18e8a
 2023-07-18 16:29:34 +00:00
dan
cb1d6b997e Merge trunk changes with this branch. 
FossilOrigin-Name: 5f66eb4e2603278dcc9dbfe4bf506cba1aa03180cfb492a0dfc3a8be32cc994b
 2023-07-18 13:55:55 +00:00
dan
263a8e8c0a Fix a buffer overrun in fts5 that could occur in 'secure-delete' mode when dealing with corrupt database records. 
FossilOrigin-Name: def41225b2f10b0294ab506f28bc87873688d4030f7056839298ff18e058ece5
 2023-07-18 13:55:19 +00:00
dan
2cccced140 Fix cases where a row is inserted into a contentless_delete=1 fts5 table and then deleted within the same transaction. 
FossilOrigin-Name: d928856a226fb7f001e55ff7e8eb58a656b982f1efa811de46c382b8b7cd778c
 2023-07-17 18:40:39 +00:00
dan
55e0fd4a9d Do not allow the 'delete' command to be used on contentless_delete=1 fts5 tables. 
FossilOrigin-Name: cc694b83408ccb5d42204cb624145c76e95329cbe1d1fe8815c70a7a00af231a
 2023-07-17 17:59:58 +00:00
dan
0dac350f3c Store large tombstone hash tables in multiple database records. Ensure the same hash tables handle rowid 0. 
FossilOrigin-Name: 4410e60d0c76e057ee962124f9239c6e17fd5ccafdbb4d9b703448eabd7781e3
 2023-07-15 18:57:57 +00:00
dan
fc158d3870 Avoid a case of an infinite loop in fts5 when dealing with corrupt records. 
FossilOrigin-Name: 0e801f11cd2d50fc710a80c2b3b805c7801e660cff1bcc87be89133d43796524
 2023-07-12 20:24:23 +00:00
dan
9f6c0d0027 Fix a crash that may occur when handling corrupt records in contentless_delete=1 mode. 
FossilOrigin-Name: 74d7610a8e32ac62a608141ff16bfe517c8d2cdfd43f81ef3a5df2b2003e541a
 2023-07-12 19:34:32 +00:00
drh
9faa3bbdbb Get access to SQL functions and virtual tables used for debugging and analysis 
of FTS5 using SQLITE_FTS5_DEBUG and without the need for SQLITE_TEST.

FossilOrigin-Name: 383de8e2259adb3f8881a618b8b5e06c1f3b2bd7ac274bd047410ad7d2a18f0f
 2023-07-12 18:38:47 +00:00
dan
b184c91076 Have contentless_delete=1 tables support REPLACE statements. 
FossilOrigin-Name: 2f553a660e00564e51bc4209c92bd3628fb1266f4a52832792fbf91e4234a0ba
 2023-07-11 20:57:19 +00:00
dan
b92669db73 Have contentless_delete=1 tables support regular DELETE statements, instead of just the special INSERT syntax. 
FossilOrigin-Name: fffb8616905501669a94231d5d9f53446bf09553353f2cdab7c43ca54bbb7fa6
 2023-07-11 20:19:46 +00:00
dan
d23f210377 Use a hash-table instead of a flat list to store tombstone rowids. 
FossilOrigin-Name: 948267b066d0dbe667881b3d26a007fa24576da6e57c112676fadeb846c13f0b
 2023-07-11 18:55:19 +00:00
dan
6788c7b7c0 Begin adding support for deleting rows from contentless fts5 tables. 
FossilOrigin-Name: e513bea84dfaf2280f7429c9a528b3a1354a46c36e58ab178ca45478975634e0
 2023-07-10 20:44:09 +00:00
dan
d679f9e5d1 Fix a buffer over-read that could occur as part of an fts5 secure-delete operation on a database containing corrupt fts5 records. 
FossilOrigin-Name: a35c50eca907d0cf373b7962c359aebfce8ac7c34ce81edeb99122d4fb5804f7
 2023-06-12 11:06:57 +00:00
dan
870fe69c72 Add extra tests for fts5, large rowids and "ORDER BY rowid DESC". No changes to code. 
FossilOrigin-Name: b5b5eaeed3ef55415e27aa04561ab446f3be881b56b7cbf9b8ba3307f0b2bf69
 2023-05-27 19:34:46 +00:00
dan
b838a229be Fix a buffer overrun that could occur in fts5 when processing corrupt records. 
FossilOrigin-Name: 4891dbd938f4bfd345eaef01f2addea9512eaa98f860844c73abb907b6a8e0e8
 2023-05-22 11:02:15 +00:00
dan
4042d2b424 Limit the number of nested NOT nodes in an fts5 expression to 256. 
FossilOrigin-Name: 0e5c1ee40a146ef8b2b3c5f53d0a45e092bc8d8e933f3819805c995819d31bae
 2023-05-15 17:14:16 +00:00
dan
cad6aae45e Allow virtual tables to prepare writes to shadow tables in defensive mode from with xRelease() callbacks - even if the xRelease() is invoked from within sqlite3_finalize(). 
FossilOrigin-Name: a17106494a8a67bc92cfe27ee8ef9bf160517d6930d796980ae12287b98c1cfe
 2023-05-05 11:47:31 +00:00
drh
62b28db5c7 Add another assert() to FTS5 to demonstrate that a pointer is never NULL. 
FossilOrigin-Name: b8a84a1bf53fa07ef01c57c6db6329ba439774a262b3adcbe94e7bd77525e296
 2023-05-03 14:10:05 +00:00
dan
3f23eb6813 Add an assert() to fts5_config to ensure that a potential OOM is being handled correctly. 
FossilOrigin-Name: fe9c207657400f9d9f4e822eb658157bc147ed538e2701322f6f973933f023ed
 2023-05-03 13:57:57 +00:00
drh
3442306989 Protect a macro argument with parentheses in FTS5. 
FossilOrigin-Name: bc07fe51fe0c6bb50ca8ae1baefcc35c8f5395b2d0de641bf0b0cedc92d754d4
 2023-05-03 13:48:33 +00:00
drh
f9b58eb4f8 Fix harmless compiler warnings. 
FossilOrigin-Name: 342af5b4fa0bd7c699e5497161db13d0cf795c7a5875ae30d666122e518f213b
 2023-05-01 20:42:15 +00:00
drh
1f02cd6cd6 Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: f6210017b3143ccbaea23655852f947f779bb438ad4fbf582f5da42f7cd1d9f5
 2023-05-01 19:59:10 +00:00
drh
59fe13e39a Fix harmless compiler warnings. 
FossilOrigin-Name: 0a926994691ccb02ea4f0ba3c2e8f728871ae67d24c705fbc0f7af81f38d5f47
 2023-05-01 18:52:35 +00:00
dan
e01e3b5b94 Fix a crash that could occur in fts5 'secure-delete' mode when operating on corrupt records. 
FossilOrigin-Name: 2e85b0e3dcae0915aa6472a3654c8ac72a6b2083c11747f3f657c79bbdaf530b
 2023-04-29 18:31:44 +00:00
dan
4d24eff8dc Avoid a crash in fts5 when combining an empty expression with another. 
FossilOrigin-Name: 33681ff3d29c9f096dbd8c5d885993f780e93974a7653adc00fa41a158356d7c
 2023-04-22 20:47:08 +00:00
dan
8cb4f0b1b2 Add tests for the progress-handler callback with the fts5 'secure-delete' feature. 
FossilOrigin-Name: 48505ad950bc0902d58210be066d4672e6085eb27c525ba2bc663fde7e932ff7
 2023-04-19 20:29:26 +00:00
dan
a5eab9579a Fix an assert() in fts5 that could fail following an OOM or IO error. 
FossilOrigin-Name: 49595083ba58e13d404b6a11c28f53768a257c59f3f36be3591438127df7e875
 2023-04-18 11:33:35 +00:00
dan
68c0da6f66 Fix some problems with using fts5 options 'secure-delete' and detail=none together. 
FossilOrigin-Name: 4d3f27ba90aa59837e49ceaae9f36cad426af3a33707d85d1bb848fc1bda6fed
 2023-04-17 18:32:22 +00:00
dan
48824f23cb Fix problems handling corrupt records in new fts5 code. 
FossilOrigin-Name: 2f63d4a8bac6dc72197df32efd84c148adbc4794943088dff08da57b15dd7900
 2023-04-17 11:41:42 +00:00
dan
5c45a9fbaf Fix an assert() in fts5 that may be true when dealing with corrupt records. 
FossilOrigin-Name: e8de2774463c07b76cd6b9a44ab32a583fe973f42a21b21af46fe82ab177c1b8
 2023-04-17 10:39:51 +00:00
dan
ae6f6ef3b0 Change a memcpy() to memmove() in new fts5 code. 
FossilOrigin-Name: 8de4542553ae20947787b4daf5e88bbe69f815679960e700e19f9c1148234b08
 2023-04-15 21:00:55 +00:00
dan
0e429cf934 Fix a use-after-free that could follow an OOM or IO error in fts5 when using 'secure-delete' mode. 
FossilOrigin-Name: 904be83d9f3dd4b21a3aa47dde337b5c045b925ffdde4bccd1f8e8fdbef68c18
 2023-04-15 19:58:58 +00:00
dan
e45317a50a Avoid dropping the error code following an OOM in fts5 when using 'secure-delete' mode. 
FossilOrigin-Name: 9ee24a499c7b4a048f269f1c03c7c0b22b2898a3c23ea3e2511c2ba0342afa30
 2023-04-15 19:13:51 +00:00
dan
112500028e Further tests for fts5 secure-delete mode. 
FossilOrigin-Name: c5a47063b7971868ebbb3f0718434dd6bd22d01c43697f48941df9aae3885abc
 2023-04-14 18:00:03 +00:00
dan
3fc123b7bc Add extra OOM test for the new code on this branch. 
FossilOrigin-Name: 846ae7e099ce67dab6b5df0ad5648a01d7bf7f2acf8312d817ed8505d4ea5aec
 2023-04-14 17:00:29 +00:00
dan
d36106c954 Add tests for the new feature on this branch. 
FossilOrigin-Name: 0268d1a59316d169927cbd4c562725e46a9023b7d7123ade911c12203e222adf
 2023-04-14 16:11:05 +00:00
dan
02877e9558 Remove temporary debugging code accidentally left on this branch. 
FossilOrigin-Name: 0a0f64870feaf95d2673efc8884ad1236d9bbf9bdb94364f8d2602221bd36ff3
 2023-04-12 18:06:43 +00:00
dan
015020cd1a Add the 'secure-delete' option to fts5. Used to configure fts5 to aggressively remove old full-text-index entries belonging to deleted or updated rows. 
FossilOrigin-Name: 4240fd09b717dbc69dffe3b88ec9149777ca4c3efa12f282af65be3af6fa5bb0
 2023-04-12 17:40:44 +00:00
drh
ebec9c475e A prepared statement that aborts due to SQLITE_SCHEMA should not invalidate 
cursors in other prepared statements that are already running.
See [forum:/forumpost/cae4367d9b|forum post cae4367d9b] for the original
trouble report.

FossilOrigin-Name: 857d0f5e16ba69ac9e5ee581befca2f7ed933edfe6e36396b10dae7979b44a57
 2023-04-06 13:35:42 +00:00
dan
5f708cc832 Fix a problem with the fts5 snippet() function that shows up when snippets just 1 token in length are requested. 
FossilOrigin-Name: 96d5116d17cbf3de6be38952881c6ddf6a226ddfd7649197bdb2df39c42b5d8e
 2023-03-10 13:36:19 +00:00
dan
a3e6192941 Fix a problem with the fts5 trigram tokenizer and LIKE or GLOB patterns for which contain runs of 2 or fewer non-wildcard characters that are 3 or more bytes when encoded as utf-8. 
FossilOrigin-Name: 00714b39b39c51519edbc0194f98c7275fecf96763a06fd95db6e1d81bb9f1f1
 2023-02-10 17:17:04 +00:00
dan
0d12a79310 Better fix the problem where optimizing an fts5 table too often causes it to become unreadable (first attempt was [35bed981]). 
FossilOrigin-Name: 459d986d38fc0ccbfd66801e0f22900cfed831268cf59ac8d1cd1e556f0d1441
 2023-02-08 17:28:08 +00:00
dan
d40a183506 Fix a problem where optimizing an fts5 table too often might cause it to become unreadable. 
FossilOrigin-Name: 35bed981416269018a0d52171d9997c43d3b097f5db1910d10d882eabe2e137f
 2023-02-03 19:28:50 +00:00
drh
3601db8f86 The json_group_array(), json_group_object(), and fts5_source_id() functions 
should all be deterministic and innocuous.

FossilOrigin-Name: edcb83fa0d3c832d044b267d608fed31a2ac841b81c7c9406b2136ac43d61a1a
 2023-01-20 13:34:14 +00:00
dan
1dff401c42 Fix a broken assert() in fts5. 
FossilOrigin-Name: ff5a9b054f4d5ad3d33787aee93b4181822c6dfc513626236d9867bb431f64da
 2022-12-27 13:52:25 +00:00
drh
3547e4997f Fix lots of harmless, nuisance compiler warnings, mostly unused parameter 
warnings in extensions.

FossilOrigin-Name: c14bbe1606c1450b709970f922b94a641dfc8f9bd09126501d7dc4db99ea4772
 2022-12-23 14:49:24 +00:00
dan
73e460c932 Fix a faulty assert() in fts5. 
FossilOrigin-Name: 569018170b928cadfa0745db631981084e595d90a8c3d1e236c42adae161018f
 2022-12-19 20:14:22 +00:00
dan
1532f9d155 Fix an assert() in fts5. Simplify memdb xLock/xUnlock some. 
FossilOrigin-Name: 7fe158aa8071acadd959b2b4a4b66e8a7f7eecec207ba681abf516b0457c2921
 2022-12-19 14:06:36 +00:00
dan
838865c0bb Better handle an error in the fts5 integrity-check code. dbsqlfuzz e87c62f9b67ea21aebdc36ab71cab7cc3eda8dc3. 
FossilOrigin-Name: ae43e97087a3207a5ca3ffae75fbe7a33c01f4a38ce0d1d7eed8591ae3083617
 2022-11-09 11:17:57 +00:00
dan
6aafd74853 Avoid some cases of signed integer overflow in fts5 by casting to unsigned values. 
FossilOrigin-Name: 46a78c8c0ed518c4521e6e0bdebeb065bab07076abc444775002e7f4361d2242
 2022-08-08 19:29:53 +00:00
dan
cf2ad7ae36 Remove a few unsuitable scripts from the "veryquick" test suite. Also have every second testrunner.tcl process favour running test scripts that contain text like "testrunner: slow" before any others. 
FossilOrigin-Name: 22d280a5cd395abbedcfffbac3d3b3a614c327be25763ca380c1338a2a7bd33a
 2022-07-18 19:32:30 +00:00
dan
f703b42dc4 Update the expected error messages in some OOM test cases to account for [44d77a7f807]. 
FossilOrigin-Name: b3d6b3c3fc68dca7e20418eefa35ce3b583322b884b88a11c6773419f027a7a4
 2022-07-11 21:40:34 +00:00
drh
de7a820fd0 Minor typo fix in a comment, for the benefit of broken IDE syntax highlighters. 
FossilOrigin-Name: 680ccecbc839373194ed2dcc1053e25512d9ea3f12f8c8963cb8fbd1a034d2f8
 2022-04-29 12:09:43 +00:00
drh
c381056e6d Fix harmless compiler warnings. 
FossilOrigin-Name: 61095b5bcb5c30d29f03efa5b4809e3a0b780ed57e35227c292aab9dd1cfe3f6
 2022-04-07 10:11:35 +00:00
dan
7ef4d75bd2 Fix a case in fts5 where a corrupt database could cause a crash. 
FossilOrigin-Name: 5e95df261cce275adb5b2226e9584f4c389bb0d53c3b4feafd5a14be8d02ce63
 2022-03-11 12:02:18 +00:00
drh
7d44b22d45 Fix test cases so that they all still work even with -DSQLITE_DQS=0. 
FossilOrigin-Name: 4883776669ee2f2310ea82b0d6df4d008eebaa7cb252102539cf21a635402ebb
 2022-01-16 19:11:13 +00:00
drh
a6ca90c38c Attempt to fix a harmless compiler warning in FTS5. 
FossilOrigin-Name: 8e619c21e2326be1538b60908e7cd211558ec840835c6eb69e768eb190e1fd0b
 2022-01-01 19:55:29 +00:00
dan
c436b3056d Fix an OOB read that could occur in fts5 when processing corrupt records. 
FossilOrigin-Name: bb9b1a15f7e80483162049dfd981d059dc69d03348b521f7ac164a8cd3ae3cc4
 2021-12-06 18:57:02 +00:00
drh
913306a53a Improved defenses against integer overflow when computing the size of a 
memory allocations.  No bugs were fixed here.  But perhaps future bugs will
be prevented.

FossilOrigin-Name: eb878c01f317f09e8ef6b1bd2ec8d6d5cd6ce0bdfe9da7fa7d92d2047cc9d9e4
 2021-11-26 17:10:18 +00:00
dan
76c3ae6848 Fix a broken assert() in fts5 that could fail if the snippet() function were used with a query phrase containing 0 tokens on a detail=none or detail=col table. 
FossilOrigin-Name: 0a5b94b99cf45df759a59cb5fc62da111ca33f8c2614769a81930d72b51f093a
 2021-11-02 07:32:13 +00:00
drh
134f544ab2 Fix a harmless scan-build warning in FTS5. 
FossilOrigin-Name: 0bf42bb5611dc3f672cb898b8be245fd25f7a3862c1e0734effd18d75e812f22
 2021-10-16 19:50:03 +00:00
dan
f7ff755651 Fix an assert() in fts5 that could fail with a corrupt database. 
FossilOrigin-Name: e99979855de937ed5ee0994b180054501400bf8776fb70acd31786d2ba1ad49a
 2021-10-16 13:59:08 +00:00
dan
33cf194218 Fix a signed integer overflow in fts5 leading to a segfault that could occur when processing corrupt records. 
FossilOrigin-Name: 69a3ff0cc159cdf84a5367eaf708882ddeda4fa65c96a5b546ae4a0114f02cb7
 2021-10-14 21:13:02 +00:00
drh
13ca72d011 Fix harmless compiler warnings, mostly caused by prior efforts to get static 
analyzers to run without warnings.

FossilOrigin-Name: cddd3b382a118bb16be3a602ae711fe2c2a7d1c673dd30dd3977b67ee555e666
 2021-10-05 18:59:05 +00:00
drh
d7ace22d8c Fix more harmless static-analyzer warnings. As of this check-in, no warnings 
are generated by scan-build for clang-6 or clang-10.

FossilOrigin-Name: a7835bead85b1b18a8affd9835240b0baf9c7af887196bbdcc3f5d58055042fc
 2021-10-05 18:33:38 +00:00
drh
7a3b4451a1 Fixes for harmless static-analyzer warnings. This also makes the code easier 
for humans to understand.

FossilOrigin-Name: 36177a62feeb4fa93ab6e3c6f4dbe1ddcf63bb02f93284abab979da0261b218e
 2021-10-05 17:41:12 +00:00
drh
640574f61a Small change to FTS5 to help static analyzers understand that it is not 
dereferencing a NULL pointer.

FossilOrigin-Name: 0d853df1b19a0270b2cb0ebdf5df8067572f19ab9a82d047cb047f3f9e850348
 2021-10-05 13:00:50 +00:00
drh
922c54206f Some #defines somehow failed to get set correctly in the previous check-in. 
Fixed here.

FossilOrigin-Name: 15bbdf9ac840a220f384411d3025ef22f949d310194b60bca8e6d6a759e6042e
 2021-10-04 18:57:42 +00:00
drh
11a9ad5669 Fix harmless static analyzer warnings in sessions, rtree, fts3 and fts5. 
Add the -DSQLITE_OMIT_AUXILIARY_SAFETY_CHECKS compile-time option to cause
ALWAYS() and NEVER() macros to be omitted from the build.

FossilOrigin-Name: 1c67f957fc77e37ce8f0d447c41ca975e8e79a35d332739c24a633649b5b0387
 2021-10-04 18:21:14 +00:00
dan
adfb203a7d Fix a buffer overread in fts5 that could occur when processing an "ORDER BY rowid DESC" query against corrupt database records. 
FossilOrigin-Name: fe098a93340b8dac15752f1be4f681e2a591f7ae1a94b177f400ccd7ef638480
 2021-10-01 15:05:19 +00:00
dan
415540ddaa Fix an out-of-bounds read in fts5 that could occur when processing corrupt records. 
FossilOrigin-Name: 9bbc01fb239b4aa752a56c584baa29655a3c845425d5c17d24a3939984d54fe2
 2021-09-29 13:36:58 +00:00
drh
dd31c033c6 Fix harmless compiler warnings. 
FossilOrigin-Name: d678ecca02698753d1b33e072566112e94ea36d0d3a8f4a24d2b09d131968d88
 2021-09-22 14:43:35 +00:00
dan
eb84c6ef35 Fix a memory leak in fts5 that could occur when writing to an fts5 table that is being scanned by an fts5vocab cursor. 
FossilOrigin-Name: 1183552de10b64a776874a23be8962c0693e5e0cd1a191ea8fa5d6a02b691e4a
 2021-09-07 16:05:48 +00:00
dan
fb8ca7de0c Fix a use-after-free error that could occur if an fts5 table is written while scanning it using an fts5vocab cursor. 
FossilOrigin-Name: e751c2ec786b5c1a1c9640fdc3fde036879a2c32db2bd67fe7c72604780f67b8
 2021-09-06 16:15:23 +00:00
drh
20de9f6cb1 Minor tweaks to various TCL build scripts so that they work even for TCL 8.4, 
which is obsolete, but is also all that is available for some legacy platforms.

FossilOrigin-Name: 2269ce64f707c4a198fcb9a72933648ea61a80b4251024b8058ff28ad81b4917
 2021-07-29 16:49:28 +00:00
dan
73983658c9 In defensive mode, allow statements that write to shadow tables to be prepared from with virtual-table xSync() calls. 
FossilOrigin-Name: c8601d83fbecf84ca7991d339449b380deb5a7620dc84756a91c2880f030b423
 2021-07-19 14:00:29 +00:00
dan
cc516af4cc Instead of disallowing writes to fts5 tables if there are fts5vocab cursors open on them (commit [c49a6ed7]), abort any fts5vocab queries if the on-disk structure of the fts5 table changes. 
FossilOrigin-Name: 9dbdc9001e3258e71ca995fbcdebf66ab95890ded87fa7125c6cb4bd43010aaf
 2021-07-07 11:51:03 +00:00
dan
b9324fea07 Do not allow writes to an fts5 table if there are any open fts5vocab cursors. 
FossilOrigin-Name: c49a6ed78a917d4972e048e2a9bbe4d400691f97ce7e022f0e4436ceaed7fb73
 2021-07-05 19:01:09 +00:00
dan
90d3b24a16 Fix a memory leak in fts5 that could occur when processing a query containing multiple classes of error. 
FossilOrigin-Name: 7c279670f544a5bb9733a88862d9e490062b07415cd790e0d6d1e2cb584d2007
 2021-06-30 14:04:25 +00:00
dan
c629e21786 Fix an assert() in fts5 that might fail when dealing with corrupt records. 
FossilOrigin-Name: 9e760e256089c0ebbf816a8de8b186f62e1ffcd6b8b0e76dab8387798cd4d0e1
 2021-06-30 12:11:19 +00:00
dan
8d2a9c3b3a Add defensive code to better handle corrupt records in fts5. 
FossilOrigin-Name: a99c29b5efaf17e34506cda5ce017ec0112b3cb24bf8a2f1831f0c14a8aa8d30
 2021-06-09 13:39:11 +00:00
drh
18f9be897d Remove two assert()s in FTS5 that checks for database corruption right before 
actual code does the same check.
dbsqlfuzz 9680db8f5338cb9b6060eb3a02d984555d4472e3.

FossilOrigin-Name: f3f82967b2e9fd4168d86300138ac4bbcd6ffd7a3ff5ecf20acdc29e5356ac3b
 2021-06-08 23:46:59 +00:00
dan
bc85a51595 Fix a buffer overread that could occur in fts5 when handling corrupt records. 
FossilOrigin-Name: 078962a2164a784b135bacee51ef10973dc2e30de04353d48698d0e72edd63d8
 2021-06-07 17:36:57 +00:00
drh
ff9186137d The fix in the previous check-in was only correct if the OOM occurs on 
the initial allocation.  This changes should make it correct for a resize
as well.

FossilOrigin-Name: 57087ab2f297e4d96da5b9f3b89a26d8dd2bc4d22c2be8045b5d956d86282f39
 2021-05-22 11:00:03 +00:00
drh
d42dbd9dfe If an FTS5 Cursor fails to enlarge the space for the aInst array, set the 
size of the aInst array to zero.
dbsqlfuzz 294254b8105cca409f27a711f1eb2e9e63cbcac5.

FossilOrigin-Name: 4ae5e5b5ecbce580d3f7f970076e828fc6fb5fbcdc3bbe8381ab38f52870a54f
 2021-05-22 01:30:30 +00:00
drh
e31cba5675 Remove an assert() in FTS5 that can (rarely) be false in the event of an 
OOM while processing a corrupt database file.

FossilOrigin-Name: 5f12f89154843e83b1d69ea1652e881d583aa329df732af72dd59029cd7eb577
 2021-05-06 13:45:37 +00:00
dan
a4cd0bbcb3 Fix a segfault that could occur when querying a corrupt data structures with an fts5token table. 
FossilOrigin-Name: cbedfa05501f3eb9750fb21e2ab8516f8f14c479c847f454e52572af278aab71
 2021-04-27 11:28:57 +00:00
dan
f3d5a68461 Fix an assert() in fts5_index.c that may fail if the database records are corrupt. 
FossilOrigin-Name: dfadb96edfef1d49e48a659264b9eb8d7f98d95d159d95c92c30dd4ba6c6e739
 2021-04-23 11:37:01 +00:00
drh
214b50314a Fix harmless compiler warnings of unused function parameters in FTS5. 
FossilOrigin-Name: 14273915979f8fbf80111e795b5511a6ee24638896b9d434d6ccf4bf7f7a7f62
 2021-04-22 19:34:31 +00:00
dan
6a4f7f0225 Remove fts5 test functions fts5_expr(), fts5_expr_tcl(), fts5_isalnum() and fts5_fold() from release builds. 
FossilOrigin-Name: c68a6c6c898d2acd1f7032ff98f8f3f99eb0b37ffbc6ee88d0dd7badf55a0e0a
 2021-04-19 16:45:09 +00:00
dan
304cbc17c2 Fix an undefined signed integer overflow in fts5. 
FossilOrigin-Name: e6f0adb00da84561e686a8db83858c7fd6b008756dd1aef807ea68f878ca3db7
 2021-04-13 17:45:36 +00:00
drh
f1f12661c3 Avoid taking the address of a NULL pointer following an OOM in FTS5. Doing 
so is harmless in actual practice, but it technically UB so we want to
avoid it.

FossilOrigin-Name: 1cfcd9dceb56b5987e6900a36a0ec092f0e1b13a7e754b8c3d8efb943e5bcc66
 2021-04-12 18:32:33 +00:00
dan
d73ade7d87 Fix issue with failing --enable-debug --enable-fts5 build. 
FossilOrigin-Name: e7491acc0c93b7b0b5a27130d956bb23f876b90ced11cdd416a86727af076ee1
 2021-04-06 17:51:18 +00:00
dan
f22c590bab Remove some test routines from release builds of fts5. 
FossilOrigin-Name: 3088bf15ca53fac3bad94f241e648836631b6d801d42837c238ff27d85074ca4
 2021-04-06 14:46:50 +00:00
dan
acbae3ba10 Add "#ifdef SQLITE_DEBUG" blocks around test variables sqlite3_fts5_may_be_corrupt and sqlite3_fts3_may_be_corrupt. 
FossilOrigin-Name: 004559544e661f9afa60306c4ac6842c29a8630b3663d62aa0cdc637f176932e
 2021-03-29 20:04:31 +00:00
drh
8210233c7b Revise tests cases to align with the new EXPLAIN QUERY PLAN output. 
FossilOrigin-Name: 50fbd532602d2c316813046ed6be8be2991c281eb5f295c4c28520a0de73862c
 2021-03-20 15:11:29 +00:00
dan
55b2e8b062 Fix further sanitizer complaints in the fts5 code. 
FossilOrigin-Name: 4cb00b9d4020a51f690720c6f51009c5be8c22ef8c70807ead1d953d19784f26
 2021-03-10 19:32:47 +00:00
dan
3be2b860d0 Fix a few sanitizer complaints in fts5. 
FossilOrigin-Name: 4fe4fee00e9294fa384bd073ed8e9a2a4ad10dcf653b6ad4851e44a64443ede9
 2021-03-10 14:57:10 +00:00
dan
8718f0b613 Fix a buffer overrun that could occur in fts5 when running a prefix query against a corrupt db. 
FossilOrigin-Name: 4da40620ac8557aba6e1e06e720418b55e9950f406288b03749ef1b117041cd9
 2021-03-03 19:36:40 +00:00
dan
5763f3d582 Fix a broken assert() in fts5 that could be triggered by corrupt database records. 
FossilOrigin-Name: b79f59f9ad897d5bd4b9d17e6219bc765b02450bfe14dc020485f221ba6b02cb
 2020-12-19 15:39:10 +00:00
dan
84b168ffa7 Change an fts5 assert() that can be triggered by a corrupt database to an if() condition. 
FossilOrigin-Name: ea0a7f103a6f6a9e57d7377140ff9f372bf2b156f86f148291fb05a7030f2b36
 2020-12-15 13:55:38 +00:00
dan
1418b9daf6 Add tests for a 'delete' command on a contentless table where values that weren't actually inserted are NULL. 
FossilOrigin-Name: 818c647cec7063b33b6c5de3e23599a1d61439fa6e9bf6c974b2522a5a9e1b44
 2020-12-09 16:49:28 +00:00
drh
aeb6bc5628 Fix harmless compiler warnings associated with the recent FTS5 enhancements. 
FossilOrigin-Name: 1db7c751912beb57a697ac8e85b9c29e30da7b6c89207e9828bf08e56c58242f
 2020-12-04 17:05:16 +00:00
dan
b9ceb833f7 Ensure that fts5 function matchinfo() is registered before running tests that use it in fts5corrupt3.test. 
FossilOrigin-Name: 932e05e093192991589e70c2cbcc8a57fb3dcc6df1c2673962bfd06ba5cfdd97
 2020-12-04 16:49:25 +00:00
dan
3190b88e75 When merging prefix lists in fts5, use 16-way merges instead of 2-way merges. This faster. 
FossilOrigin-Name: 026a93508ec392ca5cd2578ae9eab64974f58beccda088e10d4cc951f237632f
 2020-12-03 21:22:37 +00:00
dan
64f1ef6abd Allow a search for an N character prefix in fts5 to use a prefix index of size N+1, if no prefix index of size N exists. 
FossilOrigin-Name: 78a7801d8fc9e58a62e5168e35b52b7440340549123fc6a537e2abd571f6fe7b
 2020-12-02 19:08:15 +00:00
dan
0fa433b40c Improve performance for fts5 column filters that filter for more than one column. e.g. "{col1 col2 col3}:phrase". 
FossilOrigin-Name: d8de2f236d43a88fac7550a0451951dd5a945eb304e32f82e662479cea7c2684
 2020-12-02 16:23:05 +00:00
dan
cad760d16e Fix a compiler warning in fts5_aux.c. 
FossilOrigin-Name: 8edb983bc87898eff2cd2e7e672a32a47c71b2be9d818513d339e95560d45b2b
 2020-11-27 19:40:13 +00:00
dan
6d19bf9460 Add missing cast to fts5 bm25() code. 
FossilOrigin-Name: 6ff9673847c0b4174d9435e93d19af0ee7406b1a12edeb6edec98697e1646824
 2020-11-27 16:15:55 +00:00
dan
3181706895 Remove a redundant "if( rc==SQLITE_OK )" from the fts5 bm25() code. 
FossilOrigin-Name: d85f4f27f58adcc75fc7d59e63af95b2a338052d8748a11f22ec1e48d1aff4cc
 2020-11-27 16:08:15 +00:00
dan
66efc393c5 Move an "if( rc==SQLITE_OK )" to outside a loop body in the fts5 bm25() code. 
FossilOrigin-Name: 14a4dcf3474566d072007a37d214c892397c21dd3b7f8b55ad0e5edfb7130dd6
 2020-11-27 16:05:31 +00:00
dan
1c5b23f170 Fix a case in the FTS5 integrity check where a corrupt database could cause a buffer overread. 
FossilOrigin-Name: a32b4f650d2d543bd2773cbc8655c1679a20b35ac9ec4d08c7754ddf6d972acb
 2020-11-27 15:28:26 +00:00
dan
ec8962869a Update mkunicode.tcl to match the change erroneously made to machine generated file fts5_unicode2.c in [b7b7bde9]. 
FossilOrigin-Name: 326d579d777fdede6bc64f9525248767f4730de4e50260b0387e614a9d006416
 2020-11-26 20:13:54 +00:00
drh
d36f588f31 Fix harmless compiler warnings about unused function parameters. 
FossilOrigin-Name: 25d067c270966d9506db8bedf280883e32b69050b14bdbbeda4bb2d9a362619c
 2020-11-25 16:28:04 +00:00
dan
d1d43efa4f Prevent fts5 tokenizer unicode61 from considering '\0' to be a token characters, even if other characters of class "Cc" are. 
FossilOrigin-Name: b7b7bde9b7a03665e3691c6d51118965f216d2dfb1617f138b9f9e60e418ed2f
 2020-10-26 13:24:36 +00:00
dan
f6b0464e36 Fix some fts5 problems with signed integer overflow causing segfaults in -ftrapv builds. 
FossilOrigin-Name: 7e17c2f4b7dc9b563d0b4da949bb134dc7c4fc9c86ce03891432a884ca6409d5
 2020-10-12 18:09:16 +00:00
dan
f46be6a1b9 Allow fts5 trigram tables created with detail=column or detail=none to optimize LIKE and GLOB queries. Allow case-insensitive tables to optimize GLOB as well as LIKE. 
FossilOrigin-Name: 64782463be62b72b5cd0bfaa7c9b69aa487d807c5fe0e65a272080b7739fd21b
 2020-10-05 16:41:56 +00:00
dan
12a6a1eaf9 Fix a segfault caused by running "column LIKE NULL" against an fts5 table using the trigram tokenizer. Fix for [e33ee62575fc22]. 
FossilOrigin-Name: 6e72a08de764077f2bba6f7e3b99ea29001941671a971f2ccf7ceeb9c682fb1a
 2020-10-03 17:06:02 +00:00
dan
95dca8d0cf FTS5 does not handle tokens that contain embedded nul characters. Prevent the trigram tokenizer from returning such tokens. Fix for [2ba5930b2]. 
FossilOrigin-Name: b1d048748c054575425a4bebf0c5d09962f9329d5ce6a978cf54e508b238584c
 2020-10-03 14:36:06 +00:00
dan
a22c1c84cc Add the -trigram option to the fts5txt2db.tcl script. 
FossilOrigin-Name: a394dd260252e7b7dcbc90fd321dfd40d9375f0598ea52b01001c568cbdbafce
 2020-10-01 17:15:12 +00:00
dan
ccf578d435 Add tests for the trigram tokenizer. Fix minor issues. 
FossilOrigin-Name: 897ced99b44085012aa44d3264940dcbd4c77b295a894a1b58fb2c03a0f7fee8
 2020-10-01 16:10:22 +00:00
dan
33a99fad08 Add experimental unicode-aware trigram tokenizer to fts5. And support for LIKE and GLOB optimizations for fts5 tables that use said tokenizer. 
FossilOrigin-Name: 0d7810c1aea93c0a3da1ccc4911dbce8a1b6e1dbfe1ab7e800289a0c783b5985
 2020-09-30 20:35:37 +00:00
dan
7548ab20e6 In fts5 integrity checks, do not compare the contents of the index against an external content table unless specifically requested. 
FossilOrigin-Name: 782163693f37aeb65209bebbaeb6659a36881b8c4b4bec778b366658488bf966
 2020-09-21 14:53:21 +00:00
dan
f2e151aeab Fix a problem in fts5 causing the in-memory hash table to be flushed more often than necessary. 
FossilOrigin-Name: 9a55601d072bcd1ac0f6f1633216c7c8b3ff3e037451e2cfd285cbe19b60b9fc
 2020-09-17 15:16:50 +00:00
dan
3acd3fb9d8 Fix a "jump depends on unitialized value" valgrind/asan error in fts5. 
FossilOrigin-Name: 66f94d623c16b4527e42f0519e32ec78eaf9ee6fd61ee7d0eb657cc59a8f7d68
 2020-09-17 14:16:02 +00:00
dan
86f477edaa Catch fts5 index corruption caused by issuing 'delete' commands with incorrect data earlier in some cases. Also fix a couple of test script problems. 
FossilOrigin-Name: b79f19edfd33c2a75f936c352668e14e81f35acf4f07edc27a21f941a7304b38
 2020-09-11 15:01:49 +00:00
dan
832aa023c6 Add fts5 test to confirm that for a table with columns a, b, c and d, "{a b}" and "-{c d}" are handled similarly. 
FossilOrigin-Name: 1a04920998368e56276fd0b100be8343609c6ff8a731cf8e26a0490f9c6dabdf
 2020-08-28 11:19:49 +00:00
dan
bbc9e1908a Fix an integer overflow bug in fts5 triggered by a corrupt record. 
FossilOrigin-Name: 49da8bdce17ced910b414d50e8df79800513dcf03a196197dc1ea8e47deeda92
 2020-07-13 11:06:30 +00:00
drh
ac9e184e1f Avoid an infinite recursion on an illegal recursive definition of an 
fts5vocab table.

FossilOrigin-Name: 109ee07433b274a39954cef62bf67d47bcda960df9bef56127210ebf1c3c104c
 2020-02-16 17:40:35 +00:00
drh
9ca9a53a42 Modify a couple of FTS test cases so that they work on big-endian platforms. 
FossilOrigin-Name: b20503aaf5b6595afafee6aa2bc6276354a62b0f583f6d94e31fedfdfc130347
 2020-01-29 12:37:59 +00:00
mistachkin
8122c6e9be Fix variable declaration issue seen with MSVC. 
FossilOrigin-Name: 8a6fe3066cf0599fcf2960e8fb3ae39e4e7a61ec81e8ec71d0b5910aee2c16f6
 2020-01-23 00:08:25 +00:00
dan
9f3e6fadde Handle blobs that are the return values of functions being cast to text in utf16 databases in the same way as blobs read directly from the database. Fix for [771fe617]. 
FossilOrigin-Name: e782096aa06fcf410c3a0a6ee26a9e4f44333da39b28b2efe675814a8652911a
 2020-01-02 16:24:22 +00:00
dan
9d41af23a4 Do not use HIDDEN columns for NATURAL joins. Fix for [7c0e06b16]. 
FossilOrigin-Name: ab09ef427181130be09a087b7e572ad4cfb6b3e1b459769ee5ebf046b3ead682
 2019-12-30 14:32:27 +00:00
dan
e752040c09 Fix an assert() in fts5 that could fail if an xSavepoint() call on another vtab fails. 
FossilOrigin-Name: a5d7f5d24a239f729de0b8aa5becf5af35ba87160565ee0713c335b8d1bbf12d
 2019-12-26 14:36:31 +00:00
dan
4fcce747e2 Fix a problem causing fts5 integrity-check failures if numeric values are inserted into a table within a utf-16 database. Fix for [752fdbf6]. 
FossilOrigin-Name: bae060f382e4386c0793db4aa280fe9cd4893307d17a1801a17218ae4dc031c2
 2019-12-24 20:51:01 +00:00
dan
3fee9de239 Fix another case where malformed utf-8 was being mishandled in fts5. Fix for [df46a6f3]. 
FossilOrigin-Name: 1c0a05b09a97e6e2e9b11c31ed6ec7e6484686614b587ebfd0cfe27d973ba461
 2019-12-24 16:20:05 +00:00
dan
84d17bdaa7 Fix an assert() added as part of commit [a11b393dc] that can fail if fts5 database records are corrupt. 
FossilOrigin-Name: 4630c1ec013a4c2e54a34c9a64b2c803ac3912450de660497eb34ee21c91f426
 2019-12-24 15:35:53 +00:00
dan
c26e78d275 Fix a spurious report of corruption that could be made by the fts5 integrity-check in SQLITE_DEBUG builds if the fts5 index contains malformed utf text. 
FossilOrigin-Name: a11b393dc2c882cf0b3c47c3405bf43ca1d6459605bd39cccce4d32da653a72d
 2019-12-24 14:27:03 +00:00
dan
7a1d7c3968 Fix "PRAGMA data_version" so that it works the same way with locking_mode=PERSIST and journal_mode=PERSIST configured. Fix for 7a458c2a5f. 
FossilOrigin-Name: 45748e2db028ffbda5d3e747493721a5a89af3fbc06823081a2f27f570e40e73
 2019-12-22 14:29:55 +00:00
dan
42ae633ab9 Fix two more cases in fts5 where sqlite3_value_bytes() was being called before sqlite3_value_text(). Fix for e431c355. 
FossilOrigin-Name: a1ba9a37d7a68a6d31f8197c6350589ebe6a12f4e3c193a178dd7ead8bcd565a
 2019-12-20 19:41:01 +00:00
drh
b2ab3dcff0 Fix harmless compiler warnings. 
FossilOrigin-Name: 2bbd014c900e0d71fd30ec647739b82c5dc3aa237a0203f5a1db50593dae199e
 2019-12-20 14:08:34 +00:00
dan
73b21382d4 Ensure fts5 consistently calls sqlite3_value_text() before sqlite3_value_bytes(). This matters when storing blobs in fts5 tables within utf-16 databases. Fix for [cf36cb4e02]. 
FossilOrigin-Name: c16305eba0eb14368f2631b9a82b7eb4483c93f3b31257ee7c33f9b00c344379
 2019-12-20 13:24:11 +00:00
dan
db5ed35609 Avoid a buffer overread in fts5 that could occur when parsing corrupt configuration records. 
FossilOrigin-Name: 355afd77df21a2265871ca6d075f26b1fa121c7c2682cf512281944ff0c2186d
 2019-12-10 03:40:11 +00:00
dan
c19e22f397 Fix possible null pointer dereferences in the fts5_expr() scalar function. 
FossilOrigin-Name: c5d44143599f3fe98492b2b900fa3d77925c7be545096251055ceeab899a41f1
 2019-12-09 02:20:37 +00:00
drh
2bbcaee833 Fix various documentation typos suggested by Mark Benningfield. 
No changes to code.

FossilOrigin-Name: 08f09dc4f986a27f4e4e69f8226dbfaf0cfe1269cefc2830bf0fbef7b9bbda48
 2019-11-26 14:24:12 +00:00
dan
7e88d6364c Fix a memory leak in fts5 error handling code. 
FossilOrigin-Name: de6637e15d95b7aee2caea8b8d5b9865f276cbbf152ecf0401c4c7bfdf6dee6f
 2019-11-23 08:36:35 +00:00
dan
6637332988 Prevent direct and indirect recursive content= options in fts3/4 and fts5. 
FossilOrigin-Name: 2eb997327c2c369c10b7835be80112d77cb2a7cfa9fe4d564042984c21d94698
 2019-11-21 14:20:59 +00:00
dan
9930cfe878 Fix a problem in fts3 causing it to report corruption if a doclist contains consectutive rowid entries that differ by more than 2^63. 
FossilOrigin-Name: 0b0a3048f9df88bbc8979f2ac5247bee7d7bc1e585faca252610d2fcbe36d51d
 2019-10-28 13:54:59 +00:00
dan
ef2df8f343 In fts5, fix a case of overreading a buffer by 1 byte when counting characters in malformed utf-8. Fix for [dd1f67bf]. 
FossilOrigin-Name: 8d964e1c21d4cea699023e02b0616a75c5859dd083c9365cdcbc0676ebbdaae4
 2019-10-24 20:35:27 +00:00
dan
52612bec3c Fix a resource leak in fts5 that could occur if an auxiliary function is called from within a query that does not use the full-text index. 
FossilOrigin-Name: b528bdcd45db1b783ecd9739c3d3c890f04de7003f079668970eafaf8e23b2f3
 2019-10-20 08:26:08 +00:00
dan
2d82269ca8 Fix a problem with row-value IN(...) operators and virtual tables. 
FossilOrigin-Name: aa57d7abac0bb92d4d5fd4e093a11cf8efc04e4eed748b2a400d01f137250649
 2019-10-14 15:15:50 +00:00
dan
ae55737fbf Do not allow users to effectively disable fts5 crisismerge operations by setting the crisismerge threshold to higher than the maximum allowable segment b-trees on a single level. Fix for [d392017c]. 
FossilOrigin-Name: 86e497209217abb7bcb491a023cd353f3c7c9c103ebd9f58dd8661b12cf3694c
 2019-10-09 18:36:32 +00:00
dan
a6bd1871d1 Disallow fts5 page sizes greater than 65536 bytes - as there are 16-bit offsets used in the page header. 
FossilOrigin-Name: 75775c5ab44e497cb19be10397229637f1374f05c3244e8f92d6c54fcea94f5f
 2019-10-09 15:26:45 +00:00
dan
fbd79cc585 Fix problems with using the fts5 'rebuild' command inside a transaction that contains other updates of the same table. Fix for [e258f008]. 
FossilOrigin-Name: 238e0835714696aba0631f288fcc30ec5fddb43893d469c6bf017f386b3cddee
 2019-10-08 13:34:24 +00:00
dan
b186a622ee Disallow page-sizes smaller than 32 bytes in fts5. Also ensure the fts5 integrity-check works even when "PRAGMA reverse_unordered_selects" is true. Fix for [265e935b26]. 
FossilOrigin-Name: 8ab0aebdb3c2d6fb3160b2c58ce6cc0495a6ddd960878a6395958c837f3d1b71
 2019-10-07 20:36:18 +00:00
dan
03e025e217 Fix a problem with running ALTER TABLE ADD COLUMN statements within a transaction that writes to one or more virtual tables. 
FossilOrigin-Name: 31e85fbbc4cfd09a98b4a6dc6624f863816f376c6a4eef1fafc657c8e1abbc36
 2019-10-07 18:43:21 +00:00
dan
4b48c72721 Fix a "jump depends on unititialized value" valgrind error in fts5 triggered by corrupt database records. 
FossilOrigin-Name: 6b6751cd90601a1e2744a7d233c973291ecb783801b9327c3b99734dd22bcd27
 2019-09-18 11:46:34 +00:00
drh
3f3f2c7357 Ensure that the idxStr for FTS5 is always zero-terminated. 
Fix for check-in [e6db945454a0dae2]

FossilOrigin-Name: 090cd07d37904da4610d6a6787a3cc825c9a8bdcc5f051267d4608bba9b49d03
 2019-09-13 13:23:13 +00:00
drh
fbb9a5b1e3 Fix harmless compiler warnings. 
FossilOrigin-Name: a8927d14f88c44828e794787bab080455ab62705c93f65ce7d8db83478aa5a8e
 2019-09-13 12:24:58 +00:00
dan
685b2ee0c3 Allow fts5 to filter on multiple MATCH clauses in a single scan. 
FossilOrigin-Name: 9d418a7a491761eeb38a70898677a493e2631e5d62e75ee88431f52d3dfd2344
 2019-09-12 19:38:40 +00:00
dan
229ae1ae72 Fix a potential crash in fts5 caused by using an auxiliary function on a "special" query like '*id' or '*reads'. 
FossilOrigin-Name: 9490683ae883561fa347cbe54ebdd61188d849b4852d904b508250ba5d0807ef
 2019-09-02 14:46:12 +00:00
dan
50b3ea2daf Fix a potential buffer overrun in fts5 caused by corrupted database records. 
FossilOrigin-Name: c465d0eb478e42045e08efe8b6fa64d4bd4d747cdf743beae4608562b014169e
 2019-08-29 14:25:44 +00:00
dan
298680d15a Fix a potential buffer overrun in fts5 caused by corrupt database records. 
FossilOrigin-Name: 156d612800a7282fe0ebb20feb31d3fd577f4ab60fab8c00940c87143997aefb
 2019-08-24 17:11:29 +00:00
dan
35fb866ea3 Fix a problem with fts5 "ORDER BY rank" queries when the fts5 table name requires quoting. 
FossilOrigin-Name: 00e9a8f2730eb7239bf7fd107c97c409e4f9fbd968510766373440a9079898eb
 2019-08-20 19:11:29 +00:00
dan
3cbbd195ca Prevent an fts5 table from being its own content table, or part of a view that is the content table. 
FossilOrigin-Name: b6d52c9364767ff4ab7279ae981afb97799299dcfaf38a0110c40ca82c72a825
 2019-08-05 12:55:56 +00:00
dan
bfe33f80dd Fix a potential buffer overread in fts5 triggered by a corrupt database record. 
FossilOrigin-Name: 16472112b6886ada18d45cfb409cca2e83dde5a8ba8a36d481e87b61100b0186
 2019-07-25 19:29:24 +00:00
dan
98d63579be Fix an fts5 segfault that could occur after parsing a corrupt structure record. 
FossilOrigin-Name: 600daa9c27206628bf74892376ded35dac11fe0a34894fcd975ad5041a0a0fda
 2019-05-24 11:55:01 +00:00
dan
36c9301816 Add new test file fts5corrupt4.test. 
FossilOrigin-Name: 5dfc4fe98a4821cbdbfd7dfddf221b00251b2a26bf58767a3956b64e9a4e92da
 2019-05-16 16:44:47 +00:00
dan
2a71b17f2f Fix a problem with corrupt db handling in the fts5_vocab module. 
FossilOrigin-Name: 456ced57774a92acff23cad975b78ff0e5cbbf5204ea48599fe1761a018a92e4
 2019-05-14 18:33:58 +00:00
dan
c12655df87 Fix an assert() and potential buffer overrun in fts5 that could occur if the database was corrupt. 
FossilOrigin-Name: 8be8bd0d562e571b73a93f4ed18258ebd114bbab67ee3cdcd66a4c8f2987f8dc
 2019-05-14 11:33:09 +00:00
dan
d6712f3638 Fix an assert() failure in fts5 that could occur when processing a corrupt database. 
FossilOrigin-Name: f158c048be1dc7c80d38787a20495caa55b86416967efd57f4888548d42b4819
 2019-05-13 11:52:19 +00:00
dan
a27cce76bf Fix a problem in fts5 where a corrupt position list could lead to a buffer overwrite. 
FossilOrigin-Name: 516ca8945150bdc1363603e234be5f3f4a3861f581d7cc9455da3e87b9974309
 2019-04-24 15:57:25 +00:00
dan
189aa7dc96 Fix a test script problem in fts5corrupt3.test. 
FossilOrigin-Name: 734192d876e0489cb34e454aaa4a9c6858145791a49959b31dee2917a8e41a7d
 2019-04-15 13:59:08 +00:00
dan
de21ad704a Fix a test script problem in fts5corrupt3.test. 
FossilOrigin-Name: 858f62274eb727109d0452ad2d80c0e0acab0f569494b1218c404501e103ed39
 2019-04-13 16:35:02 +00:00
drh
f6ad201ac2 Fixes for harmless compiler warnings. 
FossilOrigin-Name: c28c973ad6debd63f13e5d4d3da036f680baaec9d863eda039f2747db9f1cfd5
 2019-04-13 14:07:57 +00:00
drh
3b574e4ea9 Use the 64-bit memory allocator interfaces in extensions, whenever possible. 
FossilOrigin-Name: 07ee06fd390bfebebc014b47583d489747b0423bb96c810bed5c605ce0e3be71
 2019-04-13 04:38:32 +00:00
dan
ce3c3eeda0 Fix two assert() statements in fts5 that are only true if the database is not corrupt. 
FossilOrigin-Name: edb095a9a679c8c702abd0a487e55ed4b09110b54bcd7d5275020576f2713a39
 2019-04-06 15:38:46 +00:00
mistachkin
065f3bf4f2 Fix various harmless compiler warnings seen with MSVC. 
FossilOrigin-Name: 1c0fe5b5763fe5cbace9773dcdab742e126d0bd035ab13d61f9d134afa0afc0c
 2019-03-20 05:45:03 +00:00
drh
f3e90dd276 Fix a compiler warning in FTS5. 
FossilOrigin-Name: d5acf3af65f9608d4096b9b78289d84b21cd1ea463457f858ffeb20d5bd5d123
 2019-03-19 20:13:41 +00:00
dan
d406d07262 Fix a buffer overread that could occur when running fts5 prefix queries inside a transaction. 
FossilOrigin-Name: b3fa58dd7403dbd4d2e9f3ae23d7d1337830d6fef2aa2f137ac5174de0d5828e
 2019-03-18 15:49:07 +00:00
dan
b15f19c75e Fix an fts5 problem with interleaving reads and writes in a single transaction. 
FossilOrigin-Name: 45c73deb440496e848cb24d4c1326d4105dacfee8bbafb115e567051855e6518
 2019-03-18 15:23:20 +00:00
dan
f26b145385 Verify that fts5 auxiliary functions cannot be used in aggregate queries. 
FossilOrigin-Name: 122330dba3eb2492875bd02a46442306f73e251408447af5e5914ee0c8d6a110
 2019-02-27 15:26:03 +00:00
dan
43a6e4d3df Fix another segfault that could occur in fts5 with a corrupted database. 
FossilOrigin-Name: 09e337386beb2008eba969f79601d1b8ed953951ce6e43782586e168140d78d2
 2019-02-11 16:12:09 +00:00
dan
47a30159e5 Fix typos in comments in fts5.h. No changes to code. 
FossilOrigin-Name: c19e97bc927074915a3ac8307bf89d30580cf90edd75f4001bcf589b2dfefe58
 2019-02-11 13:25:33 +00:00
dan
2fa619a3a8 Change a few assert() statements in fts3 that might fail if the database is corrupt. 
FossilOrigin-Name: db74a56af73d92b7a9d43ceda7e4540915c580c68a0266b4ddefb9e0d5cbcbeb
 2019-02-09 19:23:54 +00:00
dan
83a635a267 Fix problems with SAVEPOINT and related commands within transactions on a corrupt database that include fts5 or some other virtual tables. 
FossilOrigin-Name: b0da6e0dd1f9a9ba7646ee6bb3c14945e570303c9a42d20fa8a653570e12c3f9
 2019-02-04 16:25:40 +00:00
dan
0062ddd4c9 Fix another potential buffer overread in fts5. 
FossilOrigin-Name: 14fba4cc5db2bc206f024bfa5dd82fffdbc7577162da2c3b15a91cc1c91f3846
 2019-02-04 14:00:20 +00:00
dan
81f44cdbfb Fix another corruption case in fts5. 
FossilOrigin-Name: e608085d3f7e3cc0015fde626e205ee9842ef3cd06dc879fbbc7f2067c592965
 2019-02-04 11:54:22 +00:00
dan
8f5b14a5c2 Fix a buffer overread in fts5 debugging scalar function fts5_decode(). 
FossilOrigin-Name: 54f2399fb2a626602d405c857297f2da833f9f048cbc478f9110bed2e9bda299
 2019-02-02 13:47:25 +00:00
dan
1c7d389a23 Fix an assert() in fts5 that could fail if the database is corrupt. 
FossilOrigin-Name: 55f06aa3f890dc1dc5c2ed0e6777e46d609a6f9d223d1d4b5813097ce4f4e797
 2019-02-01 14:40:44 +00:00
dan
abd7449478 Fix another corrupt-database-handling problem in fts5. 
FossilOrigin-Name: 9876ae1b802a449aacbbdd42778653edc19e29315dc627c23ab7493d6492c800
 2019-02-01 13:34:00 +00:00
dan
b92b019fe3 Fix another buffer overrun that could occur when quering a corrupt database using an fts5vocab table. 
FossilOrigin-Name: b80119bd6044c0993210115732b6ae7b4b0b29c1471b0723bef24f2052c8e671
 2019-01-31 14:37:18 +00:00
dan
dd579233e8 Fix another buffer overread in fts5 that may occur when accessing a corrupt database. 
FossilOrigin-Name: 760d14374d40bcd9ce3a89771c18dc236c9728553c4747c9b7452ee7b24f4140
 2019-01-30 12:15:27 +00:00
dan
e25646d64c Avoid an integer overflow in the fts5 snippet() function triggered by a corrupt database record. 
FossilOrigin-Name: 7c862c4655841e1263cf548d01f5b4331685ab7d396494891cad0a5f50c63e16
 2019-01-29 16:34:45 +00:00
dan
eb2ed10ec0 Avoid a crash when the fts5 snippet() function (or similar) is used with a special query pattern, like '*id' or '*reads'. 
FossilOrigin-Name: 9d58a15737f7379fed724749cfd5b4de5b44231870e99fc664683ad483c3e6ba
 2019-01-29 15:30:03 +00:00
dan
3993397e6c Fix a buffer overrun triggered by a merge operation on a corrupt fts5 database. 
FossilOrigin-Name: 7ee3cd5b2cdf8db35f70400b3df58f7c421c78ae77c50f22b0ee22d5a3039429
 2019-01-29 11:42:43 +00:00
dan
e88609f23e Fix asan warnings in fts5 triggered by corrupt databases - passing NULL to memcmp, out-of-range left-shift values and signed integer overflow. 
FossilOrigin-Name: 93f8ec146d63af13f04e337ada4fa75e9254f72b1394df09701ae12e185f27e2
 2019-01-25 16:54:06 +00:00
dan
f8f47dd8c5 Fix another fts5 crash that can occur if the database is corrupted. 
FossilOrigin-Name: 44ce8baa47192be03c8f11777904c3c07fa5cc5c97b6d8e81572d380995ac688
 2019-01-23 19:17:05 +00:00
dan
4c7bdd1597 Fix a buffer overwrite triggered by a prefix query on a corrupt fts5 table. 
FossilOrigin-Name: 1d8172a94bce2d067027685e5aa1dc8f5d5d55b5da0832ad2836f6939de68c8f
 2019-01-23 12:19:22 +00:00
dan
ccfa550922 Fix a buffer overrun that could occur in fts5 if a prefix query is made on a corrupt database. 
FossilOrigin-Name: 1abc4415648e69362061e9f9a4f2c1d419ba33801999b377650d8b9a4d2d3a7c
 2019-01-22 21:17:40 +00:00
dan
27a037354c Avoid a division-by-zero error in fts5 caused by a corrupt database. 
FossilOrigin-Name: 928e622178bd004be52b2271ac554176aebe9d6e115ecac4e7072eccbb4262b5
 2019-01-19 15:55:27 +00:00
dan
6e0a75a2b8 Fix a crash in the fts5vocab module caused by including a "term < NULL" term 
in a WHERE clause.

FossilOrigin-Name: 9e717c4377c0116a5d36815fbc30f8b8803f14770d30be361feb27cc5b5b537b
 2019-01-18 21:12:32 +00:00
dan
a86b20820d Fix an infinite loop caused by a corrupt database in fts3. Also an undefined 
left-shift in fts5.

FossilOrigin-Name: 55c5d72af9510e2f27c33544d804a58d4282b0efb384ead38484129ce91b574f
 2019-01-18 21:03:15 +00:00
dan
4c1c79035f Avoid passing a NULL pointer to memcpy in fts5, even if the database is corrupt. 
FossilOrigin-Name: acccc9808f52009b2c7794770c8b7efe2ce861fa184d32e287e7114c74e2b522
 2019-01-18 20:15:02 +00:00
dan
934d4f7bac Fix another corruption related crash in fts5. 
FossilOrigin-Name: 4538d9afe4e6360e20c7b474e77314e5ad293453e1edbe356bf6ea84c05857ee
 2019-01-17 20:06:56 +00:00
dan
f66da6ce8e Fix a buffer overwrite that could occur when running an fts5 prefix query 
against a corrupt database.

FossilOrigin-Name: 3910b5639d5c96d1840d4feeea64d3a55073531b7365a4e75d9cda6f119f3cfc
 2019-01-17 19:11:10 +00:00
dan
25fb50674f Fix problems with joining two or more fts5_vocab tables that access the same 
underlying fts5 table.

FossilOrigin-Name: 49956395e14b61f6bf839e59ae7dd95eb32ebf32f3d16388844de6621b9c2d98
 2019-01-17 17:39:15 +00:00
dan
1a39d2bb5c Handle SQL NULL values without crashing in the fts5 snippet() and highlight() 
functions.

FossilOrigin-Name: a5e9cc794f17566160c4b2b5791601a744a21ace987b19ff8a5816fd78dce56a
 2019-01-15 18:14:27 +00:00
dan
2d4e720579 Have fts5 reject attempts to insert a non-integer, non-null value into a rowid 
column with SQLITE_MISMATCH.

FossilOrigin-Name: 4a9483f81e9ca1aa41d6ca33cb3171370418ccc71fdee9741bbf6f694bf4fbaa
 2019-01-15 15:18:58 +00:00
dan
f8a488bb6b Fix a problem causing a crash if an fts5vocab table was created to query an 
fts3/4 FTS index.

FossilOrigin-Name: 9cd64ce453e4435d80117b8610c22a0a4b53cd6e84b39815e70e684f6b0295c8
 2019-01-14 15:35:15 +00:00
dan
951d165125 Fix another problem with handling corrupt records in fts5_decode(). 
FossilOrigin-Name: 726e398b9ddc15ea1b67c40e2a12ddf5d04428a866699588c5fa1af75af846e1
 2019-01-12 20:55:36 +00:00
drh
5d4589014a Improved detection of shadow table corruption in the fts5_decode() SQL function. 
FossilOrigin-Name: b74e5f3f3057ee7a98ebcb14ca0751048eacbec8fca3e11e241883029a57ecdf
 2019-01-12 00:45:20 +00:00
dan
0ad63e5e0c Fix another problem with corrupt databases in fts5. 
FossilOrigin-Name: 9bf811ea89a133d481a86407683a7518e909eccb7aa3aa02f7932bdb33387572
 2019-01-11 19:27:55 +00:00
dan
6ca98ad6b1 Fix a memory leak in fts5 that could occur if scalar function fts5_decode() 
was passed a corrupt record.

FossilOrigin-Name: 240f1c0c92cad8a1c19438b38489aedb831f62a4603c53f22367e876d8f644ff
 2019-01-10 19:26:52 +00:00
dan
f7d687c9d3 Fix a memory leak in fts5. 
FossilOrigin-Name: ff3b011f17f4f02e486f3c8de607f62eddf3b86562e2df82dba57ea166994b65
 2019-01-10 18:35:09 +00:00
dan
25e3073741 Avoid use-after-free and double-free errors that could occur if an fts5 table 
is modified in certain ways while there are active cursors.

FossilOrigin-Name: 3291b2a6fe6f38ae91b933e5cd2bf7d97432374b4fb1fccd92b4bd759b02ee06
 2019-01-10 17:08:20 +00:00
dan
c80864d77a Fix further problems with fts5 handling corrupt databases. 
FossilOrigin-Name: 83c467d7af63bd2e7800aff4fe9b09dbd75557460b75a9e07205dfae7e28312c
 2019-01-10 15:17:32 +00:00
dan
a47e45f734 Fix an out-of-bounds read in SQL function fts5_decode() that could occur if it 
was passed a corrupt record.

FossilOrigin-Name: 931278b257b12ac14fc8fbc82c6dc88ce4ac4b8e0d668543e68f0289d825daa1
 2019-01-09 21:12:23 +00:00
drh
2d77d80a65 Use 64-bit math to compute the sizes of memory allocations in extensions. 
FossilOrigin-Name: ca67f2ec0e294384c397db438605df1b47aae5f348a8de94f97286997625d169
 2019-01-08 20:02:48 +00:00
dan
174c21ff06 Fix further problems with fts5 and corrupt databases causing integer overflow. 
FossilOrigin-Name: 673a7dd698eca05bb2909fa01f962a0667361732a6d0e61cf7780ead4a07c967
 2019-01-08 08:02:12 +00:00
dan
d9d7d253fc Fix another potential buffer overread in fts5. 
FossilOrigin-Name: d0e943d9574c7cab1af4a0fe753a0b5f91ea0e884b91a85b13239144b71f067c
 2019-01-07 16:52:00 +00:00
dan
c210c08891 Fix two more problems with corrupt database handling in fts5. 
FossilOrigin-Name: 444c7c99beb9f8b82963e9784aa2be54124c7011f4771122b4f608c02aa26408
 2019-01-05 07:17:56 +00:00
dan
2f36262dbf Fix another fts5 assert() that may fail if the database is corrupt. 
FossilOrigin-Name: 0888fc2e881d56ccd5222383d1a08cdf16fbd15c40da7ebfc14346be69ffb26a
 2019-01-04 11:20:14 +00:00
dan
e9ad07c0a3 Fix some problems with sqlite3_interrupt() handling in fts5. 
FossilOrigin-Name: c0eb839a3c23612b93002d2e1fbc41bf5db140775267989f85db52146144750b
 2019-01-03 19:12:21 +00:00
drh
8fc4a11c94 Fix harmless compiler warnings in the unicode2 logic of FTS3 and FTS5. 
FossilOrigin-Name: 703029ac6d24860230a8c30fcbf5e7e1da619e84f1cc9b9e65ebc74879a184d2
 2019-01-02 23:49:47 +00:00
dan
9b0da8319c Fix another fts5 problem caused by a corrupt database. 
FossilOrigin-Name: 911342f7512145a8a189bb0ec59e70e4c9ddc9f20f3e84816ab25139ead5dc45
 2019-01-01 19:17:42 +00:00
dan
fff1dc8e05 Fix another case in fts5 where a corrupt database could cause a buffer 
overread.

FossilOrigin-Name: f7e6cdc5625664f449d0edbe39af2d45910c4137bfd856ae1f770dd826c138ff
 2019-01-01 13:59:34 +00:00
drh
f9231c34eb Fix harmless compiler warnings. 
FossilOrigin-Name: b57c545a384ab5d62becf3164945b32b1e108b2fb4c8dbd939a1706c2079e18b
 2018-12-31 21:43:55 +00:00
dan
4177c8a2b7 Fix another problem with loading the structure record from a corrupt fts5 
database.

FossilOrigin-Name: c4d44542d259bbec11aea60ae94fcb4acd53e97e125723cae078cf0f8873f8ef
 2018-12-28 18:09:45 +00:00
dan
eefc72d12f Avoid an undefined left-shift operation in fts5 caused by malformed utf-8 
text.

FossilOrigin-Name: c3a3a11194586bef80a9d7ca54caae8af30d4e7b464b8bb3d257ba2d2ec4791f
 2018-12-28 14:33:55 +00:00
dan
666636b6de Fix a buffer overwrite in fts5 triggered by a corrupt database. 
FossilOrigin-Name: a385298df264dbfa6765f63ad8708f74bc8e8a1404239c1049890b39a1bda888
 2018-12-28 13:57:30 +00:00
dan
b163b57212 Fix problems in fts5 found by ASAN. 
FossilOrigin-Name: c564bf870106faef297594a51995619c80311d06bd5f8a0c7644f666f22ba576
 2018-12-28 07:37:22 +00:00
dan
593c716781 Fix another problem with corrupt database handling in fts5. 
FossilOrigin-Name: fb0d7fba07865ff557bfcdacfda7b625d099ac8b4dcbe0c08cd2d4b84a068bb9
 2018-12-27 20:12:02 +00:00
dan
e80834f6ff Fix another problem in fts5 caused by a corrupt schema record. 
FossilOrigin-Name: ff45f9dd69fe1e1c7875e5bb73ac3dde2a31746475d3765cda9233f8e06e26ca
 2018-12-27 15:36:16 +00:00
dan
cd80dc3667 Fix a null-pointer dereference that could occur in fts5 with a corrupt 
database.

FossilOrigin-Name: b0a49d5e6df116bb5eacb60a0bb0f482760aec38bd417d4250d9e57d43bdbb13
 2018-12-27 07:46:37 +00:00
dan
0f55a571fc Avoid an "unitialized value read" problem in fts5 that can occur with a 
corrupt database.

FossilOrigin-Name: 6cae81cd1ffb0180e0260d5ba921502c7dee2f54637bff3558a4d1cb5dfd3f32
 2018-12-26 16:57:47 +00:00
dan
c2909bf8ac Fix a problem with handling corrupt databases in the fts5 integrity-check 
function.

FossilOrigin-Name: 9bad745d373f4686a2726c3f77a2ccad60e7181645d7ed483a2e43075a4ef30c
 2018-12-26 16:04:54 +00:00
dan
1ab76166e3 Change another fts5 assert() that may fail with a corrupt database to an 
assert_nc().

FossilOrigin-Name: d9653380c455b4bf6c371540d677cbde6d17664c58662b5cc9a12929f46192dd
 2018-12-24 19:59:37 +00:00
dan
6a509d56bc Catch another case of a corrupted structure record in fts5. 
FossilOrigin-Name: fb17fea4b9779fbd3adb6ff9500da83a6ca4fa7cba379aa70074e4328814a7f2
 2018-12-22 19:54:17 +00:00
dan
862faf28bb Change a faulty assert() in fts5 that could fail on a corrupt db to an if() statement. 
FossilOrigin-Name: 1daaba9fb46706af8ed0669a661c304df999a69cf639d29d93208f422c515d8c
 2018-12-22 19:09:46 +00:00
drh
11d07ed643 Fix one more instance of excess whitespace in test result strings. 
FossilOrigin-Name: c064964deb0b7220ba784f07ca9c603680f3c8bb936d56ab4eca1962beac1359
 2018-12-22 16:14:45 +00:00
drh
3a07548b0d Remove unnecessary whitespace from testcase result strings. 
FossilOrigin-Name: ad70f03ceda74163903d247a975666b35fae722a27c48dd3efd31c1164b5627e
 2018-12-22 15:46:23 +00:00
dan
b63d6070ee Fix an assert() in fts5 that could fail with a corrupt database. 
FossilOrigin-Name: 83935b950619855fb3bf05486091804ac2a61fb993c6ec826c89f943264fabc3
 2018-12-22 15:21:31 +00:00
dan
634ac14488 Fix a "jump or move depends on uninitialized value" valgrind error that could 
occur in fts5 when dealing with a corrupt database.

FossilOrigin-Name: 3518c09b944c92974939b1de244401afd34de5d9aa75050e5ad55581a94e1e71
 2018-12-22 13:34:52 +00:00
dan
38e3853240 Add tests for the use of SQLITE_PREPARE_NO_VTAB in fts5. 
FossilOrigin-Name: 1999612010e310fc2430d1f00076f69fade0c66d202771dbb0f3d5597ac0a03c
 2018-12-21 19:30:18 +00:00
dan
1ea0443c2d Add new sqlite3_prepare_v3() flag SQLITE_PREPARE_NO_VTAB, for preparing 
statements that are not allowed to use any virtual tables. Use this to prevent
circular references in triggers on virtual table shadow tables from causing
resource leaks.

FossilOrigin-Name: 25666e3d03950caf753295cdb55df162e07dbcf6840b05875c6e0b127c469ecb
 2018-12-21 19:29:11 +00:00
dan
bef06d2637 Fix errors in comments in fts5.h. 
FossilOrigin-Name: 4002790d9418289f3e090c694f651a89d8f523119c8ce2f396999b0387148241
 2018-12-15 07:12:34 +00:00