mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
26,788 Commits 1,487 Branches 166 Tags 416 MiB
ac5e1f82ce
Commit Graph

743 Commits

Author SHA1 Message Date
dan
4d24eff8dc Avoid a crash in fts5 when combining an empty expression with another. 
FossilOrigin-Name: 33681ff3d29c9f096dbd8c5d885993f780e93974a7653adc00fa41a158356d7c
 2023-04-22 20:47:08 +00:00
dan
8cb4f0b1b2 Add tests for the progress-handler callback with the fts5 'secure-delete' feature. 
FossilOrigin-Name: 48505ad950bc0902d58210be066d4672e6085eb27c525ba2bc663fde7e932ff7
 2023-04-19 20:29:26 +00:00
dan
a5eab9579a Fix an assert() in fts5 that could fail following an OOM or IO error. 
FossilOrigin-Name: 49595083ba58e13d404b6a11c28f53768a257c59f3f36be3591438127df7e875
 2023-04-18 11:33:35 +00:00
dan
68c0da6f66 Fix some problems with using fts5 options 'secure-delete' and detail=none together. 
FossilOrigin-Name: 4d3f27ba90aa59837e49ceaae9f36cad426af3a33707d85d1bb848fc1bda6fed
 2023-04-17 18:32:22 +00:00
dan
48824f23cb Fix problems handling corrupt records in new fts5 code. 
FossilOrigin-Name: 2f63d4a8bac6dc72197df32efd84c148adbc4794943088dff08da57b15dd7900
 2023-04-17 11:41:42 +00:00
dan
5c45a9fbaf Fix an assert() in fts5 that may be true when dealing with corrupt records. 
FossilOrigin-Name: e8de2774463c07b76cd6b9a44ab32a583fe973f42a21b21af46fe82ab177c1b8
 2023-04-17 10:39:51 +00:00
dan
ae6f6ef3b0 Change a memcpy() to memmove() in new fts5 code. 
FossilOrigin-Name: 8de4542553ae20947787b4daf5e88bbe69f815679960e700e19f9c1148234b08
 2023-04-15 21:00:55 +00:00
dan
0e429cf934 Fix a use-after-free that could follow an OOM or IO error in fts5 when using 'secure-delete' mode. 
FossilOrigin-Name: 904be83d9f3dd4b21a3aa47dde337b5c045b925ffdde4bccd1f8e8fdbef68c18
 2023-04-15 19:58:58 +00:00
dan
e45317a50a Avoid dropping the error code following an OOM in fts5 when using 'secure-delete' mode. 
FossilOrigin-Name: 9ee24a499c7b4a048f269f1c03c7c0b22b2898a3c23ea3e2511c2ba0342afa30
 2023-04-15 19:13:51 +00:00
dan
112500028e Further tests for fts5 secure-delete mode. 
FossilOrigin-Name: c5a47063b7971868ebbb3f0718434dd6bd22d01c43697f48941df9aae3885abc
 2023-04-14 18:00:03 +00:00
dan
3fc123b7bc Add extra OOM test for the new code on this branch. 
FossilOrigin-Name: 846ae7e099ce67dab6b5df0ad5648a01d7bf7f2acf8312d817ed8505d4ea5aec
 2023-04-14 17:00:29 +00:00
dan
d36106c954 Add tests for the new feature on this branch. 
FossilOrigin-Name: 0268d1a59316d169927cbd4c562725e46a9023b7d7123ade911c12203e222adf
 2023-04-14 16:11:05 +00:00
dan
02877e9558 Remove temporary debugging code accidentally left on this branch. 
FossilOrigin-Name: 0a0f64870feaf95d2673efc8884ad1236d9bbf9bdb94364f8d2602221bd36ff3
 2023-04-12 18:06:43 +00:00
dan
015020cd1a Add the 'secure-delete' option to fts5. Used to configure fts5 to aggressively remove old full-text-index entries belonging to deleted or updated rows. 
FossilOrigin-Name: 4240fd09b717dbc69dffe3b88ec9149777ca4c3efa12f282af65be3af6fa5bb0
 2023-04-12 17:40:44 +00:00
drh
ebec9c475e A prepared statement that aborts due to SQLITE_SCHEMA should not invalidate 
cursors in other prepared statements that are already running.
See [forum:/forumpost/cae4367d9b|forum post cae4367d9b] for the original
trouble report.

FossilOrigin-Name: 857d0f5e16ba69ac9e5ee581befca2f7ed933edfe6e36396b10dae7979b44a57
 2023-04-06 13:35:42 +00:00
dan
5f708cc832 Fix a problem with the fts5 snippet() function that shows up when snippets just 1 token in length are requested. 
FossilOrigin-Name: 96d5116d17cbf3de6be38952881c6ddf6a226ddfd7649197bdb2df39c42b5d8e
 2023-03-10 13:36:19 +00:00
dan
a3e6192941 Fix a problem with the fts5 trigram tokenizer and LIKE or GLOB patterns for which contain runs of 2 or fewer non-wildcard characters that are 3 or more bytes when encoded as utf-8. 
FossilOrigin-Name: 00714b39b39c51519edbc0194f98c7275fecf96763a06fd95db6e1d81bb9f1f1
 2023-02-10 17:17:04 +00:00
dan
0d12a79310 Better fix the problem where optimizing an fts5 table too often causes it to become unreadable (first attempt was [35bed981]). 
FossilOrigin-Name: 459d986d38fc0ccbfd66801e0f22900cfed831268cf59ac8d1cd1e556f0d1441
 2023-02-08 17:28:08 +00:00
dan
d40a183506 Fix a problem where optimizing an fts5 table too often might cause it to become unreadable. 
FossilOrigin-Name: 35bed981416269018a0d52171d9997c43d3b097f5db1910d10d882eabe2e137f
 2023-02-03 19:28:50 +00:00
drh
3601db8f86 The json_group_array(), json_group_object(), and fts5_source_id() functions 
should all be deterministic and innocuous.

FossilOrigin-Name: edcb83fa0d3c832d044b267d608fed31a2ac841b81c7c9406b2136ac43d61a1a
 2023-01-20 13:34:14 +00:00
dan
1dff401c42 Fix a broken assert() in fts5. 
FossilOrigin-Name: ff5a9b054f4d5ad3d33787aee93b4181822c6dfc513626236d9867bb431f64da
 2022-12-27 13:52:25 +00:00
drh
3547e4997f Fix lots of harmless, nuisance compiler warnings, mostly unused parameter 
warnings in extensions.

FossilOrigin-Name: c14bbe1606c1450b709970f922b94a641dfc8f9bd09126501d7dc4db99ea4772
 2022-12-23 14:49:24 +00:00
dan
73e460c932 Fix a faulty assert() in fts5. 
FossilOrigin-Name: 569018170b928cadfa0745db631981084e595d90a8c3d1e236c42adae161018f
 2022-12-19 20:14:22 +00:00
dan
1532f9d155 Fix an assert() in fts5. Simplify memdb xLock/xUnlock some. 
FossilOrigin-Name: 7fe158aa8071acadd959b2b4a4b66e8a7f7eecec207ba681abf516b0457c2921
 2022-12-19 14:06:36 +00:00
dan
838865c0bb Better handle an error in the fts5 integrity-check code. dbsqlfuzz e87c62f9b67ea21aebdc36ab71cab7cc3eda8dc3. 
FossilOrigin-Name: ae43e97087a3207a5ca3ffae75fbe7a33c01f4a38ce0d1d7eed8591ae3083617
 2022-11-09 11:17:57 +00:00
dan
6aafd74853 Avoid some cases of signed integer overflow in fts5 by casting to unsigned values. 
FossilOrigin-Name: 46a78c8c0ed518c4521e6e0bdebeb065bab07076abc444775002e7f4361d2242
 2022-08-08 19:29:53 +00:00
dan
cf2ad7ae36 Remove a few unsuitable scripts from the "veryquick" test suite. Also have every second testrunner.tcl process favour running test scripts that contain text like "testrunner: slow" before any others. 
FossilOrigin-Name: 22d280a5cd395abbedcfffbac3d3b3a614c327be25763ca380c1338a2a7bd33a
 2022-07-18 19:32:30 +00:00
dan
f703b42dc4 Update the expected error messages in some OOM test cases to account for [44d77a7f807]. 
FossilOrigin-Name: b3d6b3c3fc68dca7e20418eefa35ce3b583322b884b88a11c6773419f027a7a4
 2022-07-11 21:40:34 +00:00
drh
de7a820fd0 Minor typo fix in a comment, for the benefit of broken IDE syntax highlighters. 
FossilOrigin-Name: 680ccecbc839373194ed2dcc1053e25512d9ea3f12f8c8963cb8fbd1a034d2f8
 2022-04-29 12:09:43 +00:00
drh
c381056e6d Fix harmless compiler warnings. 
FossilOrigin-Name: 61095b5bcb5c30d29f03efa5b4809e3a0b780ed57e35227c292aab9dd1cfe3f6
 2022-04-07 10:11:35 +00:00
dan
7ef4d75bd2 Fix a case in fts5 where a corrupt database could cause a crash. 
FossilOrigin-Name: 5e95df261cce275adb5b2226e9584f4c389bb0d53c3b4feafd5a14be8d02ce63
 2022-03-11 12:02:18 +00:00
drh
7d44b22d45 Fix test cases so that they all still work even with -DSQLITE_DQS=0. 
FossilOrigin-Name: 4883776669ee2f2310ea82b0d6df4d008eebaa7cb252102539cf21a635402ebb
 2022-01-16 19:11:13 +00:00
drh
a6ca90c38c Attempt to fix a harmless compiler warning in FTS5. 
FossilOrigin-Name: 8e619c21e2326be1538b60908e7cd211558ec840835c6eb69e768eb190e1fd0b
 2022-01-01 19:55:29 +00:00
dan
c436b3056d Fix an OOB read that could occur in fts5 when processing corrupt records. 
FossilOrigin-Name: bb9b1a15f7e80483162049dfd981d059dc69d03348b521f7ac164a8cd3ae3cc4
 2021-12-06 18:57:02 +00:00
drh
913306a53a Improved defenses against integer overflow when computing the size of a 
memory allocations.  No bugs were fixed here.  But perhaps future bugs will
be prevented.

FossilOrigin-Name: eb878c01f317f09e8ef6b1bd2ec8d6d5cd6ce0bdfe9da7fa7d92d2047cc9d9e4
 2021-11-26 17:10:18 +00:00
dan
76c3ae6848 Fix a broken assert() in fts5 that could fail if the snippet() function were used with a query phrase containing 0 tokens on a detail=none or detail=col table. 
FossilOrigin-Name: 0a5b94b99cf45df759a59cb5fc62da111ca33f8c2614769a81930d72b51f093a
 2021-11-02 07:32:13 +00:00
drh
134f544ab2 Fix a harmless scan-build warning in FTS5. 
FossilOrigin-Name: 0bf42bb5611dc3f672cb898b8be245fd25f7a3862c1e0734effd18d75e812f22
 2021-10-16 19:50:03 +00:00
dan
f7ff755651 Fix an assert() in fts5 that could fail with a corrupt database. 
FossilOrigin-Name: e99979855de937ed5ee0994b180054501400bf8776fb70acd31786d2ba1ad49a
 2021-10-16 13:59:08 +00:00
dan
33cf194218 Fix a signed integer overflow in fts5 leading to a segfault that could occur when processing corrupt records. 
FossilOrigin-Name: 69a3ff0cc159cdf84a5367eaf708882ddeda4fa65c96a5b546ae4a0114f02cb7
 2021-10-14 21:13:02 +00:00
drh
13ca72d011 Fix harmless compiler warnings, mostly caused by prior efforts to get static 
analyzers to run without warnings.

FossilOrigin-Name: cddd3b382a118bb16be3a602ae711fe2c2a7d1c673dd30dd3977b67ee555e666
 2021-10-05 18:59:05 +00:00
drh
d7ace22d8c Fix more harmless static-analyzer warnings. As of this check-in, no warnings 
are generated by scan-build for clang-6 or clang-10.

FossilOrigin-Name: a7835bead85b1b18a8affd9835240b0baf9c7af887196bbdcc3f5d58055042fc
 2021-10-05 18:33:38 +00:00
drh
7a3b4451a1 Fixes for harmless static-analyzer warnings. This also makes the code easier 
for humans to understand.

FossilOrigin-Name: 36177a62feeb4fa93ab6e3c6f4dbe1ddcf63bb02f93284abab979da0261b218e
 2021-10-05 17:41:12 +00:00
drh
640574f61a Small change to FTS5 to help static analyzers understand that it is not 
dereferencing a NULL pointer.

FossilOrigin-Name: 0d853df1b19a0270b2cb0ebdf5df8067572f19ab9a82d047cb047f3f9e850348
 2021-10-05 13:00:50 +00:00
drh
922c54206f Some #defines somehow failed to get set correctly in the previous check-in. 
Fixed here.

FossilOrigin-Name: 15bbdf9ac840a220f384411d3025ef22f949d310194b60bca8e6d6a759e6042e
 2021-10-04 18:57:42 +00:00
drh
11a9ad5669 Fix harmless static analyzer warnings in sessions, rtree, fts3 and fts5. 
Add the -DSQLITE_OMIT_AUXILIARY_SAFETY_CHECKS compile-time option to cause
ALWAYS() and NEVER() macros to be omitted from the build.

FossilOrigin-Name: 1c67f957fc77e37ce8f0d447c41ca975e8e79a35d332739c24a633649b5b0387
 2021-10-04 18:21:14 +00:00
dan
adfb203a7d Fix a buffer overread in fts5 that could occur when processing an "ORDER BY rowid DESC" query against corrupt database records. 
FossilOrigin-Name: fe098a93340b8dac15752f1be4f681e2a591f7ae1a94b177f400ccd7ef638480
 2021-10-01 15:05:19 +00:00
dan
415540ddaa Fix an out-of-bounds read in fts5 that could occur when processing corrupt records. 
FossilOrigin-Name: 9bbc01fb239b4aa752a56c584baa29655a3c845425d5c17d24a3939984d54fe2
 2021-09-29 13:36:58 +00:00
drh
dd31c033c6 Fix harmless compiler warnings. 
FossilOrigin-Name: d678ecca02698753d1b33e072566112e94ea36d0d3a8f4a24d2b09d131968d88
 2021-09-22 14:43:35 +00:00
dan
eb84c6ef35 Fix a memory leak in fts5 that could occur when writing to an fts5 table that is being scanned by an fts5vocab cursor. 
FossilOrigin-Name: 1183552de10b64a776874a23be8962c0693e5e0cd1a191ea8fa5d6a02b691e4a
 2021-09-07 16:05:48 +00:00
dan
fb8ca7de0c Fix a use-after-free error that could occur if an fts5 table is written while scanning it using an fts5vocab cursor. 
FossilOrigin-Name: e751c2ec786b5c1a1c9640fdc3fde036879a2c32db2bd67fe7c72604780f67b8
 2021-09-06 16:15:23 +00:00
drh
20de9f6cb1 Minor tweaks to various TCL build scripts so that they work even for TCL 8.4, 
which is obsolete, but is also all that is available for some legacy platforms.

FossilOrigin-Name: 2269ce64f707c4a198fcb9a72933648ea61a80b4251024b8058ff28ad81b4917
 2021-07-29 16:49:28 +00:00
dan
73983658c9 In defensive mode, allow statements that write to shadow tables to be prepared from with virtual-table xSync() calls. 
FossilOrigin-Name: c8601d83fbecf84ca7991d339449b380deb5a7620dc84756a91c2880f030b423
 2021-07-19 14:00:29 +00:00
dan
cc516af4cc Instead of disallowing writes to fts5 tables if there are fts5vocab cursors open on them (commit [c49a6ed7]), abort any fts5vocab queries if the on-disk structure of the fts5 table changes. 
FossilOrigin-Name: 9dbdc9001e3258e71ca995fbcdebf66ab95890ded87fa7125c6cb4bd43010aaf
 2021-07-07 11:51:03 +00:00
dan
b9324fea07 Do not allow writes to an fts5 table if there are any open fts5vocab cursors. 
FossilOrigin-Name: c49a6ed78a917d4972e048e2a9bbe4d400691f97ce7e022f0e4436ceaed7fb73
 2021-07-05 19:01:09 +00:00
dan
90d3b24a16 Fix a memory leak in fts5 that could occur when processing a query containing multiple classes of error. 
FossilOrigin-Name: 7c279670f544a5bb9733a88862d9e490062b07415cd790e0d6d1e2cb584d2007
 2021-06-30 14:04:25 +00:00
dan
c629e21786 Fix an assert() in fts5 that might fail when dealing with corrupt records. 
FossilOrigin-Name: 9e760e256089c0ebbf816a8de8b186f62e1ffcd6b8b0e76dab8387798cd4d0e1
 2021-06-30 12:11:19 +00:00
dan
8d2a9c3b3a Add defensive code to better handle corrupt records in fts5. 
FossilOrigin-Name: a99c29b5efaf17e34506cda5ce017ec0112b3cb24bf8a2f1831f0c14a8aa8d30
 2021-06-09 13:39:11 +00:00
drh
18f9be897d Remove two assert()s in FTS5 that checks for database corruption right before 
actual code does the same check.
dbsqlfuzz 9680db8f5338cb9b6060eb3a02d984555d4472e3.

FossilOrigin-Name: f3f82967b2e9fd4168d86300138ac4bbcd6ffd7a3ff5ecf20acdc29e5356ac3b
 2021-06-08 23:46:59 +00:00
dan
bc85a51595 Fix a buffer overread that could occur in fts5 when handling corrupt records. 
FossilOrigin-Name: 078962a2164a784b135bacee51ef10973dc2e30de04353d48698d0e72edd63d8
 2021-06-07 17:36:57 +00:00
drh
ff9186137d The fix in the previous check-in was only correct if the OOM occurs on 
the initial allocation.  This changes should make it correct for a resize
as well.

FossilOrigin-Name: 57087ab2f297e4d96da5b9f3b89a26d8dd2bc4d22c2be8045b5d956d86282f39
 2021-05-22 11:00:03 +00:00
drh
d42dbd9dfe If an FTS5 Cursor fails to enlarge the space for the aInst array, set the 
size of the aInst array to zero.
dbsqlfuzz 294254b8105cca409f27a711f1eb2e9e63cbcac5.

FossilOrigin-Name: 4ae5e5b5ecbce580d3f7f970076e828fc6fb5fbcdc3bbe8381ab38f52870a54f
 2021-05-22 01:30:30 +00:00
drh
e31cba5675 Remove an assert() in FTS5 that can (rarely) be false in the event of an 
OOM while processing a corrupt database file.

FossilOrigin-Name: 5f12f89154843e83b1d69ea1652e881d583aa329df732af72dd59029cd7eb577
 2021-05-06 13:45:37 +00:00
dan
a4cd0bbcb3 Fix a segfault that could occur when querying a corrupt data structures with an fts5token table. 
FossilOrigin-Name: cbedfa05501f3eb9750fb21e2ab8516f8f14c479c847f454e52572af278aab71
 2021-04-27 11:28:57 +00:00
dan
f3d5a68461 Fix an assert() in fts5_index.c that may fail if the database records are corrupt. 
FossilOrigin-Name: dfadb96edfef1d49e48a659264b9eb8d7f98d95d159d95c92c30dd4ba6c6e739
 2021-04-23 11:37:01 +00:00
drh
214b50314a Fix harmless compiler warnings of unused function parameters in FTS5. 
FossilOrigin-Name: 14273915979f8fbf80111e795b5511a6ee24638896b9d434d6ccf4bf7f7a7f62
 2021-04-22 19:34:31 +00:00
dan
6a4f7f0225 Remove fts5 test functions fts5_expr(), fts5_expr_tcl(), fts5_isalnum() and fts5_fold() from release builds. 
FossilOrigin-Name: c68a6c6c898d2acd1f7032ff98f8f3f99eb0b37ffbc6ee88d0dd7badf55a0e0a
 2021-04-19 16:45:09 +00:00
dan
304cbc17c2 Fix an undefined signed integer overflow in fts5. 
FossilOrigin-Name: e6f0adb00da84561e686a8db83858c7fd6b008756dd1aef807ea68f878ca3db7
 2021-04-13 17:45:36 +00:00
drh
f1f12661c3 Avoid taking the address of a NULL pointer following an OOM in FTS5. Doing 
so is harmless in actual practice, but it technically UB so we want to
avoid it.

FossilOrigin-Name: 1cfcd9dceb56b5987e6900a36a0ec092f0e1b13a7e754b8c3d8efb943e5bcc66
 2021-04-12 18:32:33 +00:00
dan
d73ade7d87 Fix issue with failing --enable-debug --enable-fts5 build. 
FossilOrigin-Name: e7491acc0c93b7b0b5a27130d956bb23f876b90ced11cdd416a86727af076ee1
 2021-04-06 17:51:18 +00:00
dan
f22c590bab Remove some test routines from release builds of fts5. 
FossilOrigin-Name: 3088bf15ca53fac3bad94f241e648836631b6d801d42837c238ff27d85074ca4
 2021-04-06 14:46:50 +00:00
dan
acbae3ba10 Add "#ifdef SQLITE_DEBUG" blocks around test variables sqlite3_fts5_may_be_corrupt and sqlite3_fts3_may_be_corrupt. 
FossilOrigin-Name: 004559544e661f9afa60306c4ac6842c29a8630b3663d62aa0cdc637f176932e
 2021-03-29 20:04:31 +00:00
drh
8210233c7b Revise tests cases to align with the new EXPLAIN QUERY PLAN output. 
FossilOrigin-Name: 50fbd532602d2c316813046ed6be8be2991c281eb5f295c4c28520a0de73862c
 2021-03-20 15:11:29 +00:00
dan
55b2e8b062 Fix further sanitizer complaints in the fts5 code. 
FossilOrigin-Name: 4cb00b9d4020a51f690720c6f51009c5be8c22ef8c70807ead1d953d19784f26
 2021-03-10 19:32:47 +00:00
dan
3be2b860d0 Fix a few sanitizer complaints in fts5. 
FossilOrigin-Name: 4fe4fee00e9294fa384bd073ed8e9a2a4ad10dcf653b6ad4851e44a64443ede9
 2021-03-10 14:57:10 +00:00
dan
8718f0b613 Fix a buffer overrun that could occur in fts5 when running a prefix query against a corrupt db. 
FossilOrigin-Name: 4da40620ac8557aba6e1e06e720418b55e9950f406288b03749ef1b117041cd9
 2021-03-03 19:36:40 +00:00
dan
5763f3d582 Fix a broken assert() in fts5 that could be triggered by corrupt database records. 
FossilOrigin-Name: b79f59f9ad897d5bd4b9d17e6219bc765b02450bfe14dc020485f221ba6b02cb
 2020-12-19 15:39:10 +00:00
dan
84b168ffa7 Change an fts5 assert() that can be triggered by a corrupt database to an if() condition. 
FossilOrigin-Name: ea0a7f103a6f6a9e57d7377140ff9f372bf2b156f86f148291fb05a7030f2b36
 2020-12-15 13:55:38 +00:00
dan
1418b9daf6 Add tests for a 'delete' command on a contentless table where values that weren't actually inserted are NULL. 
FossilOrigin-Name: 818c647cec7063b33b6c5de3e23599a1d61439fa6e9bf6c974b2522a5a9e1b44
 2020-12-09 16:49:28 +00:00
drh
aeb6bc5628 Fix harmless compiler warnings associated with the recent FTS5 enhancements. 
FossilOrigin-Name: 1db7c751912beb57a697ac8e85b9c29e30da7b6c89207e9828bf08e56c58242f
 2020-12-04 17:05:16 +00:00
dan
b9ceb833f7 Ensure that fts5 function matchinfo() is registered before running tests that use it in fts5corrupt3.test. 
FossilOrigin-Name: 932e05e093192991589e70c2cbcc8a57fb3dcc6df1c2673962bfd06ba5cfdd97
 2020-12-04 16:49:25 +00:00
dan
3190b88e75 When merging prefix lists in fts5, use 16-way merges instead of 2-way merges. This faster. 
FossilOrigin-Name: 026a93508ec392ca5cd2578ae9eab64974f58beccda088e10d4cc951f237632f
 2020-12-03 21:22:37 +00:00
dan
64f1ef6abd Allow a search for an N character prefix in fts5 to use a prefix index of size N+1, if no prefix index of size N exists. 
FossilOrigin-Name: 78a7801d8fc9e58a62e5168e35b52b7440340549123fc6a537e2abd571f6fe7b
 2020-12-02 19:08:15 +00:00
dan
0fa433b40c Improve performance for fts5 column filters that filter for more than one column. e.g. "{col1 col2 col3}:phrase". 
FossilOrigin-Name: d8de2f236d43a88fac7550a0451951dd5a945eb304e32f82e662479cea7c2684
 2020-12-02 16:23:05 +00:00
dan
cad760d16e Fix a compiler warning in fts5_aux.c. 
FossilOrigin-Name: 8edb983bc87898eff2cd2e7e672a32a47c71b2be9d818513d339e95560d45b2b
 2020-11-27 19:40:13 +00:00
dan
6d19bf9460 Add missing cast to fts5 bm25() code. 
FossilOrigin-Name: 6ff9673847c0b4174d9435e93d19af0ee7406b1a12edeb6edec98697e1646824
 2020-11-27 16:15:55 +00:00
dan
3181706895 Remove a redundant "if( rc==SQLITE_OK )" from the fts5 bm25() code. 
FossilOrigin-Name: d85f4f27f58adcc75fc7d59e63af95b2a338052d8748a11f22ec1e48d1aff4cc
 2020-11-27 16:08:15 +00:00
dan
66efc393c5 Move an "if( rc==SQLITE_OK )" to outside a loop body in the fts5 bm25() code. 
FossilOrigin-Name: 14a4dcf3474566d072007a37d214c892397c21dd3b7f8b55ad0e5edfb7130dd6
 2020-11-27 16:05:31 +00:00
dan
1c5b23f170 Fix a case in the FTS5 integrity check where a corrupt database could cause a buffer overread. 
FossilOrigin-Name: a32b4f650d2d543bd2773cbc8655c1679a20b35ac9ec4d08c7754ddf6d972acb
 2020-11-27 15:28:26 +00:00
dan
ec8962869a Update mkunicode.tcl to match the change erroneously made to machine generated file fts5_unicode2.c in [b7b7bde9]. 
FossilOrigin-Name: 326d579d777fdede6bc64f9525248767f4730de4e50260b0387e614a9d006416
 2020-11-26 20:13:54 +00:00
drh
d36f588f31 Fix harmless compiler warnings about unused function parameters. 
FossilOrigin-Name: 25d067c270966d9506db8bedf280883e32b69050b14bdbbeda4bb2d9a362619c
 2020-11-25 16:28:04 +00:00
dan
d1d43efa4f Prevent fts5 tokenizer unicode61 from considering '\0' to be a token characters, even if other characters of class "Cc" are. 
FossilOrigin-Name: b7b7bde9b7a03665e3691c6d51118965f216d2dfb1617f138b9f9e60e418ed2f
 2020-10-26 13:24:36 +00:00
dan
f6b0464e36 Fix some fts5 problems with signed integer overflow causing segfaults in -ftrapv builds. 
FossilOrigin-Name: 7e17c2f4b7dc9b563d0b4da949bb134dc7c4fc9c86ce03891432a884ca6409d5
 2020-10-12 18:09:16 +00:00
dan
f46be6a1b9 Allow fts5 trigram tables created with detail=column or detail=none to optimize LIKE and GLOB queries. Allow case-insensitive tables to optimize GLOB as well as LIKE. 
FossilOrigin-Name: 64782463be62b72b5cd0bfaa7c9b69aa487d807c5fe0e65a272080b7739fd21b
 2020-10-05 16:41:56 +00:00
dan
12a6a1eaf9 Fix a segfault caused by running "column LIKE NULL" against an fts5 table using the trigram tokenizer. Fix for [e33ee62575fc22]. 
FossilOrigin-Name: 6e72a08de764077f2bba6f7e3b99ea29001941671a971f2ccf7ceeb9c682fb1a
 2020-10-03 17:06:02 +00:00
dan
95dca8d0cf FTS5 does not handle tokens that contain embedded nul characters. Prevent the trigram tokenizer from returning such tokens. Fix for [2ba5930b2]. 
FossilOrigin-Name: b1d048748c054575425a4bebf0c5d09962f9329d5ce6a978cf54e508b238584c
 2020-10-03 14:36:06 +00:00
dan
a22c1c84cc Add the -trigram option to the fts5txt2db.tcl script. 
FossilOrigin-Name: a394dd260252e7b7dcbc90fd321dfd40d9375f0598ea52b01001c568cbdbafce
 2020-10-01 17:15:12 +00:00
dan
ccf578d435 Add tests for the trigram tokenizer. Fix minor issues. 
FossilOrigin-Name: 897ced99b44085012aa44d3264940dcbd4c77b295a894a1b58fb2c03a0f7fee8
 2020-10-01 16:10:22 +00:00
dan
33a99fad08 Add experimental unicode-aware trigram tokenizer to fts5. And support for LIKE and GLOB optimizations for fts5 tables that use said tokenizer. 
FossilOrigin-Name: 0d7810c1aea93c0a3da1ccc4911dbce8a1b6e1dbfe1ab7e800289a0c783b5985
 2020-09-30 20:35:37 +00:00
dan
7548ab20e6 In fts5 integrity checks, do not compare the contents of the index against an external content table unless specifically requested. 
FossilOrigin-Name: 782163693f37aeb65209bebbaeb6659a36881b8c4b4bec778b366658488bf966
 2020-09-21 14:53:21 +00:00
dan
f2e151aeab Fix a problem in fts5 causing the in-memory hash table to be flushed more often than necessary. 
FossilOrigin-Name: 9a55601d072bcd1ac0f6f1633216c7c8b3ff3e037451e2cfd285cbe19b60b9fc
 2020-09-17 15:16:50 +00:00