mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
23,042 Commits 1,487 Branches 166 Tags 416 MiB
b9324fea07
Commit Graph

640 Commits

Author SHA1 Message Date
dan
b9324fea07 Do not allow writes to an fts5 table if there are any open fts5vocab cursors. 
FossilOrigin-Name: c49a6ed78a917d4972e048e2a9bbe4d400691f97ce7e022f0e4436ceaed7fb73
 2021-07-05 19:01:09 +00:00
dan
90d3b24a16 Fix a memory leak in fts5 that could occur when processing a query containing multiple classes of error. 
FossilOrigin-Name: 7c279670f544a5bb9733a88862d9e490062b07415cd790e0d6d1e2cb584d2007
 2021-06-30 14:04:25 +00:00
dan
c629e21786 Fix an assert() in fts5 that might fail when dealing with corrupt records. 
FossilOrigin-Name: 9e760e256089c0ebbf816a8de8b186f62e1ffcd6b8b0e76dab8387798cd4d0e1
 2021-06-30 12:11:19 +00:00
dan
8d2a9c3b3a Add defensive code to better handle corrupt records in fts5. 
FossilOrigin-Name: a99c29b5efaf17e34506cda5ce017ec0112b3cb24bf8a2f1831f0c14a8aa8d30
 2021-06-09 13:39:11 +00:00
drh
18f9be897d Remove two assert()s in FTS5 that checks for database corruption right before 
actual code does the same check.
dbsqlfuzz 9680db8f5338cb9b6060eb3a02d984555d4472e3.

FossilOrigin-Name: f3f82967b2e9fd4168d86300138ac4bbcd6ffd7a3ff5ecf20acdc29e5356ac3b
 2021-06-08 23:46:59 +00:00
dan
bc85a51595 Fix a buffer overread that could occur in fts5 when handling corrupt records. 
FossilOrigin-Name: 078962a2164a784b135bacee51ef10973dc2e30de04353d48698d0e72edd63d8
 2021-06-07 17:36:57 +00:00
drh
ff9186137d The fix in the previous check-in was only correct if the OOM occurs on 
the initial allocation.  This changes should make it correct for a resize
as well.

FossilOrigin-Name: 57087ab2f297e4d96da5b9f3b89a26d8dd2bc4d22c2be8045b5d956d86282f39
 2021-05-22 11:00:03 +00:00
drh
d42dbd9dfe If an FTS5 Cursor fails to enlarge the space for the aInst array, set the 
size of the aInst array to zero.
dbsqlfuzz 294254b8105cca409f27a711f1eb2e9e63cbcac5.

FossilOrigin-Name: 4ae5e5b5ecbce580d3f7f970076e828fc6fb5fbcdc3bbe8381ab38f52870a54f
 2021-05-22 01:30:30 +00:00
drh
e31cba5675 Remove an assert() in FTS5 that can (rarely) be false in the event of an 
OOM while processing a corrupt database file.

FossilOrigin-Name: 5f12f89154843e83b1d69ea1652e881d583aa329df732af72dd59029cd7eb577
 2021-05-06 13:45:37 +00:00
dan
a4cd0bbcb3 Fix a segfault that could occur when querying a corrupt data structures with an fts5token table. 
FossilOrigin-Name: cbedfa05501f3eb9750fb21e2ab8516f8f14c479c847f454e52572af278aab71
 2021-04-27 11:28:57 +00:00
dan
f3d5a68461 Fix an assert() in fts5_index.c that may fail if the database records are corrupt. 
FossilOrigin-Name: dfadb96edfef1d49e48a659264b9eb8d7f98d95d159d95c92c30dd4ba6c6e739
 2021-04-23 11:37:01 +00:00
drh
214b50314a Fix harmless compiler warnings of unused function parameters in FTS5. 
FossilOrigin-Name: 14273915979f8fbf80111e795b5511a6ee24638896b9d434d6ccf4bf7f7a7f62
 2021-04-22 19:34:31 +00:00
dan
6a4f7f0225 Remove fts5 test functions fts5_expr(), fts5_expr_tcl(), fts5_isalnum() and fts5_fold() from release builds. 
FossilOrigin-Name: c68a6c6c898d2acd1f7032ff98f8f3f99eb0b37ffbc6ee88d0dd7badf55a0e0a
 2021-04-19 16:45:09 +00:00
dan
304cbc17c2 Fix an undefined signed integer overflow in fts5. 
FossilOrigin-Name: e6f0adb00da84561e686a8db83858c7fd6b008756dd1aef807ea68f878ca3db7
 2021-04-13 17:45:36 +00:00
drh
f1f12661c3 Avoid taking the address of a NULL pointer following an OOM in FTS5. Doing 
so is harmless in actual practice, but it technically UB so we want to
avoid it.

FossilOrigin-Name: 1cfcd9dceb56b5987e6900a36a0ec092f0e1b13a7e754b8c3d8efb943e5bcc66
 2021-04-12 18:32:33 +00:00
dan
d73ade7d87 Fix issue with failing --enable-debug --enable-fts5 build. 
FossilOrigin-Name: e7491acc0c93b7b0b5a27130d956bb23f876b90ced11cdd416a86727af076ee1
 2021-04-06 17:51:18 +00:00
dan
f22c590bab Remove some test routines from release builds of fts5. 
FossilOrigin-Name: 3088bf15ca53fac3bad94f241e648836631b6d801d42837c238ff27d85074ca4
 2021-04-06 14:46:50 +00:00
dan
acbae3ba10 Add "#ifdef SQLITE_DEBUG" blocks around test variables sqlite3_fts5_may_be_corrupt and sqlite3_fts3_may_be_corrupt. 
FossilOrigin-Name: 004559544e661f9afa60306c4ac6842c29a8630b3663d62aa0cdc637f176932e
 2021-03-29 20:04:31 +00:00
drh
8210233c7b Revise tests cases to align with the new EXPLAIN QUERY PLAN output. 
FossilOrigin-Name: 50fbd532602d2c316813046ed6be8be2991c281eb5f295c4c28520a0de73862c
 2021-03-20 15:11:29 +00:00
dan
55b2e8b062 Fix further sanitizer complaints in the fts5 code. 
FossilOrigin-Name: 4cb00b9d4020a51f690720c6f51009c5be8c22ef8c70807ead1d953d19784f26
 2021-03-10 19:32:47 +00:00
dan
3be2b860d0 Fix a few sanitizer complaints in fts5. 
FossilOrigin-Name: 4fe4fee00e9294fa384bd073ed8e9a2a4ad10dcf653b6ad4851e44a64443ede9
 2021-03-10 14:57:10 +00:00
dan
8718f0b613 Fix a buffer overrun that could occur in fts5 when running a prefix query against a corrupt db. 
FossilOrigin-Name: 4da40620ac8557aba6e1e06e720418b55e9950f406288b03749ef1b117041cd9
 2021-03-03 19:36:40 +00:00
dan
5763f3d582 Fix a broken assert() in fts5 that could be triggered by corrupt database records. 
FossilOrigin-Name: b79f59f9ad897d5bd4b9d17e6219bc765b02450bfe14dc020485f221ba6b02cb
 2020-12-19 15:39:10 +00:00
dan
84b168ffa7 Change an fts5 assert() that can be triggered by a corrupt database to an if() condition. 
FossilOrigin-Name: ea0a7f103a6f6a9e57d7377140ff9f372bf2b156f86f148291fb05a7030f2b36
 2020-12-15 13:55:38 +00:00
dan
1418b9daf6 Add tests for a 'delete' command on a contentless table where values that weren't actually inserted are NULL. 
FossilOrigin-Name: 818c647cec7063b33b6c5de3e23599a1d61439fa6e9bf6c974b2522a5a9e1b44
 2020-12-09 16:49:28 +00:00
drh
aeb6bc5628 Fix harmless compiler warnings associated with the recent FTS5 enhancements. 
FossilOrigin-Name: 1db7c751912beb57a697ac8e85b9c29e30da7b6c89207e9828bf08e56c58242f
 2020-12-04 17:05:16 +00:00
dan
b9ceb833f7 Ensure that fts5 function matchinfo() is registered before running tests that use it in fts5corrupt3.test. 
FossilOrigin-Name: 932e05e093192991589e70c2cbcc8a57fb3dcc6df1c2673962bfd06ba5cfdd97
 2020-12-04 16:49:25 +00:00
dan
3190b88e75 When merging prefix lists in fts5, use 16-way merges instead of 2-way merges. This faster. 
FossilOrigin-Name: 026a93508ec392ca5cd2578ae9eab64974f58beccda088e10d4cc951f237632f
 2020-12-03 21:22:37 +00:00
dan
64f1ef6abd Allow a search for an N character prefix in fts5 to use a prefix index of size N+1, if no prefix index of size N exists. 
FossilOrigin-Name: 78a7801d8fc9e58a62e5168e35b52b7440340549123fc6a537e2abd571f6fe7b
 2020-12-02 19:08:15 +00:00
dan
0fa433b40c Improve performance for fts5 column filters that filter for more than one column. e.g. "{col1 col2 col3}:phrase". 
FossilOrigin-Name: d8de2f236d43a88fac7550a0451951dd5a945eb304e32f82e662479cea7c2684
 2020-12-02 16:23:05 +00:00
dan
cad760d16e Fix a compiler warning in fts5_aux.c. 
FossilOrigin-Name: 8edb983bc87898eff2cd2e7e672a32a47c71b2be9d818513d339e95560d45b2b
 2020-11-27 19:40:13 +00:00
dan
6d19bf9460 Add missing cast to fts5 bm25() code. 
FossilOrigin-Name: 6ff9673847c0b4174d9435e93d19af0ee7406b1a12edeb6edec98697e1646824
 2020-11-27 16:15:55 +00:00
dan
3181706895 Remove a redundant "if( rc==SQLITE_OK )" from the fts5 bm25() code. 
FossilOrigin-Name: d85f4f27f58adcc75fc7d59e63af95b2a338052d8748a11f22ec1e48d1aff4cc
 2020-11-27 16:08:15 +00:00
dan
66efc393c5 Move an "if( rc==SQLITE_OK )" to outside a loop body in the fts5 bm25() code. 
FossilOrigin-Name: 14a4dcf3474566d072007a37d214c892397c21dd3b7f8b55ad0e5edfb7130dd6
 2020-11-27 16:05:31 +00:00
dan
1c5b23f170 Fix a case in the FTS5 integrity check where a corrupt database could cause a buffer overread. 
FossilOrigin-Name: a32b4f650d2d543bd2773cbc8655c1679a20b35ac9ec4d08c7754ddf6d972acb
 2020-11-27 15:28:26 +00:00
dan
ec8962869a Update mkunicode.tcl to match the change erroneously made to machine generated file fts5_unicode2.c in [b7b7bde9]. 
FossilOrigin-Name: 326d579d777fdede6bc64f9525248767f4730de4e50260b0387e614a9d006416
 2020-11-26 20:13:54 +00:00
drh
d36f588f31 Fix harmless compiler warnings about unused function parameters. 
FossilOrigin-Name: 25d067c270966d9506db8bedf280883e32b69050b14bdbbeda4bb2d9a362619c
 2020-11-25 16:28:04 +00:00
dan
d1d43efa4f Prevent fts5 tokenizer unicode61 from considering '\0' to be a token characters, even if other characters of class "Cc" are. 
FossilOrigin-Name: b7b7bde9b7a03665e3691c6d51118965f216d2dfb1617f138b9f9e60e418ed2f
 2020-10-26 13:24:36 +00:00
dan
f6b0464e36 Fix some fts5 problems with signed integer overflow causing segfaults in -ftrapv builds. 
FossilOrigin-Name: 7e17c2f4b7dc9b563d0b4da949bb134dc7c4fc9c86ce03891432a884ca6409d5
 2020-10-12 18:09:16 +00:00
dan
f46be6a1b9 Allow fts5 trigram tables created with detail=column or detail=none to optimize LIKE and GLOB queries. Allow case-insensitive tables to optimize GLOB as well as LIKE. 
FossilOrigin-Name: 64782463be62b72b5cd0bfaa7c9b69aa487d807c5fe0e65a272080b7739fd21b
 2020-10-05 16:41:56 +00:00
dan
12a6a1eaf9 Fix a segfault caused by running "column LIKE NULL" against an fts5 table using the trigram tokenizer. Fix for [e33ee62575fc22]. 
FossilOrigin-Name: 6e72a08de764077f2bba6f7e3b99ea29001941671a971f2ccf7ceeb9c682fb1a
 2020-10-03 17:06:02 +00:00
dan
95dca8d0cf FTS5 does not handle tokens that contain embedded nul characters. Prevent the trigram tokenizer from returning such tokens. Fix for [2ba5930b2]. 
FossilOrigin-Name: b1d048748c054575425a4bebf0c5d09962f9329d5ce6a978cf54e508b238584c
 2020-10-03 14:36:06 +00:00
dan
a22c1c84cc Add the -trigram option to the fts5txt2db.tcl script. 
FossilOrigin-Name: a394dd260252e7b7dcbc90fd321dfd40d9375f0598ea52b01001c568cbdbafce
 2020-10-01 17:15:12 +00:00
dan
ccf578d435 Add tests for the trigram tokenizer. Fix minor issues. 
FossilOrigin-Name: 897ced99b44085012aa44d3264940dcbd4c77b295a894a1b58fb2c03a0f7fee8
 2020-10-01 16:10:22 +00:00
dan
33a99fad08 Add experimental unicode-aware trigram tokenizer to fts5. And support for LIKE and GLOB optimizations for fts5 tables that use said tokenizer. 
FossilOrigin-Name: 0d7810c1aea93c0a3da1ccc4911dbce8a1b6e1dbfe1ab7e800289a0c783b5985
 2020-09-30 20:35:37 +00:00
dan
7548ab20e6 In fts5 integrity checks, do not compare the contents of the index against an external content table unless specifically requested. 
FossilOrigin-Name: 782163693f37aeb65209bebbaeb6659a36881b8c4b4bec778b366658488bf966
 2020-09-21 14:53:21 +00:00
dan
f2e151aeab Fix a problem in fts5 causing the in-memory hash table to be flushed more often than necessary. 
FossilOrigin-Name: 9a55601d072bcd1ac0f6f1633216c7c8b3ff3e037451e2cfd285cbe19b60b9fc
 2020-09-17 15:16:50 +00:00
dan
3acd3fb9d8 Fix a "jump depends on unitialized value" valgrind/asan error in fts5. 
FossilOrigin-Name: 66f94d623c16b4527e42f0519e32ec78eaf9ee6fd61ee7d0eb657cc59a8f7d68
 2020-09-17 14:16:02 +00:00
dan
86f477edaa Catch fts5 index corruption caused by issuing 'delete' commands with incorrect data earlier in some cases. Also fix a couple of test script problems. 
FossilOrigin-Name: b79f19edfd33c2a75f936c352668e14e81f35acf4f07edc27a21f941a7304b38
 2020-09-11 15:01:49 +00:00
dan
832aa023c6 Add fts5 test to confirm that for a table with columns a, b, c and d, "{a b}" and "-{c d}" are handled similarly. 
FossilOrigin-Name: 1a04920998368e56276fd0b100be8343609c6ff8a731cf8e26a0490f9c6dabdf
 2020-08-28 11:19:49 +00:00