mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
28,790 Commits 1,487 Branches 166 Tags 416 MiB
dd719bebfa
Commit Graph

120 Commits

Author SHA1 Message Date
dan
75f9ba32bd Ensure that the xIntegrity methods of fts3 and fts5 work on read-only databases. 
FossilOrigin-Name: b855886c4ccce0745af6957943e77be18949722f09821688725d546d3d79b4fb
 2024-01-16 16:05:21 +00:00
dan
ef6bf1bbe5 Fix a problem allowing a COMMIT following an OOM to cause fts3/4 corruption. 
FossilOrigin-Name: 7f41d7006db4225cf9b3d197d3a76842778669ac079e76361214a8023c9976e6
 2023-10-21 18:12:07 +00:00
dan
0fa2bbf9e1 Fix another problem with fts3/4 auxiliary functions and NEAR expressions that consist entirely of deferred tokens. 
FossilOrigin-Name: a8c91c132f6157b7e3649f57a799984b1d7f8a18fd434515c875617d4195db29
 2023-01-25 13:42:55 +00:00
drh
6319a8003c Make the legacy FTS3 code more robust against integer overflow during 
memory allocation.

FossilOrigin-Name: 5517bc50988b5339c2fd071b29de1b5ca03037b0b635c3b112cf7108fab54d5f
 2022-09-27 01:53:05 +00:00
dan
2999068d2c Fix fts3 and fts4 so that they can still be used even if the other is dropped by a call to sqlite3_drop_modules(). 
FossilOrigin-Name: a0a8d6c9f0e91996003626e4f80dfac278e3c5bddd9ace5d442061e6c95e49dc
 2021-12-16 15:29:49 +00:00
dan
fb8e71c584 Fix a case in the fts3 offsets() function where a corrupt database record could lead to dereferencing an uninitialized pointer. 
FossilOrigin-Name: 7b7d31a6153b1505288eb3e849d0d9ef9e88e961c7b2f918ef5582fd77990f6d
 2021-10-20 11:40:34 +00:00
drh
11a9ad5669 Fix harmless static analyzer warnings in sessions, rtree, fts3 and fts5. 
Add the -DSQLITE_OMIT_AUXILIARY_SAFETY_CHECKS compile-time option to cause
ALWAYS() and NEVER() macros to be omitted from the build.

FossilOrigin-Name: 1c67f957fc77e37ce8f0d447c41ca975e8e79a35d332739c24a633649b5b0387
 2021-10-04 18:21:14 +00:00
dan
acbae3ba10 Add "#ifdef SQLITE_DEBUG" blocks around test variables sqlite3_fts5_may_be_corrupt and sqlite3_fts3_may_be_corrupt. 
FossilOrigin-Name: 004559544e661f9afa60306c4ac6842c29a8630b3663d62aa0cdc637f176932e
 2021-03-29 20:04:31 +00:00
dan
26c48d2aab Fix a problem building fts3 separately from the amalgamation. 
FossilOrigin-Name: 2a167cfbb2abd76e1b33116b671e58a767fc972dd79bee0004aad09a1dd2b487
 2020-08-10 19:30:21 +00:00
dan
fc7f31742d Fix an integer overflow in fts3 causing a usan error. 
FossilOrigin-Name: e256f85289a78e629acdf83e5bf1f8df2a0ffb3d559738eb9e49db6c228dc8c0
 2020-04-22 11:11:17 +00:00
dan
5be1b20aec Fix problems in fts3 with processing corrupt records and undefined integer overflows. 
FossilOrigin-Name: 3b873029ef1903f76442445343af3786cd59ee66b0e5e0e6b76a0b6b1b5eb88a
 2019-12-14 14:15:21 +00:00
dan
6637332988 Prevent direct and indirect recursive content= options in fts3/4 and fts5. 
FossilOrigin-Name: 2eb997327c2c369c10b7835be80112d77cb2a7cfa9fe4d564042984c21d94698
 2019-11-21 14:20:59 +00:00
drh
feb5dcf1a6 Better detection of corruption in the %_stat and %_docsize shadow tables 
of FTS3.

FossilOrigin-Name: 1e449687881f4d388e54a0e51bcabba41ab10cf7e596ff65e31e88a23c70d497
 2019-11-17 02:41:06 +00:00
drh
c72f2fb7fe More improvements to shadow table corruption detection in FTS3. 
FossilOrigin-Name: 51525f9c3235967bc00a090e84c70a6400698c897aa4742e817121c725b8c99d
 2019-11-16 23:47:40 +00:00
drh
94febb3a77 Improved detection of corrupt shadow tables in FTS3. Enable the debugging 
special-inserts for FTS3 for both SQLITE_DEBUG and SQLITE_TEST.

FossilOrigin-Name: 04b2873be5aedeb1c4325cf36c4b5d180f929a641caf1e3829c03778adb29c8e
 2019-11-16 21:40:49 +00:00
drh
0578084507 Fix the assert_fts3_nc() macro so that it works for test builds that omit 
SQLITE_DEBUG.

FossilOrigin-Name: b10df2b569e356551cf51ca179f307e4f8827413ee9feb886db1f1e0650e6c5f
 2019-01-25 18:17:03 +00:00
dan
c1f6618217 Add an assert_fts3_nc() macro to fts3 - for assert() conditions that are only true when it is guaranteed that the fts3 database is not corrupt. 
FossilOrigin-Name: 3498908cd7f3d0e35b70796537124e3da0bd99d48750ba51bcb9eba87e28ed4d
 2019-01-25 14:48:18 +00:00
dan
a783931794 Fix a buffer overread in fts3 caused by a corrupt record. 
FossilOrigin-Name: e54efd60c20bbfc5d70ddb9e1dd6ffb68cbbcfa57b4f07cb104767ef08c1a559
 2019-01-15 16:14:31 +00:00
dan
03db962ab5 Update test script fts3expr4.test so that it always creates fts3 tokenizers in 
the "en_US" locality.

FossilOrigin-Name: 576a8f69ae25883f752e58953624e9f7126db998bebaa1f07f7c2ec47aaecabe
 2018-04-27 18:05:01 +00:00
dan
18fdde21b8 Avoid preparing a SELECT statement each time an UPDATE or DELETE 
by docid is executed against an fts3 table.

FossilOrigin-Name: 9962c10a5c6672bd82b2bf640d878fcdac0b815a
 2017-02-08 19:10:47 +00:00
drh
4553f6ea91 Automatically disable FTS3 and FTS4 when building with SQLITE_OMIT_VIRTUALTABLE. 
FossilOrigin-Name: 0beb32d20d8dd698138cdb2de6b6056de176754d
 2016-02-11 22:41:04 +00:00
dan
d943b27596 Update fts3 to use the onepass strategy for delete operations. 
FossilOrigin-Name: fffab4f70f85eeb2acbb89534064a6e397c39384
 2015-09-28 15:23:29 +00:00
dan
3fb37f097c Fix an fts4 matchinf() problem triggered by deferred tokens that are part of phrases that are part of AND expressions. 
FossilOrigin-Name: f1e942a1dda496a509741e9cc2a17e8b4dac63a3
 2015-05-25 10:57:13 +00:00
dan
e60aedc564 More optimization for the 'y' and 'b' matchinfo directives. 
FossilOrigin-Name: 8c5b9fedfcee3ac22a222819dceb981ad94a9903
 2015-05-06 17:41:19 +00:00
dan
b2bd727e52 Optimizations for the matchinfo() function, particularly the 'y' flag. 
FossilOrigin-Name: dddd7e182943a1d3a9d32830e819a63f1a228d6d
 2015-05-05 19:37:07 +00:00
drh
abf582509d Use the sqlite3Fts3ErrMsg() function to set error message text in FTS3, 
and to ensure that error messages are not overwritten and thus leaked.

FossilOrigin-Name: 605347e087ec5eb817f3c94f4616abe26c7ab66b
 2015-05-01 14:07:30 +00:00
dan
0d01ec8de8 Remove an incorrect assert() statement from sqlite3Fts3Dequote(). 
FossilOrigin-Name: bd06eeb8d06237dc2d54d8a03e8bf525cb811c9e
 2015-04-15 09:16:39 +00:00
dan
18f6ff9eb7 Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function. 
FossilOrigin-Name: 245e8730451fbdc1c729beff7295c452df604009
 2015-01-27 18:43:02 +00:00
dan
2eaf03d72b Change fts3/4 so that the "unicode61" is included in builds by default. It may now be excluded by defining SQLITE_DISABLE_FTS3_UNICODE. 
FossilOrigin-Name: 0cc0230ae9cfc9760fd8ef2c25e82576b052dbeb
 2014-07-03 12:18:22 +00:00
dan
9e83e465eb Fix various problems to do with segment promotion. Add test file fts4growth2.test, containing tests to check that the FTS index does not grow indefinitely as the table is updated. 
FossilOrigin-Name: 21491a9bc686e63bec32f1a67103622f1160a2f9
 2014-05-14 15:58:47 +00:00
dan
6bb9889ef0 Experimental code to prevent FTS indexes from growing indefinitely as the table is updated. 
FossilOrigin-Name: b3b505a4dd0c679437a4272109f1188175088cd1
 2014-05-12 20:04:48 +00:00
dan
82d25da5b0 Changes to FTS to ensure that it does not access the database from within the xConnect method. 
FossilOrigin-Name: c67a52c356416cc200f51f9168d942e78db11d34
 2014-03-12 19:20:36 +00:00
dan
85963f5474 When possible, have FTS use 32-bit operations to read varints from the database. 
FossilOrigin-Name: aa7ba302ed13aedde89b5bcbe9119799c0da8a42
 2013-11-12 17:46:44 +00:00
dan
eb26672d0e Add a test to check that the new multi-token phrase optimization is actually helping. 
FossilOrigin-Name: bc3a2ed5fb2402805928b0002457eebf06f87c47
 2013-10-02 08:04:27 +00:00
dan
19384380f3 Have fts4 full-text queries consider "docid<?" and similar constraints. 
FossilOrigin-Name: 6622424a3a149edd35ba2ba0881aa41b4536417b
 2013-09-30 11:42:19 +00:00
mistachkin
049d487e2e Modify several extensions to use the new exported function naming. Fix some shared library compilation issues. 
FossilOrigin-Name: f2ab8747825ab5131ffab174aa0ffe5e474f6811
 2013-07-04 23:53:56 +00:00
drh
adbae616bd Remove an unused function declaration from the FTS3 source code. 
FossilOrigin-Name: 096ae1d8f9a08f92daedece6b0615f4d22b05023
 2013-06-21 18:36:44 +00:00
dan
8def92bac3 Add the "notindexed" option to fts4. 
FossilOrigin-Name: 8ff2b8f5948ccddce70102e6d68d464c66e4f7ca
 2013-06-21 17:30:47 +00:00
dan
5f0b33c2cb Add the SQLITE_FTS3_MAX_EXPR_DEPTH compile time option. 
FossilOrigin-Name: 24fc9d4438a5615dd20af5419456166df83a72ea
 2013-06-11 14:22:11 +00:00
dan
e8dee7d1ed Modify the fts3tokenize table implementation so that it does not use the SQL function fts3_tokenizer. The user may have installed an authorizer callback that prohibits this. 
FossilOrigin-Name: 0ba67b64de258883e4c43db09e131bb67083855e
 2013-05-07 12:16:48 +00:00
dan
3a01b600d3 Improve the error message issued when an FTS query exceeds the maximum allowable tree depth. 
FossilOrigin-Name: f480b1fe6012f36c59cd0525efdc6df74143ccd0
 2013-04-29 18:07:37 +00:00
drh
4d648c777b Update MSCV makefile for the fts3tokenize virtual table. Clear some 
harmless compiler warnings in the fts3tokenize code.

FossilOrigin-Name: 2b3fed09eb89f1e9d6a5f7487ef526b46eb51490
 2013-04-22 17:07:56 +00:00
dan
4dc3d73d69 Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option. 
FossilOrigin-Name: e799222f3b8246e65657a758437914ece7069ba9
 2012-08-20 17:24:48 +00:00
dan
2c897e3e5f Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61. 
FossilOrigin-Name: eccd6b6580637084495b80e9232262188ba0cf8d
 2012-06-06 19:51:27 +00:00
dan
754d3adf7c Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0". 
FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0fec0ccd
 2012-06-06 19:30:38 +00:00
dan
3d403c71a8 Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators. 
FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385
 2012-05-25 17:50:19 +00:00
dan
5cfed52dd1 Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38]. 
FossilOrigin-Name: 4d3e1673b309a3d6a6cac01d67dfd79a8007f991
 2012-05-10 17:43:14 +00:00
dan
311ec02587 Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables. 
FossilOrigin-Name: 5c447e226afca0d46b9ed994dea26a16a9ae168c
 2012-03-27 15:00:06 +00:00
dan
cbcd9f5357 Add an experimental integrity-check function to FTS. 
FossilOrigin-Name: 40fc8804743dfb005991e9c5ef7b0ebcb3c2e731
 2012-03-26 10:36:55 +00:00
drh
6c2e7e19fa Enable fts3 tables to use incremental merge by automatically creating the 
%_stat table when it is needed.

FossilOrigin-Name: cc051fc0b2d89603b27b94cf2afdbda417ee9d94
 2012-03-24 17:29:05 +00:00