Commit Graph

101 Commits

Author SHA1 Message Date
dan
18fdde21b8 Avoid preparing a SELECT statement each time an UPDATE or DELETE
by docid is executed against an fts3 table.

FossilOrigin-Name: 9962c10a5c6672bd82b2bf640d878fcdac0b815a
2017-02-08 19:10:47 +00:00
drh
4553f6ea91 Automatically disable FTS3 and FTS4 when building with SQLITE_OMIT_VIRTUALTABLE.
FossilOrigin-Name: 0beb32d20d8dd698138cdb2de6b6056de176754d
2016-02-11 22:41:04 +00:00
dan
d943b27596 Update fts3 to use the onepass strategy for delete operations.
FossilOrigin-Name: fffab4f70f85eeb2acbb89534064a6e397c39384
2015-09-28 15:23:29 +00:00
dan
3fb37f097c Fix an fts4 matchinf() problem triggered by deferred tokens that are part of phrases that are part of AND expressions.
FossilOrigin-Name: f1e942a1dda496a509741e9cc2a17e8b4dac63a3
2015-05-25 10:57:13 +00:00
dan
e60aedc564 More optimization for the 'y' and 'b' matchinfo directives.
FossilOrigin-Name: 8c5b9fedfcee3ac22a222819dceb981ad94a9903
2015-05-06 17:41:19 +00:00
dan
b2bd727e52 Optimizations for the matchinfo() function, particularly the 'y' flag.
FossilOrigin-Name: dddd7e182943a1d3a9d32830e819a63f1a228d6d
2015-05-05 19:37:07 +00:00
drh
abf582509d Use the sqlite3Fts3ErrMsg() function to set error message text in FTS3,
and to ensure that error messages are not overwritten and thus leaked.

FossilOrigin-Name: 605347e087ec5eb817f3c94f4616abe26c7ab66b
2015-05-01 14:07:30 +00:00
dan
0d01ec8de8 Remove an incorrect assert() statement from sqlite3Fts3Dequote().
FossilOrigin-Name: bd06eeb8d06237dc2d54d8a03e8bf525cb811c9e
2015-04-15 09:16:39 +00:00
dan
18f6ff9eb7 Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function.
FossilOrigin-Name: 245e8730451fbdc1c729beff7295c452df604009
2015-01-27 18:43:02 +00:00
dan
2eaf03d72b Change fts3/4 so that the "unicode61" is included in builds by default. It may now be excluded by defining SQLITE_DISABLE_FTS3_UNICODE.
FossilOrigin-Name: 0cc0230ae9cfc9760fd8ef2c25e82576b052dbeb
2014-07-03 12:18:22 +00:00
dan
9e83e465eb Fix various problems to do with segment promotion. Add test file fts4growth2.test, containing tests to check that the FTS index does not grow indefinitely as the table is updated.
FossilOrigin-Name: 21491a9bc686e63bec32f1a67103622f1160a2f9
2014-05-14 15:58:47 +00:00
dan
6bb9889ef0 Experimental code to prevent FTS indexes from growing indefinitely as the table is updated.
FossilOrigin-Name: b3b505a4dd0c679437a4272109f1188175088cd1
2014-05-12 20:04:48 +00:00
dan
82d25da5b0 Changes to FTS to ensure that it does not access the database from within the xConnect method.
FossilOrigin-Name: c67a52c356416cc200f51f9168d942e78db11d34
2014-03-12 19:20:36 +00:00
dan
85963f5474 When possible, have FTS use 32-bit operations to read varints from the database.
FossilOrigin-Name: aa7ba302ed13aedde89b5bcbe9119799c0da8a42
2013-11-12 17:46:44 +00:00
dan
eb26672d0e Add a test to check that the new multi-token phrase optimization is actually helping.
FossilOrigin-Name: bc3a2ed5fb2402805928b0002457eebf06f87c47
2013-10-02 08:04:27 +00:00
dan
19384380f3 Have fts4 full-text queries consider "docid<?" and similar constraints.
FossilOrigin-Name: 6622424a3a149edd35ba2ba0881aa41b4536417b
2013-09-30 11:42:19 +00:00
mistachkin
049d487e2e Modify several extensions to use the new exported function naming. Fix some shared library compilation issues.
FossilOrigin-Name: f2ab8747825ab5131ffab174aa0ffe5e474f6811
2013-07-04 23:53:56 +00:00
drh
adbae616bd Remove an unused function declaration from the FTS3 source code.
FossilOrigin-Name: 096ae1d8f9a08f92daedece6b0615f4d22b05023
2013-06-21 18:36:44 +00:00
dan
8def92bac3 Add the "notindexed" option to fts4.
FossilOrigin-Name: 8ff2b8f5948ccddce70102e6d68d464c66e4f7ca
2013-06-21 17:30:47 +00:00
dan
5f0b33c2cb Add the SQLITE_FTS3_MAX_EXPR_DEPTH compile time option.
FossilOrigin-Name: 24fc9d4438a5615dd20af5419456166df83a72ea
2013-06-11 14:22:11 +00:00
dan
e8dee7d1ed Modify the fts3tokenize table implementation so that it does not use the SQL function fts3_tokenizer. The user may have installed an authorizer callback that prohibits this.
FossilOrigin-Name: 0ba67b64de258883e4c43db09e131bb67083855e
2013-05-07 12:16:48 +00:00
dan
3a01b600d3 Improve the error message issued when an FTS query exceeds the maximum allowable tree depth.
FossilOrigin-Name: f480b1fe6012f36c59cd0525efdc6df74143ccd0
2013-04-29 18:07:37 +00:00
drh
4d648c777b Update MSCV makefile for the fts3tokenize virtual table. Clear some
harmless compiler warnings in the fts3tokenize code.

FossilOrigin-Name: 2b3fed09eb89f1e9d6a5f7487ef526b46eb51490
2013-04-22 17:07:56 +00:00
dan
4dc3d73d69 Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option.
FossilOrigin-Name: e799222f3b8246e65657a758437914ece7069ba9
2012-08-20 17:24:48 +00:00
dan
2c897e3e5f Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61.
FossilOrigin-Name: eccd6b6580637084495b80e9232262188ba0cf8d
2012-06-06 19:51:27 +00:00
dan
754d3adf7c Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".
FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0fec0ccd
2012-06-06 19:30:38 +00:00
dan
3d403c71a8 Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.
FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385
2012-05-25 17:50:19 +00:00
dan
5cfed52dd1 Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38].
FossilOrigin-Name: 4d3e1673b309a3d6a6cac01d67dfd79a8007f991
2012-05-10 17:43:14 +00:00
dan
311ec02587 Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables.
FossilOrigin-Name: 5c447e226afca0d46b9ed994dea26a16a9ae168c
2012-03-27 15:00:06 +00:00
dan
cbcd9f5357 Add an experimental integrity-check function to FTS.
FossilOrigin-Name: 40fc8804743dfb005991e9c5ef7b0ebcb3c2e731
2012-03-26 10:36:55 +00:00
drh
6c2e7e19fa Enable fts3 tables to use incremental merge by automatically creating the
%_stat table when it is needed.

FossilOrigin-Name: cc051fc0b2d89603b27b94cf2afdbda417ee9d94
2012-03-24 17:29:05 +00:00
dan
45eddd6855 Remove the Fts3Table.mxLevel variable.
FossilOrigin-Name: 67a0cffc9d07be7f09dad9d019a18160711295cd
2012-03-24 16:11:21 +00:00
dan
4b1e4dabc3 Modify the way the number of leaves written and the maximum relative level are calculated in the auto-incr-merge code.
FossilOrigin-Name: 0d841c957c6ec4afecb49504177c6279e09d7012
2012-03-24 14:45:19 +00:00
drh
4ef9dfff97 An attempt at automatic incremental merging for FTS4.
FossilOrigin-Name: ed69434cd89084f4b57bd2cc4f5cc558904af565
2012-03-24 02:20:43 +00:00
dan
d59de73e27 Fix a spurious SQLITE_CONSTRAINT error that may be returned by an incr-merge operation.
FossilOrigin-Name: ed7c17ea165f6348506bd23ebc58c427bb65d697
2012-03-23 18:26:11 +00:00
dan
d1ab097d45 Fix various incorrect and missing comments and other style issues in and around the FTS incremental merge code.
FossilOrigin-Name: 7aabb62c8ccbd2b8d216e25226f06e5820dec38a
2012-03-17 16:56:57 +00:00
dan
5730ef599c Add tests for incremental merge code.
FossilOrigin-Name: 570473729d6561d81e6e5f8884fd18487008636e
2012-03-14 20:01:52 +00:00
dan
593c9824bc Add the 'merge=?,?' command to fts4. This still needs some work.
FossilOrigin-Name: 741b8f897750eac3c9774fd65de7e40bb89781b1
2012-03-08 18:39:03 +00:00
dan
996073b3ae Add the xLanguageid method to sqlite3_fts3_tokenizer versions 1 and greater.
FossilOrigin-Name: f8e9c445dd358c40e5a7bf3756b9f291909dbea7
2012-03-03 18:46:41 +00:00
dan
8fb662b0be Fix the FTS 'optimize' command on multi-lingual databases.
FossilOrigin-Name: 65fa693729a336e4d905ce72e6b9ccf4faa772bd
2012-03-02 12:26:19 +00:00
dan
7fcb214b93 Add the "languageid=" option to fts4. This code is still largely untested and alsmost certainly buggy.
FossilOrigin-Name: bea257f70f10dd1111d79cabd1e1462dc651704d
2012-03-01 19:44:20 +00:00
dan
585db0e23a Fix an FTS problem triggered by querying for an N character prefix using an N+1 character prefix index after rows have been deleted from the FTS table.
FossilOrigin-Name: c05c3fd20d93f430140d762ead23bacd337ffb4d
2012-01-25 16:29:45 +00:00
dan
e459b07601 Merge the fts4-content branch with the trunk.
FossilOrigin-Name: 8a4077057ddeb08e8edc5f20a75abaaba7a278ba
2011-10-19 16:20:40 +00:00
dan
9743948386 Have FTS3 ignore ^ prefixes. The ^ syntax is only supported on FTS4 tables.
FossilOrigin-Name: df36ac948179f37b432a88701b6c79299e073ce8
2011-10-19 15:52:48 +00:00
dan
d17f70a624 Change the way ^ tokens work in FTS so that the filtering is done as part of reading the FTS index instead of waiting until an entire doclist has been retrieved and then filtering it.
FossilOrigin-Name: 9b58c59eb4efaa38ce50a3ce1b52f9ba578c71d6
2011-10-19 11:57:13 +00:00
dan
3f1ea8d114 If a token within an FTS query is prefixed with a '^' character, it must be the first token in a column of data to match.
FossilOrigin-Name: 63ac33c860eb32ce96699f06bf83121cec2ffaca
2011-10-18 19:39:41 +00:00
dan
8dab211632 Handle updating the only row of an FTS table correctly. Fix for [9fd058691].
FossilOrigin-Name: 7e24645be2fe0ffe092212e7bcfa5b4500305811
2011-10-13 17:16:45 +00:00
dan
cc5b81464a Add tests to check that modifying the schema of an FTS content table does not cause a crash in the FTS module. Also disable the deferred token optimization for content=xxx FTS tables.
FossilOrigin-Name: be86c7061b68f403730bf63ea1f7dc0d9ceb0a3b
2011-10-04 16:37:35 +00:00
dan
d8e454e19a Add experimental 'content' option to FTS4.
FossilOrigin-Name: 1d27ea741f61c624e18bdc6a3b1c2d8574a64ddc
2011-10-04 11:22:59 +00:00
drh
ec661058dd Fix harmless compiler warnings on unix.
FossilOrigin-Name: 90b1aea17400bbda5ebc8ae4eb4e12127519e42e
2011-07-09 16:17:18 +00:00