e8dee7d1ed
Modify the fts3tokenize table implementation so that it does not use the SQL function fts3_tokenizer. The user may have installed an authorizer callback that prohibits this.
...
FossilOrigin-Name: 0ba67b64de258883e4c43db09e131bb67083855e
2013-05-07 12:16:48 +00:00
3a01b600d3
Improve the error message issued when an FTS query exceeds the maximum allowable tree depth.
...
FossilOrigin-Name: f480b1fe6012f36c59cd0525efdc6df74143ccd0
2013-04-29 18:07:37 +00:00
4d648c777b
Update MSCV makefile for the fts3tokenize virtual table. Clear some
...
harmless compiler warnings in the fts3tokenize code.
FossilOrigin-Name: 2b3fed09eb89f1e9d6a5f7487ef526b46eb51490
2013-04-22 17:07:56 +00:00
4dc3d73d69
Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option.
...
FossilOrigin-Name: e799222f3b8246e65657a758437914ece7069ba9
2012-08-20 17:24:48 +00:00
2c897e3e5f
Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61.
...
FossilOrigin-Name: eccd6b6580637084495b80e9232262188ba0cf8d
2012-06-06 19:51:27 +00:00
754d3adf7c
Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".
...
FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0fec0ccd
2012-06-06 19:30:38 +00:00
3d403c71a8
Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.
...
FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385
2012-05-25 17:50:19 +00:00
5cfed52dd1
Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38].
...
FossilOrigin-Name: 4d3e1673b309a3d6a6cac01d67dfd79a8007f991
2012-05-10 17:43:14 +00:00
311ec02587
Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables.
...
FossilOrigin-Name: 5c447e226afca0d46b9ed994dea26a16a9ae168c
2012-03-27 15:00:06 +00:00
cbcd9f5357
Add an experimental integrity-check function to FTS.
...
FossilOrigin-Name: 40fc8804743dfb005991e9c5ef7b0ebcb3c2e731
2012-03-26 10:36:55 +00:00
6c2e7e19fa
Enable fts3 tables to use incremental merge by automatically creating the
...
%_stat table when it is needed.
FossilOrigin-Name: cc051fc0b2d89603b27b94cf2afdbda417ee9d94
2012-03-24 17:29:05 +00:00
45eddd6855
Remove the Fts3Table.mxLevel variable.
...
FossilOrigin-Name: 67a0cffc9d07be7f09dad9d019a18160711295cd
2012-03-24 16:11:21 +00:00
4b1e4dabc3
Modify the way the number of leaves written and the maximum relative level are calculated in the auto-incr-merge code.
...
FossilOrigin-Name: 0d841c957c6ec4afecb49504177c6279e09d7012
2012-03-24 14:45:19 +00:00
4ef9dfff97
An attempt at automatic incremental merging for FTS4.
...
FossilOrigin-Name: ed69434cd89084f4b57bd2cc4f5cc558904af565
2012-03-24 02:20:43 +00:00
d59de73e27
Fix a spurious SQLITE_CONSTRAINT error that may be returned by an incr-merge operation.
...
FossilOrigin-Name: ed7c17ea165f6348506bd23ebc58c427bb65d697
2012-03-23 18:26:11 +00:00
d1ab097d45
Fix various incorrect and missing comments and other style issues in and around the FTS incremental merge code.
...
FossilOrigin-Name: 7aabb62c8ccbd2b8d216e25226f06e5820dec38a
2012-03-17 16:56:57 +00:00
5730ef599c
Add tests for incremental merge code.
...
FossilOrigin-Name: 570473729d6561d81e6e5f8884fd18487008636e
2012-03-14 20:01:52 +00:00
593c9824bc
Add the 'merge=?,?' command to fts4. This still needs some work.
...
FossilOrigin-Name: 741b8f897750eac3c9774fd65de7e40bb89781b1
2012-03-08 18:39:03 +00:00
996073b3ae
Add the xLanguageid method to sqlite3_fts3_tokenizer versions 1 and greater.
...
FossilOrigin-Name: f8e9c445dd358c40e5a7bf3756b9f291909dbea7
2012-03-03 18:46:41 +00:00
8fb662b0be
Fix the FTS 'optimize' command on multi-lingual databases.
...
FossilOrigin-Name: 65fa693729a336e4d905ce72e6b9ccf4faa772bd
2012-03-02 12:26:19 +00:00
7fcb214b93
Add the "languageid=" option to fts4. This code is still largely untested and alsmost certainly buggy.
...
FossilOrigin-Name: bea257f70f10dd1111d79cabd1e1462dc651704d
2012-03-01 19:44:20 +00:00
585db0e23a
Fix an FTS problem triggered by querying for an N character prefix using an N+1 character prefix index after rows have been deleted from the FTS table.
...
FossilOrigin-Name: c05c3fd20d93f430140d762ead23bacd337ffb4d
2012-01-25 16:29:45 +00:00
e459b07601
Merge the fts4-content branch with the trunk.
...
FossilOrigin-Name: 8a4077057ddeb08e8edc5f20a75abaaba7a278ba
2011-10-19 16:20:40 +00:00
9743948386
Have FTS3 ignore ^ prefixes. The ^ syntax is only supported on FTS4 tables.
...
FossilOrigin-Name: df36ac948179f37b432a88701b6c79299e073ce8
2011-10-19 15:52:48 +00:00
d17f70a624
Change the way ^ tokens work in FTS so that the filtering is done as part of reading the FTS index instead of waiting until an entire doclist has been retrieved and then filtering it.
...
FossilOrigin-Name: 9b58c59eb4efaa38ce50a3ce1b52f9ba578c71d6
2011-10-19 11:57:13 +00:00
3f1ea8d114
If a token within an FTS query is prefixed with a '^' character, it must be the first token in a column of data to match.
...
FossilOrigin-Name: 63ac33c860eb32ce96699f06bf83121cec2ffaca
2011-10-18 19:39:41 +00:00
8dab211632
Handle updating the only row of an FTS table correctly. Fix for [9fd058691].
...
FossilOrigin-Name: 7e24645be2fe0ffe092212e7bcfa5b4500305811
2011-10-13 17:16:45 +00:00
cc5b81464a
Add tests to check that modifying the schema of an FTS content table does not cause a crash in the FTS module. Also disable the deferred token optimization for content=xxx FTS tables.
...
FossilOrigin-Name: be86c7061b68f403730bf63ea1f7dc0d9ceb0a3b
2011-10-04 16:37:35 +00:00
d8e454e19a
Add experimental 'content' option to FTS4.
...
FossilOrigin-Name: 1d27ea741f61c624e18bdc6a3b1c2d8574a64ddc
2011-10-04 11:22:59 +00:00
ec661058dd
Fix harmless compiler warnings on unix.
...
FossilOrigin-Name: 90b1aea17400bbda5ebc8ae4eb4e12127519e42e
2011-07-09 16:17:18 +00:00
f443e317f7
Remove accidental global variable "doclist" from fts3Int.h.
...
FossilOrigin-Name: f244b2f3959e86fac555b91438cfe90528ab1648
2011-07-07 07:37:53 +00:00
b8ae1afba1
Merge latest trunk changes with fts3-changes branch.
...
FossilOrigin-Name: 226686475c28cdc87e86d2d3f10014436a9a2b8f
2011-06-28 09:51:22 +00:00
0ea2517c4e
Changes to allow FTS to be compiled as a loadable module again.
...
FossilOrigin-Name: 29e69f389cb0078e125ba5814a68b3fe8cf634c6
2011-06-28 07:15:43 +00:00
17fb042578
Fix some of the code issues (missing comments etc.) in the new FTS code.
...
FossilOrigin-Name: 8230d83120e0f4d217fde56e22c6f05aa5adee09
2011-06-23 17:09:51 +00:00
7fd3392981
More compiler warning fixes.
...
FossilOrigin-Name: ed2dda9329ca42e9c0be1986c78b091051e7598f
2011-06-20 19:00:30 +00:00
c5f21892a8
Avoid loading doclists for infrequent terms that are part of phrases twice.
...
FossilOrigin-Name: 8f939723f742329cedba8930f71dff42004f3d0d
2011-06-17 17:37:31 +00:00
10fc4d8c45
Add a missing declaration to fts3Int.h.
...
FossilOrigin-Name: 3bfd4466f50711eb71d1a13231025ff4e1e76246
2011-06-17 16:04:39 +00:00
d4d21fea8e
Fix a problem with NEAR queries executed inside a transaction that writes the FTS table.
...
FossilOrigin-Name: 051c756c367837908f6691c0a36108e088c94f99
2011-06-16 16:06:05 +00:00
6b96771b82
Changes to #ifdefs so that the build goes correctly if the only FTS macro
...
defined is SQLITE_ENABLE_FTS4.
FossilOrigin-Name: a0b43a320e6491de7070966ed7c3ec55fd660a85
2011-06-16 00:54:45 +00:00
abf2545ed9
Fix various issues to do with deferred tokens, NEAR expressions and matchinfo().
...
FossilOrigin-Name: 3972a787df5ec253b99b148385655e7b68d851fa
2011-06-08 18:39:07 +00:00
b46ee91729
Allow the "order=DESC" and "order=ASC" parameters in FTS4 "CREATE VIRTUAL TABLE" statements. Tables created with "order=DESC" store all doclists in descending order, which allows optimizations normally applied to "ORDER BY docid ASC" queries to be used with "ORDER BY docid DESC" queries instead.
...
FossilOrigin-Name: f6a0193f5a32603eb48bddc6297042dbd2ffe96e
2011-06-04 20:04:35 +00:00
126ba6c0ac
FTS changes: Remove unreachable code. Fix bugs. When processing a large doclist incrementally, read from disk incrementally too.
...
FossilOrigin-Name: a4c7e2820824e82580730c36f85aede2efa66754
2011-06-03 18:00:19 +00:00
e414854800
Changes to improve performance and support LIMIT clauses on fts3 tables. This branch is unstable for now.
...
FossilOrigin-Name: 28149a7882a1e9dfe4a75ec5b91d176ebe6284e9
2011-06-02 19:57:24 +00:00
382874fc5c
Minor changes made while planning a larger change.
...
FossilOrigin-Name: 84097a4c759b1d65890af885f137d3cb16eef584
2011-05-28 15:57:40 +00:00
9d1f874ab3
Change fts4 so that the prefix= parameter is passes a comma-separated list of integers. For each integer N, a separate index of all prefixes of length N bytes is created.
...
FossilOrigin-Name: be59bf49402d2e2f4b95fb6668849f3745cb7bf2
2011-05-25 18:34:53 +00:00
09643ab745
If the fts4 option prefix=1 is specified, have the fts4 module maintain an index of prefixes as well as terms.
...
FossilOrigin-Name: b5bdc639898ee22eebedeb560810e94e74de8aa4
2011-05-24 18:49:45 +00:00
e485522462
Do not invoke the xRollbackTo or xRelease methods of a virtual table without
...
having first invoked an appropriate xSavepoint method. Add assert() statements
to FTS3/4 to verify that this is happening in all cases.
FossilOrigin-Name: 651ef24249d8c22c4f13e4c0bb98a60099cfd23a
2011-05-24 15:36:01 +00:00
807d0fcaba
Avoid including fts3_term.c in the amalgamation, as it contains test code only.
...
FossilOrigin-Name: f392b7ae0266b4c694836583cb91b10f2b6c0752
2011-05-17 14:41:36 +00:00
0f599faa29
Optimize "ORDER BY rowid/docid DESC/ASC" clauses on FTS tables.
...
FossilOrigin-Name: 13395121e3d17ab6581dc5f6736ea324321a374c
2011-05-04 12:52:59 +00:00
20d5f354e9
Optimize handling of equality and range constraints on the "term" column of an fts4aux table.
...
FossilOrigin-Name: 386701ded2bcee5309ec9f285d94e6eb1eade193
2011-02-02 17:30:43 +00:00
dan