mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
23,300 Commits 1,487 Branches 166 Tags 416 MiB
dee0359ddb
Commit Graph

627 Commits

Author SHA1 Message Date
mistachkin
48864df97d Many spelling fixes in comments. No changes to code. 
FossilOrigin-Name: 6f6e2d50941e444ebc83604daddcc034137a05b7
 2013-03-21 21:20:32 +00:00
drh
7b6be6f7bf Fix an uninitialized variable detected by valgrind. 
FossilOrigin-Name: ffd1e51490286abfaea68fd4b4b4cb967d87b04b
 2012-12-04 00:53:08 +00:00
dan
bba44d0d1b Fix a problem causing the "number-of-documents" field maintained by FTS4 to be set incorrectly. 
FossilOrigin-Name: e38fb02d5ea5daa6992df4dfbbeec92bf7b525f6
 2012-11-27 15:56:38 +00:00
dan
e62c0694de Fix a bug in fts3 causing it to return SQLITE_NOMEM when NEAR, OR and some auxiliary functions were used together. Also a segfault in vtab.c that could follow an OOM condition. 
FossilOrigin-Name: 33f4f9817e8b3cb97ce02a4c49bd586332da37b2
 2012-10-25 19:25:39 +00:00
drh
8749c1831a Make sure substructure elements have proper alignment in the ICU tokenizers 
of FTS2 and FTS3.

FossilOrigin-Name: aaa2d9b0db74d8452d9294de17cff786ab4ec7c8
 2012-10-19 02:10:53 +00:00
dan
1276cd0ba8 Add a comment to fts3_snippet.c. 
FossilOrigin-Name: b5f5ba3103e6fcf117fb69d5379fa70f691a652d
 2012-10-17 20:28:52 +00:00
dan
6c42aa4a0e Initialize variables used as outputs of custom tokenizer method calls in fts4. 
FossilOrigin-Name: 514505a84bcc677518d60a42896598f85ed78265
 2012-10-17 20:15:10 +00:00
mistachkin
d5578433ff Fix all known instances of 'repeated the' style typos in comments. No changes to code. 
FossilOrigin-Name: 9b19b847533f944f289d93dcdba29c0d67bf251c
 2012-08-25 10:01:29 +00:00
dan
4dc3d73d69 Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option. 
FossilOrigin-Name: e799222f3b8246e65657a758437914ece7069ba9
 2012-08-20 17:24:48 +00:00
mistachkin
fe7b2bc9ff Merge several compiler compatibility fixes to trunk. 
FossilOrigin-Name: d3d491a5461e21d84c6425977b80deedc7fa8a3a
 2012-06-21 14:05:15 +00:00
dan
2ae26b759d Fix a problem with identifying white-space characters outside of the ascii range in the ICU tokenizer. 
FossilOrigin-Name: 892b74116a3b23268895b96433d18ef00c1433d8
 2012-06-18 20:52:32 +00:00
mistachkin
c8bde37d19 Fix a few compilation issues that can occur with certain compilers (e.g. GCC 2.95.3, MSVC). 
FossilOrigin-Name: f970a3de61fe2ebaf1778c4a3383cfdc52299162
 2012-06-18 08:00:56 +00:00
drh
1c30debd41 Fix an obscure memory leak in FTS3 that can come about when two memory 
allocations are immediately adjacent to one another.

FossilOrigin-Name: 025227be5495f950c466dfabac140cba69e498be
 2012-06-08 14:01:53 +00:00
dan
25cdf46ae4 Add the "tokenchars=" and "separators=" options, for customizing the set of characters considered to be token separators, to the unicode61 tokenizer. 
FossilOrigin-Name: e56fb462aa1f11bb23303ae0dc62815c21e26a52
 2012-06-07 15:53:48 +00:00
dan
2c897e3e5f Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61. 
FossilOrigin-Name: eccd6b6580637084495b80e9232262188ba0cf8d
 2012-06-06 19:51:27 +00:00
dan
754d3adf7c Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0". 
FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0fec0ccd
 2012-06-06 19:30:38 +00:00
drh
a9cfaba95a Omit the fts3 unicode character class routines from the build if fts3/4 
is disabled.

FossilOrigin-Name: c00bb5d4601efc15933f222349e96a043b610a19
 2012-05-28 12:22:00 +00:00
dan
7946c53009 If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer. 
FossilOrigin-Name: e71495a817b479bc23c5403d99255e3f098eb054
 2012-05-26 18:28:14 +00:00
dan
501c74d3e1 Change the format of the tables used by sqlite3FtsUnicodeTolower() to make them a little smaller. 
FossilOrigin-Name: b89d3834f6690073fca0fc22c18afa1fb280ea7d
 2012-05-26 17:57:02 +00:00
dan
cf9f6f1375 Add fault-injection tests that use the unicode61 tokenizer. Fix a problem revealed by the same. 
FossilOrigin-Name: ed28c48a3dd7e766e60db0d96ef5460bf9913e6b
 2012-05-26 15:44:08 +00:00
dan
ab322bd21e Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1. 
FossilOrigin-Name: 8f3e60aa2253f21bcee5d03982cfdd7f16c00060
 2012-05-26 14:54:50 +00:00
dan
1c7016c9a5 Add special fast paths to sqlite3FtsUnicodeTolower() and Isalnum() for codepoints in the ASCII range. 
FossilOrigin-Name: cf7b25d47687635a04f4347d45f135c686b9d758
 2012-05-25 19:50:12 +00:00
dan
80ed5a56a5 Fix comments in generated file fts3_unicode2.c. 
FossilOrigin-Name: 3dc567ef4702d9a63d78d11ff705cb7f7359f7a6
 2012-05-25 18:48:48 +00:00
dan
3d403c71a8 Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators. 
FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385
 2012-05-25 17:50:19 +00:00
dan
5cfed52dd1 Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38]. 
FossilOrigin-Name: 4d3e1673b309a3d6a6cac01d67dfd79a8007f991
 2012-05-10 17:43:14 +00:00
drh
83cc139231 Fix harmless compiler warnings on x64 MSVC, mostly in test code, but also in 
tclsqlite.c and in the FTS4 module.

FossilOrigin-Name: 3281972eaa46cb57fd9f0387063f47430dc0a3b4
 2012-04-19 18:04:28 +00:00
drh
85e7243acf Fix harmless static-analysis warnings, mosting having to do with memory 
leaks in the command-line shell.  Add a clang analysis of the command-line
shell to the "warnings-clang.sh" script.  Other minor cleanups to the
command-line shell code.

FossilOrigin-Name: 93a0f452a7023898ad3d62ee81b39a80477c332f
 2012-04-11 11:38:53 +00:00
drh
3b06a2a056 Add #ifdefs to allow a test build to succeed even if SQLITE_ENABLE_FTS3 is 
not defined.

FossilOrigin-Name: fb121980e48af368353431fd04924e414b65c852
 2012-04-02 17:18:23 +00:00
dan
a25a42cfe7 Change the ICU tokenizer so that it does not attempt to call strlen(NULL). 
FossilOrigin-Name: 04298f1ac42c40cb2a48092b415acf96a08954b7
 2012-03-31 11:58:23 +00:00
drh
7da5fcb0b7 Fix MSVC compiler warnings in test code. 
FossilOrigin-Name: cb7a850439c9a4a7887650d6b81d95ab8025de5b
 2012-03-30 14:59:43 +00:00
drh
4e245a4c35 Fix compiler warnings on GCC and MSVC and fix a C89-ism that 
broke the build for MSVC.

FossilOrigin-Name: b451c0f97f0abe78ebe6c62ff489ec1ad8a1f767
 2012-03-30 00:00:36 +00:00
drh
cdb86dc437 Add output of PRAGMAs auto_vacuum and encoding to the "schema" command of the 
fts3view utility program.

FossilOrigin-Name: e31076319363a46905836880765bae3bf204ed19
 2012-03-27 18:00:05 +00:00
dan
311ec02587 Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables. 
FossilOrigin-Name: 5c447e226afca0d46b9ed994dea26a16a9ae168c
 2012-03-27 15:00:06 +00:00
drh
a1a9f0a0d8 In the fts3view utility, label the blank segments used to mark the end of a 
segment sequence for a level/idx as "null".  Improve the alignment of root
segment names.

FossilOrigin-Name: 04aea0245e4183fef3664609f5a6353b65d71a85
 2012-03-27 14:54:44 +00:00
drh
1ecb59a1e5 Enhance the fts3view tool with the big-segment command and fix a bug in the 
display of doclists.

FossilOrigin-Name: e9436d8038e5a0d1ba992a77d1064d4a55595f57
 2012-03-27 13:51:31 +00:00
dan
5da0aa1603 Allow multiple incremental merges to proceed concurrently. This is required to prevent a large crisis-merge from occuring while an even larger incremental-merge is underway. 
FossilOrigin-Name: 7ed9d2f24a650b424b97dfc19b8042c4cf09c82c
 2012-03-27 11:48:02 +00:00
drh
40aab4c56d Minor correct errors in the file format description for FTS3/4 contained in 
the fts3.c header comment.

FossilOrigin-Name: fb8aacdd8fbdc946cb271cc589f76b806387937d
 2012-03-27 00:38:33 +00:00
drh
280d5c33cd Enhance fts3view to show decodes of segments and doclists. 
FossilOrigin-Name: 6d09de231b68dd9520d99c65d133f26e90eb784f
 2012-03-27 00:34:04 +00:00
drh
36f6b891e6 Add the fts3view utility program. 
FossilOrigin-Name: f936c8ea16d21345fd1622272dc7e9850acb2493
 2012-03-26 21:57:53 +00:00
drh
cbf9450404 Fix FTS3 so that it works even without SQLITE_DEBUG. 
FossilOrigin-Name: a18c103121529c2e3c6a8ada16a4c40d14080670
 2012-03-26 14:36:42 +00:00
dan
6f4df8ac83 Modify the FTS integrity-check so that the checksums do not depend on the results of signed integer overflow, which is undefined in C. 
FossilOrigin-Name: f907fc3fb387e74bb66babcbf050748cb253a6fa
 2012-03-26 10:57:31 +00:00
dan
84bce14f77 Add a comment to explain how the FTS integrity-check works. 
FossilOrigin-Name: 64e8a116f39434a3b7347f01a47f88eef3276742
 2012-03-26 10:47:03 +00:00
dan
cbcd9f5357 Add an experimental integrity-check function to FTS. 
FossilOrigin-Name: 40fc8804743dfb005991e9c5ef7b0ebcb3c2e731
 2012-03-26 10:36:55 +00:00
drh
6c2e7e19fa Enable fts3 tables to use incremental merge by automatically creating the 
%_stat table when it is needed.

FossilOrigin-Name: cc051fc0b2d89603b27b94cf2afdbda417ee9d94
 2012-03-24 17:29:05 +00:00
dan
61fa09dbf2 Fix a failing assert() in the FTS3_LOG_MERGES related code. 
FossilOrigin-Name: 4220d52cb3426f1680b72d57ecc9f4ade029357d
 2012-03-24 17:09:11 +00:00
dan
0dfcdeb4f0 Add a comment to fts3SyncMethod() to justify the nMinMerge=64 constant. 
FossilOrigin-Name: af55ca5fc6778cb6d1a79a17dfa2d4e567ea1ccc
 2012-03-24 16:43:55 +00:00
drh
790964ebc4 Fix a bug in debugging code enabled when FTS3_LOG_MERGES is defined. 
FossilOrigin-Name: 2e06babf49de844d0e878d68114dbcc5ad4d6e54
 2012-03-24 16:18:08 +00:00
dan
45eddd6855 Remove the Fts3Table.mxLevel variable. 
FossilOrigin-Name: 67a0cffc9d07be7f09dad9d019a18160711295cd
 2012-03-24 16:11:21 +00:00
dan
4b1e4dabc3 Modify the way the number of leaves written and the maximum relative level are calculated in the auto-incr-merge code. 
FossilOrigin-Name: 0d841c957c6ec4afecb49504177c6279e09d7012
 2012-03-24 14:45:19 +00:00
drh
4ef9dfff97 An attempt at automatic incremental merging for FTS4. 
FossilOrigin-Name: ed69434cd89084f4b57bd2cc4f5cc558904af565
 2012-03-24 02:20:43 +00:00
dan
d59de73e27 Fix a spurious SQLITE_CONSTRAINT error that may be returned by an incr-merge operation. 
FossilOrigin-Name: ed7c17ea165f6348506bd23ebc58c427bb65d697
 2012-03-23 18:26:11 +00:00
dan
3501a91677 Remove an incorrect assert() statement. Fix a const-related warning. 
FossilOrigin-Name: 96ed47493b3d46344fd2105642f31690aee06674
 2012-03-23 14:38:49 +00:00
dan
e81eaec754 Following an incr-merge operation that does not completely consume its input segments, store context in the rowid==1 row of the %_stat table that allows the next incr-merge to pick up where the previous left off. 
FossilOrigin-Name: ab0a4f44fb67e9f0cb82297b80e728ca58cdb0fb
 2012-03-22 16:48:12 +00:00
drh
e239233982 Merge trunk changes into the fts4-incr-merge branch. 
FossilOrigin-Name: f61d5fb0281381228eb1a12a233bacaeb26b12a3
 2012-03-20 17:04:17 +00:00
drh
d13a3bc772 Fix one more compiler warning missed by the previous check-in. 
FossilOrigin-Name: bc03d99a78e90c02b69037e5f5f81537b5a3ac60
 2012-03-19 14:57:49 +00:00
drh
01ea399a84 Suppress harmless compiler warnings on windows in FTS4 and RTREE. 
FossilOrigin-Name: 4fd68647c8d4b120e04d054617cef31001f44c6f
 2012-03-19 14:51:19 +00:00
drh
bbe0520329 Avoid a compiler warning (an incorrect compiler warning, at that) in vs2010. 
FossilOrigin-Name: 7dd97f12cd268cac1241f0f3e8de94bb629b97c7
 2012-03-19 14:28:43 +00:00
dan
d1ab097d45 Fix various incorrect and missing comments and other style issues in and around the FTS incremental merge code. 
FossilOrigin-Name: 7aabb62c8ccbd2b8d216e25226f06e5820dec38a
 2012-03-17 16:56:57 +00:00
drh
c8d7b24e93 Cherrypick the [3475092cff] fix for 32-bit overflow with large language-ids 
into trunk.

FossilOrigin-Name: 2755edc7f1186ba8955f2c3550a90f40a55ddae2
 2012-03-16 16:52:54 +00:00
dan
81b35dc66c Fix some integer overflow problems that can occur when using large langauge id values. 
FossilOrigin-Name: 3475092cff862080a020d386076d739f0d22c9b2
 2012-03-16 15:54:19 +00:00
dan
e931b7f45f Add a comment to the FTS getAbsoluteLevel() function. No actual code changes. 
FossilOrigin-Name: 7e0f861beda4d74d0c3c9fb4abb3ddb5fee346bd
 2012-03-16 14:54:07 +00:00
drh
7a48480d07 Fix harmless compiler warnings. 
FossilOrigin-Name: 74eadeec34c4b19cf5f8b7f648db3b7ad601a00e
 2012-03-16 00:28:11 +00:00
dan
4ab6f2b9c1 Modify incremental merge code to merge nMin segments at a time. 
FossilOrigin-Name: cd34bc1af4ba608ea3b52bab55bcfe0086711900
 2012-03-15 17:45:50 +00:00
dan
5730ef599c Add tests for incremental merge code. 
FossilOrigin-Name: 570473729d6561d81e6e5f8884fd18487008636e
 2012-03-14 20:01:52 +00:00
dan
a2af0aecdb Avoid allocating a large object on the stack in the incremental merge code. Use sqlite3_malloc() instead. 
FossilOrigin-Name: 36ae510de45be44efd34cff242d02fb21b7419ac
 2012-03-14 12:17:40 +00:00
dan
8af1fc721e Fix another bug in the incremental merge code. 
FossilOrigin-Name: f97b12e0955c4c29f9c31a186d72d87f7407782e
 2012-03-14 11:51:31 +00:00
dan
604f58a686 Fix some bugs in the incremental merge code. Some remain. 
FossilOrigin-Name: bff21683705a61b8b8672e0b44c287d1dc7c32a9
 2012-03-13 19:56:34 +00:00
drh
ab4f6385de Minor commenting and stylistic changes only. 
FossilOrigin-Name: a1747086c5e0c152fcf4bd9fa80a61b6f03f4a94
 2012-03-09 12:52:43 +00:00
dan
593c9824bc Add the 'merge=?,?' command to fts4. This still needs some work. 
FossilOrigin-Name: 741b8f897750eac3c9774fd65de7e40bb89781b1
 2012-03-08 18:39:03 +00:00
dan
fd6ddf91b0 Fix a problem compiling the test code in fts3_test.c when SQLITE_ENABLE_FTS3 is not defined. 
FossilOrigin-Name: b00ccda307caae597c143ab0586f90acb77f79cf
 2012-03-05 16:24:26 +00:00
dan
3a1abde7c8 Merge the fts4-languageid branch with the trunk. 
FossilOrigin-Name: 99a9073b5e411ce94f38ce49608baaa15de8b850
 2012-03-05 15:33:32 +00:00
dan
996073b3ae Add the xLanguageid method to sqlite3_fts3_tokenizer versions 1 and greater. 
FossilOrigin-Name: f8e9c445dd358c40e5a7bf3756b9f291909dbea7
 2012-03-03 18:46:41 +00:00
drh
9db1d56f02 Remove an obsolete comment from FTS3. No code changes. 
FossilOrigin-Name: 74d224b14467d8a6915d3cf372f45e0d28708fa2
 2012-03-02 20:34:57 +00:00
dan
7395599031 Fix problems with combining content= and languageid= in a single fts4 table. 
FossilOrigin-Name: 22491e7bc38aee43819b888e04241cb6a6ef73a3
 2012-03-02 19:53:02 +00:00
dan
8fb662b0be Fix the FTS 'optimize' command on multi-lingual databases. 
FossilOrigin-Name: 65fa693729a336e4d905ce72e6b9ccf4faa772bd
 2012-03-02 12:26:19 +00:00
dan
e3ddd1ac42 Fix a bug in merging FTS language tables for languages other than language 0. 
FossilOrigin-Name: d281cb8984c911a4c0cce2ec299e1351d8e580e4
 2012-03-02 11:48:50 +00:00
dan
7fcb214b93 Add the "languageid=" option to fts4. This code is still largely untested and alsmost certainly buggy. 
FossilOrigin-Name: bea257f70f10dd1111d79cabd1e1462dc651704d
 2012-03-01 19:44:20 +00:00
drh
548af8b69e Fix a compiler warning in FTS4. 
FossilOrigin-Name: b4cad1ce78e6bb967c472bec5c562a24b4dfebaf
 2012-01-31 23:34:21 +00:00
dan
585db0e23a Fix an FTS problem triggered by querying for an N character prefix using an N+1 character prefix index after rows have been deleted from the FTS table. 
FossilOrigin-Name: c05c3fd20d93f430140d762ead23bacd337ffb4d
 2012-01-25 16:29:45 +00:00
drh
45d29309a7 Fix typos in commands. Combine the ExpandBlob and expandBlob macros into one. 
FossilOrigin-Name: fc9179e154751b51ddef429600d30a89d4a098bd
 2012-01-08 22:18:33 +00:00
drh
64803bd2fa Fix minor issues with FTS and RTREE discovered by coverity. 
FossilOrigin-Name: 1c27d842163e27c39bbe9409f50657b9de9ade6e
 2011-12-22 17:31:17 +00:00
dan
0c8cda6e0d Fix a problem in fts3_write.c causing stack memory to be referenced after it is out of scope. 
FossilOrigin-Name: f9c4a7c8f4e5821b47c1393d6272e32416d8886c
 2011-12-22 15:30:46 +00:00
dan
689e38f7cf Add a missing va_end() macro to fts3.c. 
FossilOrigin-Name: e6f825748a1d510c8f5529c79656bb5db9216231
 2011-11-04 10:37:21 +00:00
dan
042f88e5b4 Add new file ext/fts3/README.content, describing the experimental FTS4 content option. 
FossilOrigin-Name: 13a9d085e1a5654a97b8d26bae7182ca6c0c237b
 2011-10-31 11:36:29 +00:00
drh
6ed18b7110 Fix harmless compiler warnings in FTS4. 
FossilOrigin-Name: 1af4a25631c4077a9bfc64e15eb92fbf5146c89c
 2011-10-19 18:21:47 +00:00
dan
e459b07601 Merge the fts4-content branch with the trunk. 
FossilOrigin-Name: 8a4077057ddeb08e8edc5f20a75abaaba7a278ba
 2011-10-19 16:20:40 +00:00
dan
9743948386 Have FTS3 ignore ^ prefixes. The ^ syntax is only supported on FTS4 tables. 
FossilOrigin-Name: df36ac948179f37b432a88701b6c79299e073ce8
 2011-10-19 15:52:48 +00:00
dan
d17f70a624 Change the way ^ tokens work in FTS so that the filtering is done as part of reading the FTS index instead of waiting until an entire doclist has been retrieved and then filtering it. 
FossilOrigin-Name: 9b58c59eb4efaa38ce50a3ce1b52f9ba578c71d6
 2011-10-19 11:57:13 +00:00
dan
50a7544d6f Fix a problem in FTS to do with ^ tokens and the snippet() function. 
FossilOrigin-Name: 2c03b24f4cc6f2c28c9d5b9984320d41b8486c32
 2011-10-19 09:40:49 +00:00
dan
3f1ea8d114 If a token within an FTS query is prefixed with a '^' character, it must be the first token in a column of data to match. 
FossilOrigin-Name: 63ac33c860eb32ce96699f06bf83121cec2ffaca
 2011-10-18 19:39:41 +00:00
dan
a986d33fd3 Cherrypick patch [3126754c72] from the trunk into the content= branch. 
FossilOrigin-Name: f9b5b217088a6aeb25eba184ab92d1a842a680a6
 2011-10-18 12:49:59 +00:00
dan
42495cd46f Fix a floating-point exception that can occur when an FTS4 query contains a large number of tokens connected by AND or NEAR operators. 
FossilOrigin-Name: 3126754c72351c724be29d75a194bfc3e7b67205
 2011-10-18 12:44:04 +00:00
drh
30ddce6ff4 Added the tool/warnings-clang.sh script. Changes so that there are no 
warnings with either gcc or clang even including FTS4 and RTREE and
both with and without SQLITE_THREADSAFE=0.

FossilOrigin-Name: 39408702a989f907261c298bf0947f3e68bd10fe
 2011-10-15 00:16:30 +00:00
drh
086e4913b5 Fix a few minor and harmless clang warnings in FTS3 and RTREE. 
FossilOrigin-Name: b3324f6cc27c3bfb32b12eacace2fc731c2dd644
 2011-10-14 22:57:03 +00:00
dan
8dab211632 Handle updating the only row of an FTS table correctly. Fix for [9fd058691]. 
FossilOrigin-Name: 7e24645be2fe0ffe092212e7bcfa5b4500305811
 2011-10-13 17:16:45 +00:00
dan
deb9473250 Change FTS4 so that if both the content=xxx option and column names are specified, the virtual table assumes that the named columns correspond to columns of table xxx. 
FossilOrigin-Name: 289ee43179369fce2fde50870d72c445e184e896
 2011-10-05 15:11:30 +00:00
dan
8361b189c9 Fix a problem with IO error handling in the rebuild-index code. 
FossilOrigin-Name: c6ba81fcad32192674bd510e607f787adc1f7038
 2011-10-05 06:07:00 +00:00
dan
49fc336438 Improve test coverage of fts3.c. 
FossilOrigin-Name: 0f439944ab49a5691615bc170fdcf652055573df
 2011-10-04 19:41:50 +00:00
dan
cc5b81464a Add tests to check that modifying the schema of an FTS content table does not cause a crash in the FTS module. Also disable the deferred token optimization for content=xxx FTS tables. 
FossilOrigin-Name: be86c7061b68f403730bf63ea1f7dc0d9ceb0a3b
 2011-10-04 16:37:35 +00:00
dan
d8e454e19a Add experimental 'content' option to FTS4. 
FossilOrigin-Name: 1d27ea741f61c624e18bdc6a3b1c2d8574a64ddc
 2011-10-04 11:22:59 +00:00
dan
f9556a1a3d Allocate the correct size for the output buffer in fts3DoclistOrMerge(). Fix for [56be976859]. 
FossilOrigin-Name: 07788c0f7f3740c1c280f6ce4dc68401c30bae6e
 2011-09-13 19:08:43 +00:00
drh
ec661058dd Fix harmless compiler warnings on unix. 
FossilOrigin-Name: 90b1aea17400bbda5ebc8ae4eb4e12127519e42e
 2011-07-09 16:17:18 +00:00
dan
f443e317f7 Remove accidental global variable "doclist" from fts3Int.h. 
FossilOrigin-Name: f244b2f3959e86fac555b91438cfe90528ab1648
 2011-07-07 07:37:53 +00:00
dan
da7b45615d Add a fix and tests for the FTS deferred token logic. 
FossilOrigin-Name: 91daea7d2ec41f014fb30c6371aae447cc07f287
 2011-06-28 11:58:09 +00:00
dan
b8ae1afba1 Merge latest trunk changes with fts3-changes branch. 
FossilOrigin-Name: 226686475c28cdc87e86d2d3f10014436a9a2b8f
 2011-06-28 09:51:22 +00:00
dan
0ea2517c4e Changes to allow FTS to be compiled as a loadable module again. 
FossilOrigin-Name: 29e69f389cb0078e125ba5814a68b3fe8cf634c6
 2011-06-28 07:15:43 +00:00
dan
5b217c308f Changes to improve the selection of deferred tokens within phrases. 
FossilOrigin-Name: 2c4bbd90e2fca593c186bf412b608aff8c9f9061
 2011-06-27 11:15:53 +00:00
dan
17fb042578 Fix some of the code issues (missing comments etc.) in the new FTS code. 
FossilOrigin-Name: 8230d83120e0f4d217fde56e22c6f05aa5adee09
 2011-06-23 17:09:51 +00:00
drh
45de8d5f0f If "PRAGMA page_size" commands are not authorized, the FTS module will assume 
a page size of 1024.

FossilOrigin-Name: ba39382ef546b4c4ae15ed90ef0b6c87762bffef
 2011-06-23 00:59:41 +00:00
shaneh
2e2f36b4ed Fix a couple of compiler issues on Windows due to warning clean up. 
FossilOrigin-Name: 81e6f151d384fa1f1474175b39ad6dfa64f8da0d
 2011-06-20 20:50:49 +00:00
drh
7fd3392981 More compiler warning fixes. 
FossilOrigin-Name: ed2dda9329ca42e9c0be1986c78b091051e7598f
 2011-06-20 19:00:30 +00:00
drh
051eb38a26 Fix various harmless compiler warnings in FTS3 and RTREE. 
FossilOrigin-Name: fe62179efdea40887e23a16f898060b879013824
 2011-06-20 17:24:29 +00:00
dan
6764a70086 Fix a problem where FTS test code was not being included in the testfixture build when SQLITE_ENABLE_FTS4 was defined. 
FossilOrigin-Name: e539d08a4dda073dc67dc39a104f89374d3bd8e9
 2011-06-20 11:15:06 +00:00
dan
c5f21892a8 Avoid loading doclists for infrequent terms that are part of phrases twice. 
FossilOrigin-Name: 8f939723f742329cedba8930f71dff42004f3d0d
 2011-06-17 17:37:31 +00:00
dan
10fc4d8c45 Add a missing declaration to fts3Int.h. 
FossilOrigin-Name: 3bfd4466f50711eb71d1a13231025ff4e1e76246
 2011-06-17 16:04:39 +00:00
dan
d4d21fea8e Fix a problem with NEAR queries executed inside a transaction that writes the FTS table. 
FossilOrigin-Name: 051c756c367837908f6691c0a36108e088c94f99
 2011-06-16 16:06:05 +00:00
drh
6b96771b82 Changes to #ifdefs so that the build goes correctly if the only FTS macro 
defined is SQLITE_ENABLE_FTS4.

FossilOrigin-Name: a0b43a320e6491de7070966ed7c3ec55fd660a85
 2011-06-16 00:54:45 +00:00
drh
bf4bc8c254 Fix so that the TCL test harness works even if SQLITE_ENABLE_FTS is omitted. 
FossilOrigin-Name: 63ebcb52a1909aca80f2fef3e982f8fb5929b73b
 2011-06-15 13:02:21 +00:00
drh
1aa4f3e529 Fix a couple of compiler warnings. 
FossilOrigin-Name: 3899f3b95ed50773a72d907b39b451fefce69c9e
 2011-06-15 12:43:36 +00:00
dan
376d1e5c4f Add an interface to better test incremental loading of doclists by FTS4. Also some tests for this and term prefix queries. 
FossilOrigin-Name: 7a3813138d1a5c5d99f4756c79998831d779774f
 2011-06-15 08:30:29 +00:00
dan
2cf1a1de4b Fix another bug caused by NEAR/matchinfo/order=DESC interaction. 
FossilOrigin-Name: 04907fbadeb743c95cc9f3529e63ef388684799f
 2011-06-14 09:00:27 +00:00
dan
c72f9d98cb Merge recent trunk changes into fts3-prefix-search branch. 
FossilOrigin-Name: 135ce30f62ebd6a1b239c18dbbd9c926ea507db4
 2011-06-14 07:22:30 +00:00
dan
806cbfc214 Remove unused parameters from internal fts3 function. 
FossilOrigin-Name: 06de3f2cbc27cdfd9f83218c9ea576f74f60d07b
 2011-06-14 07:14:43 +00:00
dan
2f77934385 Add tests for deferred tokens to fts3auto.test. Fix a problem with OR queries and deferred tokens. 
FossilOrigin-Name: b9fb69e55bb05a8819688ee63987f2a05c33d59b
 2011-06-13 17:00:12 +00:00
dan
99ebad90e3 Fix a bug exposed by combining matchinfo(), NEAR and "ORDER BY rowid DESC". 
FossilOrigin-Name: 5f6b87f420f21749aa7c72e020c50aca74890086
 2011-06-13 09:11:01 +00:00
dan
786b068967 Fix problems to do with using both OR and NEAR operators in a single expression. 
FossilOrigin-Name: 4e8dd19eef04777d800977faf1859a405e396f30
 2011-06-09 10:48:02 +00:00
dan
abf2545ed9 Fix various issues to do with deferred tokens, NEAR expressions and matchinfo(). 
FossilOrigin-Name: 3972a787df5ec253b99b148385655e7b68d851fa
 2011-06-08 18:39:07 +00:00
dan
3eabcf5f46 Have NEAR queries use incremental merging. Fix issues surrounding the deferred token optimization. 
FossilOrigin-Name: 9d10a6846b12a9cc8fd4fdc3affd931a27218b5a
 2011-06-07 18:35:45 +00:00
drh
ebcdc73f53 Merge the latest trunk changes into the fts3-prefix-search branch. 
FossilOrigin-Name: 567dd84359218245d4e6887547e2a48881f2c8e0
 2011-06-06 18:14:25 +00:00
dan
5289b01524 Modify fts3rnd.test to run tests for both "ORDER BY docid ASC" and "ORDER BY docid DESC" with both order=ASC and order=DESC FTS tables. Fixes for some bugs found. 
FossilOrigin-Name: 89f2f482e077241ac29a58eadf44a72a9c01f98c
 2011-06-06 14:51:50 +00:00
dan
9aab071780 Clean up the code for processing FTS4 options a bit. 
FossilOrigin-Name: 0425138a2365d23b07d88fda2b1f458f112f389d
 2011-06-06 06:55:38 +00:00
dan
76e04431ef Remove some unreachable code. 
FossilOrigin-Name: 650e1a79eda5a2134a1fbd305ab1f205a57c0892
 2011-06-04 20:13:24 +00:00
dan
b46ee91729 Allow the "order=DESC" and "order=ASC" parameters in FTS4 "CREATE VIRTUAL TABLE" statements. Tables created with "order=DESC" store all doclists in descending order, which allows optimizations normally applied to "ORDER BY docid ASC" queries to be used with "ORDER BY docid DESC" queries instead. 
FossilOrigin-Name: f6a0193f5a32603eb48bddc6297042dbd2ffe96e
 2011-06-04 20:04:35 +00:00
dan
126ba6c0ac FTS changes: Remove unreachable code. Fix bugs. When processing a large doclist incrementally, read from disk incrementally too. 
FossilOrigin-Name: a4c7e2820824e82580730c36f85aede2efa66754
 2011-06-03 18:00:19 +00:00
dan
e414854800 Changes to improve performance and support LIMIT clauses on fts3 tables. This branch is unstable for now. 
FossilOrigin-Name: 28149a7882a1e9dfe4a75ec5b91d176ebe6284e9
 2011-06-02 19:57:24 +00:00
dan
382874fc5c Minor changes made while planning a larger change. 
FossilOrigin-Name: 84097a4c759b1d65890af885f137d3cb16eef584
 2011-05-28 15:57:40 +00:00
drh
7ee4fdd7ba Fix some minor and harmless compiler warnings. 
FossilOrigin-Name: a4755e7088c3cc7c5ea191ce37f3950472f523ec
 2011-05-25 23:18:02 +00:00
dan
98b08e71f2 If a prefix index of size N is not present, use a prefix index of size N+1 along with the terms index for queries for prefixes of length N. 
FossilOrigin-Name: cc83991caae7c7d647432d5711b6cd80228c3002
 2011-05-25 19:17:32 +00:00
dan
73ae6e5e76 Merge trunk changes into experimental fts3-prefix-search branch. 
FossilOrigin-Name: f0f0a03db214b68a37069f64c27ae8520220c900
 2011-05-25 18:47:26 +00:00
dan
9d1f874ab3 Change fts4 so that the prefix= parameter is passes a comma-separated list of integers. For each integer N, a separate index of all prefixes of length N bytes is created. 
FossilOrigin-Name: be59bf49402d2e2f4b95fb6668849f3745cb7bf2
 2011-05-25 18:34:53 +00:00
drh
346506f97b Changes to savepoint in virtual tables for simpler and more consistent 
operation.

FossilOrigin-Name: 92f26a8b8f18832cb1f8ba7dd8e5b020e71a5883
 2011-05-25 01:16:42 +00:00
dan
09643ab745 If the fts4 option prefix=1 is specified, have the fts4 module maintain an index of prefixes as well as terms. 
FossilOrigin-Name: b5bdc639898ee22eebedeb560810e94e74de8aa4
 2011-05-24 18:49:45 +00:00
drh
e485522462 Do not invoke the xRollbackTo or xRelease methods of a virtual table without 
having first invoked an appropriate xSavepoint method.  Add assert() statements
to FTS3/4 to verify that this is happening in all cases.

FossilOrigin-Name: 651ef24249d8c22c4f13e4c0bb98a60099cfd23a
 2011-05-24 15:36:01 +00:00
dan
133d7dab17 Add extended return code SQLITE_CORRUPT_VTAB. Returned when the tcontents of the sqlite tables used internally by a virtual table module are invalid or inconsistent. 
FossilOrigin-Name: 8844e8bfb87314fb40ecb92705e8fff88f72bb38
 2011-05-17 15:56:16 +00:00
dan
807d0fcaba Avoid including fts3_term.c in the amalgamation, as it contains test code only. 
FossilOrigin-Name: f392b7ae0266b4c694836583cb91b10f2b6c0752
 2011-05-17 14:41:36 +00:00
drh
e578b5967c Increase the iVersion for sqlite3_module needed for the xSavepoint, 
xRelease, and xRollbackTo methods from 1 to 2.  This is so that it will
conform to published documentation.

FossilOrigin-Name: 0f4954e9793332573d4ffa0c1e199180f4cb990a
 2011-05-06 00:19:57 +00:00
dan
7b0ced74c0 Merge vtab-conflict branch with trunk. 
FossilOrigin-Name: 8f9666af5f9459cbd0305a86281d745ea6163acf
 2011-05-04 17:23:19 +00:00
dan
62d6c7e32b Fix a couple of compiler warnings in the FTS code. 
FossilOrigin-Name: 1a113359705d307efa1833b1dfa5542b045dae43
 2011-05-04 16:30:05 +00:00
dan
e1c8e13bf6 Fix a performance problem in queries that use "ORDER BY rowid DESC" and one or more FTS auxiliary functions. 
FossilOrigin-Name: 95e09b20e9aad28f829c8950f3632debe473070a
 2011-05-04 15:41:18 +00:00
dan
0f599faa29 Optimize "ORDER BY rowid/docid DESC/ASC" clauses on FTS tables. 
FossilOrigin-Name: 13395121e3d17ab6581dc5f6736ea324321a374c
 2011-05-04 12:52:59 +00:00
dan
3480a01dad Add documentation for the newly introduced sqlite3_vtab_config() and on_conflict() API functions. Test that encountering an SQLITE_MISMATCH in fts3 does not corrupt the full text index. 
FossilOrigin-Name: abdd70ae0424ccadb7edaf16e970c78b5257d23c
 2011-04-27 16:02:46 +00:00
dan
a311b80392 Extra tests for fts3. And fixes for conflict-handling related problems in fts3. 
FossilOrigin-Name: fb4a355871d9482ccb28b6ba03b842b3cc87b696
 2011-04-26 19:21:34 +00:00
dan
b061d058cb Add support for on conflict clauses to fts3/fts4. 
FossilOrigin-Name: 6d2633a6d0a9bb88fb1a6adac0827dc51df2d4d2
 2011-04-25 18:49:57 +00:00
dan
355c53b6ef Avoid passing NULL to the xOpen method of an FTS3/4 tokenizer. 
FossilOrigin-Name: 0dd09fc034c127718366d3a3183e367d2f9fd82d
 2011-04-19 06:43:22 +00:00
drh
fcd71b6010 Suppress many harmless compiler warnings, mostly signed/unsigned comparisons 
within asserts or unused parameters in extensions.

FossilOrigin-Name: 3eeb0ff78d04891b5fd1a3d99a9fb8cfbed77a81
 2011-04-05 22:08:24 +00:00
dan
0370493760 Fix a bug in fts4 to do with matchinfo and deferred tokens. 
FossilOrigin-Name: 30d42dc66f19ef5cc5b27d6273eadf56619a3ce8
 2011-03-23 17:10:43 +00:00
dan
2fe88b7ca4 Ensure fts4aux can handle a table name in single or double quotes as a constructor argument. 
FossilOrigin-Name: 929d62e496bb36a3ee0e19ec4609329d79aaeddc
 2011-02-05 15:47:12 +00:00
dan
bb7e77c758 Add test cases for fts4aux. Fix a bug affecting fts3 tables with multiple columns. 
FossilOrigin-Name: dc511e60a65232a7087e12ff40b63506cf37a634
 2011-02-05 14:37:57 +00:00
dan
4c600ac389 Change fts4aux to (additionally) report on term frequency in individual columns of an fts table. 
FossilOrigin-Name: 3996f92a9aca9ac2628be003eca83c8f954c71de
 2011-02-04 18:56:25 +00:00
dan
767f9a8f9e Extra tests for fts4 compress/uncompress hooks. Fix some minor problems with the same. 
FossilOrigin-Name: 80225abe79b0a7723b922ec129954438af95855a
 2011-02-03 12:48:17 +00:00
dan
16690ed167 Extra tests for the fts4aux module. 
FossilOrigin-Name: cfc475690d85ea7e3547424289d9837f46ab7ef4
 2011-02-03 10:56:00 +00:00
dan
20d5f354e9 Optimize handling of equality and range constraints on the "term" column of an fts4aux table. 
FossilOrigin-Name: 386701ded2bcee5309ec9f285d94e6eb1eade193
 2011-02-02 17:30:43 +00:00
dan
ee85a51092 Add missing file fts3_aux.c. 
FossilOrigin-Name: 0147d9739f318a02721d3ae0e7be339a19231f60
 2011-02-02 04:26:46 +00:00
dan
0bfda98155 Fix a case in fts4 where a corrupt %_stat table could lead to a crash. 
FossilOrigin-Name: 4ade96ce974244fc34bb97713d3cba10e3d33056
 2011-02-01 17:55:48 +00:00
dan
a240fd01b6 Add virtual table module "fts4aux", used to inspect the full-text index of an fts4 table directly. Also add the "compress" and "uncompress" fts4 options. 
FossilOrigin-Name: b010ddcc52889160af2183a33c5f483bb0ae91b9
 2011-02-01 16:34:32 +00:00
dan
a8ab60c648 Fix a couple of crashes in fts3 that can occur if the database contents are inconsistent. 
FossilOrigin-Name: 811e12cddfb3246c6cf3d5085bd9b72b12e05550
 2011-01-13 11:20:04 +00:00
dan
86596e552e Fix a segfault that can occur in matchinfo if an fts4 table contains mostly zero-length documents. Specifically, if the table contains more rows than it does bytes of text. 
FossilOrigin-Name: fe9047668eaaf76e7aa1ef1f32dec7c7c4226e45
 2011-01-13 10:58:26 +00:00
dan
b9a93d70b8 Reduce the number of calls to sqlite3_realloc() made by fts3 when querying for position information of a term prefix. 
FossilOrigin-Name: 7088d9450f403f12f67eed558e368573101245d6
 2010-12-24 15:49:27 +00:00
dan
d3842765ae Fix a performance regression in matchinfo(). 
FossilOrigin-Name: fa37d8eb7cd8049689068af2a80fe2cbb4a603f1
 2010-12-10 17:06:48 +00:00
dan
014153f092 Fix a memory leak in fts3 that can occur in an OOM condition. 
FossilOrigin-Name: 507027b70f71689f77dee3e849ebc66a1c4c3166
 2010-12-04 18:20:18 +00:00
dan
4d8d27813c Fix a bug causing fts3 to incorrectly return SQLITE_CORRUPT. 
FossilOrigin-Name: 099195b14829f375055345b8322905ccd073d442
 2010-12-02 17:39:26 +00:00
shaneh
556f6bbc92 Fix some warnings under MSVC in fts3 module. 
FossilOrigin-Name: c7771c0b22f2b45a47070cf84b9ecf1011e40404
 2010-12-01 15:36:00 +00:00
dan
0cb3a1eca4 Fix compiler warnings discovered while building SQLite on [http://www.devio.us/]. 
FossilOrigin-Name: 5602ec95aa2a74d0624bb6c7d53e7a0d35536253
 2010-11-29 17:55:18 +00:00
drh
6ea28d6d84 Fix various compiler warnings. 
FossilOrigin-Name: c412f61229b6ab1ac90b932afd56f7c5e3ba1cfe
 2010-11-26 16:49:59 +00:00
dan
102b7de0e9 Fix an uninitialized variable in fts3.c. 
FossilOrigin-Name: 3c3d076b42da36cd5413749ec022d0349325edfa
 2010-11-26 16:31:43 +00:00
dan
72de061fb1 Add a couple of comments to fts3_snippet.c. 
FossilOrigin-Name: ad8df6c5ee0055cf1161a3b700d69dbe07d969bc
 2010-11-26 15:13:31 +00:00
dan
c2f16cb12b Add coverage tests (and associated fixes) for new matchinfo() code. 
FossilOrigin-Name: 70495ceccc793d608930f59e330777f287ba1ede
 2010-11-25 17:49:28 +00:00
dan
cf56554f42 Fix bugs in fts3 function matchinfo() when used with deferred tokens. 
FossilOrigin-Name: ddc2b7ec2618b010c981ecfa05b3e53a9fac686f
 2010-11-25 10:33:54 +00:00
dan
f9376540fa Add code for the matchinfo 'longest common substring' feature. 
FossilOrigin-Name: 71011a4f9baf09ec6935ad591145252bf3c286ed
 2010-11-24 19:26:18 +00:00
dan
d6e12384bd Fix crashes that can occur when queries are run on an FTS4 table containing zero rows. 
FossilOrigin-Name: ed61fd20adef44d0f6b2345e0205b25f3641a15e
 2010-11-24 15:02:22 +00:00
dan
ad3acbbfb3 Remove some unused code from fts3. Add tests to fts3matchinfo.test. 
FossilOrigin-Name: ae40b34cf7c24c9601bdfb5cbe5b20f05a376ea8
 2010-11-24 11:51:56 +00:00
dan
1e66e40eb9 Experimental changes to fts3 function matchinfo(). 
FossilOrigin-Name: 9cf0f2b76bc68c168e3fa861b7235f384db21d38
 2010-11-23 19:16:47 +00:00
drh
3b691b5466 Restrict the scope of the fts3ExprCost() subroutine inside of FTS3. 
FossilOrigin-Name: 76681870a4705179f78976e7fda638a6fea0b9f9
 2010-11-18 13:52:34 +00:00
drh
437261d077 Fix compiler warnings. 
FossilOrigin-Name: 6c4f1d5c24522d1f541a2b96b229ad0374f99c19
 2010-11-18 13:47:50 +00:00
dan
af4c214e1f Changes to allow FTS4 tables to be created without the underlying %_docsize table (in order to save space). 
FossilOrigin-Name: 31989b18f53d97eddfb39660ef04fbf9463583e0
 2010-11-02 17:41:52 +00:00
dan
cfddb09e3b Test cases and minor changes to make fts3 more robust in the face of a corrupt database. 
FossilOrigin-Name: b770290561f5450e4d985ca0050ef5eb01657c80
 2010-10-30 15:21:13 +00:00
dan
797f3ee87c Add extra test cases and changes to fts3 to avoid crashing on a corrupt database. 
FossilOrigin-Name: 252f0e457d3e33404df87d3e6c44ede61b78319c
 2010-10-29 18:45:10 +00:00
dan
8aaa252d8c Fix a buffer overread in fts3 that can occur if the database is corrupt. 
FossilOrigin-Name: 84194c4195d7144ff7f9cedcdc74fdd908f3bfcd
 2010-10-27 16:52:27 +00:00
dan
8c30711208 In fts4, store the total number of bytes of for all records in the table in the %_stat table. 
FossilOrigin-Name: 941647d121ac60e2eabc998cfe79b157fb918d7e
 2010-10-27 10:55:54 +00:00
dan
c7c91257d5 More coverage tests for fts3.c. 
FossilOrigin-Name: 7a2f2864002d4e9a487e2779ae05513880ecb4d1
 2010-10-26 07:14:25 +00:00
dan
e25ac099f1 Further improvements to coverage of fts3.c. Fixes for bugs revealed by the same. 
FossilOrigin-Name: 918b609290127f54326c638d82837d117398eade
 2010-10-25 19:01:25 +00:00
dan
ddf80eb8ce Test coverage improvements for fts3.c. 
FossilOrigin-Name: a8b1d99899678b72c2a487909eabed321593d55f
 2010-10-25 12:47:43 +00:00
dan
378d0ab97b Fixes for the matchinfo() function related to FTS4 common token handling. 
FossilOrigin-Name: deb80eac9112d21835dfd3cee08ed8f09d975bf7
 2010-10-23 19:07:30 +00:00
dan
789cb8f5cf Add missing comments and fix compiler warnings in new FTS3/4 code. Other minor fixes too. 
FossilOrigin-Name: 1c9c70fec3c88319f7b2efe5316694a6ce0ab1a5
 2010-10-22 16:44:39 +00:00
dan
ca3b8f9c86 Merge trunk changes into experimental branch. 
FossilOrigin-Name: fd1e5cade04961c2f5438a1dfcc2e15eafb4503f
 2010-10-21 15:49:47 +00:00
dan
4e76cc3650 Updates to FTS4 to improve performance and make more accurate cost estimates for prefix terms. 
FossilOrigin-Name: d0a450ce78e99f55c862f26f9332786660007a0a
 2010-10-20 18:56:04 +00:00
dan
4f7c5e684a Experimental changes to fts4 to try to selectively avoid loading very large doclists. 
FossilOrigin-Name: 5ae0ba447a561e3b6637b52f9b83a9fc683d2572
 2010-10-19 14:07:59 +00:00
dan
8786f7f589 Have all FTS3 queries obtain a read or write table-lock at the shared-cache level before doing anything else. 
FossilOrigin-Name: 018e82c775d0fb8c0d90cddf7a87c20c8c8172a9
 2010-09-17 15:28:41 +00:00
dan
a235d0ca63 Fixes to allow fts3 tables to be renamed mid-transaction. 
FossilOrigin-Name: d1c875320a045c3938c765ceb543dfba1a0ecf0b
 2010-08-24 16:59:47 +00:00
drh
f053d5b6c7 Fix two comment typos that were reported on the mailing list. 
FossilOrigin-Name: 88bf9016277c133dfbf4f4b2be0c35c9fe9fabd6
 2010-08-09 14:26:32 +00:00
drh
c78db070a3 Remove all vestiges of ctype.h from FTS3. 
FossilOrigin-Name: b8b465ed2c56b23b50334680a2a940885b1ac530
 2010-08-06 19:00:12 +00:00
dan
a69f7d5aa5 Re-introduce the prefix-search optimization of [feef1b15d6], which was lost in a reorganization of FTS3 code. 
FossilOrigin-Name: d692434b4935e8e7858230af1c126b0be8203077
 2010-07-19 11:16:36 +00:00
shaneh
8a300f8095 Fix a couple more compiler warnings under MSVC. 
FossilOrigin-Name: 26bc27e3f4c8a666f3358f73fc21eccdec3508f6
 2010-07-02 18:15:31 +00:00
dan
6d3cb2fbae Fix for [9861b74ab9]. Correctly handle strings with zero tokens in the fts3 offsets() function. 
FossilOrigin-Name: d37034f7fc46b83be681247fde97823736c994cc
 2010-03-24 15:57:33 +00:00
drh
7bca0a3660 More commenting and documentation enhancements in FTS3. 
FossilOrigin-Name: 892e286709c4a72aae364b85dd43bea525bb1b4d
 2010-03-23 18:24:06 +00:00
drh
d6291c9155 Fix up obsolete comments in FTS3 to conform to the latest nomenclature. 
Add new comments to better explain FTS3 operation.

FossilOrigin-Name: 3e4a0082170155b5b779afd075a3ee650530ca68
 2010-03-23 15:46:41 +00:00
dan
3b6f4c06ac Rewrite a couple of queries used internally by FTS3 to take advantage of the "SELECT max(x) FROM ..." optimization. 
FossilOrigin-Name: b7e42ae77443b33e0ab83265064830594094dc7c
 2010-03-19 06:59:19 +00:00
drh
71663ad0c1 Mark internal routines as "static" in FTS3. 
Ticket [b1552d0fa7].

FossilOrigin-Name: 39b34ac4a7bd06144b3630c0ad71bad0615418b6
 2010-03-18 16:34:44 +00:00
drh
3a3dd3ac8d Improved error message when the right-hand operand of MATCH in FTS3 is 
not a valid search expression.  Ticket [170872f1c3].

FossilOrigin-Name: bea9258643edfeb98ebf59d93fa18a14e4221fea
 2010-03-15 20:53:01 +00:00
drh
9396a41e6e Fix FTS3 so that it does not run illegal SQL and cause a spurious log 
message.  Ticket [42d45a693e6].

FossilOrigin-Name: f186b6a61909be1d65b76a6fbaa42f57bbd0d1e5
 2010-03-15 19:27:55 +00:00
drh
40d238e09f Disable FTS4 (which is incomplete and not ready for publication) to prevent 
accidental use.

FossilOrigin-Name: 87a6a9970b8cea25fc5f8ff03d87762dd1166899
 2010-03-09 12:35:21 +00:00
dan
89374d2309 Fix a one byte buffer overread in fts3. 
FossilOrigin-Name: 6e3e014af91601ed1f3a9cbe23f7c4260a4d177f
 2010-03-05 17:27:01 +00:00
shaneh
84f4b2f237 Minor changes to silence some compiler warnings with MSVC. 
FossilOrigin-Name: 4927c62d6214d93e582bf1266215ed3519630c15
 2010-02-26 01:46:54 +00:00
dan
56f2996b48 Fix a segfault in fts3 that may occur if the snippet, offsets or matchinfo functions are used in a query that does not contain a MATCH clause. 
FossilOrigin-Name: 14ab536a96f81595ab8c3dcd4b72075f1a827c8b
 2010-02-10 05:33:17 +00:00
drh
b4a1fed2ea Add a new full-text search variant that tracks the total number of documents 
and document sizes, to make ranking search results easier.  Currently
called FTS4.

FossilOrigin-Name: 1b6e6094c88214e02c9e3638932997ac20bfe413
 2010-02-03 19:55:13 +00:00
dan
d3789b98a7 Fix two similar problems in fts3 that meant that an OOM error could cause a memory leak. 
FossilOrigin-Name: 701ef64b3dbf45e52043e79e528002bd4b7a21e2
 2010-01-22 15:48:18 +00:00
dan
3996a06236 Add coverage tests for fts3_snippet.c. Also fixes related to the same. 
FossilOrigin-Name: 5e9d8ccae9731f380527463ef87ddcd216b4b721
 2010-01-15 17:25:52 +00:00
dan
f596da5cc6 Update comments in fts3_snippet.c. 
FossilOrigin-Name: dd9689b7c31861484f8d8062478ae73d3c940902
 2010-01-12 17:57:30 +00:00
dan
9a34020f05 Modify snippets code to run more efficiently. And to avoid a bug relating to snippets based on full-text queries that contain duplicate terms. 
FossilOrigin-Name: a2b1183d9e9898d06d623b342bbb552e85a9b3f6
 2010-01-11 12:00:47 +00:00
dan
1a54b8ecfa Fix handling of an OOM error in the fts3 offsets() function. Fix a couple of snippet related test cases in e_fts3.test. 
FossilOrigin-Name: 14dc46a74aafe44c0bf7dffd26268395b2c5edb2
 2010-01-09 07:33:54 +00:00
drh
819443e506 Update comments in fts3.c to more accurately describe the doclist format. 
FossilOrigin-Name: e424a0307359fee6875424c10ecad1a10acfba0e
 2010-01-08 23:01:32 +00:00
dan
3174598a64 Fixes to problems in FTS3 snippet() function found by th3 tests. 
FossilOrigin-Name: 3b5ccd2682176929f4da8a3f39a7e8f58b179f18
 2010-01-07 10:54:28 +00:00
dan
c06013dca7 Fix a segfault that can occur following an OOM in the FTS3 snippet() function 
FossilOrigin-Name: c7e5966e3b031672f149d0b6e1f75f9bc40868fa
 2010-01-06 18:36:27 +00:00
dan
b023b04fcb Change the fts3 snippet function to return (hopefully) more relevant snippets in less time. 
FossilOrigin-Name: 8a208223a74d451f60d9cd707d63fb7d157d1737
 2010-01-06 17:19:21 +00:00
shaneh
bb6b9878d7 Removed compiler warnings in MSVC. 
FossilOrigin-Name: 4ecb4725b76042ac414caf90039d5d0a5f8f7068
 2010-01-05 14:18:54 +00:00
dan
a98af17af6 Add experimental implementation of FTS3 functions matchinfo() and snippet() (not enabled by default). 
FossilOrigin-Name: 51f7ee844057086789dcfcdcba7daf45343cae62
 2010-01-02 19:02:02 +00:00
shaneh
46aae3c0da Changes to remove warnings in MSVC build. 
FossilOrigin-Name: 6cf76c2ae25d6e58926637ecd42eed6b300b1a25
 2009-12-31 19:06:23 +00:00
dan
8718183ea5 Fix a bug triggered by optimizing an FTS3 table when there are no segments on disk but pending terms in the hash table. 
FossilOrigin-Name: a3b50e4f80ca2dacc1f72435b8c72d55ffdb64e7
 2009-12-29 10:32:36 +00:00
dan
3540c1f7b8 Add the experimental FTS3 matchinfo() function. Provides details of the match that may be used for result ranking and other purposes. 
FossilOrigin-Name: 37a1de02d1d8a34604f1bee896eaf579d4ba149a
 2009-12-22 18:56:19 +00:00
dan
e61fc3b6a8 Fix a problem in FTS3 phrase queries. Add tests to check that it really is fixed. 
FossilOrigin-Name: 956de051f43dfc2f178a7820981da4bf69927190
 2009-12-20 15:00:19 +00:00
dan
c007646318 Handle an unhandled OOM error code that occurs in fts3 with SQLITE_TEST defined. 
FossilOrigin-Name: 82b069998e10b1a9a1d5e6bd85877d7b654c78bd
 2009-12-14 15:17:27 +00:00
dan
971742058f Fix a broken assert() in fts3_write.c. Also fix a couple of compiler warnings in fts3 code. 
FossilOrigin-Name: a730a05b39c45138765df98d34b9db1ab4e72152
 2009-12-14 14:49:57 +00:00
dan
ca6be31b69 Tests to cover a few extra branches in fts3.c. 
FossilOrigin-Name: 06b72b007393dc34d75a8bb16ce0e4507d5f8faa
 2009-12-12 19:15:27 +00:00
dan
55be7449f9 Further fts3 coverage tests. 
FossilOrigin-Name: d2a8c0f683271f5fb8c9badfb13e4e46fd78db71
 2009-12-12 16:04:32 +00:00
dan
45bcd6c274 Extra tests for coverage of fts3 code. 
FossilOrigin-Name: eee921a99e69a9cd868a89de620bf47c4e26e4b5
 2009-12-12 13:16:09 +00:00
dan
f5fff2a7b2 Add coverage test cases for fts3. 
FossilOrigin-Name: 8fcb0478c82507403165719724b62a308cb83b57
 2009-12-12 09:51:25 +00:00
dan
601cd9aa24 Change the fts3 test interface used to configure the advisory node size paraemeter. 
FossilOrigin-Name: 87fc0ce151c89beb5a43b65c0b1706f340c0c173
 2009-12-11 16:03:45 +00:00
dan
4f887abc4c Rationalize some code in fts3 used by optimize operations, queries of the pending-terms hash table and segment merges. Add the "INSERT INTO tbl(tbl) VALUES('optimize')" syntax. 
FossilOrigin-Name: 29476da353df4c67fe744c1c5f466ba5b9c1a54b
 2009-12-11 12:29:04 +00:00
shaneh
ef4114dd24 Avoid declaring and freeing test variables if SQLITE_TEST not defined. 
FossilOrigin-Name: c2d22960f652264c48ec41b9ca4047aa142d86f7
 2009-12-10 21:11:28 +00:00
dan
dcf3cc1f91 Remove some unused macros and types from fts3.c. 
FossilOrigin-Name: 83a80efe8a6c3d53a7f80b4c5f4e0f40aa0aa52f
 2009-12-10 18:38:54 +00:00
dan
0502e56415 Fix handling of "WHERE col MATCH ? AND docid = ?" clauses in fts3. 
FossilOrigin-Name: 6cbbae849990d99b7ffe252b642d6be49d0c7235
 2009-12-10 18:29:04 +00:00
dan
2c11d597c7 Fix an OOM related problem in the snippet() and offsets() functions of fts3. 
FossilOrigin-Name: 61efff414254f209f1c69728ae2da3b3888b7290
 2009-12-10 18:20:31 +00:00
dan
b893721a45 Changes to fts3 to avoid flushing data to disk within a SELECT statement. 
FossilOrigin-Name: 48c0db0eb2d134bb302bb5eca6beb0ec46736257
 2009-12-10 16:04:25 +00:00
dan
18ff7fa6b4 Improve test coverage of fts3.c. 
FossilOrigin-Name: 56b6432f8622d53ffd3a4d9a2244114f8531ed71
 2009-12-09 14:39:41 +00:00
dan
6d67e33776 Mark fts3ReallocOrFree and fts3InitVtab as static. Ticket [ff44d82f3b]. 
FossilOrigin-Name: a9038306c33c88120d8bef27209d8f0641c85c9b
 2009-12-09 05:30:36 +00:00
dan
84db21ec6a Add tests to improve coverage of fts3. Associated bugfixes. 
FossilOrigin-Name: f0eac4175aee6c50ee68acc253f76fbe44574250
 2009-12-08 19:05:53 +00:00
dan
7bf44fc018 Remove a redundant line from fts3. 
FossilOrigin-Name: cd50acf37fd1e3b388f98fb2df7ed03cff454b24
 2009-12-07 16:26:52 +00:00
shaneh
0d935576a4 Move some SQLITE_TEST code down to avoid mixing code and variable declarations. 
Fix a test for high-order bit handling in sqlite3Fts3InitTokenizer().

FossilOrigin-Name: fad43d290f9489726aaa2e999a17ea17ed78b27b
 2009-12-07 16:18:37 +00:00
dan
ff32e39c8e Add some tests for OR, AND and NOT operations to fts3rnd.test. Add tests to check that errors are returned when bad arguments are passed to fts3 functions snippet, offsets and optimize. Minor fix for the same 
FossilOrigin-Name: 5811df3f0412598d189d46b58de4deff24573651
 2009-12-07 12:34:51 +00:00
dan
28f372f6d9 Fix another bug in 3-way NEAR queries. 
FossilOrigin-Name: 3bb13a06521b54194c9f3eb44e0dc42bacf016a4
 2009-12-05 14:29:22 +00:00
dan
6e053f9c23 Fix some problems with FTS3 and 3-way NEAR queries. 
FossilOrigin-Name: 23dc6fb5b28712d1ba18dc7ddb3f2ef3b469d611
 2009-12-05 11:37:19 +00:00
dan
165b67cb36 Fix a problem involving a 3-way NEAR query. 
FossilOrigin-Name: 507890a9139875b1b594225c432c714f67312c0e
 2009-12-04 19:07:24 +00:00
dan
acf28fbdd8 Modify [2ad1461f25] to avoid leaving a prepared statement in "active" state following an OOM error in FTS3. 
FossilOrigin-Name: 69c21ee46aeeb624fd8638b17ff7259a3e5f9a46
 2009-12-04 14:11:33 +00:00
drh
406a15ad8b Fix an FTS3 problem where an OOM error was not being propagated back 
out to the top-level interface.

FossilOrigin-Name: 2ad1461f255c2499367b706a5ec65b44c1fc1618
 2009-12-04 13:42:59 +00:00
dan
e2e5145441 Fix an incorrect assert() in fts3.c. Add further fts3 tests. 
FossilOrigin-Name: 75863c2d55e0801add5b8dcf88d575c5c870af04
 2009-12-03 17:36:22 +00:00
shaneh
e585b8f05c Updates to FTS3 to correct compiler warnings under MSVC. 
FossilOrigin-Name: 37495b55ffbdc2db4482367ac7d8e32d4d71d58e
 2009-12-03 06:26:46 +00:00
drh
c12075b3b2 Change an error message in FTS3 to avoid using an uninitialized variable. 
FossilOrigin-Name: 620a8a2b38f5f2ad3db304b2bc88360285c174db
 2009-12-02 20:25:57 +00:00
dan
bc7c039ce2 Clear the Pager.dbModified flag when unlocking the database. Assert that it is clear when locking the database. 
FossilOrigin-Name: d17ec16b7c5051c904c09580a856593b2fb85edc
 2009-12-02 14:44:32 +00:00
dan
6bd3b2aa64 Use #include "sqlite3.h" instead of <sqlite3.h> in fts3Int.h. 
FossilOrigin-Name: 7737db490ceba02c29c36fe181b4e7895b40aa75
 2009-12-01 17:08:09 +00:00
dan
8c4499764b Add typedefs for i16 and u8 to fts3Int.h when not building via the amalgamation method. 
FossilOrigin-Name: fa56c1c47296c2f9ba1de9450d421dd06fde5a6a
 2009-12-01 17:05:50 +00:00
drh
65e8c82e1a Changes to the TCL interface header to allow it to be compiled independently 
from the amalgamation.

FossilOrigin-Name: 58113932d93926b4aa037a7487105a55f883cd0a
 2009-12-01 13:57:48 +00:00
dan
3acb07d6c3 Open a savepoint within the FTS3 optimize() function. 
FossilOrigin-Name: 4924fbb244bd1b7103e29e045812cb1c4d2d81c8
 2009-12-01 13:48:13 +00:00
dan
612b1d5cd0 Fix a segfault that can occur when querying an empty FTS3 table. Also restore the rowid/docid conflict handling to work as it did in version 3.6.20. 
FossilOrigin-Name: c022f66b5a65aa54d5ebd55cfe941118a2042280
 2009-12-01 12:00:22 +00:00
drh
ff3f307cd0 Test coverage improvements in the FTS3 porter stemmer. 
FossilOrigin-Name: 6d112bfd53998b8f6693d3f2edbcd5ab4cdf5fb1
 2009-11-30 19:48:16 +00:00
drh
0a62730d3f Updates to snippet() and offsets() functions of FTS3 so that they work 
sanely following an OOM fault.

FossilOrigin-Name: b939a37a8ce296785a300e79ab9d3d87ad91343f
 2009-11-28 21:33:21 +00:00
drh
9287d93c17 Change FTS3 to detect when the RHS of the MATCH opertor encounters an OOM 
during string format conversion and report back an SQLITE_NOMEM error.

FossilOrigin-Name: 31eed4f8f95f0799d634eccbd9e09cb58172d250
 2009-11-28 17:23:47 +00:00
drh
44c1e5a13c Remove all benign OOM failure opportunities from the FTS3 hash table 
implementation.  All OOM faults cause SQLITE_NOMEM to be returned.

FossilOrigin-Name: 80754d383a0e890ea3f315dab941b9f166481ddd
 2009-11-28 17:07:42 +00:00
dan
8e9f6aedae Add a test case for creating an FTS3 table with no module arguments or opening/closing brackets in the CREATE VIRTUAL TABLE statement. 
FossilOrigin-Name: a9cba7ea0a06efa7a63a3069b219cc30fb127e98
 2009-11-28 15:35:16 +00:00
dan
81fa6dc319 Fix a bug introduced by the fts3 refactoring (segfault when creating a table with zero module args). Also a fix to handle an OOM error. 
FossilOrigin-Name: eada284bc10cafcab9beb3473bb0c70b3b4de2f9
 2009-11-28 12:40:32 +00:00
dan
f6d7b055fa Remove a C++ism accidentally added to fts3.c. 
FossilOrigin-Name: 97d332416069d2fbce323740b276d0e7523eeee5
 2009-11-27 12:14:47 +00:00
dan
7eee299243 Add some missing comments to fts3 files. No source code changes. 
FossilOrigin-Name: b6402b2065b844acb3f1bb94ad964568706bcb86
 2009-11-21 03:03:21 +00:00
dan
e6828f5503 Merge leaf accidentally created by [1c4984c62f]. 
FossilOrigin-Name: cae949ce971ca216e0f8880b2f93866619fa05be
 2009-11-20 10:23:12 +00:00
dan
d1414c58e5 Improve comments and other things in fts3_write.c. 
FossilOrigin-Name: 1cf0e3cc14bad22867e740736c2886dc1c4a48dc
 2009-11-20 05:05:19 +00:00
dan
d313865550 Minor optimizations to fts3 code. 
FossilOrigin-Name: b456eacbbb16513d1b27e90015ea58a6dc92cc3b
 2009-11-20 02:24:15 +00:00
dan
8f4a4f24dd Fix a performance regression introduced while reworking the fts3 code. 
FossilOrigin-Name: 7cd178a72ab99c94fdacffb19aad819ae600e57d
 2009-11-19 18:28:45 +00:00
dan
16708c4a0d Fix some fts3 related issues with the autoconf and amalgamation build systems. 
FossilOrigin-Name: 3b17924754343c0163464dabf01a9c46ffccef28
 2009-11-19 15:25:25 +00:00
dan
5dc842ddf7 Fix problems introduced into fts3 as part of the refactoring. 
FossilOrigin-Name: fa0998e19d984ee57f4f506c34eb858026cc49c3
 2009-11-19 00:15:27 +00:00
dan
bd61689382 Add some missing comments and fix some other issues in fts3 code. 
FossilOrigin-Name: 2fe579e778b75fbf503c02e01e5424c1926f2b49
 2009-11-18 15:35:58 +00:00
dan
f13b704ee6 Improvements to the way fts3 reads the full-text index. 
FossilOrigin-Name: 45c051e78651d8204c17cecdda2bde705698881f
 2009-11-17 12:52:10 +00:00
dan
948a5f88ea Add a few extra coverage test cases for fts3. 
FossilOrigin-Name: f29c8fcade4aadeae3824975cf59f306c11c906b
 2009-11-16 16:36:23 +00:00
dan
91f0ce39e4 Further OOM testing for fts3 code. Add Tcl code implementing an integrity-check for fts3. 
FossilOrigin-Name: c27d46b33e8596b45c562c2742b05030e8899092
 2009-11-14 11:41:00 +00:00
dan
09977bb9f0 Start reworking fts3 code to match the rest of SQLite (code conventions, malloc-failure handling etc.). 
FossilOrigin-Name: 30a92f1132801c7582007ee625c577ea2ac31cdf
 2009-11-13 10:36:20 +00:00
danielk1977
ee0484c1b5 Add the experimental API sqlite3_strnicmp(). Modify fts3 so that in terms like 'column_name:token' the column_name is interpreted in a case-insenstive fashion. Ticket #3996. (CVS 6950) 
FossilOrigin-Name: 4571aa9e9142db465ae8250b0adf27e0a094331a
 2009-07-28 16:44:26 +00:00
danielk1977
e932ba260e Fix a double-free that can occur when using the fts3 legacy syntax '-' operator. Add tests for the same operator. Ticket #3960. (CVS 6874) 
FossilOrigin-Name: c19d419e8cf94a26d9bb6ad478e84841168a882e
 2009-07-10 09:24:43 +00:00
danielk1977
2fe5cb1809 Avoid fts3 crash on (MATCH '""') expressions. Ticket #3717. (CVS 6343) 
FossilOrigin-Name: 03679857a320517a7b89e5214e948bce9af896a9
 2009-03-12 15:43:47 +00:00
shane
be21779385 Corrected typos and misspellings. Ticket #3702. (CVS 6336) 
FossilOrigin-Name: 6404afa0c515a6536fc2e878d4fb451e4dc06942
 2009-03-05 04:20:31 +00:00
danielk1977
e1d3ac9cd0 Add a comment to fts3_tokenizer.h to make it clear how the xNext() method is supposed to set its output variables. Make sure the output variables of xNext() are only used if SQLITE_OK is returned. Ticket #3604. (CVS 6198) 
FossilOrigin-Name: 5b3c075f96be9671d0bcffe928589b211559e835
 2009-01-21 17:45:33 +00:00
drh
d162988b47 Fix typos in comments in FTS3 implementation. (CVS 6178) 
FossilOrigin-Name: b0f066630c35c4947d3ecd29d32d91036da19e94
 2009-01-14 18:59:41 +00:00
drh
be90df0b3e Do not display matches against 
the right-hand side of a NOT operator in the output
of the FTS snippet() or offsets() functions. (CVS 6097)

FossilOrigin-Name: d44c84c0f77bd0fc4a9942177b6cae6d109b89b7
 2009-01-02 01:10:42 +00:00
danielk1977
fc8c9f84ab Fix some problems in the fts3 expression parser with mismatched parenthesis. (CVS 6095) 
FossilOrigin-Name: ccfe4580ac7ba9add0e69c786a9a3a43d69b7753
 2009-01-01 14:06:13 +00:00
drh
b39187ae89 Additional test cases and cleanup of FTS3 parenthesis processing. (CVS 6094) 
FossilOrigin-Name: afac4293000f81410d105a99956605bf7102fa62
 2009-01-01 12:34:45 +00:00
danielk1977
5973e6a30b Add pseudo-random tests of the fts3 expression parser. Revise the fix in (6091). (CVS 6092) 
FossilOrigin-Name: 11c2d4686197fb3f0d601651d5bbb3492af8f0dd
 2009-01-01 07:08:54 +00:00
danielk1977
49b4b4d84a Fix a bug parsing "<expr> AND (abc NEAR def)" in fts3_expr.c. (CVS 6091) 
FossilOrigin-Name: d1a6a2edd799d65ff88510df951e909919e35b6b
 2009-01-01 04:19:51 +00:00
drh
42128b9e33 Fix the name in the documentation of the compile-time macro for 
enabling FTS3 parenthesis processing. (CVS 6089)

FossilOrigin-Name: ac8258da6ecd3ea37f394dc3b48834eb57832cf4
 2008-12-31 19:27:53 +00:00
drh
757b178100 Fix the FTS3 expression parser so that it works in the amalgamation when 
FTS3 is disabled. (CVS 6088)

FossilOrigin-Name: 7e238e8604b9a9f786d84a47d21c6b42f1585755
 2008-12-31 16:27:58 +00:00
drh
aeba020bea Fix the FTS3 module with parenthesis syntax so that it will work in 
the amalgamation. (CVS 6087)

FossilOrigin-Name: c2b9891fc05ec05b270f108f61ab81b2df874e01
 2008-12-31 16:01:04 +00:00
danielk1977
d597e08b23 Fix a bug in README.tokenizers. Ticket #3559. (CVS 6075) 
FossilOrigin-Name: b8898d132e84888dc7c51b2f1ab67f78cc21f31b
 2008-12-30 06:36:50 +00:00
danielk1977
d34c03a946 Add the file ext/fts3/README.syntax, containing documentation describing the two query syntaxes now supported by fts3. (CVS 6042) 
FossilOrigin-Name: ed81ad5a5d22304a4d96e778e8e9094f74c461c0
 2008-12-19 11:37:38 +00:00
danielk1977
f0f9f75443 Fix some strict-aliasing problems in fts3_expr.c. (CVS 6035) 
FossilOrigin-Name: 20a4ca5d361ecbb982129171f10cccac4f5ad093
 2008-12-17 15:49:51 +00:00
danielk1977
33e8903540 Modify fts3 to support a more complex expression syntax that allows parenthesis. The new syntax is not entirely backwards compatible, so is disabled by default. Use -DSQLITE_ENABLE_FTS3_PARENTHESIS to enable it. (CVS 6034) 
FossilOrigin-Name: 7389b9ecb80294569845c40a23e0c832d07f7a45
 2008-12-17 15:18:17 +00:00
danielk1977
777da0848d Fix a couple of memory leaks that may follow malloc failures. (CVS 5906) 
FossilOrigin-Name: 4cf8a8e1bf22e1d8f7166e64328a95fe36c75033
 2008-11-13 19:12:34 +00:00
drh
8578611b95 Fix the NEAR connector in FTS3 so that it can take ranges in excess of 9. 
The maximum range is now 32767. (CVS 5695)

FossilOrigin-Name: 8e9b9553115c42dae38cad0612d98d9a0c453a5c
 2008-09-12 18:25:30 +00:00
danielk1977
075c23af26 Begin adding support for the SQLITE_OMIT_WSD macro. Some (many) WSD variables still need wrappers added to them. (CVS 5652) 
FossilOrigin-Name: 573d92abb9adb1c321ebc2fcadcf14374213b093
 2008-09-01 18:34:20 +00:00
shess
b5f94870c2 Re-used prepared statement from fts3 cursor. Previously, each call to 
fulltextFilter() finalized any existing prepared statement and
prepared a new one.  In the case where idxNum has not changed, simply
reseting the statement suffices.  This provides an order of magnitude
speedup in incoming joins against docid. (CVS 5489)

FossilOrigin-Name: a08a5f2b1256b8a93beca5a359ccfc28d403efa3
 2008-07-29 01:13:02 +00:00
shess
7d9ef0d0fc Implement optimize() function. This merges all segments in the fts 
index into a single segment, including dropping delete cookies. (CVS 5417)

FossilOrigin-Name: b22e187bc2b38bd219dd0feba19b97279bd83089
 2008-07-15 21:32:07 +00:00
shess
c2c66a030d Delete all fts3 index data the table becomes empty. Previously, 
deleting all rows from an fts3 table would leave a bunch of index data
describing the terms of the original data, plus deletions of those
terms, perhaps with some amount of it merged together so the deletions
knocked out the originals.  Even when all rows were deleted that
original data would hang out, though eventually it would mostly be
overwritten if new data contained the same set of terms. (CVS 5413)

FossilOrigin-Name: 8b872e426091d9ef108e52dbec0d968ed7452907
 2008-07-14 20:43:15 +00:00
shess
6c106e3f3b fts3 functions for testing scripts. These are a first step towards 
being able to write test script which verify that fts3 is internally
building indices in the expected way.  Both new functions are only
defined if fts3.c is compiled with SQLITE_TEST defined, as when
building testfixture.  These functions are not intended to be part of
the exposed fts3 API.

dump_terms() generates a TEXT result of all the terms in the index (or
a specified segment), sorted and joined with spaces.

dump_doclist() generates a TEXT representation of the doclist
associated with a given term in the index (or a specified segment). (CVS 5340)

FossilOrigin-Name: a48e3d95f7a656285e959cef595cbe6d53428ad9
 2008-07-03 19:53:21 +00:00
drh
8a29dfdea0 Remove all instances of sprintf() from the FTS modules. Ticket #3049. (CVS 4996) 
FossilOrigin-Name: 062bf5d44d53ae0ee2bf96eddcc8de09157aa789
 2008-04-12 13:06:09 +00:00
drh
dd95535f20 Minor fixes to FTS3 so that it works better when appended to the end 
of the amalgamation. (CVS 4769)

FossilOrigin-Name: 62ede6699d8f116921a5a0baddca5e7e63740cd3
 2008-02-01 15:34:09 +00:00
drh
820a90694e Version number to 3.5.5. Include FTS3 in the amalgamation by default 
(but disabled unless compiled with -DSQLITE_ENABLE_FTS3).  Fix a memory
allocation problem. (CVS 4757)

FossilOrigin-Name: 72411043e60d5358d5a7adf566d662d65d3b3336
 2008-01-31 13:35:48 +00:00
drh
85b623f2f9 Change all instances of "it's" in comments to either "its" or "it is", 
as appropriate, in case the comments are ever again read by a pedantic
grammarian.  Ticket #2840. (CVS 4629)

FossilOrigin-Name: 4e91a267febda572e7239f0f1cc66b3102558c36
 2007-12-13 21:54:09 +00:00
shess
b6a75606ed Change prefix search from O(N*M) to O(NlogM). The previous code 
linearly merged the doclists, so as the accumulated list got large,
things got slow (the M term, a fucntion of the number of documents in
the index).  This change does pairwise merges until a single doclist
remains.  A test search of 't*' against a database of RFC text
improves from 1m16s to 4.75s. (CVS 4599)

FossilOrigin-Name: feef1b15d645d638b4a05742f214b0445fa7e176
 2007-12-07 23:47:53 +00:00
drh
8255feca02 The FTS3 amalgamation can now be appended to the SQLite amalgamation to 
generate a single source file that contains both components. (CVS 4558)

FossilOrigin-Name: 0fc61f99b54bd269fcc011f448b9b971e902cb01
 2007-11-24 00:41:52 +00:00
drh
ac320cc14a Add a #include of sqlite3.h to fts3_hash.c. Tickets #2762 and #2777. (CVS 4555) 
FossilOrigin-Name: c8485eb8bc62c810ec9f73e103468c57116fd94c
 2007-11-23 18:01:07 +00:00
drh
613a0fe455 Changes fts3 to use only sqlite3_malloc() and not system malloc. 
Ticket #2762. (CVS 4554)

FossilOrigin-Name: 460af6bb668094c99a1d4dc1540b44b6d1d036b6
 2007-11-23 17:31:17 +00:00
shess
cd7274ceb0 Don't do anything when input doclists are both empty. Ticket #2774 (CVS 4546) 
FossilOrigin-Name: 75cb46f82a6a95dbe9e279dede299bafa2e91cae
 2007-11-16 00:23:07 +00:00
shess
adafd5747f Be a bit more susicious of invalid results from the tokenizer. (CVS 4514) 
FossilOrigin-Name: deb8f56d3adea0025d28b8effabec7c7b7fe3026
 2007-10-24 23:24:22 +00:00
shess
b6d78dc7bb fts3.c buildTerms() passes -1 for nInput. (CVS 4511) 
FossilOrigin-Name: e87c883a1235ac47ee340a31051dcd5deb369d4e
 2007-10-24 21:52:37 +00:00
danielk1977
1c1764ae66 Add the NEAR operator to fts3. (CVS 4502) 
FossilOrigin-Name: aef7720e0bb49d52332ddebe6f698feb926ef7d7
 2007-10-22 18:02:20 +00:00
drh
8a07c7a414 Cleanup the hash functions in FTS3. (CVS 4440) 
FossilOrigin-Name: ac645c8f30aac0d98fc481260084c9bd3975a845
 2007-09-20 12:53:27 +00:00
shess
961303c1e7 Drop the forced error from fts3.c and add forced errors to fts2.c and 
fts1.c. (CVS 4427)

FossilOrigin-Name: fec6567a0f8a868cda9bba2a473491dfb17b6c88
 2007-09-13 18:16:08 +00:00
shess
d83ae45639 Add an implicit (HIDDEN) docid column. This works as an alias to 
rowid, similar to how things work in SQLite tables with INTEGER
PRIMARY KEY.  Add tests to verify operation. (CVS 4426)

FossilOrigin-Name: c8d2345200f9ece1af712543982097d0b6f348c7
 2007-09-13 18:14:49 +00:00
shess
0ec85ae216 Mark the table-named column HIDDEN. Add tests to make sure it's 
working as expected. (CVS 4425)

FossilOrigin-Name: ca669eaf1b4af441741129bee4af02f32a7c74b8
 2007-09-13 18:12:09 +00:00
shess
27a770e044 Fix memory leak of InteriorReader.term. Comes up when doing queries 
against large segments. (CVS 4315)

FossilOrigin-Name: 6c617bd89fc57881a2a308a6360e8ebb42835d46
 2007-08-28 20:36:53 +00:00
shess
bae37537b0 Make comments and variable naming more consistent WRT rowid versus 
docid/blockid.  This should have no code impact. (CVS 4281)

FossilOrigin-Name: 76f1e18ebc25d692f122784e87d202992c4cfed2
 2007-08-23 20:28:49 +00:00
shess
6beeb0329a Fix fts3 to not have the VACUUM bug from fts2. %_content.docid is an 
alias to fix the rowid for documents, %_segments.blockid is an alias
to fix the rowid for segment blocks.  Unit test for the problem. (CVS 4280)

FossilOrigin-Name: 6eb2d74a8cfce322930f05c97d4ec255f3711efb
 2007-08-23 20:23:37 +00:00
shess
acce22f5c7 Copy fts2 to fts3, renaming, and replacing references to fts2 with 
fts3, including capitalization variants. (CVS 4249)

FossilOrigin-Name: 216c91d2fc49792d9ff53596746f1162f5b7f8d4
 2007-08-20 17:37:02 +00:00