Commit Graph

211 Commits

Author SHA1 Message Date
drh
4f99189051 Fix various harmless compiler warnings. Change the "warnings.sh" script to
work with STAT4 instead of STAT3.

FossilOrigin-Name: 7df06684ab36bfdad9e9aca6940b7a665c2a0cb5
2013-10-11 15:05:05 +00:00
mistachkin
86f72f0721 Fix compilation issue with MSVC.
FossilOrigin-Name: 36d64dc36f18c166b2c93c43579fa3bbb5cd545f
2013-10-07 21:49:16 +00:00
dan
9211f8a6ce Allow FTS4 multi-token phrases to use a combination of in-memory and incrementally loaded doclists. This allows phrases to (partially) benefit from incremental doclists without disabling the deferred token optimization.
FossilOrigin-Name: f6819c5f3363d358e7ef65fe6978f13991bd44af
2013-10-03 19:27:14 +00:00
dan
eb26672d0e Add a test to check that the new multi-token phrase optimization is actually helping.
FossilOrigin-Name: bc3a2ed5fb2402805928b0002457eebf06f87c47
2013-10-02 08:04:27 +00:00
dan
ff574f4c76 Allow multi-token phrases to load doclists from the database incrementally. This allows queries that feature such phrases to benefit from the "docid<?" optimization.
FossilOrigin-Name: ea543f081d93ed1bf66c21ce2108ec94e349f4c5
2013-10-01 20:02:32 +00:00
dan
73632e37c9 Merge trunk changes with this branch.
FossilOrigin-Name: e294a9c7c525d1da698259c2fef089579bfd1c0b
2013-09-30 18:16:07 +00:00
dan
1f03e609ff Fix a performance problem in the FTS4 auxiliary functions triggered by an OR clause in the full-text query.
FossilOrigin-Name: fa0f2f0e3e79ae653118b901e1cca7725dfaf249
2013-09-30 18:14:45 +00:00
dan
19384380f3 Have fts4 full-text queries consider "docid<?" and similar constraints.
FossilOrigin-Name: 6622424a3a149edd35ba2ba0881aa41b4536417b
2013-09-30 11:42:19 +00:00
drh
1f10c28618 Modify several extensions to use the new exported function naming.
Fix some shared library compilation issues.

FossilOrigin-Name: 1e39f85077f1f2b96c3a656c5b6334bafb005908
2013-07-09 12:36:30 +00:00
dan
ed20073b8d Make a couple of implicit casts explicit to fix compiler warnings.
FossilOrigin-Name: 9676280a9387b8bfba82cfc0087318863d05d4f3
2013-07-05 10:46:08 +00:00
mistachkin
049d487e2e Modify several extensions to use the new exported function naming. Fix some shared library compilation issues.
FossilOrigin-Name: f2ab8747825ab5131ffab174aa0ffe5e474f6811
2013-07-04 23:53:56 +00:00
drh
4580cfb93e Cut over the next generation query planner. Increase the version number
to 3.8.0.

FossilOrigin-Name: 0fe31f60cadc5fe5a9d87e110bfaed5fd026cba1
2013-06-26 11:43:18 +00:00
dan
8def92bac3 Add the "notindexed" option to fts4.
FossilOrigin-Name: 8ff2b8f5948ccddce70102e6d68d464c66e4f7ca
2013-06-21 17:30:47 +00:00
drh
73843cac73 Increase the estimated full-scan cost for an FTS3 table.
FossilOrigin-Name: 8d9f29f847f38612447c79a5fb366a9143ef6205
2013-06-04 13:07:44 +00:00
dan
87ddfeb079 When writing to an FTS table, take an exclusive shared-cache lock on the %_segdir table before writing. Otherwise, an xCommit() call may report an SQLITE_LOCKED error.
FossilOrigin-Name: 3cd2da42e9403b1e6243ad53f3f2bbf89c0fb9b0
2013-05-27 10:11:53 +00:00
dan
e8dee7d1ed Modify the fts3tokenize table implementation so that it does not use the SQL function fts3_tokenizer. The user may have installed an authorizer callback that prohibits this.
FossilOrigin-Name: 0ba67b64de258883e4c43db09e131bb67083855e
2013-05-07 12:16:48 +00:00
dan
3a01b600d3 Improve the error message issued when an FTS query exceeds the maximum allowable tree depth.
FossilOrigin-Name: f480b1fe6012f36c59cd0525efdc6df74143ccd0
2013-04-29 18:07:37 +00:00
dan
d7a959c694 Add virtual table module 'fts3tokenize' to fts3. fts3tokenize provides SQL access to FTS tokenizer implementations.
FossilOrigin-Name: c5a4b21a15bd8a184380817ff41657acbccb866d
2013-04-22 15:30:37 +00:00
mistachkin
48864df97d Many spelling fixes in comments. No changes to code.
FossilOrigin-Name: 6f6e2d50941e444ebc83604daddcc034137a05b7
2013-03-21 21:20:32 +00:00
dan
e62c0694de Fix a bug in fts3 causing it to return SQLITE_NOMEM when NEAR, OR and some auxiliary functions were used together. Also a segfault in vtab.c that could follow an OOM condition.
FossilOrigin-Name: 33f4f9817e8b3cb97ce02a4c49bd586332da37b2
2012-10-25 19:25:39 +00:00
dan
4dc3d73d69 Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option.
FossilOrigin-Name: e799222f3b8246e65657a758437914ece7069ba9
2012-08-20 17:24:48 +00:00
dan
2c897e3e5f Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61.
FossilOrigin-Name: eccd6b6580637084495b80e9232262188ba0cf8d
2012-06-06 19:51:27 +00:00
dan
7946c53009 If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer.
FossilOrigin-Name: e71495a817b479bc23c5403d99255e3f098eb054
2012-05-26 18:28:14 +00:00
dan
ab322bd21e Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1.
FossilOrigin-Name: 8f3e60aa2253f21bcee5d03982cfdd7f16c00060
2012-05-26 14:54:50 +00:00
dan
3d403c71a8 Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.
FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385
2012-05-25 17:50:19 +00:00
dan
5cfed52dd1 Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38].
FossilOrigin-Name: 4d3e1673b309a3d6a6cac01d67dfd79a8007f991
2012-05-10 17:43:14 +00:00
drh
4e245a4c35 Fix compiler warnings on GCC and MSVC and fix a C89-ism that
broke the build for MSVC.

FossilOrigin-Name: b451c0f97f0abe78ebe6c62ff489ec1ad8a1f767
2012-03-30 00:00:36 +00:00
dan
311ec02587 Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables.
FossilOrigin-Name: 5c447e226afca0d46b9ed994dea26a16a9ae168c
2012-03-27 15:00:06 +00:00
drh
40aab4c56d Minor correct errors in the file format description for FTS3/4 contained in
the fts3.c header comment.

FossilOrigin-Name: fb8aacdd8fbdc946cb271cc589f76b806387937d
2012-03-27 00:38:33 +00:00
drh
cbf9450404 Fix FTS3 so that it works even without SQLITE_DEBUG.
FossilOrigin-Name: a18c103121529c2e3c6a8ada16a4c40d14080670
2012-03-26 14:36:42 +00:00
dan
cbcd9f5357 Add an experimental integrity-check function to FTS.
FossilOrigin-Name: 40fc8804743dfb005991e9c5ef7b0ebcb3c2e731
2012-03-26 10:36:55 +00:00
drh
6c2e7e19fa Enable fts3 tables to use incremental merge by automatically creating the
%_stat table when it is needed.

FossilOrigin-Name: cc051fc0b2d89603b27b94cf2afdbda417ee9d94
2012-03-24 17:29:05 +00:00
dan
0dfcdeb4f0 Add a comment to fts3SyncMethod() to justify the nMinMerge=64 constant.
FossilOrigin-Name: af55ca5fc6778cb6d1a79a17dfa2d4e567ea1ccc
2012-03-24 16:43:55 +00:00
dan
45eddd6855 Remove the Fts3Table.mxLevel variable.
FossilOrigin-Name: 67a0cffc9d07be7f09dad9d019a18160711295cd
2012-03-24 16:11:21 +00:00
dan
4b1e4dabc3 Modify the way the number of leaves written and the maximum relative level are calculated in the auto-incr-merge code.
FossilOrigin-Name: 0d841c957c6ec4afecb49504177c6279e09d7012
2012-03-24 14:45:19 +00:00
drh
4ef9dfff97 An attempt at automatic incremental merging for FTS4.
FossilOrigin-Name: ed69434cd89084f4b57bd2cc4f5cc558904af565
2012-03-24 02:20:43 +00:00
dan
d59de73e27 Fix a spurious SQLITE_CONSTRAINT error that may be returned by an incr-merge operation.
FossilOrigin-Name: ed7c17ea165f6348506bd23ebc58c427bb65d697
2012-03-23 18:26:11 +00:00
drh
d13a3bc772 Fix one more compiler warning missed by the previous check-in.
FossilOrigin-Name: bc03d99a78e90c02b69037e5f5f81537b5a3ac60
2012-03-19 14:57:49 +00:00
drh
01ea399a84 Suppress harmless compiler warnings on windows in FTS4 and RTREE.
FossilOrigin-Name: 4fd68647c8d4b120e04d054617cef31001f44c6f
2012-03-19 14:51:19 +00:00
drh
bbe0520329 Avoid a compiler warning (an incorrect compiler warning, at that) in vs2010.
FossilOrigin-Name: 7dd97f12cd268cac1241f0f3e8de94bb629b97c7
2012-03-19 14:28:43 +00:00
dan
3a1abde7c8 Merge the fts4-languageid branch with the trunk.
FossilOrigin-Name: 99a9073b5e411ce94f38ce49608baaa15de8b850
2012-03-05 15:33:32 +00:00
dan
996073b3ae Add the xLanguageid method to sqlite3_fts3_tokenizer versions 1 and greater.
FossilOrigin-Name: f8e9c445dd358c40e5a7bf3756b9f291909dbea7
2012-03-03 18:46:41 +00:00
drh
9db1d56f02 Remove an obsolete comment from FTS3. No code changes.
FossilOrigin-Name: 74d224b14467d8a6915d3cf372f45e0d28708fa2
2012-03-02 20:34:57 +00:00
dan
7395599031 Fix problems with combining content= and languageid= in a single fts4 table.
FossilOrigin-Name: 22491e7bc38aee43819b888e04241cb6a6ef73a3
2012-03-02 19:53:02 +00:00
dan
7fcb214b93 Add the "languageid=" option to fts4. This code is still largely untested and alsmost certainly buggy.
FossilOrigin-Name: bea257f70f10dd1111d79cabd1e1462dc651704d
2012-03-01 19:44:20 +00:00
dan
585db0e23a Fix an FTS problem triggered by querying for an N character prefix using an N+1 character prefix index after rows have been deleted from the FTS table.
FossilOrigin-Name: c05c3fd20d93f430140d762ead23bacd337ffb4d
2012-01-25 16:29:45 +00:00
dan
689e38f7cf Add a missing va_end() macro to fts3.c.
FossilOrigin-Name: e6f825748a1d510c8f5529c79656bb5db9216231
2011-11-04 10:37:21 +00:00
drh
6ed18b7110 Fix harmless compiler warnings in FTS4.
FossilOrigin-Name: 1af4a25631c4077a9bfc64e15eb92fbf5146c89c
2011-10-19 18:21:47 +00:00
dan
e459b07601 Merge the fts4-content branch with the trunk.
FossilOrigin-Name: 8a4077057ddeb08e8edc5f20a75abaaba7a278ba
2011-10-19 16:20:40 +00:00
dan
9743948386 Have FTS3 ignore ^ prefixes. The ^ syntax is only supported on FTS4 tables.
FossilOrigin-Name: df36ac948179f37b432a88701b6c79299e073ce8
2011-10-19 15:52:48 +00:00