Commit Graph

91 Commits

Author SHA1 Message Date
dan
a2af0aecdb Avoid allocating a large object on the stack in the incremental merge code. Use sqlite3_malloc() instead.
FossilOrigin-Name: 36ae510de45be44efd34cff242d02fb21b7419ac
2012-03-14 12:17:40 +00:00
dan
8af1fc721e Fix another bug in the incremental merge code.
FossilOrigin-Name: f97b12e0955c4c29f9c31a186d72d87f7407782e
2012-03-14 11:51:31 +00:00
dan
604f58a686 Fix some bugs in the incremental merge code. Some remain.
FossilOrigin-Name: bff21683705a61b8b8672e0b44c287d1dc7c32a9
2012-03-13 19:56:34 +00:00
drh
ab4f6385de Minor commenting and stylistic changes only.
FossilOrigin-Name: a1747086c5e0c152fcf4bd9fa80a61b6f03f4a94
2012-03-09 12:52:43 +00:00
dan
593c9824bc Add the 'merge=?,?' command to fts4. This still needs some work.
FossilOrigin-Name: 741b8f897750eac3c9774fd65de7e40bb89781b1
2012-03-08 18:39:03 +00:00
dan
996073b3ae Add the xLanguageid method to sqlite3_fts3_tokenizer versions 1 and greater.
FossilOrigin-Name: f8e9c445dd358c40e5a7bf3756b9f291909dbea7
2012-03-03 18:46:41 +00:00
dan
8fb662b0be Fix the FTS 'optimize' command on multi-lingual databases.
FossilOrigin-Name: 65fa693729a336e4d905ce72e6b9ccf4faa772bd
2012-03-02 12:26:19 +00:00
dan
e3ddd1ac42 Fix a bug in merging FTS language tables for languages other than language 0.
FossilOrigin-Name: d281cb8984c911a4c0cce2ec299e1351d8e580e4
2012-03-02 11:48:50 +00:00
dan
7fcb214b93 Add the "languageid=" option to fts4. This code is still largely untested and alsmost certainly buggy.
FossilOrigin-Name: bea257f70f10dd1111d79cabd1e1462dc651704d
2012-03-01 19:44:20 +00:00
drh
548af8b69e Fix a compiler warning in FTS4.
FossilOrigin-Name: b4cad1ce78e6bb967c472bec5c562a24b4dfebaf
2012-01-31 23:34:21 +00:00
dan
585db0e23a Fix an FTS problem triggered by querying for an N character prefix using an N+1 character prefix index after rows have been deleted from the FTS table.
FossilOrigin-Name: c05c3fd20d93f430140d762ead23bacd337ffb4d
2012-01-25 16:29:45 +00:00
drh
64803bd2fa Fix minor issues with FTS and RTREE discovered by coverity.
FossilOrigin-Name: 1c27d842163e27c39bbe9409f50657b9de9ade6e
2011-12-22 17:31:17 +00:00
dan
0c8cda6e0d Fix a problem in fts3_write.c causing stack memory to be referenced after it is out of scope.
FossilOrigin-Name: f9c4a7c8f4e5821b47c1393d6272e32416d8886c
2011-12-22 15:30:46 +00:00
drh
6ed18b7110 Fix harmless compiler warnings in FTS4.
FossilOrigin-Name: 1af4a25631c4077a9bfc64e15eb92fbf5146c89c
2011-10-19 18:21:47 +00:00
dan
e459b07601 Merge the fts4-content branch with the trunk.
FossilOrigin-Name: 8a4077057ddeb08e8edc5f20a75abaaba7a278ba
2011-10-19 16:20:40 +00:00
dan
d17f70a624 Change the way ^ tokens work in FTS so that the filtering is done as part of reading the FTS index instead of waiting until an entire doclist has been retrieved and then filtering it.
FossilOrigin-Name: 9b58c59eb4efaa38ce50a3ce1b52f9ba578c71d6
2011-10-19 11:57:13 +00:00
dan
3f1ea8d114 If a token within an FTS query is prefixed with a '^' character, it must be the first token in a column of data to match.
FossilOrigin-Name: 63ac33c860eb32ce96699f06bf83121cec2ffaca
2011-10-18 19:39:41 +00:00
dan
8dab211632 Handle updating the only row of an FTS table correctly. Fix for [9fd058691].
FossilOrigin-Name: 7e24645be2fe0ffe092212e7bcfa5b4500305811
2011-10-13 17:16:45 +00:00
dan
deb9473250 Change FTS4 so that if both the content=xxx option and column names are specified, the virtual table assumes that the named columns correspond to columns of table xxx.
FossilOrigin-Name: 289ee43179369fce2fde50870d72c445e184e896
2011-10-05 15:11:30 +00:00
dan
8361b189c9 Fix a problem with IO error handling in the rebuild-index code.
FossilOrigin-Name: c6ba81fcad32192674bd510e607f787adc1f7038
2011-10-05 06:07:00 +00:00
dan
d8e454e19a Add experimental 'content' option to FTS4.
FossilOrigin-Name: 1d27ea741f61c624e18bdc6a3b1c2d8574a64ddc
2011-10-04 11:22:59 +00:00
drh
7fd3392981 More compiler warning fixes.
FossilOrigin-Name: ed2dda9329ca42e9c0be1986c78b091051e7598f
2011-06-20 19:00:30 +00:00
dan
d4d21fea8e Fix a problem with NEAR queries executed inside a transaction that writes the FTS table.
FossilOrigin-Name: 051c756c367837908f6691c0a36108e088c94f99
2011-06-16 16:06:05 +00:00
drh
6b96771b82 Changes to #ifdefs so that the build goes correctly if the only FTS macro
defined is SQLITE_ENABLE_FTS4.

FossilOrigin-Name: a0b43a320e6491de7070966ed7c3ec55fd660a85
2011-06-16 00:54:45 +00:00
dan
376d1e5c4f Add an interface to better test incremental loading of doclists by FTS4. Also some tests for this and term prefix queries.
FossilOrigin-Name: 7a3813138d1a5c5d99f4756c79998831d779774f
2011-06-15 08:30:29 +00:00
dan
abf2545ed9 Fix various issues to do with deferred tokens, NEAR expressions and matchinfo().
FossilOrigin-Name: 3972a787df5ec253b99b148385655e7b68d851fa
2011-06-08 18:39:07 +00:00
dan
5289b01524 Modify fts3rnd.test to run tests for both "ORDER BY docid ASC" and "ORDER BY docid DESC" with both order=ASC and order=DESC FTS tables. Fixes for some bugs found.
FossilOrigin-Name: 89f2f482e077241ac29a58eadf44a72a9c01f98c
2011-06-06 14:51:50 +00:00
dan
b46ee91729 Allow the "order=DESC" and "order=ASC" parameters in FTS4 "CREATE VIRTUAL TABLE" statements. Tables created with "order=DESC" store all doclists in descending order, which allows optimizations normally applied to "ORDER BY docid ASC" queries to be used with "ORDER BY docid DESC" queries instead.
FossilOrigin-Name: f6a0193f5a32603eb48bddc6297042dbd2ffe96e
2011-06-04 20:04:35 +00:00
dan
126ba6c0ac FTS changes: Remove unreachable code. Fix bugs. When processing a large doclist incrementally, read from disk incrementally too.
FossilOrigin-Name: a4c7e2820824e82580730c36f85aede2efa66754
2011-06-03 18:00:19 +00:00
dan
e414854800 Changes to improve performance and support LIMIT clauses on fts3 tables. This branch is unstable for now.
FossilOrigin-Name: 28149a7882a1e9dfe4a75ec5b91d176ebe6284e9
2011-06-02 19:57:24 +00:00
dan
382874fc5c Minor changes made while planning a larger change.
FossilOrigin-Name: 84097a4c759b1d65890af885f137d3cb16eef584
2011-05-28 15:57:40 +00:00
dan
98b08e71f2 If a prefix index of size N is not present, use a prefix index of size N+1 along with the terms index for queries for prefixes of length N.
FossilOrigin-Name: cc83991caae7c7d647432d5711b6cd80228c3002
2011-05-25 19:17:32 +00:00
dan
9d1f874ab3 Change fts4 so that the prefix= parameter is passes a comma-separated list of integers. For each integer N, a separate index of all prefixes of length N bytes is created.
FossilOrigin-Name: be59bf49402d2e2f4b95fb6668849f3745cb7bf2
2011-05-25 18:34:53 +00:00
dan
09643ab745 If the fts4 option prefix=1 is specified, have the fts4 module maintain an index of prefixes as well as terms.
FossilOrigin-Name: b5bdc639898ee22eebedeb560810e94e74de8aa4
2011-05-24 18:49:45 +00:00
dan
133d7dab17 Add extended return code SQLITE_CORRUPT_VTAB. Returned when the tcontents of the sqlite tables used internally by a virtual table module are invalid or inconsistent.
FossilOrigin-Name: 8844e8bfb87314fb40ecb92705e8fff88f72bb38
2011-05-17 15:56:16 +00:00
dan
62d6c7e32b Fix a couple of compiler warnings in the FTS code.
FossilOrigin-Name: 1a113359705d307efa1833b1dfa5542b045dae43
2011-05-04 16:30:05 +00:00
dan
3480a01dad Add documentation for the newly introduced sqlite3_vtab_config() and on_conflict() API functions. Test that encountering an SQLITE_MISMATCH in fts3 does not corrupt the full text index.
FossilOrigin-Name: abdd70ae0424ccadb7edaf16e970c78b5257d23c
2011-04-27 16:02:46 +00:00
dan
a311b80392 Extra tests for fts3. And fixes for conflict-handling related problems in fts3.
FossilOrigin-Name: fb4a355871d9482ccb28b6ba03b842b3cc87b696
2011-04-26 19:21:34 +00:00
dan
b061d058cb Add support for on conflict clauses to fts3/fts4.
FossilOrigin-Name: 6d2633a6d0a9bb88fb1a6adac0827dc51df2d4d2
2011-04-25 18:49:57 +00:00
dan
355c53b6ef Avoid passing NULL to the xOpen method of an FTS3/4 tokenizer.
FossilOrigin-Name: 0dd09fc034c127718366d3a3183e367d2f9fd82d
2011-04-19 06:43:22 +00:00
dan
4c600ac389 Change fts4aux to (additionally) report on term frequency in individual columns of an fts table.
FossilOrigin-Name: 3996f92a9aca9ac2628be003eca83c8f954c71de
2011-02-04 18:56:25 +00:00
dan
20d5f354e9 Optimize handling of equality and range constraints on the "term" column of an fts4aux table.
FossilOrigin-Name: 386701ded2bcee5309ec9f285d94e6eb1eade193
2011-02-02 17:30:43 +00:00
dan
0bfda98155 Fix a case in fts4 where a corrupt %_stat table could lead to a crash.
FossilOrigin-Name: 4ade96ce974244fc34bb97713d3cba10e3d33056
2011-02-01 17:55:48 +00:00
dan
a240fd01b6 Add virtual table module "fts4aux", used to inspect the full-text index of an fts4 table directly. Also add the "compress" and "uncompress" fts4 options.
FossilOrigin-Name: b010ddcc52889160af2183a33c5f483bb0ae91b9
2011-02-01 16:34:32 +00:00
dan
a8ab60c648 Fix a couple of crashes in fts3 that can occur if the database contents are inconsistent.
FossilOrigin-Name: 811e12cddfb3246c6cf3d5085bd9b72b12e05550
2011-01-13 11:20:04 +00:00
dan
86596e552e Fix a segfault that can occur in matchinfo if an fts4 table contains mostly zero-length documents. Specifically, if the table contains more rows than it does bytes of text.
FossilOrigin-Name: fe9047668eaaf76e7aa1ef1f32dec7c7c4226e45
2011-01-13 10:58:26 +00:00
dan
b9a93d70b8 Reduce the number of calls to sqlite3_realloc() made by fts3 when querying for position information of a term prefix.
FossilOrigin-Name: 7088d9450f403f12f67eed558e368573101245d6
2010-12-24 15:49:27 +00:00
shaneh
556f6bbc92 Fix some warnings under MSVC in fts3 module.
FossilOrigin-Name: c7771c0b22f2b45a47070cf84b9ecf1011e40404
2010-12-01 15:36:00 +00:00
drh
6ea28d6d84 Fix various compiler warnings.
FossilOrigin-Name: c412f61229b6ab1ac90b932afd56f7c5e3ba1cfe
2010-11-26 16:49:59 +00:00
dan
f9376540fa Add code for the matchinfo 'longest common substring' feature.
FossilOrigin-Name: 71011a4f9baf09ec6935ad591145252bf3c286ed
2010-11-24 19:26:18 +00:00