Commit Graph

119 Commits

Author SHA1 Message Date
dan
2a6ecb3141 Fix a memory leak in fts5_expr.c.
FossilOrigin-Name: 399932a181437d84cd57234e76c4b65da1e4ac5f
2015-09-03 10:27:02 +00:00
dan
9c671b741c Further tests to raise coverage of fts5 synonym code to 100%. Fix a dropped error code in the same.
FossilOrigin-Name: bdedd838bb3028c586bcc9f643852ce1364adb49
2015-09-02 19:48:55 +00:00
dan
50ea76e748 Fix an issue with fts5 synonyms and NEAR(...) queries.
FossilOrigin-Name: f2e590700d8eb30da8ac3a92f6ca92a2bbe9c1fe
2015-09-02 18:56:01 +00:00
dan
bea34fc53a Fix a problem with fts5 synonyms and phrase queries. Also fix an OOM handling bug in fts5.
FossilOrigin-Name: a4c35fa2c94fe34b376670244fe72303c99868c1
2015-09-02 17:34:22 +00:00
dan
583611df65 Fix a problem handling OOM conditions within fts5 queries that feature synonyms.
FossilOrigin-Name: 11fa980897c6c7be218bbd9c4cd8253272d2c300
2015-09-02 14:17:38 +00:00
dan
df998c3d4f Fix a problem with fts5 synonyms and the xQueryPhrase() auxiliary function API.
FossilOrigin-Name: cf3e45e76d23e10ee06296c3561a341591597a04
2015-09-02 08:22:41 +00:00
dan
d917ad019c Fix a problem that occurs when more than 4 synonyms for a term appear within a single row.
FossilOrigin-Name: cd359550bdc2bf7be4c294b60130c9fc3f583902
2015-09-01 18:44:33 +00:00
dan
3faa07ee04 Add tests for fts5 synonyms implemented by adding extra terms to queries. And fixes for the same.
FossilOrigin-Name: dbcb73802b88f76be17f09f3eb83ffac542de633
2015-09-01 18:08:36 +00:00
dan
cb8b58caf7 Begin changes to allow synonym support by adding multiple terms to a query (an alternative to adding multiple terms to the FTS index).
FossilOrigin-Name: ad7feaed4cd6b1d6e6376bb82d1f5664ddd083f3
2015-08-31 20:06:06 +00:00
dan
ee0c0a8de3 Another change to the fts5 tokenizer API.
FossilOrigin-Name: fc71868496f45f9c7a79ed2bf2d164a7c4718ce1
2015-08-29 15:44:27 +00:00
dan
57e0add3f9 Change the fts5 tokenizer API to allow more than one token to occupy a single position within a document.
FossilOrigin-Name: 90b85b42f2b2dd3e939b129b7df2b822a05e243d
2015-08-28 19:56:47 +00:00
mistachkin
0cbb513a67 Fix harmless compiler warning in FTS5.
FossilOrigin-Name: 02448a9fccd4532daa667e4ba6ff1dbb7f68008a
2015-08-18 16:32:45 +00:00
dan
50b5491771 Add tests for the matchinfo-like test function. Fix problems found in test and fts5 code by doing so.
FossilOrigin-Name: 9e3aafe44a0813aa2a0c6172fdba1440b8a973ec
2015-08-05 19:35:59 +00:00
dan
f9419d17ff Remove all references to "docid" within fts5 source code and comments. Replace with "rowid".
FossilOrigin-Name: dffd358f6cbf575d3b1045b1ce53429d15bade2a
2015-08-05 07:43:46 +00:00
dan
dbac3ece0f Return an error message (instead of segfaulting) if test function fts5_expr() is invoked with no arguments.
FossilOrigin-Name: 8e818b8985c0196cd9671a6491796faaeebeb16e
2015-07-30 11:26:10 +00:00
dan
08406bf064 Fix warnings in fts5 code.
FossilOrigin-Name: bd4f156d07d0db1f063d42891965920fc2229352
2015-07-24 20:34:40 +00:00
dan
8694d6049f Fix compiler warnings in fts5 code.
FossilOrigin-Name: e9bf275cd969eca6fb41384d3637528d6a19f819
2015-07-16 20:17:57 +00:00
dan
3f09beda45 Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c.
FossilOrigin-Name: 7819002ed85497bbd0f9cf4d39df641573324436
2015-07-02 15:52:21 +00:00
mistachkin
ed52f9ff48 Initial changes to get FTS5 working with MSVC.
FossilOrigin-Name: ef2052f81e33ca98e85a60f8a78cdd19a7c1c35c
2015-06-26 04:34:36 +00:00
dan
e4449454c5 Make use of range constraints on the rowid field of an fts5 table in full-text queries.
FossilOrigin-Name: 32cbc0ed3699cc21302f0b6a159493117ad4bd4f
2015-06-05 19:05:57 +00:00
dan
6f277eb6bc Fix an fts5 problem in extracting columns from position lists containing large varints.
FossilOrigin-Name: 4ea015ab983300d420ef104cca550b22a6395866
2015-06-03 11:23:30 +00:00
dan
f67bb4a21b Change the fts5 multi-column syntax to use parenthesis instead of square brackets.
FossilOrigin-Name: ab85a6fc4f7580278fc9d1f0090fdcf0a90d065b
2015-06-02 19:38:15 +00:00
dan
167fd757f0 Merge latest trunk changes with this branch.
FossilOrigin-Name: c9ffda4abb4390bbc5719e269196e2807b254f97
2015-06-02 18:07:58 +00:00
dan
dbef6e752e Reimplement [ec69e09a] so that each call to the xNext() method does not involve two iterations of the match expression tree (only one).
FossilOrigin-Name: 80fe305b3eefb17310a9d6185d1c8cd73ee38b1e
2015-06-02 17:57:01 +00:00
dan
5350d91ed3 Improve performance of the fts5 AND operator.
FossilOrigin-Name: b43e9a5b7a0483ccb102316a4dbc5e32b5bc69ec
2015-06-01 19:17:06 +00:00
dan
34efc82eed Change fts5 expression processing to avoid linear scans of long doclists caused by phrases that match specific columns only.
FossilOrigin-Name: ec69e09a55b4daf1c40aeaaf9ee95091fe86f5c0
2015-06-01 09:15:20 +00:00
dan
3f3074e0c1 Remove the "#include sqlite3Int.h" from fts5Int.h.
FossilOrigin-Name: e008c3c8e29c843ec945ddad54b9688bbf2bdb44
2015-05-30 11:49:58 +00:00
dan
a8c024905c Add extra tests and fixes for multi-column matches.
FossilOrigin-Name: ae6794ffa23ef6191bd8834422abf322d978c11b
2015-05-29 19:00:22 +00:00
dan
6d21f42db2 Add syntax to fts5 used to specify that a phrase or NEAR group should match a subset of columns. For example "[col1 col2 ...] : <phrase>".
FossilOrigin-Name: 0fc0ea20920615f3e48ea2dbe2b7dcd979b0993e
2015-05-29 15:55:30 +00:00
dan
e96efb6426 Optimizations for fts5 queries that match against a specific column.
FossilOrigin-Name: b29ac50af0491a780a5a4c0985d88d0e5e014ba3
2015-05-28 19:57:12 +00:00
dan
494016a424 Avoid redundant loads from the %_data table in the fts5 code.
FossilOrigin-Name: 02069782f8b7896a582582c79185b50418622736
2015-05-25 11:46:33 +00:00
dan
2e7d35e2fe Avoid making redundant copies of position-lists within the fts5 code.
FossilOrigin-Name: 5165de548b84825cb000d33e5d3de12b0ef112c0
2015-05-23 15:43:05 +00:00
dan
21b7d2a9b8 Improve test coverage of fts5_unicode2.c.
FossilOrigin-Name: fea8a4db9d8c7b9a946017a0dc984cbca6ce240e
2015-05-22 06:08:25 +00:00
dan
27aac274b9 Improve test coverage of fts5_config.c.
FossilOrigin-Name: 47dbfadb994814c9349d4c9c113b862c2e97c01a
2015-05-18 17:50:17 +00:00
dan
e0569a4aae Further test coverage improvements for fts5.
FossilOrigin-Name: 927d9a64e13c6b768f0a15475713192fcfaaf9e7
2015-05-16 20:04:43 +00:00
dan
4591334dd4 Change to storing all keys in a single merge-tree structure instead of one main structure and a separate one for each prefix index. This is a file-format change. Also introduce a mechanism for managing file-format changes.
FossilOrigin-Name: a684b5e2d9d52cf4700e7e5f9dd547a2ba54e8e9
2015-05-07 19:29:46 +00:00
dan
7c479d51e5 Reorganize some of the fts5 expression parsing code. Improve test coverage of the same.
FossilOrigin-Name: c4456dc5f5f8f45f04e3bbae53b6bcc209fc27d5
2015-05-02 20:35:24 +00:00
dan
0501b2492a Further improvements to test coverage of fts5 code.
FossilOrigin-Name: d4331943dff259380c4025bb740d8aba6972d351
2015-05-01 20:38:57 +00:00
dan
7b2ec1ae41 Improve fts5 tests.
FossilOrigin-Name: c1f07a3aa98eac87e2747527d15e5e5562221ceb
2015-04-29 20:54:08 +00:00
dan
90dd70226f Fix an fts5 bug in handling writes while there are active cursors.
FossilOrigin-Name: 07f70955392697556ca2951c9b6c3a5204cd5ec0
2015-04-28 20:24:50 +00:00
dan
206480992d Add extra OOM tests for fts5.
FossilOrigin-Name: 2dd59b5762c944b3bdd270e95c6739cd4f530bfa
2015-04-22 20:58:24 +00:00
dan
47c467c80e Fix a couple of build problems.
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
2015-03-04 08:29:24 +00:00
dan
57fec54b53 Fix some problems with building fts5 and fts3 together using the amalgamation.
FossilOrigin-Name: fb10bbb9f9c4481e6043d323a3018a4ec68eb0ff
2015-02-02 11:32:20 +00:00
dan
8ac3025419 Have fts5 store rowids in ascending order. Query speed is virtually the same regardless of rowid order, and ascending order makes some insert optimizations easier.
FossilOrigin-Name: 5206ca6005bfa9dfc7346d4b89430c9748d32c10
2015-01-24 19:57:03 +00:00
dan
aa4d380a42 Handle the case where a tokenizer determines that there are zero tokens in an fts5 query term.
FossilOrigin-Name: 75f3d17f864072dfa2caee182b86cc4b9972d691
2015-01-19 11:15:36 +00:00
dan
27277c4e3c Fix some documentation issues in fts5.
FossilOrigin-Name: 512e1bdb4093b59d1494dfc63391476eadd52aea
2015-01-10 20:34:27 +00:00
dan
aacf3d1a3b Remove the iPos parameter from the tokenizer callback. Fix the "tokenchars" and "separators" options on the simple tokenizer.
FossilOrigin-Name: 65f0262fb82dbfd9f80233ac7c3108e2f2716c0a
2015-01-06 19:08:26 +00:00
dan
0fbc269fef Add support for external content tables to fts5.
FossilOrigin-Name: 17ef5b59f789e9fa35c4f053246d819987fd06f8
2015-01-03 20:44:58 +00:00
dan
e4bec37900 Fix various problems in fts5 revealed by fault-injection tests.
FossilOrigin-Name: e358c3de5c916f2c851ab9324ceaae4e4e7a0fbd
2014-12-18 18:25:48 +00:00
dan
845d0ab323 Begin testing fts5 OOM and IO error handling.
FossilOrigin-Name: 2037dba62fdd995ad15b642abe499a790f5ffe5c
2014-12-03 17:27:35 +00:00
dan
48d7014067 Fix the customization interfaces so that they match the documentation.
FossilOrigin-Name: fba0b5fc7eead07a4853e78e02d788e7c714f6cd
2014-11-15 20:07:31 +00:00
dan
192d413e95 Use doclist indexes for AND queries as well as phrases.
FossilOrigin-Name: 5d38e6edc40ef188fbf96505073797036aa6783a
2014-08-05 19:35:20 +00:00
dan
9af0705e84 Start changing things to use doclist indexes as required. code is not activated yet.
FossilOrigin-Name: b8864da95db2c0e611116304d607e35a86c9247d
2014-08-02 20:49:36 +00:00
dan
fdb4a30af2 Add hidden column "rank". Currently this always returns the same value as the bm25() function.
FossilOrigin-Name: 4cc048c3651e830a6aeded924c7f3a60b634e133
2014-07-30 19:41:58 +00:00
dan
92e497e517 Add the "loadfts" program, for performance testing the loading of data into fts3/fts4/fts5 tables.
FossilOrigin-Name: 770b9540c19ad1e3d24adff382332bf032065efd
2014-07-28 20:14:02 +00:00
dan
700b33d7a5 Add extension apis xRowCount, xQueryPhrase, xSetAuxdata and xGetAuxdata. And a ranking function that uses all of the above.
FossilOrigin-Name: c4d50428ab97f77e6721c4f8d03eaaf3ea91f3eb
2014-07-25 20:30:47 +00:00
dan
b30860abe2 Add a snippet() function to fts5.
FossilOrigin-Name: bdc58fd28a63ac9632c3df6c7768a9a236566605
2014-07-23 19:31:56 +00:00
dan
30b650e2f4 Fix issues with position lists and NEAR constraints.
FossilOrigin-Name: 16352d3654d5672cd0251db51dbe19f779373feb
2014-07-18 19:59:00 +00:00
dan
c5b44f3d90 Fix a problem with position list processing for OR queries.
FossilOrigin-Name: 5808f30fae0d844c52a785bf18872be371d4af68
2014-07-17 15:14:07 +00:00
dan
9cfd51f587 Begin adding interface for auxiliary functions.
FossilOrigin-Name: 1e2a7ba0889093416455f488fca893eaeb195d45
2014-07-16 19:15:57 +00:00
dan
48eecfb8b9 Support "ORDER BY rowid ASC".
FossilOrigin-Name: b96b5e166990e4ec363b24f66e04cfa5f00f6342
2014-07-10 20:21:12 +00:00
dan
a2b569f955 Add support for prefix queries to fts5.
FossilOrigin-Name: 75ebd3cd5904a4f89f7f3a9b25d32b2a42a31310
2014-07-08 16:27:37 +00:00
dan
103576dd75 Add support for AND, OR and NOT to fts5.
FossilOrigin-Name: 8682b87e794767cefcaa080fd53c8973c24c556a
2014-07-05 15:15:41 +00:00
dan
c2642d7c1e Add support for the "colname : <nearset>" syntax to fts5.
FossilOrigin-Name: 004667106e552e832a564b77e242b86f183d4441
2014-07-05 07:54:01 +00:00
dan
c18a8fe99c Add support for NEAR expressions to fts5.
FossilOrigin-Name: 250ae8d40115e2e47cc5a1e8a427fa8c0a89124d
2014-07-03 20:39:39 +00:00
dan
1dfacb4675 Add support for phrase queries to fts5.
FossilOrigin-Name: 2e5652e6526b8fb3f5c163168d95bc0bb4c93686
2014-07-02 20:18:49 +00:00
dan
e369fe4352 Fix minor problems in term matching.
FossilOrigin-Name: 94eeb077d08a1d2607f3ff3a9fbf18229ba475bb
2014-06-26 12:31:41 +00:00
dan
22d43ec4e8 Begin adding query support to fts5.
FossilOrigin-Name: 47a9f3cc92deefe163108e3507bd4614bf1f5da7
2014-06-25 20:28:38 +00:00
dan
e0fa4107c2 Add some code for an experimental fts5 module. Does not work yet.
FossilOrigin-Name: 1e0648dcf283d4f1f6159db4d2433b6cc635992e
2014-06-23 11:33:22 +00:00