Commit Graph

131 Commits

Author SHA1 Message Date
dan
5211e54f95 Add a test for an untested branch in fts5_expr.c.
FossilOrigin-Name: ce08206b5c5bf6a81decf20c99a524dd92c3715b
2015-05-18 18:03:06 +00:00
dan
27aac274b9 Improve test coverage of fts5_config.c.
FossilOrigin-Name: 47dbfadb994814c9349d4c9c113b862c2e97c01a
2015-05-18 17:50:17 +00:00
dan
e0569a4aae Further test coverage improvements for fts5.
FossilOrigin-Name: 927d9a64e13c6b768f0a15475713192fcfaaf9e7
2015-05-16 20:04:43 +00:00
dan
369e4129b9 Improve test coverage of fts5_index.c.
FossilOrigin-Name: 7aea8c6d99737c6c72078e0b4b9c5f8186021aa0
2015-05-15 18:13:14 +00:00
dan
415ae725cb Add test cases. And some fixes.
FossilOrigin-Name: adee788586197445672013d434e7ba47ce510b59
2015-05-15 12:18:39 +00:00
dan
23d538885e Change fts5 doclist-index structures to be trees instead of flat lists. This only makes a difference for databases that contain millions of instances of the same token.
FossilOrigin-Name: aa34bf666c384cf32a8d8166ab6d9afbca26a256
2015-05-13 17:15:32 +00:00
dan
71ab324066 Allow the fts5vocab table to optionally provide data on a per-column basis.
FossilOrigin-Name: 3922276135a7825d0ede8d9c757e9cfe492f803a
2015-05-09 18:28:27 +00:00
dan
5e38f1c9bf Add the fts5vocab module, for direct access to the fts5 index.
FossilOrigin-Name: 6bf93e3b56e6705b7d12bab5024fc615f373b36c
2015-05-08 20:21:24 +00:00
dan
76724372ae Improve the error message returned by FTS5 if it encounters an unknown file format.
FossilOrigin-Name: f369caec145f311bb136cf7af144e2695badcb9b
2015-05-08 09:21:05 +00:00
dan
4591334dd4 Change to storing all keys in a single merge-tree structure instead of one main structure and a separate one for each prefix index. This is a file-format change. Also introduce a mechanism for managing file-format changes.
FossilOrigin-Name: a684b5e2d9d52cf4700e7e5f9dd547a2ba54e8e9
2015-05-07 19:29:46 +00:00
dan
7c479d51e5 Reorganize some of the fts5 expression parsing code. Improve test coverage of the same.
FossilOrigin-Name: c4456dc5f5f8f45f04e3bbae53b6bcc209fc27d5
2015-05-02 20:35:24 +00:00
dan
0501b2492a Further improvements to test coverage of fts5 code.
FossilOrigin-Name: d4331943dff259380c4025bb740d8aba6972d351
2015-05-01 20:38:57 +00:00
dan
557085a5a1 Improve test coverage of fts5.c.
FossilOrigin-Name: add4f4681c648dcbecaa68d08f7b2f4e6d63003c
2015-05-01 12:14:23 +00:00
dan
7b2ec1ae41 Improve fts5 tests.
FossilOrigin-Name: c1f07a3aa98eac87e2747527d15e5e5562221ceb
2015-04-29 20:54:08 +00:00
dan
90dd70226f Fix an fts5 bug in handling writes while there are active cursors.
FossilOrigin-Name: 07f70955392697556ca2951c9b6c3a5204cd5ec0
2015-04-28 20:24:50 +00:00
dan
cdbd150170 Improve coverage of fts5 tests.
FossilOrigin-Name: 8e8136f2dc08082c2984462719d9cba0f212c92a
2015-04-28 18:35:28 +00:00
dan
a3bdec7ee4 Change the fts5 content= option so that it matches fts5 columns with the underlying table columns by name, not by their position within the CREATE TABLE statement.
FossilOrigin-Name: e38e2bb637844dae8ae5d5f3e23d8369e1b91e45
2015-04-27 16:21:49 +00:00
dan
c1cea8f731 Further tests for fts5.
FossilOrigin-Name: ffeb3ef3cfec3681b72bb28cfa612aa15e07887d
2015-04-27 11:31:56 +00:00
dan
26c84373c4 Improve coverage of fts5_index.c slightly.
FossilOrigin-Name: e5aaa01306597ffd2475dcb83ae889393f68d315
2015-04-25 20:29:46 +00:00
dan
b33e64c83e Add tests for fts5.
FossilOrigin-Name: e748651c940eae2389fe826cf5c25f1166a5e611
2015-04-25 18:56:48 +00:00
dan
ef8b74324d Merge latest trunk changes with this branch.
FossilOrigin-Name: 1c78d8920fb59da3cb97dd2eb09b3e08dfd14259
2015-04-24 20:18:21 +00:00
dan
df5bd1fed2 Add the "unindexed" column option to fts5.
FossilOrigin-Name: 86309961344f4076ddcf55d730d3600ec3b6e45c
2015-04-24 19:41:43 +00:00
dan
def90aae18 Add extra tests for corrupt database handling in fts5.
FossilOrigin-Name: 41449f7a0b5da6332eef48386c91ef63382c4783
2015-04-24 15:56:09 +00:00
dan
206480992d Add extra OOM tests for fts5.
FossilOrigin-Name: 2dd59b5762c944b3bdd270e95c6739cd4f530bfa
2015-04-22 20:58:24 +00:00
dan
c51f5b3fd9 Fix problems with doclist-indexes involving very large rowids.
FossilOrigin-Name: 1e5994097e4c740c5173ea9718c3935728fdb86f
2015-04-22 20:14:46 +00:00
dan
5bb97c2b80 Fix a problem in fts5fault1.test.
FossilOrigin-Name: a21d60cb2ac6463c012d82d1970d90da5da2a14a
2015-04-21 20:13:38 +00:00
dan
d7889b26ac Fix an fts5 problem with large deletes.
FossilOrigin-Name: e50e8031d6f804ebe50e0eec9a8b6e7f0152ecc3
2015-04-21 19:07:39 +00:00
dan
80d3ea080a Fix some fts5 problems with very large position lists.
FossilOrigin-Name: 2ea8f9cbe67dac60c1a0a661c95a03ecfa9a0b9a
2015-04-20 18:48:57 +00:00
dan
687c5124e0 Logically store updates as (insert+delete) within the FTS tree. This allows keys to be annihilated more quickly under some circumstances.
FossilOrigin-Name: 50fae1f0006c0e946b5214e73eedf2687a0016f9
2015-04-15 18:49:20 +00:00
dan
dbca0ab9b0 Fix a problem preventing doclist indexes from being loaded.
FossilOrigin-Name: b29109a083e5cd442cdd19f29d7be45b09c4f661
2015-04-15 16:01:42 +00:00
dan
f1e58b7e8d Begin changing fts5 to use a delete flag so that delete markers may be annihilated more quickly.
FossilOrigin-Name: 9341c070bb6140dbf559680952909674aa83fa55
2015-04-14 20:15:41 +00:00
dan
929b2cfc2e Have fts5 integrity check verify that prefix indexes contain the same values as returned by prefix queries on the main terms index.
FossilOrigin-Name: bdb8e82ab683f2438cde9f0b63e497dbf0141dcf
2015-04-11 18:25:04 +00:00
dan
56e2497aaa Improve fts5 integrity-check so that it checks that DESC queries return the same as ASC. Change the poslist format slightly to make room for a delete-flag.
FossilOrigin-Name: 49c1e74522a26e5dbe6f8305bc96487279b80dfb
2015-04-11 16:23:31 +00:00
dan
8885708c3c When scanning the full-text index as part of the fts5 integrity-check, also run a point query for every term and verify that these results are consistent with those found by the linear scan.
FossilOrigin-Name: ce972f6aab90f6929d018696f1ab3c2649eca802
2015-03-21 15:37:19 +00:00
dan
f5fab92d82 Add an optimization to the fts5 unicode tokenizer code.
FossilOrigin-Name: f5db489250029678fce845dfb2b1109fde46bea5
2015-03-11 14:51:39 +00:00
dan
c9ca6fb85d Avoid redundant string comparisons while merging fts5 segment b-trees.
FossilOrigin-Name: 5c46820d9b4aae791a8704b69145bd81f1e6780d
2015-03-10 19:24:30 +00:00
dan
92faed663a Fix some compiler warnings caused by signed/unsigned pointer conversions.
FossilOrigin-Name: cccee7b5b1e84523f1c549d3052fd170e32bde80
2015-03-07 15:46:41 +00:00
dan
6f876c339d Fix the bm25() function so that it multiplies scores by -1 before returning them. This means better matches have a lower numerical score, so "ORDER BY rank" (not "ORDER BY rank DESC") does what you want.
FossilOrigin-Name: 3ee7b5a9f987c269251620ae7cc0fc7876b58ee5
2015-03-07 11:50:31 +00:00
dan
47c467c80e Fix a couple of build problems.
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
2015-03-04 08:29:24 +00:00
dan
f400282bdf Further minor optimizations to flushing fts5 data to disk.
FossilOrigin-Name: a07dcca9ef3821a6719ef9dbbc8ed861fa005035
2015-02-27 09:41:10 +00:00
dan
5db7e2ca83 Fix suffix and prefix compression of terms in top-level fts5 segments. And a crash that could follow an OOM condition.
FossilOrigin-Name: bb104b3646c6f07ed002be7360b08433ee7980d4
2015-02-27 07:23:26 +00:00
dan
dfdc4b4613 Optimize copying data from fts5 in-memory hash tables to top level segments.
FossilOrigin-Name: 8e3ca6323a2beab5f04250e24ae15b159d2aa0ac
2015-02-26 20:49:09 +00:00
dan
4a7e11c629 Fix an fts5 bug in large incremental merges.
FossilOrigin-Name: 208e3cb6b6dc8c7d824b64dec2034004c9fcbba5
2015-02-26 14:54:03 +00:00
dan
6d8f42ec81 Instead of the 4-byte fields, use regular varints for the poslist-size field in fts5_hash.c.
FossilOrigin-Name: 7eb022d7e5fdb180af823c82c47c938e4a7a355f
2015-02-25 19:24:37 +00:00
dan
57fec54b53 Fix some problems with building fts5 and fts3 together using the amalgamation.
FossilOrigin-Name: fb10bbb9f9c4481e6043d323a3018a4ec68eb0ff
2015-02-02 11:32:20 +00:00
dan
5082140311 Minor optimizations to fts5 writes.
FossilOrigin-Name: 1fffe51fa92f1784365140d5b163ab6c690981ae
2015-01-31 15:23:44 +00:00
dan
719b0f66e5 Fix some problems with transactions that both read and write an fts5 table.
FossilOrigin-Name: 0e225b15357765f132c3364b222f9931a608a5b2
2015-01-29 20:59:34 +00:00
dan
900f1922ac Fix a problem with fts5 doclist-indexes that occured if the first rowid of the first non-term page of a doclist is zero.
FossilOrigin-Name: f704bc059e06b01f1d68fa7dad89e33eace6c389
2015-01-27 20:41:00 +00:00
dan
8ac3025419 Have fts5 store rowids in ascending order. Query speed is virtually the same regardless of rowid order, and ascending order makes some insert optimizations easier.
FossilOrigin-Name: 5206ca6005bfa9dfc7346d4b89430c9748d32c10
2015-01-24 19:57:03 +00:00
dan
641cb4360a Fix compression of keys stored on internal segment b-tree nodes by fts5.
FossilOrigin-Name: 51444f67c0cc58a3023eb1cd78e7cf889da6c80f
2015-01-23 17:43:21 +00:00