mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
26,030 Commits 1,487 Branches 166 Tags 416 MiB
addc7e0e5b
Commit Graph

334 Commits

Author SHA1 Message Date
dan
4e9d0d5418 Fix a problem in fts5 where a corrupt db could lead to a (huge) buffer overread. 
FossilOrigin-Name: c9a30e117f2c6c9ef0cc0c6ca5227d2961715b8f
 2016-02-29 17:34:16 +00:00
dan
437a54eaba Fix an fts5 problem causing 'optimize' to corrupt the fts index under some circumstances. 
FossilOrigin-Name: 251d6473f7c9ad95adbdcc49cb8eaf7c0956764b
 2016-02-29 17:16:26 +00:00
drh
7ff89807e0 Fix a harmless compiler warning in FTS5. 
FossilOrigin-Name: e52c90758c53cbb0f74e0d8582e4c3c1bee04150
 2016-02-15 23:46:36 +00:00
drh
5d2e2f1267 Remove the unused fts5BlobCompare() routine. 
FossilOrigin-Name: defc762dd11144402abd591278819768a080d5ea
 2016-02-11 21:53:57 +00:00
dan
2a44b3d877 Avoid a buffer overread when reading a corrupt fts5 structure record. 
FossilOrigin-Name: facbc424e555061135aced7b134bf6c19f54e484
 2016-02-11 18:08:38 +00:00
dan
22e8356368 Handle parser stack overflow when parsing fts5 query expressions. Fix some compiler warnings in fts5 code. 
FossilOrigin-Name: bc3f7900d5a06829d123814a5ac7b951bcfc1560
 2016-02-11 17:01:32 +00:00
dan
d120a1e3ab Add tests for and remove unreachable branches from fts5 in order to restore test coverage. 
FossilOrigin-Name: 22589018ac3321f7bd89ce9fa69e57eae395e324
 2016-02-05 21:09:26 +00:00
dan
b80b8f482c Further streamlining of fts5 prefix query code. 
FossilOrigin-Name: ca91bd8ac70a5b3fef127364f73ec675e58bb92c
 2016-02-05 19:18:02 +00:00
dan
4786cf5a20 Further improve performance of unindexed fts5 prefix queries. 
FossilOrigin-Name: c9c6457d8ea911f6cc63967127e58da3146fd3ef
 2016-02-04 19:45:19 +00:00
dan
eb715c62f9 Improve performance of fts5 prefix queries on detail=col tables. 
FossilOrigin-Name: ca11f46db047e7f131cef3893f73824758a2076b
 2016-02-03 20:04:59 +00:00
dan
735ff4a809 Enhance the performance of fts5 AND and OR queries. 
FossilOrigin-Name: 62ea9e5ab8bc1a20245beebceb5ea62dcd7ec84e
 2016-02-02 17:40:41 +00:00
dan
52f5d1ba6b Performance enhancement for fts5 column filter queries on detail=full tables. 
FossilOrigin-Name: 13fb4aa6a87c5c6258979953da82eedc1a7bf14f
 2016-01-30 19:16:11 +00:00
dan
18205c781d Further minor performance improvements and code-size reductions related to fts5 column filters on detail=col tables. 
FossilOrigin-Name: b4ac61aeee976296e7719949cd4fb496147a29e8
 2016-01-26 20:08:50 +00:00
dan
1c16bccb79 Improve the performance of fts5 column filters on detail=col tables. 
FossilOrigin-Name: 249a2d070c34bf884a04cb248b9691e239f2871c
 2016-01-26 19:30:49 +00:00
dan
e9eb1593f5 Fix an fts5 problem with using both xPhraseFirst() and xPhraseFirstColumn() within a single statement in detail=col mode. 
FossilOrigin-Name: 72d53699bf0dcdb9d2a22e229989d7435f061399
 2016-01-23 18:51:59 +00:00
dan
8498f387be Fix a problem involving detail=col and column filters. 
FossilOrigin-Name: 7558a0ad2276e91f2faced8ea405d9fdb4fa0c6e
 2016-01-23 14:45:36 +00:00
dan
795502d26c Experimental performance enhancements for fts5. 
FossilOrigin-Name: b5a57b812fd6a734cf2a342bf0b730ae18912d73
 2016-01-22 19:48:34 +00:00
dan
6d3c2889ec Add tests for fts5. Fix a crash that can occur in fts5 if the database content is corrupted. 
FossilOrigin-Name: acaf426449bf6fd3140fd63141750ff69d1119a5
 2016-01-18 17:48:28 +00:00
dan
0536a07c0e Add a debugging function to print human-readable versions of fts5 detail=none leaf pages. 
FossilOrigin-Name: 8358af3658d888516cdef5f8c8d89e9bdee53f91
 2016-01-18 09:08:56 +00:00
mistachkin
1e38927c12 Fix a couple C99-isms in FTS5 that cause compile errors on MSVC. 
FossilOrigin-Name: 584848d4396d7755027fab377c97e40fbcc90d21
 2016-01-15 05:39:55 +00:00
drh
0ea94db612 Yet another change to FTS5 trying to get it to merge successfully into sessions. 
FossilOrigin-Name: 8dedff3b9ac3e6bf9c131fee19f7d26dc1ebd61f
 2016-01-14 14:33:36 +00:00
dan
17535393b9 Fix another compiler warning in fts5 code. 
FossilOrigin-Name: d17bc2c92f4d086280e49a3cc72993be7fee2da7
 2016-01-14 14:19:50 +00:00
dan
f705e9deab Fix compiler warnings in fts5. 
FossilOrigin-Name: 5a343cc0336bba056df4449e6cd2e3fb9e75a105
 2016-01-14 14:15:54 +00:00
dan
fd7601a125 Improve performance of fts5 low level iterators. 
FossilOrigin-Name: bc5118f40a11f64ffb4e1c086277fa80b9764745
 2016-01-12 19:28:51 +00:00
dan
4364e5c9f7 Fix an fts5 problem involving detail=none, "ORDER BY rowid DESC" and deleted items. Also add tests to verify that the documented operator precedences are correct. 
FossilOrigin-Name: 8d05cfd48db1462dfd3efd502d53bd33214403fb
 2016-01-06 18:53:57 +00:00
dan
fe0c3cfee1 Changes to run many fts5 tests with detail=none and detail=col tables as well as the default detail=full. Also fixes for the bugs uncovered by running said tests. 
FossilOrigin-Name: 6322a1d984e7946735bace8a069ef24b31754b3b
 2016-01-02 19:01:56 +00:00
dan
811501e237 Fix the fts5 integrity-check code so that it works with detail=none tables. 
FossilOrigin-Name: 3a1df69e58e5830da1dff158eedbe3817743f58f
 2015-12-31 18:39:14 +00:00
dan
841243156f Fix some problems with fts5 detail=none tables. Some still remain. 
FossilOrigin-Name: 6a6f7bc40d6b4c8a4a254a9098c9d2f31fde69ca
 2015-12-31 17:36:58 +00:00
dan
d11e397b2c Updates to fts5 to support detail=none mode. As of this commit, many cases are still broken. 
FossilOrigin-Name: ac8f4cf0cede6bcbe47eeefb85d80a27e9278212
 2015-12-30 19:58:57 +00:00
dan
9a3a5275cd Add the xPhraseFirstColumn() and xPhraseNextColumn() API functions to fts5. For iterating through the set of columns that contain intances of a phrase. 
FossilOrigin-Name: 8c30605bcd0a78a5015948171145bc6f640b8358
 2015-12-29 19:35:03 +00:00
dan
9f44deed93 Change the name of the offsets=0 option to "detail=column". Have the xInst, xPhraseFirst and other API functions work by parsing the original text for detail=column tables. 
FossilOrigin-Name: 228b4d10e38f7d70e1b008c3c9b4a1ae3e32e30d
 2015-12-28 19:55:00 +00:00
dan
159fd77e0f Fix an fts5 integrity-check problem that affects offsets=0 tables with prefix indexes. 
FossilOrigin-Name: 609a0bc7f34e6dae74ce756aff920f3df78fe828
 2015-12-21 18:45:09 +00:00
dan
c58b9eeaaa Fix a problem with prefix queries on fts5 offsets=0 tables. 
FossilOrigin-Name: ad0987d83c252dd8d6a69321893629d7be805c28
 2015-12-18 19:07:14 +00:00
dan
b12dc84fbb Add the "offsets=0" option to fts5, to create a smaller index without term offset information. A few things are currently broken on this branch. 
FossilOrigin-Name: 40b5bbf02a824ca73b33aa4ae1c7d5f65b7cda10
 2015-12-17 20:36:13 +00:00
mistachkin
b9becaa268 Fix even more harmless compiler warnings. 
FossilOrigin-Name: 1d0e6aa119da8e15d35508f5d75ffc729979da92
 2015-12-16 23:30:30 +00:00
dan
dbbda39453 Have fts5 load its configuration from the xConnect() method is invoked. This ensures that the very first query run uses the correct value of the 'rank' option. 
FossilOrigin-Name: 33e6606f5e497e81119ec491cf2370f60bddafc0
 2015-11-06 12:50:57 +00:00
dan
d82211db56 Add the 'hashsize' configuration option to fts5, for configuring the amount of memory allocated to the in-memory hash table while writing. 
FossilOrigin-Name: 445480095e6877cce8220b1c095f334bbb04c1c3
 2015-11-05 18:09:16 +00:00
dan
24b8457911 Fix problems with prefix queries in fts5. 
FossilOrigin-Name: 3b5758c647530bd5c2e68d0ee3e9f58a96347ca4
 2015-10-27 17:48:57 +00:00
mistachkin
16158eeb49 Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: aa4e01ea1af327d1f8398ebea1c5bacc46698c3d
 2015-10-21 22:08:36 +00:00
dan
99b25309d0 Remove some branches made unreachable by recent changes from fts5. 
FossilOrigin-Name: ae350bfbfd59f912c469a51bf7d1409f3c8d958a
 2015-10-21 20:56:27 +00:00
dan
f2c9995dbe Minor optimization for fts5 queries. 
FossilOrigin-Name: 363b36d50b6f3e3d3e79c538be1167b071a5a710
 2015-10-21 20:07:08 +00:00
mistachkin
6d7734c27d Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: 0a903ec26bfbbe9bfd18bc34934e689ffdb01ce5
 2015-10-20 23:27:14 +00:00
dan
219f4d48a2 More optimizations for fts5 prefix queries. 
FossilOrigin-Name: b8fb263ed1b36d085437131795505e53af103d26
 2015-10-20 21:05:53 +00:00
dan
9cc4ae6f2a Fix #line directives added to generated file fts5.c. 
FossilOrigin-Name: 1f5f5804cd394a81b1c0712ce6d1052f228d0a40
 2015-10-20 19:55:35 +00:00
dan
8789fbb2bd Another optimization for fts5 prefix (and other) queries. 
FossilOrigin-Name: 60a8bde055a960c5b8cb4e231802c75617c942d8
 2015-10-19 20:49:10 +00:00
dan
5e5b454cd8 Another tweak to improve performance of fts5 prefix queries. 
FossilOrigin-Name: 69be427c864dc3de804ca3c5f1c6addcd33cd188
 2015-10-19 17:43:24 +00:00
dan
05903e6a87 Handle equality and range constraints on the "term" column of fts5vocab tables. Also have the same module report columns using names instead of indexes. 
FossilOrigin-Name: 6020d96ab47a0bebe41f691a3f44fd55d8b2d2b7
 2015-10-17 19:49:01 +00:00
dan
5f93ccd7fc Tests and minor fixes to improve coverage of FTS5 code. 
FossilOrigin-Name: f4de6d450e143cb1dd5447800d4b178e3db114c8
 2015-10-14 21:08:48 +00:00
mistachkin
cdabd7bd50 Fix harmless compiler warnings. 
FossilOrigin-Name: 1c46c194a2da24fe613d77b5a8d727cc2fc9faa4
 2015-10-14 20:34:57 +00:00
dan
df705d8185 Further tweaks to improve fts5 prefix query performance. 
FossilOrigin-Name: 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
 2015-10-12 19:12:29 +00:00
dan
98e713bb8d Further optimizations for fts5 prefix queries without a prefix index. 
FossilOrigin-Name: 83dc1ff7fa010715ca7f406a572f4ee444a967d7
 2015-10-07 19:06:21 +00:00
dan
22c7cca21f Allow the subsitute character (codepoint 26 - 0x1A) to appear in fts5 barewords. 
FossilOrigin-Name: baec1b96cb64a6a6317143a8df841e502afe3914
 2015-10-07 17:01:22 +00:00
dan
d46cc33131 Add tests for fts5 phrase queries with column filters. 
FossilOrigin-Name: f20f9f813f00cefcd427e506a09b0b973c782e3f
 2015-10-07 13:24:27 +00:00
dan
92099b0753 Add further tests for fts5 prefix queries. 
FossilOrigin-Name: accdc98b1291f07b802fd23f3ebc7dbc02ba09d3
 2015-10-07 09:02:50 +00:00
dan
a2507137f3 Optimizations for fts5 expressions that filter on column. More still to come. 
FossilOrigin-Name: bf1607ac155018573ca40fb58aca62c5fea7e60b
 2015-10-06 20:53:26 +00:00
dan
6f832d3dff Improve performance of prefix queries without a prefix index on fts5 tables. 
FossilOrigin-Name: f2f0184e9e1c9f121ee2ac864cd28c8cd8efecb5
 2015-10-05 19:41:16 +00:00
dan
bca189d5d8 Update fts5 to avoid using a statement journal for UPDATE and DELETE operations that affect at most a single row. 
FossilOrigin-Name: 5c83b9db46d61cfa76a1abed50467e2f02db0eb0
 2015-10-02 20:04:30 +00:00
dan
4ad0971919 Fix an off-by-one error in test function fts5_decode(). 
FossilOrigin-Name: 3a9f076250d9559d8ea94ba44095ba3ddbc5542d
 2015-09-29 12:19:51 +00:00
dan
6609bab24d Add test cases to cover fts5 integrity-check code. 
FossilOrigin-Name: 1d018c35b9e81982df036f5e62a4a42219b54e02
 2015-09-15 14:39:17 +00:00
dan
128ea696ea Fix a problem with fts5 "ORDER BY rowid DESC" queries and large terms. 
FossilOrigin-Name: b26d8f79c62cb71b4c3ea5df9d0e09558bbbd794
 2015-09-15 11:58:33 +00:00
dan
dafeda754b Improve test coverage of fts5_index.c. 
FossilOrigin-Name: c1f76686cee3918b1be785a4071d68cb3afda0ef
 2015-09-14 19:51:05 +00:00
mistachkin
16a93128f8 Fix harmless compiler warnings. 
FossilOrigin-Name: bfc7b84b766860d2e410702ba7c1166d7328309a
 2015-09-11 18:05:01 +00:00
dan
bcb04b6128 Fix some compiler warnings in fts5 code. 
FossilOrigin-Name: 0dc436116e55e3fd55eb6085ada71e099069b32d
 2015-09-11 14:15:46 +00:00
dan
56c8634853 Fix a segfault in fts5 that could occur if the database contents were corrupt. 
FossilOrigin-Name: 4931e37da4d2c26d7afc5432f7f0d534b51a85fa
 2015-09-10 16:19:01 +00:00
dan
b97efa0ba3 Update description of on-disk format in fts5_index.c. 
FossilOrigin-Name: 85aac7b8b6731e2f6880b80cfd62d431ea059799
 2015-09-10 15:49:16 +00:00
dan
f679d97d18 Fix an fts5 problem that could occur if a term and the first associated rowid are on different leaf pages. 
FossilOrigin-Name: ffe2796ac9244c62325fce4960f26c653321623c
 2015-09-10 10:01:30 +00:00
dan
a5aa8e1db2 Change the array of 16-bit offsets at the end of each page to an array of varints. 
FossilOrigin-Name: fab245bea4f283714c17bca22428d5eb4db5935a
 2015-09-10 05:40:17 +00:00
dan
fe8e2eba0a Remove the 0x00 terminators from the end of doclists stored on disk. 
FossilOrigin-Name: 00d990061dec3661b0376bd167082942d5563bfe
 2015-09-08 19:55:26 +00:00
dan
60f8139a96 Use macros to make the code in fts5_index.c easier to read. 
FossilOrigin-Name: 67ff5ae81357eb7fa28049bb724a22cb6f52e076
 2015-09-07 08:14:30 +00:00
dan
e386a1ba25 Experiment with a different fts5 leaf page format that allows faster seeks. 
FossilOrigin-Name: a1f4c3b543eed84e808f6b901a38179786fffe16
 2015-09-05 19:52:08 +00:00
dan
4dfe98a8c4 Enhance showfts5.tcl so that it can optionally display the number of terms in each segment. 
FossilOrigin-Name: d648ddd93de039820f5abe064c7bc1318cd9d6b1
 2015-09-04 11:13:00 +00:00
dan
e1c28d7229 Remove some more code from fts5_index.c by consolidating similar functions. 
FossilOrigin-Name: 59ae30b97b40faa363c55aa2664dead9eaeeddc0
 2015-09-03 14:22:27 +00:00
dan
5d144875bc Remove dead code from fts5_index.c. 
FossilOrigin-Name: 8a0a9b01e74072ee52fe393311ad591208fbbf7c
 2015-09-03 11:17:52 +00:00
mistachkin
7e968a7d0a Fix harmless compiler warnings in FTS5 code. 
FossilOrigin-Name: 0ddb2532b2daaaf1b0109ac360822f84cb999b7f
 2015-08-06 03:19:14 +00:00
dan
50b5491771 Add tests for the matchinfo-like test function. Fix problems found in test and fts5 code by doing so. 
FossilOrigin-Name: 9e3aafe44a0813aa2a0c6172fdba1440b8a973ec
 2015-08-05 19:35:59 +00:00
dan
f9419d17ff Remove all references to "docid" within fts5 source code and comments. Replace with "rowid". 
FossilOrigin-Name: dffd358f6cbf575d3b1045b1ce53429d15bade2a
 2015-08-05 07:43:46 +00:00
dan
291d090947 Merge the fts5-btree-index branch with this one. FTS5 changes only. 
FossilOrigin-Name: 9f1dcdc3e1644c21762dacf619fac70afe6318c5
 2015-07-27 12:19:11 +00:00
dan
8d90087146 Minor tweak to the SQL schema used by fts5. 
FossilOrigin-Name: 427bf09ba41158041d774aac2b5ffbd062c6eabe
 2015-07-27 11:43:36 +00:00
dan
44198f30b5 Merge latest trunk changes with this branch. 
FossilOrigin-Name: 5ec933c257884019484db7f533ce920012e47a1c
 2015-07-27 10:46:48 +00:00
dan
08406bf064 Fix warnings in fts5 code. 
FossilOrigin-Name: bd4f156d07d0db1f063d42891965920fc2229352
 2015-07-24 20:34:40 +00:00
dan
3baaacae03 Merge trunk changes, including fixes for compiler warnings in fts5 code, with this branch. 
FossilOrigin-Name: 7190d79ba452ceb1af77ce1375278b097816a8be
 2015-07-16 20:24:42 +00:00
dan
8694d6049f Fix compiler warnings in fts5 code. 
FossilOrigin-Name: e9bf275cd969eca6fb41384d3637528d6a19f819
 2015-07-16 20:17:57 +00:00
dan
e3229c19cb Use a WITHOUT ROWID table to index fts5 btree leaves. This is faster to query and only slightly larger than storing btree nodes within an intkey table. 
FossilOrigin-Name: 862418e3506d4b7cca9c44d58c2eb9dc915d75c9
 2015-07-15 19:46:02 +00:00
mistachkin
532f179cab Fix some harmless compiler warnings. 
FossilOrigin-Name: 3de030c904d125ccf41fa1929646b8a002b5341b
 2015-07-14 17:18:05 +00:00
dan
d2454f47e7 Improve the performance of docid merges in fts5. 
FossilOrigin-Name: b2de77a01cc5edcea2f98f7916e64cb33e6bd414
 2015-07-09 20:46:35 +00:00
dan
3dbd2397f4 Reduce the number of calls to malloc() made by fts5. 
FossilOrigin-Name: 898618ccf61d3ad166d9fc742e132d518338b5c3
 2015-07-09 19:02:19 +00:00
dan
bcf2163771 Merge the Fts5MultiSegIter and Fts5IndexIter objects. 
FossilOrigin-Name: 6ace73541ae48aeb19e0c2d9f6772d032d0a7eae
 2015-07-07 19:35:11 +00:00
dan
b2fe306a2e Further optimizations for fts5 b-tree seeks. 
FossilOrigin-Name: f37899686c032145f431f81c1072738536c61c7e
 2015-07-07 08:29:32 +00:00
dan
656773b675 Speed up seek operations on fts5 b-tree structures. 
FossilOrigin-Name: 7b7da1eb435d321fc4283f6aa2161fa1e16f2cf3
 2015-07-06 20:27:19 +00:00
dan
30ae503167 Optimize seek operations on fts5 b-trees. 
FossilOrigin-Name: 8cf02090ce53ec150492d77d9e5e5f27665bd34f
 2015-07-04 18:44:07 +00:00
dan
70c8885417 Fix a problem in the fts5 integrity-check. 
FossilOrigin-Name: 4e473559c9bdcf4d0b823e995a3642cd50ebd337
 2015-07-04 15:35:02 +00:00
dan
28a10c3c1f Rework the Fts5MultiSegIter structure a bit to make it more efficient. 
FossilOrigin-Name: 0778825d0ec9315c70659fae8d0640b209049dd8
 2015-07-03 20:47:18 +00:00
dan
00a1351e55 Speed up eof checks on fts5 cursors. 
FossilOrigin-Name: 3df4af5d8c28863783b0bc867abfbe31cc96f1b9
 2015-07-03 19:13:56 +00:00
dan
3f09beda45 Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c. 
FossilOrigin-Name: 7819002ed85497bbd0f9cf4d39df641573324436
 2015-07-02 15:52:21 +00:00
dan
6fbeb22596 Fix some cases in the fts5 code where a corrupt database could cause a buffer overread. 
FossilOrigin-Name: 360c57bbb8bad6cc84af140f61e0282fa06462b4
 2015-06-26 18:50:29 +00:00
mistachkin
a468027c8d Merge upstream changes. Get the test fixture working with FTS5. Fix compiler warnings. 
FossilOrigin-Name: bfcd16089cf8c65130c011a9718bf7812685c841
 2015-06-26 16:55:55 +00:00
mistachkin
ed52f9ff48 Initial changes to get FTS5 working with MSVC. 
FossilOrigin-Name: ef2052f81e33ca98e85a60f8a78cdd19a7c1c35c
 2015-06-26 04:34:36 +00:00
dan
ed2589f555 Add a script to combine all fts5 code into a single file - fts5.c - that can be used to build an SQLite loadable extension. 
FossilOrigin-Name: 46e86b0637248fb4d623c97778cc041eabe3636c
 2015-06-25 20:10:24 +00:00
dan
51ef0f57c7 Improve test coverage of fts5. 
FossilOrigin-Name: df5ccea80e8f0da83af5e595b539687006085120
 2015-06-23 18:47:55 +00:00
dan
3f3074e0c1 Remove the "#include sqlite3Int.h" from fts5Int.h. 
FossilOrigin-Name: e008c3c8e29c843ec945ddad54b9688bbf2bdb44
 2015-05-30 11:49:58 +00:00
dan
6d21f42db2 Add syntax to fts5 used to specify that a phrase or NEAR group should match a subset of columns. For example "[col1 col2 ...] : <phrase>". 
FossilOrigin-Name: 0fc0ea20920615f3e48ea2dbe2b7dcd979b0993e
 2015-05-29 15:55:30 +00:00
dan
af5cd564f0 Remove some dead code from fts5. Add auxiliary function api tests to the same. 
FossilOrigin-Name: 0f9df202cc58097afddb8dad662b7c7fdc2c7d0c
 2015-05-28 14:37:26 +00:00
dan
7f64db1343 Simplifications and minor optimizations to fts5 prefix queries that cannot use a prefix index. 
FossilOrigin-Name: aef89d9f6aa536efee347367558cf5d4ea81b238
 2015-05-26 18:22:01 +00:00
dan
494016a424 Avoid redundant loads from the %_data table in the fts5 code. 
FossilOrigin-Name: 02069782f8b7896a582582c79185b50418622736
 2015-05-25 11:46:33 +00:00
dan
2e7d35e2fe Avoid making redundant copies of position-lists within the fts5 code. 
FossilOrigin-Name: 5165de548b84825cb000d33e5d3de12b0ef112c0
 2015-05-23 15:43:05 +00:00
dan
27aac274b9 Improve test coverage of fts5_config.c. 
FossilOrigin-Name: 47dbfadb994814c9349d4c9c113b862c2e97c01a
 2015-05-18 17:50:17 +00:00
dan
369e4129b9 Improve test coverage of fts5_index.c. 
FossilOrigin-Name: 7aea8c6d99737c6c72078e0b4b9c5f8186021aa0
 2015-05-15 18:13:14 +00:00
dan
415ae725cb Add test cases. And some fixes. 
FossilOrigin-Name: adee788586197445672013d434e7ba47ce510b59
 2015-05-15 12:18:39 +00:00
dan
23d538885e Change fts5 doclist-index structures to be trees instead of flat lists. This only makes a difference for databases that contain millions of instances of the same token. 
FossilOrigin-Name: aa34bf666c384cf32a8d8166ab6d9afbca26a256
 2015-05-13 17:15:32 +00:00
dan
5e38f1c9bf Add the fts5vocab module, for direct access to the fts5 index. 
FossilOrigin-Name: 6bf93e3b56e6705b7d12bab5024fc615f373b36c
 2015-05-08 20:21:24 +00:00
dan
4591334dd4 Change to storing all keys in a single merge-tree structure instead of one main structure and a separate one for each prefix index. This is a file-format change. Also introduce a mechanism for managing file-format changes. 
FossilOrigin-Name: a684b5e2d9d52cf4700e7e5f9dd547a2ba54e8e9
 2015-05-07 19:29:46 +00:00
dan
cdbd150170 Improve coverage of fts5 tests. 
FossilOrigin-Name: 8e8136f2dc08082c2984462719d9cba0f212c92a
 2015-04-28 18:35:28 +00:00
dan
a3bdec7ee4 Change the fts5 content= option so that it matches fts5 columns with the underlying table columns by name, not by their position within the CREATE TABLE statement. 
FossilOrigin-Name: e38e2bb637844dae8ae5d5f3e23d8369e1b91e45
 2015-04-27 16:21:49 +00:00
dan
c1cea8f731 Further tests for fts5. 
FossilOrigin-Name: ffeb3ef3cfec3681b72bb28cfa612aa15e07887d
 2015-04-27 11:31:56 +00:00
dan
26c84373c4 Improve coverage of fts5_index.c slightly. 
FossilOrigin-Name: e5aaa01306597ffd2475dcb83ae889393f68d315
 2015-04-25 20:29:46 +00:00
dan
b33e64c83e Add tests for fts5. 
FossilOrigin-Name: e748651c940eae2389fe826cf5c25f1166a5e611
 2015-04-25 18:56:48 +00:00
dan
df5bd1fed2 Add the "unindexed" column option to fts5. 
FossilOrigin-Name: 86309961344f4076ddcf55d730d3600ec3b6e45c
 2015-04-24 19:41:43 +00:00
dan
def90aae18 Add extra tests for corrupt database handling in fts5. 
FossilOrigin-Name: 41449f7a0b5da6332eef48386c91ef63382c4783
 2015-04-24 15:56:09 +00:00
dan
206480992d Add extra OOM tests for fts5. 
FossilOrigin-Name: 2dd59b5762c944b3bdd270e95c6739cd4f530bfa
 2015-04-22 20:58:24 +00:00
dan
c51f5b3fd9 Fix problems with doclist-indexes involving very large rowids. 
FossilOrigin-Name: 1e5994097e4c740c5173ea9718c3935728fdb86f
 2015-04-22 20:14:46 +00:00
dan
5bb97c2b80 Fix a problem in fts5fault1.test. 
FossilOrigin-Name: a21d60cb2ac6463c012d82d1970d90da5da2a14a
 2015-04-21 20:13:38 +00:00
dan
d7889b26ac Fix an fts5 problem with large deletes. 
FossilOrigin-Name: e50e8031d6f804ebe50e0eec9a8b6e7f0152ecc3
 2015-04-21 19:07:39 +00:00
dan
80d3ea080a Fix some fts5 problems with very large position lists. 
FossilOrigin-Name: 2ea8f9cbe67dac60c1a0a661c95a03ecfa9a0b9a
 2015-04-20 18:48:57 +00:00
dan
687c5124e0 Logically store updates as (insert+delete) within the FTS tree. This allows keys to be annihilated more quickly under some circumstances. 
FossilOrigin-Name: 50fae1f0006c0e946b5214e73eedf2687a0016f9
 2015-04-15 18:49:20 +00:00
dan
dbca0ab9b0 Fix a problem preventing doclist indexes from being loaded. 
FossilOrigin-Name: b29109a083e5cd442cdd19f29d7be45b09c4f661
 2015-04-15 16:01:42 +00:00
dan
f1e58b7e8d Begin changing fts5 to use a delete flag so that delete markers may be annihilated more quickly. 
FossilOrigin-Name: 9341c070bb6140dbf559680952909674aa83fa55
 2015-04-14 20:15:41 +00:00
dan
929b2cfc2e Have fts5 integrity check verify that prefix indexes contain the same values as returned by prefix queries on the main terms index. 
FossilOrigin-Name: bdb8e82ab683f2438cde9f0b63e497dbf0141dcf
 2015-04-11 18:25:04 +00:00
dan
56e2497aaa Improve fts5 integrity-check so that it checks that DESC queries return the same as ASC. Change the poslist format slightly to make room for a delete-flag. 
FossilOrigin-Name: 49c1e74522a26e5dbe6f8305bc96487279b80dfb
 2015-04-11 16:23:31 +00:00
dan
8885708c3c When scanning the full-text index as part of the fts5 integrity-check, also run a point query for every term and verify that these results are consistent with those found by the linear scan. 
FossilOrigin-Name: ce972f6aab90f6929d018696f1ab3c2649eca802
 2015-03-21 15:37:19 +00:00
dan
c9ca6fb85d Avoid redundant string comparisons while merging fts5 segment b-trees. 
FossilOrigin-Name: 5c46820d9b4aae791a8704b69145bd81f1e6780d
 2015-03-10 19:24:30 +00:00
dan
92faed663a Fix some compiler warnings caused by signed/unsigned pointer conversions. 
FossilOrigin-Name: cccee7b5b1e84523f1c549d3052fd170e32bde80
 2015-03-07 15:46:41 +00:00
dan
47c467c80e Fix a couple of build problems. 
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
 2015-03-04 08:29:24 +00:00
dan
f400282bdf Further minor optimizations to flushing fts5 data to disk. 
FossilOrigin-Name: a07dcca9ef3821a6719ef9dbbc8ed861fa005035
 2015-02-27 09:41:10 +00:00
dan
5db7e2ca83 Fix suffix and prefix compression of terms in top-level fts5 segments. And a crash that could follow an OOM condition. 
FossilOrigin-Name: bb104b3646c6f07ed002be7360b08433ee7980d4
 2015-02-27 07:23:26 +00:00
dan
dfdc4b4613 Optimize copying data from fts5 in-memory hash tables to top level segments. 
FossilOrigin-Name: 8e3ca6323a2beab5f04250e24ae15b159d2aa0ac
 2015-02-26 20:49:09 +00:00
dan
4a7e11c629 Fix an fts5 bug in large incremental merges. 
FossilOrigin-Name: 208e3cb6b6dc8c7d824b64dec2034004c9fcbba5
 2015-02-26 14:54:03 +00:00
dan
6d8f42ec81 Instead of the 4-byte fields, use regular varints for the poslist-size field in fts5_hash.c. 
FossilOrigin-Name: 7eb022d7e5fdb180af823c82c47c938e4a7a355f
 2015-02-25 19:24:37 +00:00
dan
57fec54b53 Fix some problems with building fts5 and fts3 together using the amalgamation. 
FossilOrigin-Name: fb10bbb9f9c4481e6043d323a3018a4ec68eb0ff
 2015-02-02 11:32:20 +00:00
dan
5082140311 Minor optimizations to fts5 writes. 
FossilOrigin-Name: 1fffe51fa92f1784365140d5b163ab6c690981ae
 2015-01-31 15:23:44 +00:00
dan
719b0f66e5 Fix some problems with transactions that both read and write an fts5 table. 
FossilOrigin-Name: 0e225b15357765f132c3364b222f9931a608a5b2
 2015-01-29 20:59:34 +00:00
dan
900f1922ac Fix a problem with fts5 doclist-indexes that occured if the first rowid of the first non-term page of a doclist is zero. 
FossilOrigin-Name: f704bc059e06b01f1d68fa7dad89e33eace6c389
 2015-01-27 20:41:00 +00:00
dan
8ac3025419 Have fts5 store rowids in ascending order. Query speed is virtually the same regardless of rowid order, and ascending order makes some insert optimizations easier. 
FossilOrigin-Name: 5206ca6005bfa9dfc7346d4b89430c9748d32c10
 2015-01-24 19:57:03 +00:00
dan
641cb4360a Fix compression of keys stored on internal segment b-tree nodes by fts5. 
FossilOrigin-Name: 51444f67c0cc58a3023eb1cd78e7cf889da6c80f
 2015-01-23 17:43:21 +00:00
dan
626d9e3062 Remove some redundant code from fts5. 
FossilOrigin-Name: 939b7a5de25e064bdf08e03864c35ab718da6f6f
 2015-01-23 06:50:33 +00:00
dan
0b520cc5f9 Add further tests and fixes for fts5. 
FossilOrigin-Name: 5b295897153e9b26cd0d2e7ea112a4d461d0a665
 2015-01-22 19:13:08 +00:00
dan
4c2871bead Further tests and fixes for fts5. 
FossilOrigin-Name: c020a291ed293a66d21c5885e50a7fee04aa6366
 2015-01-21 20:30:14 +00:00
dan
dd8a501019 Fix an fts5 issue with loading doclist-indexes for a term that is the last thing on its leaf page. 
FossilOrigin-Name: e0d614425f7f5ffe266fdc03642931b1ec19ad25
 2015-01-21 18:23:25 +00:00
dan
b461380333 Add further tests for fts5 backend. 
FossilOrigin-Name: 09dabb3b9e140eec6cfda83bcb86b6b9f5cf54b6
 2015-01-21 16:10:59 +00:00
dan
fd0b436e10 Add extra fault injection tests to fts5. 
FossilOrigin-Name: f45a0dc0a8911c8aac5a1028ac4f543a709656e7
 2015-01-20 20:34:17 +00:00
dan
d8736bc3b8 Ensure an up to date copy of the fts5 configuration has been loaded into memory before attempting to modify the same configuration. 
FossilOrigin-Name: f30afd209aa4ce42766b1493750c4f5b5f1e9502
 2015-01-17 20:01:52 +00:00
dan
851ca6e715 Fix prefix indexes so that they work in characters, not bytes. 
FossilOrigin-Name: af8d43a4a08528bbae25ee38fe25de8a86f8a21c
 2015-01-13 17:25:08 +00:00
dan
d6b1c880d2 Add the fts5 'optimize' command. 
FossilOrigin-Name: e749be563d8e738af113bd301770e2f22763ab77
 2015-01-07 19:33:11 +00:00
dan
96a32bb51c Add the 'rebuild' and 'delete-all' commands. 
FossilOrigin-Name: 0cb2fed525778d96237b5b0943047665e1f636d1
 2015-01-07 17:11:11 +00:00
dan
aacf3d1a3b Remove the iPos parameter from the tokenizer callback. Fix the "tokenchars" and "separators" options on the simple tokenizer. 
FossilOrigin-Name: 65f0262fb82dbfd9f80233ac7c3108e2f2716c0a
 2015-01-06 19:08:26 +00:00
dan
2a28e507f7 Further fixes and test cases related to external content tables. 
FossilOrigin-Name: ce6a899baff7265a60c880098a9a57ea352b5415
 2015-01-06 14:38:34 +00:00
dan
2ae0f4c931 Fix a problem with prefix queries and the AND operator. 
FossilOrigin-Name: 38b3c65e3ee95eb7afadb76e0110570fbbc41e1b
 2014-12-18 20:01:15 +00:00
dan
e4bec37900 Fix various problems in fts5 revealed by fault-injection tests. 
FossilOrigin-Name: e358c3de5c916f2c851ab9324ceaae4e4e7a0fbd
 2014-12-18 18:25:48 +00:00
dan
845d0ab323 Begin testing fts5 OOM and IO error handling. 
FossilOrigin-Name: 2037dba62fdd995ad15b642abe499a790f5ffe5c
 2014-12-03 17:27:35 +00:00
dan
ca7fad3d2c Add a cookie mechanism to ensure that the %_config table is re-read as required. 
FossilOrigin-Name: bb4a37b53de60da9ec8b9317eec14afa99690828
 2014-11-28 20:01:13 +00:00
dan
cb11e73fc2 Add a %_config table to fts5. 
FossilOrigin-Name: 83491c56661ca78f96020ba68184bb3fb19e674f
 2014-11-27 20:03:45 +00:00
dan
cb62aae034 Add the auxiliary highlight() function to fts5. 
FossilOrigin-Name: 059092379f981eb919b500ce447006f9e645fc5a
 2014-11-24 16:24:33 +00:00
dan
6885bbc713 Add an "automerge=0" mode that disables auto-merging and falls back to fts4-style crisis merges. 
FossilOrigin-Name: 2397404e152b908d838e6491294b263b05943b3f
 2014-08-18 19:30:01 +00:00
dan
e2fb318e34 Automatically resize the hash table used by fts5. 
FossilOrigin-Name: f1cb48f412a5f200f1fe04f91072864f379db08f
 2014-08-12 08:36:00 +00:00
dan
1a669f84a1 Simplify the way position lists are copied when merging data. 
FossilOrigin-Name: 9f8d678a0ea75e169daf8b3f00bd05f52a050ea6
 2014-08-11 20:26:34 +00:00
dan
c7fe7a969b Replace the hash table borrowed from fts3. 
FossilOrigin-Name: 617e2fac1c128212254f71b1a8fddaf0d1d90262
 2014-08-11 19:44:52 +00:00
dan
b191db8390 Use multiple memory allocations for a single Fts5Structure object. This is probably less efficient but much easier to get right. 
FossilOrigin-Name: 2821825f7a481755a333dcdcad780b3e24448f20
 2014-08-09 18:02:27 +00:00
dan
1d3ed1f46a Add "segment promotion" to fts5. This prevents the FTS index from growing indefinitely as data is added and deleted. 
FossilOrigin-Name: ba359d78e166d78e0dc89e3c63a9a41e9ffea989
 2014-08-07 18:47:33 +00:00
dan
d90aab8f94 Avoid writing delete markers to the oldest segment in an FTS index. 
FossilOrigin-Name: 1baeb1cee61d9c56c718b50af034a24f1018a322
 2014-08-06 20:04:14 +00:00
dan
7b71fbaf93 Add support for savepoints to fts5. 
FossilOrigin-Name: 3b19eba042bb2eeb1be60f8d58ebaa0a045d6a5c
 2014-08-06 16:30:21 +00:00
dan
d5997d80cb Use doclist-indexes with "ORDER BY rowid ASC" fts5 queries as well. 
FossilOrigin-Name: d028ba6589f3122b635474c2683c0f93d5bc6c7c
 2014-08-05 19:00:22 +00:00
dan
9a67058a63 Fix fts5_index.c to use doclist-indexes when possible. Only some cases work so far. 
FossilOrigin-Name: 90b82d3ef613b2915e0e280dc1d2e5a2b617d59c
 2014-08-04 20:07:40 +00:00
dan
9af0705e84 Start changing things to use doclist indexes as required. code is not activated yet. 
FossilOrigin-Name: b8864da95db2c0e611116304d607e35a86c9247d
 2014-08-02 20:49:36 +00:00
dan
ff31da03e5 Add a special case to the integrity-check code to check that the final integer in a doclist index is as expected. 
FossilOrigin-Name: c98934155cb48adfda57bd0fd1b950226d45f67a
 2014-08-01 20:13:49 +00:00
dan
d34742901d Have the fts5 integrity-check verify that doclist indexes match the contents of the leaf pages that they index. 
FossilOrigin-Name: 37a7d3035eb4bbad7e32fe550321ac9fae611a57
 2014-08-01 19:27:07 +00:00
dan
a29284d65f Add "doclist index" records to the database. These are to make navigating within very large doclists faster. They are not yet used by queries. 
FossilOrigin-Name: 89377421ff69f2450364987afe781b6d8bcbf087
 2014-08-01 11:16:25 +00:00
dan
84d6fb394f Fix the xColumnSize() extension API. 
FossilOrigin-Name: 19504c4108472d2ad1281221642b8bd06eb69f4e
 2014-07-21 11:44:47 +00:00
dan
48eecfb8b9 Support "ORDER BY rowid ASC". 
FossilOrigin-Name: b96b5e166990e4ec363b24f66e04cfa5f00f6342
 2014-07-10 20:21:12 +00:00
dan
a2b569f955 Add support for prefix queries to fts5. 
FossilOrigin-Name: 75ebd3cd5904a4f89f7f3a9b25d32b2a42a31310
 2014-07-08 16:27:37 +00:00
dan
c18a8fe99c Add support for NEAR expressions to fts5. 
FossilOrigin-Name: 250ae8d40115e2e47cc5a1e8a427fa8c0a89124d
 2014-07-03 20:39:39 +00:00
dan
1dfacb4675 Add support for phrase queries to fts5. 
FossilOrigin-Name: 2e5652e6526b8fb3f5c163168d95bc0bb4c93686
 2014-07-02 20:18:49 +00:00
dan
acf6642819 Change the position list format so that its size in bytes is stored at the start of the list itself. 
FossilOrigin-Name: 62f2ff20418702ed0fbf708369edf5638445b51b
 2014-07-01 20:45:18 +00:00
dan
e369fe4352 Fix minor problems in term matching. 
FossilOrigin-Name: 94eeb077d08a1d2607f3ff3a9fbf18229ba475bb
 2014-06-26 12:31:41 +00:00
dan
22d43ec4e8 Begin adding query support to fts5. 
FossilOrigin-Name: 47a9f3cc92deefe163108e3507bd4614bf1f5da7
 2014-06-25 20:28:38 +00:00
dan
e0fa4107c2 Add some code for an experimental fts5 module. Does not work yet. 
FossilOrigin-Name: 1e0648dcf283d4f1f6159db4d2433b6cc635992e
 2014-06-23 11:33:22 +00:00